Difference between revisions of "SAC Service Status"

From OSGeo
Jump to navigation Jump to search
m (GitHub GitHub GitHub)
 
(183 intermediate revisions by 8 users not shown)
Line 5: Line 5:
  
 
= Servers at OSL =
 
= Servers at OSL =
[[OSL | Open Source Labs]] - 4 physical machines containing ''x'' virtual machines.
+
[[OSL | Open Source Labs]] - 7 physical machines of which 5 ar lxd hosts containing ''x'' virtual machines/containers.
  
  
== Logging into VMS ==
+
== Logging into Physical Machines ==
Note this does not apply to osgeo4 and osgeo7 lxd hosts.  Refer to those sections for instructions on logging in.
 
If VM hanging, see [[OSL]] for how to open a ticket with OSUOSL's support.
 
  
All [[SAC#Members|SAC administrators]] have LDAP auth to the OSL Virtual Machines.  
+
Currently we have osgeo6 and backup.osgeo.osuosl.org that are physical machines under LDAP control.
 +
 
 +
'''Note this does not apply to osgeo3, osgeo4, osgeo7, osgeo8, osgeo9 lxd hosts'''.  Refer to those sections for instructions on logging in.
 +
If hanging, see [[OSL]] for how to open a ticket with OSUOSL's support.
 +
 
 +
This section currently only applies to '''osgeo6''' and '''backup'''.
 +
 
 +
All [[SAC#Members|SAC administrators]] have LDAP auth to the OSL Machines.  
  
 
To ssh into a server using your LDAP account, you can do the following replacing '''your_osgeo_login''' with your OSGeo login and '''vmname''' with the vm name of the server at OSL.
 
To ssh into a server using your LDAP account, you can do the following replacing '''your_osgeo_login''' with your OSGeo login and '''vmname''' with the vm name of the server at OSL.
  
   ssh '''your_osgeo_login'''@'''vmname'''.osgeo.osuosl.org
+
   ssh '''your_osgeo_login'''@'''servername'''.osgeo.osuosl.org
  
 
When prompted for password, use your OSGeo Login password.
 
When prompted for password, use your OSGeo Login password.
Line 23: Line 28:
 
[[SAC:Primary Administrators]] also have ssh key access in case LDAP is down and that will also apply to the physical machines. Worst case scenario use the information on [[OSL | Open Source Labs]] to file a ticket (SAC members only). Direct connection to virtual machines is by appending it's vm alias to .osgeo.osuosl.org.
 
[[SAC:Primary Administrators]] also have ssh key access in case LDAP is down and that will also apply to the physical machines. Worst case scenario use the information on [[OSL | Open Source Labs]] to file a ticket (SAC members only). Direct connection to virtual machines is by appending it's vm alias to .osgeo.osuosl.org.
  
* Live Status via [http://webextra.osgeo.osuosl.org/munin/index.html Munin]
+
== Logging into LXD Hosts ==
 +
 
 +
OSGeo3, OSGeo4, OSGeo7, OSGeo8, and OSGeo9 are all Ubuntu servers running LXD.
 +
LXD is a management system for LXC containers and QEMU VMS. LXD has a [https://www.youtube.com/channel/UCuP6xPt0WTeZu32CkQPpbvA channel] that covers its features.
 +
 
 +
To directly access the host, you go thru port 2222
 +
 
 +
  ssh tech_dev@''server_name''.osgeo.osuosl.org -p 2222
 +
 
 +
Only [[SAC:Primary Administrators]] have their ssh key installed under that account.  In order to access via KVM of these in event servers do not come up on a reboot, you need to go thru OSU OSL OpenVPN. To get an OpenVPN account, you need to put in a support ticket to support@osuosl.org.  In order to qualify for an OpenVPN account, you need to be an OSGeo SAC administrator. You will also need to install [https://openvpn.net/community-downloads/ OpenVPN client]) to use your OpenVPN account.
 +
 
 +
Each host on the private KVM side is named https://'''osgeo8'''.osuosl.oob -- where replace '''osgeo8''' with the relevant host. The .oob is the private network, so doesn't work unless you are connected to via OpenVPN.
 +
 
 +
The browser interface is sometimes clunky, so you might want to use  '''ipmitool''' installable on linux/unix or wsl using relevant package manager. KVM passwords are stored in [https://git.osgeo.org/gitea/sac/password-store SAC password-store].
 +
 
  
== Logging into LXC ==
+
A convenient block to add to your ~/.ssh/config to easily login to osgeo's LXD hosts follows:
  
A convenient block to add to your ~/.ssh/config to easily login to osgeo's LXC hosts is the following:
+
    Host osgeo?
 +
      User tech_dev
 +
      HostName %h.osgeo.osuosl.org
 +
      Port 2222
  
 +
Then you would be able to log into those hosts with commands like:
 +
 +
    ssh osgeo7
 +
 +
== Logging into LXD Containers and VMs ==
 +
 +
A convenient block to add to your ~/.ssh/config to easily login to osgeo's LXD hosted containers and vms is the following:
 +
 +
  # This stanza is only needed if you have an IdentityFile configured below.
 +
  # The IdentityFile from a target host is not automatically applied to the hop host, so we need to make it explicit:
 +
  Host hop.*.osgeo.org
 +
    IdentityFile "path/to/your/private/key"
 +
 
 
   Host osgeo*-*
 
   Host osgeo*-*
 
     ProxyCommand ssh hop.$(sed -e "s/-.*//" <<< "%h").osgeo.org -W $(sed -e "s/^osgeo[^-*]-//;s/$/.lxd/" <<< "%h"):%p
 
     ProxyCommand ssh hop.$(sed -e "s/-.*//" <<< "%h").osgeo.org -W $(sed -e "s/^osgeo[^-*]-//;s/$/.lxd/" <<< "%h"):%p
Line 34: Line 69:
 
     IdentityFile "path/to/your/private/key"
 
     IdentityFile "path/to/your/private/key"
  
Then you'll be able to access an LXC host on machine `osgeo3` with:
+
Then you'll be able to access a LXC Container or QEMU VM on machine `osgeo3` with:
  
   ssh osgeo3-matrix
+
   ssh yourusername@osgeo3-matrix
  
 
And one on machine `osgeo7` with:
 
And one on machine `osgeo7` with:
  
   ssh osgeo7-secure
+
   ssh yourusername@osgeo7-download
 +
 
 +
Note you still need to know where each LXC host is hosted... See successive sections to know what's on which machine.
 +
 
 +
 
 +
'''Troubleshooting:''' In case of "Permission denied (publickey)." after an update to a modern openSSH version, it might well be that your ssh key (RSH key) is disabled in your client in favour of more modern cyphers.
 +
 
 +
Ugly workaround: add one line `PubkeyAcceptedKeyTypes ...` in `.ssh/config`, to re-enable RSA keys for now (consider to generate a new key):
 +
 
 +
  vim .ssh/config
 +
  ...
 +
  Host *
 +
    ...
 +
    PubkeyAcceptedKeyTypes +ssh-rsa
 +
 
 +
... but better read e.g. [https://dev.to/bowmanjd/upgrade-ssh-client-keys-and-remote-servers-after-fedora-33-s-new-crypto-policy-47ag here]!
 +
 
 +
== osgeo 8 ==
 +
Server added April 2021. Intended to provide additional LXD capacity and backup
 +
[[osgeo8|Configuration Details]]
 +
 
 +
=== Services running on osgeo8 ===
 +
 
 +
==== hop ====
 +
hop.osgeo8.osgeo.org - jump host for accessing containers/vms on osgeo8
 +
 
 +
==== nginx ====
 +
http, https Proxy for all containers on osgeo8 and also provides mirror proxy for download.osgeo.org
 +
 
 +
==== centtie-7-pgrouting ====
 +
Centos 7 running PostgreSQL 15, PostGIS 3.3.2, gcc-4.8.5, cmake 3
 +
Configured to be a github self-hosted runner for testing centos for pgrouting project
 +
 
 +
[https://github.com/pgRouting/admin/wiki/CI%3A-Centos-7-GHA-runner Details of Github Action runner setup]
 +
 
 +
==== download8 ====
 +
Replica of download that is on osgeo7. Mirrors download and home folders from osgeo7.
 +
https://download-cache.osgeo.org
 +
 
 +
==== dronie-client ====
 +
a ci bot for dronie.osgeo.org which is used for git.osgeo.org/gitea ci jobs
 +
 
 +
==== grass-wiki ====
 +
Debian 10 (copy of wiki) home of https://grasswiki.osgeo.org upgraded to Bullseye debian 11.
 +
 
 +
==== grass ====
 +
https://grass.osgeo.org upgraded to Bullseye debian 11.
 +
 
 +
GRASS GIS server
 +
 
 +
Current DNS name: grass.osgeo.org
  
Note you still need to know where each LXC host is hosted... See successiv section sto know what's on which machine.
+
Debian 11 Bullseye
 +
 
 +
Web: Apache + Hugo (generated through cronjob from https://github.com/OSGeo/grass-website/), see https://github.com/OSGeo/grass-addons/tree/grass8/utils/cronjobs_osgeo_lxd
 +
 
 +
`unattended-upgrades` for automatic installation of security upgrades is installed and running
 +
 
 +
ssh: reachable via jumphost.
 +
 
 +
==== meshcentral ====
 +
https://remote.osgeo.org
 +
This is a remoting tool currently setup to test livecd vms via a web browser.
 +
 
 +
4 VMS currently set up on osgeo8 accessible from this. Currently based on livecd 16rc1 snapshots, with wm install script run.
 +
 
 +
 
 +
==== pgrouting-dev ====
 +
For pgrouting development use to do things like pushing docker images on a scheduled basis.
 +
Perhaps later for demo sites.  WIP.
 +
 
 +
==== woodie-client ====
 +
 
 +
Separate agent for woodie-server
 +
 
 +
==== woodie-server ====
 +
Ubuntu 22.04 with docker.  https://woodie.osgeo.org
 +
A CI server + agent running https://woodpecker-ci.org and authentication/repo for https://git.osgeo.org/gitea
 +
 
 +
woodpecker server and agent are running on this each in a docker container using a docker-compose yaml running under account woodie.
 +
 
 +
Setup details [https://git.osgeo.org/gitea/sac/osgeo8/wiki/woodie-server OSGeo8 Woodie setup]
 +
 
 +
== osgeo 9 ==
 +
Server added April 2021. Is an LXD host.  Also Stores lxd images used by other lxd hosts.
 +
[[osgeo9|Configuration Details]]
 +
 
 +
=== Services running on osgeo9 ===
 +
==== hop ====
 +
hop.osgeo9.osgeo.org. For LDAP users allows them to hop thru to get to other containers.
 +
 
 +
==== Secure (LDAP )  ====
 +
[https://git.osgeo.org/gitea/sac/osgeo9/wiki/secure-container secure] -- ldap.osgeo.org [[SAC:LDAP]] used for ldap service (a rebuild of old secure.osgeo.osuosl.org) now on Debian 11
 +
Moved from osgeo7
 +
 
 +
==== ldap-web ====
 +
 
 +
Currently housing https://id.osgeo.org/ for LDAP management.
 +
Deployed via ansible
 +
Moved from osgeo9
 +
 
 +
* id.osgeo.org
 +
 
 +
==== jitsi ====
 +
https://meet.osgeo.org.  Requires LDAP for moderation of a room, but other users are allowed in as guests.
 +
 
 +
==== nextcloud  ====
 +
https://nextcloud.osgeo.org
 +
 
 +
Ubuntu 22.04 LXD/nginx/postgresql 14 container for document sharing similar to dropbox/google drive - nextcloud.lxd - https://nextcloud.osgeo.org [https://git.osgeo.org/gitea/sac/osgeo9/wiki/Nextcloud-container Nextcloud Setup]
 +
 
 +
home of https://nextcloud.osgeo.org
 +
This server does not use ssh osgeo-ldap as it was the first container built.  However nextcloud.osgeo.org does authenticate with osgeo ldap.
 +
 
 +
TODO: add special page for this
 +
 
 +
==== nginx ====
 +
nginx (for web proxy of traffic of osgeo9 containers) additional mirror proxy for download.osgeo.org
 +
 
 +
==== adventure (WIP)====
 +
https://adventure.osgeo.org runs https://github.com/thecodingmachine/workadventure software
 +
 
 +
==== dronie-client ====
 +
a ci bot for dronie.osgeo.org which is used for git.osgeo.org/gitea ci jobs
 +
 
 +
==== limesurvey ====
 +
Debian 10, PostgreSQL 13, PHP 8 with ldap/ssh. https://limesurvey.osgeo.org
 +
Setup detailed on [https://git.osgeo.org/gitea/sac/osgeo3/wiki/limesurvey-container limesurvey container]
 +
 
 +
==== pixelfed ====
 +
 
 +
SHUT OFF (both container and website) cause of lack of interest.  Container is still there.
 +
[[Pixelfed]] instance reachable on https://photo.osgeo.org to house community photos
 +
 
 +
==== peertube ====
 +
 
 +
[[Peertube]] instance reachable on https://video.osgeo.org
 +
 
 +
==== pretalx  ====
 +
Ubuntu 20.04 with OSGeo LDAP and Docker installed.  pretalx software runs in Docker.
 +
https://talks.osgeo.org - for OSGeo Talk collection and voting See [[Pretalx]]
 +
 
 +
==== weblate ====
 +
'''Container Name:''' weblate (for doc translation)
 +
 
 +
Houses: https://weblate.osgeo.org  (for document translation to different languages)
 +
For further details refer to [[SAC:Weblate]]
 +
 
 +
==== wordpress ====
 +
Houses: https://www.osgeo.org, https://2018.foss4g.org, https://2020.europe.foss4g.org websites
 +
Runs Debian 11 with OSGeo LDAP access
 +
 
 +
[https://git.osgeo.org/gitea/sac/osgeo9/wiki/wordpress Setup details of wordpress container]
 +
 
 +
==== wiki ====
 +
(wiki.osgeo.org)  - debian10 lxd container with ldap/ssh (MediaWiki 1.34, 10.3.22-MariaDB, PHP 7.3)
 +
 
 +
Note that this is a rebuild of wiki.osgeo.org that used to be on osgeo7 old-wiki container.  Move 2020-05-22
 +
Setup steps are at [https://git.osgeo.org/gitea/sac/osgeo9/wiki/wiki-container wiki container setup]
 +
 
 +
==== discourse ====
 +
Ubuntu 22.04 VM running docker that hosts an instance of discourse software
 +
https://discourse.osgeo.org
 +
It current allows OSGeo LDAP, OSGeo Gitea, GitHub, and sign up
 +
repo is at [https://git.osgeo.org/gitea/sac/discourse/wiki/discourse - Discourse-VM]
  
 
== osgeo 7 ==
 
== osgeo 7 ==
Line 56: Line 253:
  
 
Only the download.osgeo.org is directly exposed ssh via port 22.  To access the other containers, you can tunnel thru  
 
Only the download.osgeo.org is directly exposed ssh via port 22.  To access the other containers, you can tunnel thru  
download.osgeo.org -- You will need an ssh key on download already to do so.  If you do not already have a key installed, put in a [https://trac.osgeo.org/osgeo/newticket SAC Ticket Request]
+
download.osgeo.org -- You need to be in the shell group to be able to access download and the other servers.  If you are not already put in a [https://trac.osgeo.org/osgeo/newticket SAC Ticket Request].  You also need to have your public key registered. To do so edit your profile [https://id.osgeo.org/ldap/edit]  (and put in your public key)
  
 
A convenient block to add to your own `.ssh/config` file follows:
 
A convenient block to add to your own `.ssh/config` file follows:
Line 62: Line 259:
  
 
  Host osgeo7-*
 
  Host osgeo7-*
   ProxyCommand ssh your_osgeo_id@download.osgeo.org -W $(sed -e "s/^osgeo7-//;s/$/.lxd/" <<< "%h"):%p
+
   ProxyCommand ssh your_osgeo_id@hop.osgeo7.osgeo.org -W $(sed -e "s/^osgeo7-//;s/$/.lxd/" <<< "%h"):%p
 
   IdentityFile "path/to/your/private/key"
 
   IdentityFile "path/to/your/private/key"
  
Line 69: Line 266:
 
   ssh your_id@osgeo7-<container_name>
 
   ssh your_id@osgeo7-<container_name>
  
 +
'''Troubleshooting:''' In case of "Permission denied (publickey)." after an update to a modern openSSH version, it might well be that your ssh key (RSH key) is disabled in your client in favour of more modern cyphers.
  
Services currently on:
+
Ugly workaround: add one line `PubkeyAcceptedKeyTypes ...` in `.ssh/config`, to re-enable RSA keys for now (consider to generate a new key):
  
=== Download ===
+
  vim .ssh/config
'''Container Name:''' download
+
  ...
 +
  Host *
 +
    ...
 +
    PubkeyAcceptedKeyTypes +ssh-rsa
  
Houses: https://download.osgeo.org, https://bottle.download.osgeo.org download sites
+
... but better read e.g. [https://dev.to/bowmanjd/upgrade-ssh-client-keys-and-remote-servers-after-fedora-33-s-new-crypto-policy-47ag here]!
'''Moved 5/5/2019 from osgeo3'''
 
https://download.osgeo.org Setup detailed in [https://git.osgeo.org/gitea/sac/osgeo7/wiki/Download-Container OSGeo7 Download container]  
 
  
This is new home of download.osgeo.org, it's a debian 9 container (now locked down to only allow ssh key access / ldap auth) for ssh.  It is running nginx instead of apache that the old ran. It has webdav with single local htaccess account geotools for geotools bot use.
+
 
 +
=== Services on osgeo7 ===
 +
 
 +
==== Download ====
 +
* Houses: https://upload.osgeo.org, https://bottle.download.osgeo.org download sites
 +
** '''Moved 5/5/2019 from osgeo3'''
 +
* Setup of https://upload.osgeo.org detailed in [https://git.osgeo.org/gitea/sac/osgeo7/wiki/Download-Container OSGeo7 Download container]
 +
* https://download.osgeo.org goes thru osgeo8 and osgeo9, but data is pulled from this container
 +
* It is also the official hop container for osgeo7.  Accessible via ssh hop.osgeo7.osgeo.org
 +
* It's a debian 10 container (now locked down to only allow ssh key access / ldap auth) for ssh.  It is running nginx instead of apache that the old ran. <del>It has webdav with single local htaccess account geotools for geotools bot use.</del>
  
 
In order to be able to log in or sftp  
 
In order to be able to log in or sftp  
Line 86: Line 294:
 
* Once you are in shell group, go to https://id.osgeo.org/ldap/edit and put in your public key.
 
* Once you are in shell group, go to https://id.osgeo.org/ldap/edit and put in your public key.
  
You should then be able to log into download with your private key.
+
You should then be able to log into download (and all other hop servers on the other hosts) with your private key.
  
 
You can put in a [https://trac.osgeo.org/osgeo/newticket ticket] to request such access.
 
You can put in a [https://trac.osgeo.org/osgeo/newticket ticket] to request such access.
  
=== WWW ===
+
==== nginx  ====
'''Container Name:''' wordpress
+
Proxy that routes all http/https traffic for the other containers (can be accessed via osgeo7 host lxc or ubuntu@osgeo7-nginx if your key is installed on ubuntu user).
 
+
The nginx container holds the letsencrypt https SSL certs for all the containers and handles the renewal of the letsencrypt certs using certbot renew cronjob.
Houses: https://www.osgeo.org, https://2018.foss4g.org, https://2020.europe.foss4g.org websites
+
Prometheus server to collect all monitoring logs from OSGeo7 (only accessible by OSGeo3), these get queried via monitor.osgeo.org (running on osgeo3) via grafana server.
 
 
'''Moved 2019-09-03 from web18a.osgeo.osuosl.org to osgeo7 wordpress container'''
 
 
 
Note this is a replica of web18a VM server provided by OSUOSL
 
 
 
[https://git.osgeo.org/gitea/sac/osgeo7/wiki/wordpress-container Setup details of wordpress container]
 
  
=== Secure (LDAP ) ldap.osgeo.org ===
+
==== tracsvn (trac, svn, git) ====
'''Container Name:''' secure
+
https://trac.osgeo.org, https://git.osgeo.org/gitea, https://svn.osggeo.org
[https://git.osgeo.org/gitea/sac/osgeo7/wiki/secure secure] -- ldap.osgeo.org [[SAC:LDAP]] used for ldap service (a rebuild of old secure.osgeo.osuosl.org) now on Debian 9
 
 
 
=== TracSVN (trac, svn, git) ===
 
 
This used to be housed on osgeo3 in VM TracSVN VM, and was moved 2019-10-12 to osgeo7 as tracsvn container.
 
This used to be housed on osgeo3 in VM TracSVN VM, and was moved 2019-10-12 to osgeo7 as tracsvn container.
* See [[TracsvnVM]] for full details, and some notes on services running here
+
It has since been upgraded to Debian 9 (Stretch with plans to upgrade more)
 +
* See [[TracSVN]] for full details, and some notes on services running here
  
'''Container Name:''' tracsvn
+
See [[TracSVN]] for info about what it contains (spoiler: trac, svn, gitea)
  
This houses the following:
+
==== old-wiki (stopped) ====
 +
This used to be housed on osgeo3, and was moved 2019-09-14 to osgeo7 as old-wiki container.
 +
wiki.osgeo.org moved back to osgeo3 on 2020-05-22 and in wiki container. The wiki container is a complete rebuild with files and database restored and upgraded.
 +
Refer to the osgeo3 section for more details.
  
https://git.osgeo.org/gitea
+
[https://git.osgeo.org/gitea/sac/osgeo7/wiki/old-wiki-container old wiki container] -- used for wiki service (it is an lxd2pc created image of wiki.osgeo.osuosl.org VM that was on osgeo3)
https://trac.osgeo.org
 
https://svn.osgeo.org
 
  
=== old-wiki (stopped) ===
+
See [[OSGeo Wiki]]
This used to be housed on osgeo3, and was moved 2019-09-14 to osgeo7 as old-wiki container.
 
wiki.osgeo.org moved back to osgeo3 on 2020-05-22 and in wiki container. The wiki container is a complete rebuild with files and database restored and upgraded
 
  
'''Container Name:''' old-wiki
+
==== nextcloud-ubuntu (stopped) ====
 +
Moved to osgeo9
  
[https://git.osgeo.org/gitea/sac/osgeo7/wiki/old-wiki-container old wiki container] -- used for wiki service (it is an lxd2pc created image of wiki.osgeo.osuosl.org VM that was on osgeo3)
+
==== collabora (stopped) ====
 +
# https://collabora.osgeo.org
 +
Ubuntu 18.04 LXD container for  LibreOffice/MS Office online document editor currently used exclusively by nextcloud.osgeo.org.  Setup detailed in Nextcloud setup.
 +
No longer needed now that nextcloud is using a self-contained collabora instance.
  
See [[OSGeo Wiki]]
+
==== live ====
 +
Home of live.osgeo.org (created 2021-10-05ish
 +
Running Ubuntu 20.04 with OSGeo LDAP SSH
  
=== Nextcloud nextcloud.osgeo.org ===
+
==== dronie-server ====
'''Container Name:''' nextcloud-ubuntu,  collabora (for document/view/editing)
+
*  2.0 server of drone.io runs in an LXD container details [[Dronie]] https://dronie.osgeo.org
  
Nextcloud - Ubuntu 18.04 LXD/nginx/postgresql 11 container for document sharing similar to dropbox/google drive - nextcloud-ubuntu.lxd - https://nextcloud.osgeo.org [https://git.osgeo.org/gitea/sac/osgeo7/wiki/Nextcloud-Ubuntu-Setup Nextcloud Setup]
+
==== old-projects (stopped) ====
 +
-- this is the old projects.osgeo.osuosl.org migrated from osgeo4 as an lxd container, so more or less the same as it was before, with the exception that all the websites are now proxied thru the nginx container. Websites on it are community-review.foss4g.org and spatialreference.org
  
Collabora - Ubuntu 18.04 LXD container for  LibreOffice/MS Office online document editor currently used exclusively by nextcloud.osgeo.org.  Setup detailed in Nextcloud setup.
+
To access you need to go thru download.osgeo.org -> old-projects
  
=== GRASS GIS server ===
 
'''Container Name:''' grass
 
  
Current DNS name: staging.grass.osgeo.org (to be migrated to grass.osgeo.org once the CMS will be retired in 2020 to old.grass.osgeo.org)
+
==== old-web (stopped) ====
 +
The old web.osgeo.osuosl.org (was on osgeo3)  
  
Debian 10 Buster
+
* mapguide.osgeo.org
  
Web: Apache + Hugo (generated through cronjob from https://github.com/OSGeo/grass-website/), see https://github.com/OSGeo/grass-addons/tree/master/tools/cronjobs_osgeo_lxd
+
==== old-webextra ====
 +
This is a replica of webextra.osgeo.osuosl.org that was hosted on osgeo3
  
Will replace old grass.osgeo.org (on osgeo6) in 2020.
+
Started move on November 29th 2019 and completed December 8th, 2019
 +
* foss4g.org
 +
* europe.foss4g.org
 +
* video.foss4g.org
 +
* planet.osgeo.org
 +
* various old foss4g.org years
 +
* <del>live.osgeo.org</del> moved to dedicated container
 +
* journal.osgeo.org (not sure what this is for, should be retired?)
 +
* <del>vmap0.tiles.osgeo.org</del> #removed site
  
=== NGINX proxy  ===
+
==== pycsw ====
'''Container Name:''' nginx
+
'''Container Name:''' pycsw
  
Proxy that routes all http/https traffick for the other containers (can be accessed via osgeo7 host lxc or ubuntu@osgeo7-nginx if your key is installed on ubuntu user).
+
* https://demo.pycsw.org
The nginx container holds the letsencrypt https SSL certs for all the containers and handles the renewal of the letsencrypt certs using certbot renew cronjob.
+
* '''OGC CSW Reference Implementation and Server demo'''
Prometheus server to collect all monitoring logs from OSGeo7 (only accessible by OSGeo3), these get queried via monitor.osgeo.org (running on osgeo3) via grafana server.
+
* deployment setup at https://github.com/geopython/demo.pycsw.org
 +
* running hourly teardown/setup cron via docker-compose
 +
* migrated from [[AdhocVM#Existing_services_hosted_on_the_Ad-hoc_VM:|Adhoc VM]] thanks to [https://trac.osgeo.org/osgeo/ticket/2452 SAC] (May 2020)
  
=== Dronie Server dronie.osgeo.org ===
+
==== mapserver ====
'''Container Name:''' dronie-server
+
'''Container Name:''' mapserver
*  1.0 server of drone.io runs in an LXD container details [[Dronie]] https://dronie.osgeo.org
 
  
=== Old Projects ===
+
* https://demo.mapserver.org  (Moved from old-adhoc 2021-11-07)
'''Container Name:''' old-projects
+
* host of many services for the MapServer community: see https://wiki.osgeo.org/wiki/MapServer_at_osgeo7
 +
* migrated from [https://wiki.osgeo.org/wiki/MapServer_at_AdhocVM Adhoc VM] thanks to [https://trac.osgeo.org/osgeo/ticket/2459 SAC] + jmckenna (July 2020)
  
-- this is the old projects.osgeo.osuosl.org migrated from osgeo4 as an lxd container, so more or less the same as it was before, with the exception that all the websites are now proxied thru the nginx container.  Websites like on it are community-review.foss4g.org and spatialreference.org
+
=== osgeo7 decommissioned containers ===
  
To access you need to go thru download.osgeo.org -> old-projects
+
==== <del>old-adhoc</del> ====
  
=== Adhoc ===
+
'''SHUTOFF as of 2022-01-29'''
'''Container Name:''' old-adhoc
 
  
[[AdhocVM|old-adhoc]] -- this is the old adhoc.osgeo.osuosl.org migrated 2019-05-08 from osgeo4 as an lxd container.  Used by osgeo-live for there test docs and by grass for earthquake, and mapserver for demo
+
[[AdhocVM|old-adhoc]] -- this is the old adhoc.osgeo.osuosl.org migrated 2019-05-08 from osgeo4 as an lxd container.   
 +
Used by osgeo-live for there test docs and by grass for earthquake, and mapserver for demo.
 +
Note that there is a new live (container that osgeo-live will more to), there is also a mapserver container (which mapserver have started to move their demo to)
  
 
To access via ssh you should go thru download.osgeo.org -> old-adhoc.lxd
 
To access via ssh you should go thru download.osgeo.org -> old-adhoc.lxd
Line 174: Line 391:
 
* See [[AdhocVM]] for full details, and some notes on services running here.
 
* See [[AdhocVM]] for full details, and some notes on services running here.
 
* eg http://adhoc.osgeo.osuosl.org/livedvd/docs/en/quickstart/
 
* eg http://adhoc.osgeo.osuosl.org/livedvd/docs/en/quickstart/
 
=== Old Web ===
 
'''Container Name''': old-web --- the old web.osgeo.osuosl.org (was on osgeo3) currently housing https://id.osgeo.org/ for LDAP management.
 
* old.www.osgeo.org, osgeo.org
 
* id.osgeo.org
 
* mapguide.osgeo.org
 
 
=== Old WebExtra ===
 
 
'''Container Name:'''  old-webextra
 
This is a replica of webextra.osgeo.osuosl.org that was hosted on osgeo3
 
 
Started move on November 29th 2019 and completed December 8th, 2019
 
* foss4g.org, europe.foss4g.org, planet.osgeo.org various old foss4g.org years, live.osgeo.org, journal.osgeo.org (not sure what this is for should be retired?)
 
 
 
=== pycsw ===
 
'''Container Name:''' pycsw
 
 
* https://demo.pycsw.org
 
* '''OGC CSW Reference Implementation and Server demo'''
 
* deployment setup at https://github.com/geopython/demo.pycsw.org
 
* running hourly teardown/setup cron via docker-compose
 
* migrated from [[AdhocVM#Existing_services_hosted_on_the_Ad-hoc_VM:|Adhoc VM]] thanks to [https://trac.osgeo.org/osgeo/ticket/2452 SAC] (May 2020)
 
  
 
== osgeo6 ==
 
== osgeo6 ==
  
 
* Mail
 
* Mail
 +
* lists.osgeo.org
 
See [[Osgeo6]] for full details
 
See [[Osgeo6]] for full details
 
  
 
== Backup (osgeo5) ==
 
== Backup (osgeo5) ==
Line 222: Line 415:
 
hop container has port 22 open but requires ssh access so users who’ve been granted rights can hop thru it to other containers using hop.osgeo4.osgeo.org as name.
 
hop container has port 22 open but requires ssh access so users who’ve been granted rights can hop thru it to other containers using hop.osgeo4.osgeo.org as name.
  
A convenient block to add to your own .ssh/config file follows
+
A convenient block to add to your own .ssh/config file follows where your_id could be your osgeo id or a local account on that container
  
 
  Host osgeo4-*
 
  Host osgeo4-*
 
   ProxyCommand ssh your_osgeo_id@hop.osgeo4.osgeo.org -W $(sed -e "s/^osgeo4-//;s/$/.lxd/" <<< "%h"):%p
 
   ProxyCommand ssh your_osgeo_id@hop.osgeo4.osgeo.org -W $(sed -e "s/^osgeo4-//;s/$/.lxd/" <<< "%h"):%p
 
   IdentityFile "path/to/your/private/key"
 
   IdentityFile "path/to/your/private/key"
 +
  User your_id
  
Then to access say the wordpress-dev container, you'd do the below where your_id could be your osgeo id or a local account on that container
+
Then to access say the wordpress-dev container, you'd do the below
  
  ssh your_id@osgeo4-wordpress-dev
+
  ssh osgeo4-wordpress-dev
  
 
=== osgeo4 baremetal features ===
 
=== osgeo4 baremetal features ===
Line 270: Line 464:
 
</pre>
 
</pre>
  
Nightly backups of osgeo7 containers (secure, nextcloud) will be here and named <container>-backup and be kept in a stopped state.
+
Nightly backups of osgeo3, osgeo7, and osgeo4 containers are kept here and named <container>-backup and be kept in a stopped state.
 +
 
 +
== Services running on osgeo4 ==
  
Services currently on:
 
 
=== hop ===
 
=== hop ===
 
'''Container Name:''' hop - this is the only container with direct ssh access via ssh hop.osgeo4.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access
 
'''Container Name:''' hop - this is the only container with direct ssh access via ssh hop.osgeo4.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access
 +
 +
=== ansible-dev ===
 +
'''Container Name:''' ansible-dev, has ansible 2.9.27 installed and all plugins needed to manage OSGeo ansible infrastructure.
 +
DEPRECATED, use `ansible-dev`
 +
 +
=== ansible-control ===
 +
'''Container Name:''' ansible-control, can be used to deploy OSGeo ansible infrastructure. Replaces `ansible-dev`
  
 
=== osgeo4-nginx ===
 
=== osgeo4-nginx ===
 
'''Container Name:''' osgeo4-nginx ->> all web traffick from other containers on osgeo4 get proxied thru here
 
'''Container Name:''' osgeo4-nginx ->> all web traffick from other containers on osgeo4 get proxied thru here
 +
 +
=== old-web-staging  ===
 +
'''Container Name:''' old-web-staging - used primarily for experimenting with changes to id.osgeo.org (old-web on osgeo7) like testing out OS and software upgrade etc, changes to LDAP forms and registration, before applying to id.osgeo.org. - https://id.staging.osgeo.org
 +
 +
=== pretalx-staging  ===
 +
'''Container Name:''' pretalx-staging - used primarily for experimenting with changes to talks.osgeo.org (pretalx on osgeo3) like testing out Docker builds and software upgrade etc, before applying to talks.osgeo.org. - https://talks.staging.osgeo.org
  
 
=== wordpress-dev  ===
 
=== wordpress-dev  ===
 
'''Container Name:''' wordpress-dev - used primarily for osgeo.org main website development - https://staging.www.osgeo.org, https://dev.www.osgeo.org
 
'''Container Name:''' wordpress-dev - used primarily for osgeo.org main website development - https://staging.www.osgeo.org, https://dev.www.osgeo.org
  
=== old-wiki-dev  ===
+
=== wiki-dev  ===
'''Container Name:''' old-wiki-dev - used primarily for experimenting with changes to wiki.osgeo.org like testing out OS and softwere upgrade etc before appying to wiki.osgeo.org. - https://dev.wiki.osgeo.org
+
'''Container Name:''' wiki-dev - used primarily for experimenting with changes to wiki.osgeo.org like testing out OS and software upgrade etc before appying to wiki.osgeo.org. - https://dev.wiki.osgeo.org
 +
 
 +
=== wiki-staging  ===
 +
'''Container Name:''' wiki-staging - used primarily for upgrade changes to wiki.osgeo.org like testing out OS and software upgrade etc before applying to wiki.osgeo.org. - https://staging.wiki.osgeo.org.  The construction of this container is managed by sac ansible-deployment.
  
 
=== tracsvn-dev  ===
 
=== tracsvn-dev  ===
'''Container Name:''' tracsvn-dev - This is a 2019-09-05 lxd2pc image of tracsvn.osgeo.osuosl.org (now on osgeo7 as tracsvn) used primarily for experimenting like testing out OS, git and software upgrade etc before appying to production.
+
'''Container Name:''' tracsvn-dev - This is a 2019-09-05 lxd2pc image of tracsvn.osgeo.osuosl.org (now on osgeo7 as tracsvn) used primarily for experimenting like testing out OS, git and software upgrade etc before appying to production. -- https://dev.git.osgeo.org, https://dev.tracsvn.osgeo.org Has the following sites: https://dev.trac.osgeo.org, https://dev.git.osgeo.org/gitea, https://dev.svn.osgeo.org.
 +
 
 +
It was upgraded to Debian 11 on 2024-08-21.
  
 
=== dronie-client  ===
 
=== dronie-client  ===
 
'''Container Name:''' dronie-client - This is a debian 10 machine, with OSGeo LDAP authentication and a drone-agent docker running.  To be used with https://dronie.osgeo.org
 
'''Container Name:''' dronie-client - This is a debian 10 machine, with OSGeo LDAP authentication and a drone-agent docker running.  To be used with https://dronie.osgeo.org
 
=== gallery ===
 
'''Container Name:''' gallery - work in progress currently contains lots of pictures of OSGeo events.  Investigating various different applications we can use to serve these - currently under consideration are photophrism and nextcloud (both installed on this server).
 
Talk to Brian Hamlin if you want to help out. https://gallery.osgeo.org, https://nextcloud.gallery.osgeo.org
 
 
Has the following sites: https://dev.trac.osgeo.org, https://dev.git.osgeo.org/gitea, https://dev.svn.osgeo.org
 
  
 
== osgeo3 ==
 
== osgeo3 ==
osgeo3 physical server was rebuilt March 2020 with new disks and power supply, as an Ubuntu 18.04 with LXD. It will be used to run production, but moderately risky things. Refer to [[SAC:Old-osgeo3]] for past history before osgeo3 was rebuilt.
+
osgeo3 physical server refer to [[osgeo3|Configuration Details]] for hardware specs. It is used to run production, but moderately risky things. Refer to [[SAC:Old-osgeo3]] for past history before osgeo3 was rebuilt.
 
osgeo3 is a hosted by OSUOSL - can be accessed via ssh tech_dev@osgeo3.osgeo.osuosl.org -p 2222  (only people with their access keys installed can log in and doesn't allow password access) - password for tech_dev is in the secure container (on osgeo7) / access folder.
 
osgeo3 is a hosted by OSUOSL - can be accessed via ssh tech_dev@osgeo3.osgeo.osuosl.org -p 2222  (only people with their access keys installed can log in and doesn't allow password access) - password for tech_dev is in the secure container (on osgeo7) / access folder.
  
 
=== sshing into osgeo3 containers ===
 
=== sshing into osgeo3 containers ===
 
Note that all the containers are closed off from direct ssh access except for the hop.osgeo3.osgeo.org.  To access the other containers, you need to hop through hop.osgeo3.osgeo.org.
 
Note that all the containers are closed off from direct ssh access except for the hop.osgeo3.osgeo.org.  To access the other containers, you need to hop through hop.osgeo3.osgeo.org.
hop container has port 22 open but requires ssh access (and ssh keys installed) so users who’ve been granted rights can hop thru it to other containers using hop.osgeo3.osgeo.org as name.  Other containers may or may not allow password access. It's up to the those who manage the internal containers.
+
hop container has port 22 open but requires ssh key access, you need to be added to shell group and have your ssh keys registered on your profile https://id.osgeo.org/ldap/edit). Users who’ve been granted rights can hop thru it to other containers using hop.osgeo3.osgeo.org as name.  Other containers may or may not allow password access. It's up to the those who manage the internal containers.
  
 
A convenient block to add to your own .ssh/config file follows
 
A convenient block to add to your own .ssh/config file follows
Line 318: Line 525:
 
  ssh '''your_id'''@osgeo3-nexus
 
  ssh '''your_id'''@osgeo3-nexus
  
=== Baremetal setup of osgeo3 ===
+
=== hop ===
It's makeup is as follows:
+
This is the only container with direct ssh access via ssh hop.osgeo3.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access which you can register by editing your ldap profile.
{| class="wikitable"
 
|-
 
! Item !! Settings
 
|-
 
| Disks || 6 1.8 TB drives
 
|-
 
| Memory || 48 GB
 
|-
 
| CPUs || 8 Intel(R) Xeon(R) CPU E5540  @ 2.53GHz (8192kb cache)
 
|}
 
<pre>lsblk -i
 
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
 
loop0            7:0    0 91.4M  1 loop /snap/core/8689
 
loop1            7:1    0  67M  1 loop /snap/lxd/13901
 
sda              8:0    0  1.8T  0 disk
 
|-sda1            8:1    0  953M  0 part /boot
 
`-sda2            8:2    0  1.8T  0 part
 
  |-osgeo3-root 253:0    0  800G  0 lvm  /
 
  `-osgeo3-swap 253:1    0  32G  0 lvm  [SWAP]
 
sdb              8:16  0  1.8T  0 disk
 
|-sdb1            8:17  0  1.8T  0 part
 
`-sdb9            8:25  0    8M  0 part
 
sdc              8:32  0  1.8T  0 disk
 
|-sdc1            8:33  0  1.8T  0 part
 
`-sdc9            8:41  0    8M  0 part
 
sdd              8:48  0  1.8T  0 disk
 
|-sdd1            8:49  0  1.8T  0 part
 
`-sdd9            8:57  0    8M  0 part
 
sde              8:64  0  1.8T  0 disk
 
|-sde1            8:65  0  1.8T  0 part
 
`-sde9            8:73  0    8M  0 part
 
sr0              11:0    1 1024M  0 rom
 
  
</pre>
+
=== osgeo3-nginx ===
 +
nginx proxy all web-traffic via the  (using IP: 140.211.15.6 web-osgeo3.osgeo.osuosl.org), the second ip is not in use, but may be used for a secondary nginx, to temporarily host osgeo7/osgeo8/osgeo9 containers in case of major hardware upgrades
  
<pre>
+
=== dronie-client ===
sdb,sdc,sdd,sdd  form a zfs osgeo3_lxd partition (sdb,sdc) mirrors sdd,sdf for total lxd capacity of 3.62 TB
+
This is a debian 10 lxd container running docker. Currently has just one running docker osgeo-drone-agent to serve as a client for dronie-server (dronie.osgeo.org running on osgeo7)  
sda (composed of 2 1.8TB disks) is under hardware RAID1 - root lvm partition is 800GB, 1TB unallocated for future use
 
</pre>
 
  
Services currently on:
+
=== geo-docs ===
=== hop ===
+
Debian 11 houses the following:
'''Container Name:''' hop - this is the only container with direct ssh access via ssh hop.osgeo3.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access
 
  
=== osgeo3-nginx ===
+
* lastools.osgeo.org (as of 2021-10-06 snapshot of  https://www.cs.unc.edu/~isenburg/lastools/ see [https://trac.osgeo.org/osgeo/ticket/2649 2649]
'''Container Name:''' nginx proxy all web-traffick via the  (using IP: 140.211.15.6 web-osgeo3.osgeo.osuosl.org), the second ip is not in use, but may be used for a secondary nginx, to temporarily host osgeo7/osgeo4 containers in case of major hardware upgrades
+
* https://planet.osgeo.org
 +
* https://docs.geoserver.org
 +
* https://docs.geotools.org [https://trac.osgeo.org/osgeo/ticket/2944 2944]
 +
* https://geotools.org [https://trac.osgeo.org/osgeo/ticket/2944 2944]
  
=== nexus (repo.osgeo.org, docker.osgeo.org)  ===
+
planned home of ? docs.geowebcache.org
'''Container Name:''' nexus - see [[SAC:Repo]] this is a debian 10 lxd container running docker 19.  It currently has one docker container running within it called nexus -- exposed as repo.osgeo.org on nginx.
 
 
 
=== dronie-client ===
 
'''Container Name:''' dronie-client - this is a debian 10 lxd container running docker. Currently has just one running docker osgeo-drone-agent to serve as a client for dronie-server (dronie.osgeo.org running on osgeo7) 
 
  
 
=== matrix ===
 
=== matrix ===
'''Container Name:''' matrix - debian10 lxd container with ldap/ssh.
+
'''Container Name:''' matrix - lxd container with ldap/ssh.
 +
Hosts [[Matrix]] homeserver ([[SAC:MatrixSynapse]]) and IRC bridges ([[SAC:Heisenbridge]])
  
See [[SAC:MatrixSynapse]] for full detail.
+
See https://git.osgeo.org/gitea/sac/osgeo3/wiki/matrix-container for full detail on how the container is setup
  
=== wiki ===
+
=== nexus (repo.osgeo.org, docker.osgeo.org)  ===
'''Container Name:''' wiki (wiki.osgeo.org)  - debian10 lxd container with ldap/ssh (MediaWiki 1.34, 10.3.22-MariaDB, PHP 7.3)
+
See [[SAC:Repo]] this is a debian 10 lxd container running docker 19.
 +
It currently has one docker container running within it called nexus -- exposed as repo.osgeo.org on nginx.
  
Note that this is a rebuild of wiki.osgeo.org that used to be on osgeo7 old-wiki container. Move 2020-05-22
+
Also exposed as project dockers for pushing images:  postgis-docker.osgeo.org, geoserver-docker.osgeo.org, geos-docker.osgeo.org, sac-docker.osgeo.org
  
 
=== monitor ===
 
=== monitor ===
'''Container Name:''' monitor - debian10 lxd container with ldap/ssh. https://monitor.osgeo.org (houses grafana dashboard (for all servers) and prometheus server for osgeo3 containers and pulls basic container metrics using node exporters pulled via prometheus servers. Requirs ldap to log into the web console.
+
debian10 lxd container with ldap/ssh. https://monitor.osgeo.org (houses grafana dashboard (for all servers) and prometheus server for osgeo3 containers and pulls basic container metrics using node exporters pulled via prometheus servers. Requirs ldap to log into the web console.
  
 
Configuring servers for monitoring is detailed [https://git.osgeo.org/gitea/sac/prometheus-config Git Prometheus Config]
 
Configuring servers for monitoring is detailed [https://git.osgeo.org/gitea/sac/prometheus-config Git Prometheus Config]
  
=== Base ===
 
* Template VM used when creating new VMs.
 
* Not normally running - make a special request to OSL to start it up.
 
  
=== WebExtra ===
+
=== gallery ===
 +
 
 +
Picture gallery.
 +
See [[Gallery]]
 +
 
 +
== WebExtra ==
 
** Retired December 8th, 2019 -- and moved to osgeo7 as container old-webextra
 
** Retired December 8th, 2019 -- and moved to osgeo7 as container old-webextra
  
Line 408: Line 584:
 
[[SAC_Shared_Building_Services|SAC Shared Building Services]]
 
[[SAC_Shared_Building_Services|SAC Shared Building Services]]
  
 
== OSGeo funtoo ==
 
 
<del>For lxd experimentation it's an lxd container running other lxd containers and provided by funtoo.org.</del>
 
 
OSGeo is paying funtoo via treasurer at osgeo.org.
 
 
* [https://git.osgeo.org/gitea/sac/osgeo_funtoo OSGeo Funtoo] osgeo.host.funtoo.org
 
 
=== LimeSurvey ===
 
 
* funtoo LXDs currently running:
 
** [https://limesurvey.osgeo.org LimeSurvey] -this may be in future migrated to osgeo7 or osgeo3
 
  
 
== Atlantic.net ==
 
== Atlantic.net ==
  
* host.postgis.net -p 2222 is an LXD Debian 10 8GB RAM
+
* host.postgis.net -p 2222 is an LXD Ubuntu 18.04 16GB RAM/ 6 vCPU, 350GB data, 250GB block storage
 
* Currenlty running two lxd containers:
 
* Currenlty running two lxd containers:
     debbie: postgis.net, planet.postgis.net, debbie.postgis.net (jenkins build bot)   
+
     debbie: debian 10 postgis.net, planet.postgis.net, debbie.postgis.net (jenkins build bot)   
 
     debbie-docker.host.postgis.net - runs docker and serves as a 1.0 agent for dronie.osgeo.org
 
     debbie-docker.host.postgis.net - runs docker and serves as a 1.0 agent for dronie.osgeo.org
  
Line 449: Line 612:
 
* Hosts wordpress sites staging.www.osgeo.org,www.osgeo.org, staging.grass.osgeo.org, foss4g2018.osgeo.org
 
* Hosts wordpress sites staging.www.osgeo.org,www.osgeo.org, staging.grass.osgeo.org, foss4g2018.osgeo.org
 
* Setup details on [https://git.osgeo.org/gitea/osgeo/www_apache_configs/wiki/Web18a-setup Web18a setup]
 
* Setup details on [https://git.osgeo.org/gitea/osgeo/www_apache_configs/wiki/Web18a-setup Web18a setup]
 +
 +
 +
== OSGeo funtoo ==
 +
 +
For lxd experimentation it's an lxd container running other lxd containers and provided by funtoo.org.
 +
 +
OSGeo is paying funtoo via treasurer at osgeo.org.
 +
 +
* [https://git.osgeo.org/gitea/sac/osgeo_funtoo OSGeo Funtoo] osgeo.host.funtoo.org
 +
 +
 +
* funtoo LXDs currently running:
 +
** <del>[https://limesurvey.osgeo.org LimeSurvey] -this may be in future migrated to osgeo7 or osgeo3</del>
 +
Migrated to osgeo3  2020-11-28
  
  
 
[[Category:Infrastructure]]
 
[[Category:Infrastructure]]
 +
[[Category:Services]]

Latest revision as of 08:33, 25 September 2024

Infrastructure of OSGeo System Administration Committee (SAC)

For emergency plans see: SAC:Admin and Troubleshooting


Servers at OSL

Open Source Labs - 7 physical machines of which 5 ar lxd hosts containing x virtual machines/containers.


Logging into Physical Machines

Currently we have osgeo6 and backup.osgeo.osuosl.org that are physical machines under LDAP control.

Note this does not apply to osgeo3, osgeo4, osgeo7, osgeo8, osgeo9 lxd hosts. Refer to those sections for instructions on logging in. If hanging, see OSL for how to open a ticket with OSUOSL's support.

This section currently only applies to osgeo6 and backup.

All SAC administrators have LDAP auth to the OSL Machines.

To ssh into a server using your LDAP account, you can do the following replacing your_osgeo_login with your OSGeo login and vmname with the vm name of the server at OSL.

 ssh your_osgeo_login@servername.osgeo.osuosl.org

When prompted for password, use your OSGeo Login password.


SAC:Primary Administrators also have ssh key access in case LDAP is down and that will also apply to the physical machines. Worst case scenario use the information on Open Source Labs to file a ticket (SAC members only). Direct connection to virtual machines is by appending it's vm alias to .osgeo.osuosl.org.

Logging into LXD Hosts

OSGeo3, OSGeo4, OSGeo7, OSGeo8, and OSGeo9 are all Ubuntu servers running LXD. LXD is a management system for LXC containers and QEMU VMS. LXD has a channel that covers its features.

To directly access the host, you go thru port 2222

  ssh tech_dev@server_name.osgeo.osuosl.org -p 2222

Only SAC:Primary Administrators have their ssh key installed under that account. In order to access via KVM of these in event servers do not come up on a reboot, you need to go thru OSU OSL OpenVPN. To get an OpenVPN account, you need to put in a support ticket to support@osuosl.org. In order to qualify for an OpenVPN account, you need to be an OSGeo SAC administrator. You will also need to install OpenVPN client) to use your OpenVPN account.

Each host on the private KVM side is named https://osgeo8.osuosl.oob -- where replace osgeo8 with the relevant host. The .oob is the private network, so doesn't work unless you are connected to via OpenVPN.

The browser interface is sometimes clunky, so you might want to use ipmitool installable on linux/unix or wsl using relevant package manager. KVM passwords are stored in SAC password-store.


A convenient block to add to your ~/.ssh/config to easily login to osgeo's LXD hosts follows:

   Host osgeo?
     User tech_dev
     HostName %h.osgeo.osuosl.org
     Port 2222

Then you would be able to log into those hosts with commands like:

   ssh osgeo7

Logging into LXD Containers and VMs

A convenient block to add to your ~/.ssh/config to easily login to osgeo's LXD hosted containers and vms is the following:

  # This stanza is only needed if you have an IdentityFile configured below.
  # The IdentityFile from a target host is not automatically applied to the hop host, so we need to make it explicit:
  Host hop.*.osgeo.org
    IdentityFile "path/to/your/private/key"
  
  Host osgeo*-*
    ProxyCommand ssh hop.$(sed -e "s/-.*//" <<< "%h").osgeo.org -W $(sed -e "s/^osgeo[^-*]-//;s/$/.lxd/" <<< "%h"):%p
    # this is only needed if you you use different private keys for different servers
    IdentityFile "path/to/your/private/key"

Then you'll be able to access a LXC Container or QEMU VM on machine `osgeo3` with:

  ssh yourusername@osgeo3-matrix

And one on machine `osgeo7` with:

  ssh yourusername@osgeo7-download

Note you still need to know where each LXC host is hosted... See successive sections to know what's on which machine.


Troubleshooting: In case of "Permission denied (publickey)." after an update to a modern openSSH version, it might well be that your ssh key (RSH key) is disabled in your client in favour of more modern cyphers.

Ugly workaround: add one line `PubkeyAcceptedKeyTypes ...` in `.ssh/config`, to re-enable RSA keys for now (consider to generate a new key):

 vim .ssh/config
 ...
 Host *
    ...
    PubkeyAcceptedKeyTypes +ssh-rsa

... but better read e.g. here!

osgeo 8

Server added April 2021. Intended to provide additional LXD capacity and backup Configuration Details

Services running on osgeo8

hop

hop.osgeo8.osgeo.org - jump host for accessing containers/vms on osgeo8

nginx

http, https Proxy for all containers on osgeo8 and also provides mirror proxy for download.osgeo.org

centtie-7-pgrouting

Centos 7 running PostgreSQL 15, PostGIS 3.3.2, gcc-4.8.5, cmake 3 Configured to be a github self-hosted runner for testing centos for pgrouting project

Details of Github Action runner setup

download8

Replica of download that is on osgeo7. Mirrors download and home folders from osgeo7. https://download-cache.osgeo.org

dronie-client

a ci bot for dronie.osgeo.org which is used for git.osgeo.org/gitea ci jobs

grass-wiki

Debian 10 (copy of wiki) home of https://grasswiki.osgeo.org upgraded to Bullseye debian 11.

grass

https://grass.osgeo.org upgraded to Bullseye debian 11.

GRASS GIS server

Current DNS name: grass.osgeo.org

Debian 11 Bullseye

Web: Apache + Hugo (generated through cronjob from https://github.com/OSGeo/grass-website/), see https://github.com/OSGeo/grass-addons/tree/grass8/utils/cronjobs_osgeo_lxd

`unattended-upgrades` for automatic installation of security upgrades is installed and running

ssh: reachable via jumphost.

meshcentral

https://remote.osgeo.org This is a remoting tool currently setup to test livecd vms via a web browser.

4 VMS currently set up on osgeo8 accessible from this. Currently based on livecd 16rc1 snapshots, with wm install script run.


pgrouting-dev

For pgrouting development use to do things like pushing docker images on a scheduled basis. Perhaps later for demo sites. WIP.

woodie-client

Separate agent for woodie-server

woodie-server

Ubuntu 22.04 with docker. https://woodie.osgeo.org A CI server + agent running https://woodpecker-ci.org and authentication/repo for https://git.osgeo.org/gitea

woodpecker server and agent are running on this each in a docker container using a docker-compose yaml running under account woodie.

Setup details OSGeo8 Woodie setup

osgeo 9

Server added April 2021. Is an LXD host. Also Stores lxd images used by other lxd hosts. Configuration Details

Services running on osgeo9

hop

hop.osgeo9.osgeo.org. For LDAP users allows them to hop thru to get to other containers.

Secure (LDAP )

secure -- ldap.osgeo.org SAC:LDAP used for ldap service (a rebuild of old secure.osgeo.osuosl.org) now on Debian 11 Moved from osgeo7

ldap-web

Currently housing https://id.osgeo.org/ for LDAP management. Deployed via ansible Moved from osgeo9

  • id.osgeo.org

jitsi

https://meet.osgeo.org. Requires LDAP for moderation of a room, but other users are allowed in as guests.

nextcloud

https://nextcloud.osgeo.org

Ubuntu 22.04 LXD/nginx/postgresql 14 container for document sharing similar to dropbox/google drive - nextcloud.lxd - https://nextcloud.osgeo.org Nextcloud Setup

home of https://nextcloud.osgeo.org This server does not use ssh osgeo-ldap as it was the first container built. However nextcloud.osgeo.org does authenticate with osgeo ldap.

TODO: add special page for this

nginx

nginx (for web proxy of traffic of osgeo9 containers) additional mirror proxy for download.osgeo.org

adventure (WIP)

https://adventure.osgeo.org runs https://github.com/thecodingmachine/workadventure software

dronie-client

a ci bot for dronie.osgeo.org which is used for git.osgeo.org/gitea ci jobs

limesurvey

Debian 10, PostgreSQL 13, PHP 8 with ldap/ssh. https://limesurvey.osgeo.org Setup detailed on limesurvey container

pixelfed

SHUT OFF (both container and website) cause of lack of interest. Container is still there. Pixelfed instance reachable on https://photo.osgeo.org to house community photos

peertube

Peertube instance reachable on https://video.osgeo.org

pretalx

Ubuntu 20.04 with OSGeo LDAP and Docker installed. pretalx software runs in Docker. https://talks.osgeo.org - for OSGeo Talk collection and voting See Pretalx

weblate

Container Name: weblate (for doc translation)

Houses: https://weblate.osgeo.org (for document translation to different languages) For further details refer to SAC:Weblate

wordpress

Houses: https://www.osgeo.org, https://2018.foss4g.org, https://2020.europe.foss4g.org websites Runs Debian 11 with OSGeo LDAP access

Setup details of wordpress container

wiki

(wiki.osgeo.org) - debian10 lxd container with ldap/ssh (MediaWiki 1.34, 10.3.22-MariaDB, PHP 7.3)

Note that this is a rebuild of wiki.osgeo.org that used to be on osgeo7 old-wiki container. Move 2020-05-22 Setup steps are at wiki container setup

discourse

Ubuntu 22.04 VM running docker that hosts an instance of discourse software https://discourse.osgeo.org It current allows OSGeo LDAP, OSGeo Gitea, GitHub, and sign up repo is at - Discourse-VM

osgeo 7

Server added June 2018. Intended to replace osgeo3 and old osgeo4 (before reformat). Configuration Details

Container setup of all the osgeo7 servers is located in https://git.osgeo.org/gitea/sac/osgeo7/wiki/_pages

Running LXD 3 snap based container management -- LXD version 3.17 as of 2019-09-15

Accessing osgeo7 containers via ssh

Only the download.osgeo.org is directly exposed ssh via port 22. To access the other containers, you can tunnel thru download.osgeo.org -- You need to be in the shell group to be able to access download and the other servers. If you are not already put in a SAC Ticket Request. You also need to have your public key registered. To do so edit your profile [1] (and put in your public key)

A convenient block to add to your own `.ssh/config` file follows:


Host osgeo7-*
  ProxyCommand ssh your_osgeo_id@hop.osgeo7.osgeo.org -W $(sed -e "s/^osgeo7-//;s/$/.lxd/" <<< "%h"):%p
  IdentityFile "path/to/your/private/key"

With the above in place, you can connect to any container using:

 ssh your_id@osgeo7-<container_name>

Troubleshooting: In case of "Permission denied (publickey)." after an update to a modern openSSH version, it might well be that your ssh key (RSH key) is disabled in your client in favour of more modern cyphers.

Ugly workaround: add one line `PubkeyAcceptedKeyTypes ...` in `.ssh/config`, to re-enable RSA keys for now (consider to generate a new key):

 vim .ssh/config
 ...
 Host *
    ...
    PubkeyAcceptedKeyTypes +ssh-rsa

... but better read e.g. here!


Services on osgeo7

Download

In order to be able to log in or sftp

  • You must be a member of the OSGeo shell group which can be granted from another person in shell group - Shell
  • Once you are in shell group, go to https://id.osgeo.org/ldap/edit and put in your public key.

You should then be able to log into download (and all other hop servers on the other hosts) with your private key.

You can put in a ticket to request such access.

nginx

Proxy that routes all http/https traffic for the other containers (can be accessed via osgeo7 host lxc or ubuntu@osgeo7-nginx if your key is installed on ubuntu user). The nginx container holds the letsencrypt https SSL certs for all the containers and handles the renewal of the letsencrypt certs using certbot renew cronjob. Prometheus server to collect all monitoring logs from OSGeo7 (only accessible by OSGeo3), these get queried via monitor.osgeo.org (running on osgeo3) via grafana server.

tracsvn (trac, svn, git)

https://trac.osgeo.org, https://git.osgeo.org/gitea, https://svn.osggeo.org This used to be housed on osgeo3 in VM TracSVN VM, and was moved 2019-10-12 to osgeo7 as tracsvn container. It has since been upgraded to Debian 9 (Stretch with plans to upgrade more)

  • See TracSVN for full details, and some notes on services running here

See TracSVN for info about what it contains (spoiler: trac, svn, gitea)

old-wiki (stopped)

This used to be housed on osgeo3, and was moved 2019-09-14 to osgeo7 as old-wiki container. wiki.osgeo.org moved back to osgeo3 on 2020-05-22 and in wiki container. The wiki container is a complete rebuild with files and database restored and upgraded. Refer to the osgeo3 section for more details.

old wiki container -- used for wiki service (it is an lxd2pc created image of wiki.osgeo.osuosl.org VM that was on osgeo3)

See OSGeo Wiki

nextcloud-ubuntu (stopped)

Moved to osgeo9

collabora (stopped)

  1. https://collabora.osgeo.org

Ubuntu 18.04 LXD container for LibreOffice/MS Office online document editor currently used exclusively by nextcloud.osgeo.org. Setup detailed in Nextcloud setup. No longer needed now that nextcloud is using a self-contained collabora instance.

live

Home of live.osgeo.org (created 2021-10-05ish Running Ubuntu 20.04 with OSGeo LDAP SSH

dronie-server

old-projects (stopped)

-- this is the old projects.osgeo.osuosl.org migrated from osgeo4 as an lxd container, so more or less the same as it was before, with the exception that all the websites are now proxied thru the nginx container. Websites on it are community-review.foss4g.org and spatialreference.org

To access you need to go thru download.osgeo.org -> old-projects


old-web (stopped)

The old web.osgeo.osuosl.org (was on osgeo3)

  • mapguide.osgeo.org

old-webextra

This is a replica of webextra.osgeo.osuosl.org that was hosted on osgeo3

Started move on November 29th 2019 and completed December 8th, 2019

  • foss4g.org
  • europe.foss4g.org
  • video.foss4g.org
  • planet.osgeo.org
  • various old foss4g.org years
  • live.osgeo.org moved to dedicated container
  • journal.osgeo.org (not sure what this is for, should be retired?)
  • vmap0.tiles.osgeo.org #removed site

pycsw

Container Name: pycsw

mapserver

Container Name: mapserver

osgeo7 decommissioned containers

old-adhoc

SHUTOFF as of 2022-01-29

old-adhoc -- this is the old adhoc.osgeo.osuosl.org migrated 2019-05-08 from osgeo4 as an lxd container. Used by osgeo-live for there test docs and by grass for earthquake, and mapserver for demo. Note that there is a new live (container that osgeo-live will more to), there is also a mapserver container (which mapserver have started to move their demo to)

To access via ssh you should go thru download.osgeo.org -> old-adhoc.lxd It is accessible via https://adhoc.osgeo.org and http://adhoc.osgeo.osuosl.org

osgeo6

  • Mail
  • lists.osgeo.org

See Osgeo6 for full details

Backup (osgeo5)

  • Backup now runs on dedicated hardware
  • Provides Rsync backups of download.osgeo.org
  • Provides Bacula backups of various VMs.
  • See SAC:Backups for details.

osgeo4

osgeo4 is a real server managed by OSUOSL - can be access via ssh tech_dev@osgeo4.osgeo.osuosl.org -p 2222 (only people with their access keys installed can log in and doesn't allow password access) - password for tech_dev is in the secure container (on osgeo7) / access folder.

In August 2019 the server had new power supply put in and replacement disks. It was reformatted with Ubuntu 18.04.3 to serve as secondary LXD host to osgeo7 zfsutils-linux was installed so lxd can use zfs for storage.

sshing into osgeo4 containers

Note that all the containers are closed off from direct ssh access except for the hop.osgeo4.osgeo.org. To access the other containers, you need to hop through hop. hop container has port 22 open but requires ssh access so users who’ve been granted rights can hop thru it to other containers using hop.osgeo4.osgeo.org as name.

A convenient block to add to your own .ssh/config file follows where your_id could be your osgeo id or a local account on that container

Host osgeo4-*
  ProxyCommand ssh your_osgeo_id@hop.osgeo4.osgeo.org -W $(sed -e "s/^osgeo4-//;s/$/.lxd/" <<< "%h"):%p
  IdentityFile "path/to/your/private/key"
  User your_id

Then to access say the wordpress-dev container, you'd do the below

ssh osgeo4-wordpress-dev

osgeo4 baremetal features

It's makeup is as follows:

Item Settings
Disks 6 1.8 TB drives
Memory 48 GB
CPUs 8 Intel(R) Xeon(R) CPU E5540 @ 2.53GHz (8192kb cache)
lsblk -i
NAME           MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda              8:0    0  1.8T  0 disk  
|-sda1           8:1    0  953M  0 part  
| `-md0          9:0    0  952M  0 raid1 /boot
`-sda2           8:2    0 46.6G  0 part  
  `-md1          9:1    0 46.5G  0 raid1 
	|-lvm-root 253:0    0 37.3G  0 lvm   /
	`-lvm-swap 253:1    0  7.5G  0 lvm   [SWAP]
sdb              8:16   0  1.8T  0 disk  
|-sdb1           8:17   0  953M  0 part  
| `-md0          9:0    0  952M  0 raid1 /boot
`-sdb2           8:18   0 46.6G  0 part  
  `-md1          9:1    0 46.5G  0 raid1 
	|-lvm-root 253:0    0 37.3G  0 lvm   /
	`-lvm-swap 253:1    0  7.5G  0 lvm   [SWAP]
sdc              8:32   0  1.8T  0 disk  
sdd              8:48   0  1.8T  0 disk  
sde              8:64   0  1.8T  0 disk  
sdf              8:80   0  1.8T  0 disk 
sdc,sdd,sde,sdf  form a zfs osgeo4_lxd partition (sdc,sdd) mirrors sde,sdf for total lxd capacity of 3.62 TB

Nightly backups of osgeo3, osgeo7, and osgeo4 containers are kept here and named <container>-backup and be kept in a stopped state.

Services running on osgeo4

hop

Container Name: hop - this is the only container with direct ssh access via ssh hop.osgeo4.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access

ansible-dev

Container Name: ansible-dev, has ansible 2.9.27 installed and all plugins needed to manage OSGeo ansible infrastructure. DEPRECATED, use `ansible-dev`

ansible-control

Container Name: ansible-control, can be used to deploy OSGeo ansible infrastructure. Replaces `ansible-dev`

osgeo4-nginx

Container Name: osgeo4-nginx ->> all web traffick from other containers on osgeo4 get proxied thru here

old-web-staging

Container Name: old-web-staging - used primarily for experimenting with changes to id.osgeo.org (old-web on osgeo7) like testing out OS and software upgrade etc, changes to LDAP forms and registration, before applying to id.osgeo.org. - https://id.staging.osgeo.org

pretalx-staging

Container Name: pretalx-staging - used primarily for experimenting with changes to talks.osgeo.org (pretalx on osgeo3) like testing out Docker builds and software upgrade etc, before applying to talks.osgeo.org. - https://talks.staging.osgeo.org

wordpress-dev

Container Name: wordpress-dev - used primarily for osgeo.org main website development - https://staging.www.osgeo.org, https://dev.www.osgeo.org

wiki-dev

Container Name: wiki-dev - used primarily for experimenting with changes to wiki.osgeo.org like testing out OS and software upgrade etc before appying to wiki.osgeo.org. - https://dev.wiki.osgeo.org

wiki-staging

Container Name: wiki-staging - used primarily for upgrade changes to wiki.osgeo.org like testing out OS and software upgrade etc before applying to wiki.osgeo.org. - https://staging.wiki.osgeo.org. The construction of this container is managed by sac ansible-deployment.

tracsvn-dev

Container Name: tracsvn-dev - This is a 2019-09-05 lxd2pc image of tracsvn.osgeo.osuosl.org (now on osgeo7 as tracsvn) used primarily for experimenting like testing out OS, git and software upgrade etc before appying to production. -- https://dev.git.osgeo.org, https://dev.tracsvn.osgeo.org Has the following sites: https://dev.trac.osgeo.org, https://dev.git.osgeo.org/gitea, https://dev.svn.osgeo.org.

It was upgraded to Debian 11 on 2024-08-21.

dronie-client

Container Name: dronie-client - This is a debian 10 machine, with OSGeo LDAP authentication and a drone-agent docker running. To be used with https://dronie.osgeo.org

osgeo3

osgeo3 physical server refer to Configuration Details for hardware specs. It is used to run production, but moderately risky things. Refer to SAC:Old-osgeo3 for past history before osgeo3 was rebuilt. osgeo3 is a hosted by OSUOSL - can be accessed via ssh tech_dev@osgeo3.osgeo.osuosl.org -p 2222 (only people with their access keys installed can log in and doesn't allow password access) - password for tech_dev is in the secure container (on osgeo7) / access folder.

sshing into osgeo3 containers

Note that all the containers are closed off from direct ssh access except for the hop.osgeo3.osgeo.org. To access the other containers, you need to hop through hop.osgeo3.osgeo.org. hop container has port 22 open but requires ssh key access, you need to be added to shell group and have your ssh keys registered on your profile https://id.osgeo.org/ldap/edit). Users who’ve been granted rights can hop thru it to other containers using hop.osgeo3.osgeo.org as name. Other containers may or may not allow password access. It's up to the those who manage the internal containers.

A convenient block to add to your own .ssh/config file follows

# this is only needed if you you use different private keys for different servers
Host hop.osgeo3.osgeo.org
  IdentityFile "path/to/your/private/key"
Host osgeo3-*
  ProxyCommand ssh your_osgeo_id@hop.osgeo3.osgeo.org -W $(sed -e "s/^osgeo3-//;s/$/.lxd/" <<< "%h"):%p
  IdentityFile "path/to/your/private/key"

Then to access say the nexus container, you'd do the below where your_id could be your osgeo id or a local account on that container

ssh your_id@osgeo3-nexus

hop

This is the only container with direct ssh access via ssh hop.osgeo3.osgeo.org. To get to other containers, you need to hop thru this one. Requires ssh key access which you can register by editing your ldap profile.

osgeo3-nginx

nginx proxy all web-traffic via the (using IP: 140.211.15.6 web-osgeo3.osgeo.osuosl.org), the second ip is not in use, but may be used for a secondary nginx, to temporarily host osgeo7/osgeo8/osgeo9 containers in case of major hardware upgrades

dronie-client

This is a debian 10 lxd container running docker. Currently has just one running docker osgeo-drone-agent to serve as a client for dronie-server (dronie.osgeo.org running on osgeo7)

geo-docs

Debian 11 houses the following:

planned home of ? docs.geowebcache.org

matrix

Container Name: matrix - lxd container with ldap/ssh. Hosts Matrix homeserver (SAC:MatrixSynapse) and IRC bridges (SAC:Heisenbridge)

See https://git.osgeo.org/gitea/sac/osgeo3/wiki/matrix-container for full detail on how the container is setup

nexus (repo.osgeo.org, docker.osgeo.org)

See SAC:Repo this is a debian 10 lxd container running docker 19. It currently has one docker container running within it called nexus -- exposed as repo.osgeo.org on nginx.

Also exposed as project dockers for pushing images: postgis-docker.osgeo.org, geoserver-docker.osgeo.org, geos-docker.osgeo.org, sac-docker.osgeo.org

monitor

debian10 lxd container with ldap/ssh. https://monitor.osgeo.org (houses grafana dashboard (for all servers) and prometheus server for osgeo3 containers and pulls basic container metrics using node exporters pulled via prometheus servers. Requirs ldap to log into the web console.

Configuring servers for monitoring is detailed Git Prometheus Config


gallery

Picture gallery. See Gallery

WebExtra

    • Retired December 8th, 2019 -- and moved to osgeo7 as container old-webextra

Cloud Hosted Servers and other external under SAC Control

Future Hosting Plans for Windows / Mac Building

SAC Shared Building Services


Atlantic.net

  • host.postgis.net -p 2222 is an LXD Ubuntu 18.04 16GB RAM/ 6 vCPU, 350GB data, 250GB block storage
  • Currenlty running two lxd containers:
   debbie: debian 10 postgis.net, planet.postgis.net, debbie.postgis.net (jenkins build bot)  
   debbie-docker.host.postgis.net - runs docker and serves as a 1.0 agent for dronie.osgeo.org

QGIS off OSGeo

Services on separated machines rented and managed by the QGIS project at hetzner

  • website including documentation http://www.qgis.org
  • website building, documentation building, debian/ubuntu nightlies, plugins.qgis.org
  • issues.qgis.org: redmine

Historical servers (not more in use)

- Telascience Blades (Historical)

web18a.osgeo.osuosl.org

NO LONGER USED - turned off 2019-09-03 Production services www.osgeo.org, 2018.foss4g.org moved to wordpress container on osgeo7 Staging services (staging.www.osgeo.org, dev.www.osgeo.org move to wordpress-dev container on osgeo4 Grass wordpress is disabled as grass decided to go with another solution, so have grass container on osgeo7 (Cloud hosted server on OSUOSL hardware (not ours) )

  • Debian 9.3 4GB server, host name: web18a.osgeo.osuosl.org require ssh key to log in.
  • Hosts wordpress sites staging.www.osgeo.org,www.osgeo.org, staging.grass.osgeo.org, foss4g2018.osgeo.org
  • Setup details on Web18a setup


OSGeo funtoo

For lxd experimentation it's an lxd container running other lxd containers and provided by funtoo.org.

OSGeo is paying funtoo via treasurer at osgeo.org.


  • funtoo LXDs currently running:
    • LimeSurvey -this may be in future migrated to osgeo7 or osgeo3

Migrated to osgeo3 2020-11-28