Revision as of 19:13, 31 August 2019

Infrastructure of OSGeo System Administration Committee (SAC)

For emergency plans see: SAC:Admin and Troubleshooting

Servers at OSL

Open Source Labs - 4 physical machines containing x virtual machines.

If VM hanging, see OSL for how to open a ticket with OSUOSL's support.

All SAC administrators have LDAP auth to the OSL Virtual Machines.

To ssh into a server using your LDAP account, you can do the following replacing your_osgeo_login with your OSGeo login and vmname with the vm name of the server at OSL.

 ssh your_osgeo_login@vmname.osgeo.osuosl.org

When prompted for password, use your OSGeo Login password.

SAC:Primary Administrators also have ssh key access in case LDAP is down and that will also apply to the physical machines. Worst case scenario use the information on Open Source Labs to file a ticket (SAC members only). Direct connection to virtual machines is by appending it's vm alias to .osgeo.osuosl.org.

Live Status via Munin

osgeo 7

New server just came in as of June 2018. Intended to replace osgeo3 and whatever left on osgeo4. Configuration Details

Container setup of all the osgeo7 servers is located in https://git.osgeo.org/gitea/sac/osgeo7/wiki/_pages

Running LXD 3 snap based container management -- currently at LXD version 3.16

Accessing containers via ssh

Only the download.osgeo.org is directly exposed ssh via port 22. To access the other containers, you can tunnel thru download.osgeo.org -- You will need an ssh key on download already to do so.

A convenient block to add to your own `.ssh/config` file follows:

Host osgeo-*
  ProxyCommand ssh download.osgeo.org -W $(sed -e "s/^osgeo-//;s/$/.lxd/" <<< "%h"):%p

With the above in place, you can connect to any container using:

 ssh osgeo-<container_name>

Services currently on:

Download

Container Name: download

Moved 5/5/2019 from osgeo3 https://download.osgeo.org Setup detailed in OSGeo7 Download container This is new home of download.osgeo.org, it's a debian 9 container (now locked down to only allow ssh key access / ldap auth) for ssh. It is running nginx instead of apache that the old ran. It has webdav with single local htaccess account geotools for geotools bot use.

Secure

Container Name: secure secure -- ldap.osgeo.org SAC:LDAP used for ldap service (a rebuild of old secure.osgeo.osuosl.org) now on Debian 9

Nextcloud

Container Name: nextcloud, collabora (for document/view/editing) Nextcloud - Ubuntu 18.04 LXD/nginx/postgresql 11 container for document sharing similar to dropbox/google drive - nextcloud-ubuntu.lxd - https://nextcloud.osgeo.org Nextcloud Setup

Collabora - Ubuntu 18.04 LXD container for LibreOffice/MS Office online document editor currently used exclusively by nextcloud.osgeo.org. Setup detailed in Nextcloud setup

NGINX proxy

nginx.lxd - proxy that routes all http/https traffick for the other containers (can only be accessed via osgeo7 host)

Dronie Server

Container Name: dronie-server

1.0 server of drone.io runs in an LXD container details Dronie https://dronie.osgeo.org

Old Projects

Container Name: old-projects -- this is the old projects.osgeo.osuosl.org migrated as an lxd container, so more or less the same as it was before, with the exception that all the websites are now proxied thru the nginx container. Websites like on it are community-review.foss4g.org and spatialreference.org

To access you need to go thru download.osgeo.org -> old-projects.lxd

Adhoc

old-adhoc -- this is the old adhoc.osgeo.osuosl.org migrated as an lxd container. Used by osgeo-live for there test docs and by grass for earthquake, and mapserver for demo

Adhoc 2019-05-08 THIS VM has been migrated to osgeo7 as an LXD container old-adhoc

To access via ssh you should go thru download.osgeo.org -> old-adhoc.lxd It is accessible via https://adhoc.osgeo.org and http://adhoc.osgeo.osuosl.org

VM used for projects for various adhoc purposes. Risks to system stability that would be unacceptable on the Projects VM may be ok here.
See AdhocVM for full details, and some notes on services running here.
eg http://adhoc.osgeo.osuosl.org/livedvd/docs/en/quickstart/

Old Web

old-web --- the old web.osgeo.osuosl.org (was on osgeo3) currently only housing https://id.osgeo.org/ for LDAP management. The rest are planned to be moved here

osgeo6

Mail

See Osgeo6 for full details

Backup (osgeo5)

Backup now runs on dedicated hardware
Provides Rsync backups of download.osgeo.org
Provides Bacula backups of various VMs.
See SAC:Backups for details.

osgeo4

osgeo4 is a real server managed by OSUOSL - can be access via ssh tech_dev@osgeo4.osgeo.osuosl.org -p 2222 (only people with their access keys installed can log in and doesn't allow password access) - password for tech_dev is in the secure container (on osgeo7) / access folder.

In August 2019 the server had new power supply put in and replacement disks. It was reformatted with Ubuntu 18.04.3 to serve as secondary LXD host to osgeo7 zfsutils-linux was installed so lxd can use zfs for storage.

Work is in process to make this a secondary LXD host.

It's makeup is as follows:

Item	Settings
Disks	6 1.8 TB drives
Memory	48 GB
CPUs	8 Intel(R) Xeon(R) CPU E5540 @ 2.53GHz (8192kb cache)

lsblk -i
NAME           MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sda              8:0    0  1.8T  0 disk  
|-sda1           8:1    0  953M  0 part  
| `-md0          9:0    0  952M  0 raid1 /boot
`-sda2           8:2    0 46.6G  0 part  
  `-md1          9:1    0 46.5G  0 raid1 
	|-lvm-root 253:0    0 37.3G  0 lvm   /
	`-lvm-swap 253:1    0  7.5G  0 lvm   [SWAP]
sdb              8:16   0  1.8T  0 disk  
|-sdb1           8:17   0  953M  0 part  
| `-md0          9:0    0  952M  0 raid1 /boot
`-sdb2           8:18   0 46.6G  0 part  
  `-md1          9:1    0 46.5G  0 raid1 
	|-lvm-root 253:0    0 37.3G  0 lvm   /
	`-lvm-swap 253:1    0  7.5G  0 lvm   [SWAP]
sdc              8:32   0  1.8T  0 disk  
sdd              8:48   0  1.8T  0 disk  
sde              8:64   0  1.8T  0 disk  
sdf              8:80   0  1.8T  0 disk

sdc,sdd,sde,sdf  form a zfs osgeo4_lxd partition (sdc,sdd) mirrors sde,sdf for total lxd capacity of 3.62 TB

Eventually nightly backup of osgeo7 containers will be here and named <container>-backup and be kept in a stopped state.

Services currently on:

osgeo-nginx

Container Name: osgeo-nginx ->> still in progress and all web traffick will proxy thru here

wordpress-dev

Container Name: wordpress-dev - used primarily for osgeo.org main website development - https://staging.www.osgeo.org, https://dev.www.osgeo.org

osgeo3

osgeo3 is a real server managed by OSUOSL and running a virtualization solution (ganeti+kvm) to provide Virtual Machines which are the ones SAC can access and manage. SAC can _not_ access osgeo3 directly, but has to contact OSUOSL for that.

Each of the sections below represent a Virtual Machine and describes which services are running on it, and by which name they are reachable.

Wiki

See OSGeo Wiki

TracSvnGit

See TracsvnVM for full details, and some notes on services running here

Web

* Drupal instances for the main OSGeo web site and projects like MapGuide and FDO are hosted here (??). DEFUNCT- moved to wordpress on web18a.osgeo.osuosl.org

Virtual Hosts (on same server)
- http://fdo.osgeo.org
- http://mapguide.osgeo.org
LDAP Python Admin Scripts in /usr/lib/cgi-bin - accessed via https://id.osgeo.org (this current reside on osgeo7 lxd container called old-web)

See Infrastructure Transition Plan 2010#Final_Plan for full details. * Infrastructure Transition Plan 2010#osgeo3_.26_osgeo4 * Lots of config information in the Migration Documentation.

SAC:Backups describes backup strategy.

Base

Template VM used when creating new VMs.
Not normally running - make a special request to OSL to start it up.

WebExtra

See WebExtraVM for full details (server: http://webextra.osgeo.osuosl.org)
hosts http://planet.osgeo.org, http://mum03.mapserver.org, http://live.osgeo.org
http://foss4g.org (main portal) and archive of old sites 2006-2014
http://conference.osgeo.org - Conference System (also: SAC:Setup_OCS)
http://journal.osgeo.org / osgeo.org/ojs - Journal System
Redirects for many chapter and other urls handled via /etc/httpd/conf.d/rewrite.conf

Cloud Hosted Servers and other external under SAC Control

osgeo funtoo

For lxd experimentation it's an lxd container running other lxd containers and provided by funtoo.org.

OSGeo Funtoo osgeo.host.funtoo.org

LXDs currently running

LimeSurvey -this may be in future migrated to osgeo7

web18a.osgeo.osuosl.org

(Cloud hosted server on OSUOSL hardware (not ours) )

Debian 9.3 4GB server, host name: web18a.osgeo.osuosl.org require ssh key to log in.
Hosts wordpress sites staging.www.osgeo.org,www.osgeo.org, staging.grass.osgeo.org, foss4g2018.osgeo.org
Setup details on Web18a setup

Atlantic.net

host.postgis.net -p 2222 is an LXD Debian 10 8GB RAM
Currenlty running two lxd containers:

   debbie: postgis.net, planet.postgis.net, debbie.postgis.net (jenkins build bot)  
   debbie-docker.host.postgis.net - runs docker and serves as a 1.0 agent for dronie.osgeo.org

QGIS off OSGeo

Services on separated machines rented and managed by the QGIS project at hetzner

website including documentation http://www.qgis.org
website building, documentation building, debian/ubuntu nightlies, plugins.qgis.org
issues.qgis.org: redmine

Historical servers (not more in use)

- Telascience Blades (Historical)

@@ Line 158: / Line 158: @@
 === wordpress-dev  ===
-'''Container Name:''' wordpress-dev - still in progress
+'''Container Name:''' wordpress-dev - used primarily for osgeo.org main website development - https://staging.www.osgeo.org, https://dev.www.osgeo.org
 == osgeo3 ==

Difference between revisions of "SAC Service Status"