SAC:Backups

Backups

backup.osgeo.org (aka osgeo5, mirror)

This was the OSUOSL VM used for backups starting in 2010/04 and is now upgraded to dedicated hardware (very) early in 2014. It is used for Bacula backups of OSU OSL VMs and Rsync backups of the public space at "download.osgeo.org".

• Admins: Frank Warmerdam (rsync), Martin Spott (bacula), Sandro Santilli (bacula)
• access is via SAC shell group.
• download.osgeo.org rsync mirroring runs nightly and is externally available at download2.osgeo.org.
• /backup/bacula/ contains the Bacula storage device
• /backup/rsync/ is for rsync backups
• /backup/longterm/ is for long term backups of static material (ie conference web sites)
• /backup/pgdump/ contains PostgreSQL dumps of the Bacula director database
• /etc/bacula contains Bacula configuration, is under local git repo

Hardware

CPU: Intel Xeon E3-1220v2, 3.1GHz (4-Core, 8MB Cache, 69W) 22nm
RAM: 16GB (2 x 8GB DDR3-1600 ECC Unbuffered 2R DIMMs) Operating at 1600 MT/s Max
NIC: Two Gigabit Ethernet Controllers (Intel 82574L) - Integrated
Management: Integrated IPMI 2.0 & KVM with Dedicated LAN
Controller: 2 Ports 6Gb/s SATA, and 4 Ports 3Gb/s SATA via Intel C216 Chipset
PCIe 3.0 x16: No Item Selected
PCIe 2.0 x4 (x8): No Item Selected
NOTE: Drives will be connected to onboard SATA controller unless otherwise specified
Hot-Swap Drive - 1: 3TB Seagate Constellation ES.3 (6Gb/s, 7.2K RPM, 128MB Cache) 3.5-inch SATA
Hot-Swap Drive - 2: 3TB Seagate Constellation ES.3 (6Gb/s, 7.2K RPM, 128MB Cache) 3.5-inch SATA
Hot-Swap Drive - 3: 3TB Seagate Constellation ES.3 (6Gb/s, 7.2K RPM, 128MB Cache) 3.5-inch SATA
Hot-Swap Drive - 4: 3TB Seagate Constellation ES.3 (6Gb/s, 7.2K RPM, 128MB Cache) 3.5-inch SATA
Optical Drive: Blanking Panel, No Optical Drive
Front Input: Blanking Panel - No Front Inputs
Power Supply: Redundant 500W Power Supply with PMBus - 80 PLUS Platinum Certified
Rail Kit: 1U Sliding Rail Kit
OS: No Item Selected
Warranty: Standard 3-Year Warranty
Notes:
New quote with redundant power supply. Can I get a 2.5 inch spinning
disk instead of SSD in the Front Input slot (500 GB Seagate
Constellation 6 GB/s 2.5")?

OS installation

• Create an empty partition at the beginning of every hard drive, use "Reserved BIOS boot area" on GPT-partitioned disks (typical with 2 GByte disks or bigger) or just "Unused" on DOS-type partition tables. By tradition I reserve 10 MByte on each disk, maybe 1 MByte would be sufficient as well, but these 9 MByte of difference don't cost that much and you're on the safe side ....
• Create a second partition on every disk - for use by MD - allocating all the remaining disk space
• MD RAID5 over all the 'second' partitions for use by LVM - 8 TB usable space
• LVM volume group on top of the MD physical volume
• Volumes
  • "boot" (256 MByte, Ext2, /boot/),
  • "root" (20 GByte, XFS, /root/),
  • "swap" (8 GByte),
  • "var" (20 GByte, XFS, /var/)
• Install the base operating system, don't forget to set up SSH ;-)
• Volumes
  • "backup" (2 TByte, XFS, /backup/),
  • "mirror" (1 TByte, XFS, /mirror/)

Bacula

See SAC:Bacula

osgeo1 (primary peer1 host)

current cron jobs for backups

/etc/cron.d/backup.cron

#### Backup cron jobs 
# min hour day month dayofweek user command
#
# 2am: Daily rsync to /home/back
05 2 * * * root /root/scripts/daily.sh
# Every 3 hours 
# /etc/mysql-zrm
#33 */3 * * * root /usr/bin/rsync -a --delete /etc/mysql-zrm/ /home/back/etc/mysql-zrm/
# /var/lib/mysql-zrm
#34 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mysql-zrm/ /home/back/mysql-zrm/
# call backup_trac_svn.sh to backup 
# subversion /var/www/svn/repos
# trac /var/www/trac
45 */3 * * * root /root/scripts/backup_svn_incremental.sh
#45 */3 * * * root /root/scripts/trac_backup.pl
#45 */3 * * * root /root/scripts/backup_trac_svn.sh
# Thisdoes some hourly drupal/mysql optimization
39 * * * * root /root/scripts/drupal_cron.sh

backup script are in /root/scripts

osgeo3 & osgeo7

osgeo3 and osgeo7 are production LXD Hosts. Using the built in LXD scheduler snapshots of each container are done daily with a retention of 30 days. This means each container in case of disaster such as virus/exploit/serious misconfiguration can be restored to up to 30 days before.

In addition the lastest snapshots of each container are made on a daily or weekly basis by osgeo4. Smaller containers such as secure are snapshot nightly and pushed to osgeo4 using cronjob - /etc/cron.d/osgeo4-lxd-maint on osgeo4. Larger containers such as repo (on osgeo3) and download and tracsvn on osgeo7 are snapshotted weekly and pushed to osgeo4. These containers are prefixed with -backup. So you will see on osgeo4 containers such as download-backup, grass-backup etc.

Right now a manual process - backups of key containers are made monthly and stored on backup.osgeo.org in the /backup/osgeo7_containers/image_backups/ folder. There is currently not enough space there to hold the download or repo backups. download is however rsync'ed to ftp.osuosl.org/osgeo and folder in backup.osgeo.org.

download.osgeo.org

As noted in Download Server the contents of download.osgeo.org are rsync'ed to osgeo2.

ProjectsVM

Bacula - we need details.

??? How does it work?

svn.osgeo.org

As noted in Subversion the subversion repository is mirrored using svn mirroring and does not need to be backed up otherwise.