Revision as of 09:35, 16 December 2010

Backups

backup.osgeo.org

This the OSUOSL VM used for backups starting in 2010/04. It is used for rsync backups of various things (download.osgeo.org, osgeo1, ...), and backula backups of OSU OSL VMs.

Admins: Frank Warmerdam (rsync), Martin Spott (backula)
access is via SAC shell group.
download.osgeo.org rsync mirroring runs nightly and is externally available at download2.osgeo.org.
/osgeo/backup/backula is for backula backups
/osgeo/backup/rsync is for rsync backups
/osgeo/longterm is for long term backups of static material (ie conference web sites)

More documentation is required on Backula.

osgeo1 (primary peer1 host)

current cron jobs for backups

/etc/cron.d/backup.cron

#### Backup cron jobs 
# min hour day month dayofweek user command
#
# 2am: Daily rsync to /home/back
05 2 * * * root /root/scripts/daily.sh
# Every 3 hours 
# /etc/mysql-zrm
#33 */3 * * * root /usr/bin/rsync -a --delete /etc/mysql-zrm/ /home/back/etc/mysql-zrm/
# /var/lib/mysql-zrm
#34 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mysql-zrm/ /home/back/mysql-zrm/
# call backup_trac_svn.sh to backup 
# subversion /var/www/svn/repos
# trac /var/www/trac
45 */3 * * * root /root/scripts/backup_svn_incremental.sh
#45 */3 * * * root /root/scripts/trac_backup.pl
#45 */3 * * * root /root/scripts/backup_trac_svn.sh
# Thisdoes some hourly drupal/mysql optimization
39 * * * * root /root/scripts/drupal_cron.sh

backup script are in /root/scripts

osgeo3 & osgeo4

See: Infrastructure Transition Plan 2010

download.osgeo.org

As noted in Download Server the contents of download.osgeo.org are rsync'ed to osgeo2.

ProjectsVM

Bacula - we need details.

??? How does it work?

svn.osgeo.org

As noted in Subversion the subversion repository is mirrored using svn mirroring and does not need to be backed up otherwise.

@@ Line 14: / Line 14: @@
 == osgeo1 (primary peer1 host) ==
-     '''TODO''': add in which time zone cron is running - Atlanta [http://www.timeanddate.com/worldclock/city.html?n=25 EST]?
-      WISH: please don't run heavy backups during European day, but preferably
-      during Pacific Ocean day   ([http://www.timeanddate.com/worldclock/meetingtime.html?month=12&day=30&year=2007&p1=770&p2=248&p3=-1&p4=-1 optimize backup time])
 ==== current cron jobs for backups ====
@@ Line 44: / Line 38: @@
 backup script are in /root/scripts
-backup_svn_incremental.sh runs every 3 hours and only dumps what has changed in the repository in the last three hours to /home/back/svn_backup/project/incremental
-backup-svn_full.sh runs once a day at 1 am and does a 'svnadmin hotcopy' of the repository in /home/back/svn_backup/project/full.  14 copies of the repository are kept.
-When adding a new svn repository, make sure to edit both the _full and _incremental scripts to add backups for your projects.  You will also have to create the /home/back/svn_backup/project/{full,incremental} directories or the scripts will not work.
-'''rsync_back''' rsyncs www.osgeo.org/home/back with test.osgeo.net/home/back (the second PEER1 server)
-=== Daily PEER1 Backup ===
-PEER1 hosting service is doing daily backup of identified parts of our system using their [http://en.wikipedia.org/wiki/Storage_area_network#SAN_infrastructure SANS] infrastructure.
-* This is a service we pay for on both osgeo1 and osgeo2.  Do we need it on both?
-* At present it is backing up the '''portions identified below''' <s>complete file systems</s> on both servers, so we maintain a reasonable sized backup without incurring storage costs.  Update the list by sending them the list through their support ticket system.
-The backup daemon can be disabled with:
- /sbin/service dsmcad stop
- /sbin/service dsmc stop
-and restarted with:
- /sbin/service dsmc start
- /sbin/service dsmcad start
-added note (december 2009): the peer1 admin staff seem determined to restart this service so I have renamed the executables:
- mv /usr/bin/dsmc /usr/bin/dsmc_hidden
- mv /usr/bin/dsmcad /usr/bin/dsmcad_hidden
-==== Backup Folders / Files ====
-Please list locations to be backed up here, preferably the same on both servers.  Unless we feel we can drop the service on one of them.  Storage stats are from early 2008 and out of date by now.
-# /etc - 10MB
-# /var/www/trac - 550MB
-# /root - 3MB
-# /home/back/mailman - 1370MB
-# /var/lib/mysql-zrm/daily - 401MB
-# /usr/local/awstats/data - 52MB
-# /var/www/- 6GB (SVN 7GB excluded, below)
-'''running total: < 9GB'''
-===== Exclude =====
-# /var/www/svn - 7GB
-=== Mysql ===
-MySQL is being backed up using [http://mysqlbackup.zmanda.com MySQL-zrm]
-* Daily recovery files saved at: /var/lib/mysql-zrm/daily - other files needed?
-=== LDAP ===
-* Anything needed for this?  i.e. /var/lib/ldap ?
-== osgeo2 (test.osgeo.net) ==
-* '''Daily PEER1 Backup''' above also applies
 == osgeo3 & osgeo4 ==
@@ Line 103: / Line 48: @@
 == [[ProjectsVM]] ==
+Bacula - we need details.
 ??? How does it work?
+== svn.osgeo.org ==
+As noted in [[Subversion]] the subversion repository is mirrored using svn mirroring and does not need to be backed up otherwise.
 [[Category:Infrastructure]]

Difference between revisions of "SAC:Backups"