Migration Documentation

= Description of current layout and installed software =

Notes from Shawn on our PEER1 system:


 * OS Version: Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
 * For most elements of system i tried to stay with the default redhat locations and red hat el 4 rpms installed from peer 1's up2date repository. Reasoning, the servers are updated automatically against this repository and supported by PEER1 - reduce sysadmin load on keeping packages updated by using packages not available through PEER1 repository

Installed software using up2date

 * apache - httpd-2.0.52-28.ent.i386
 * postfix - postfix-2.2.10-1.RHEL4.2.i386
 * php - php-4.3.9-3.22PIDH.i386
 * python - python-2.3.4-14.3.i386
 * mailman - mailman-2.1.5.1-34.rhel4.5.i386

Red Hat EL 4 rpms installed manually (rpm -i)

 * MySQL-client-standard-5.0.27-0.rhel4.i386.rpm
 * MySQL-server-standard-5.0.27-0.rhel4.i386.rpm
 * MySQL-devel-standard-5.0.27-0.rhel4.i386.rpm
 * MySQL-shared-compat-5.0.27-0.rhel4.i386.rpm
 * clearsilver-0.10.1-1.2.el4.rf.i386.rpm
 * sqlite-2.8.16-1.2.el4.rf.i386.rpm
 * python-clearsilver-0.10.1-1.2.el4.rf.i386.rpm
 * python-sqlite-1.0.1-12.el4.rf.i386.rpm
 * subversion-1.4.3-0.1.el4.rf.i386.rpm
 * mod_dav_svn-1.4.3-0.1.el4.rf.i386.rpm

Not specific to Red Hat EL 4 rpms installed manually

 * MySQL-zrm-1.1.2-1.noarch.rpm

Source Install

 * drupal-4.7.4.tar.gz
 * phpldapadmin-0.9.8.3.tar.gz
 * trac-0.10.3.tar.gz

= Paths to services directories =

Apache root directory /var/www/html/

Subversion parent directory /var/www/svn/repos/

Trac parent directory /var/www/trac/

= IP tables custom =

All iptables rules must be written in /etc/sysconfig/iptables-custom

Current custom rules are:

/etc/sysconfig/iptables-custom/SSH *filter -A OUTPUT -p tcp --dport 22 -j ACCEPT COMMIT

/etc/sysconfig/iptables-custom/VERITAS

note: this file was placed here by PEER1 *filter -A INPUT -s 10.0.48.0/24 -d 0/0 -p tcp -j ACCEPT -A OUTPUT -s 0/0 -d 10.0.48.0/24 -p tcp -j ACCEPT COMMIT

= System login and maintenance procedures =

'''Note: if change root password must let PEER1 know the new password for backup/restore and tickets. i would refer no root ssh login but, PEER1 admins need root access.'''

= Subversion maintanence = Subversion Book

Creating a repository
$ sudo mkdir /var/www/svn/repos/ $ sudo svnadmin /var/www/svn/repos/ $ sudo chown -R apache:apache /var/www/svn/repos/

Loading a repository from a dumpfile
$ sudo svnadmin load /var/www/svn/repos/ < /path/to/dumpfile $ sudo chown -R apache:apache /var/www/svn/repos/

Dumping a repository
$ sudo svnadmin dump /var/www/svn/repos/ > dumpfile

Recovering / unlocking repository
priviledged access is needed (may need to stop/start apache to drop requests to repository before recover)
 * subversion may lock if user ctl-c during checkout or checkout is interupted with an apache restart
 * Need policy on who to contact and who can run 'svnadmin recover' as

Subversion was upgraded to version 1.4.3 (30 Jan. 2007) to reduce the repository locking problem if ctl-c used to end a checkout.

If the repository locks the following command should be used to recover the repository:

$ sudo svnadmin --wait recover /var/www/svn/repos/ $ sudo chown -R apache:apache /var/www/svn/repos/

In most cases this will work in the odd case that you are waiting a very long time for the command to run then apache may have to be restarted to drop anything accessing the repository and preventing the repository from being recovered

$ sudo /sbin/service httpd restart $ sudo svnadmin --wait recover /var/www/svn/repos/ $ sudo chown -R apache:apache /var/www/svn/repos/

= TRAC Maintenance = Trac Guide

Creating a trac instance
$ sudo trac-admin /var/www/trac/ initenv $ sudo chown -R apache:apache /var/www/trac/

Hotcopy a trac instance
$ sudo trac-admin /var/www/html/trac/ hotcopy /path/to/copy/trac/to

= Mailman Maintenance =

create a new mailing list
$sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd

renaming a list
$sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd $ sudo cp /var/lib/mailman/oldlist/oldlist.mbox/oldlist.mbox \ /var/lib/mailman/newlist/newlist.mbox/ $ sudo /usr/lib/mailman/bin/arch --wipe newlist $ sudo /usr/lib/mailman/bin/list_members -r listname > listname-regular.txt $ sudo /usr/lib/mailman/bin/list_members -d listname > listname-digest.txt $ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y listname $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y listname $ sudo /usr/lib/mailman/bin/rmlist listname $ sudo /usr/lib/mailman/bin/genaliases $ sudo /usr/lib/mailman/bin/check_perms -f
 * create new list
 * move original lists archive to newlists archive location
 * create archive
 * export subscribers from old list regular and digest members
 * import subscribers into new list
 * remove old list
 * update aliases and check that proper permissions are set

= Backups =

Daily tape backup

 * PEER1 is doing daily tape backup of entire system

Mysql
MySQL is being backed up using MySQL-zrm

current cron jobs for backups
/etc/cron.d/backup.cron

#### Backup cron jobs # min hour day month dayofweek user command #  # Daily rsync to /home/back # /etc 05 10 * * * root /usr/bin/rsync -a --delete /etc/ /home/back/etc/ # /var/www/html 08 10 * * * root /usr/bin/rsync -a --delete /var/www/html/ /home/back/html/ # Every 3 hours # /var/lib/mailman 20 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mailman/ /home/back/mailman/ # /etc/mysql-zrm 33 */3 * * * root /usr/bin/rsync -a --delete /etc/mysql-zrm/ /home/back/etc/mysql-zrm/ # /var/lib/mysql-zrm 34 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mysql-zrm/ /home/back/mysql-zrm/ # call backup_trac_svn.sh to backup # subversion /var/www/svn/repos # trac /var/www/trac 45 */3 * * * root /root/scripts/backup_trac_svn.sh  # once a day rsync /home/back to osgeo2.osgeo.net 0 22 * * * root /root/scripts/rsync_back.pl

backup script are in /root/scripts

backup_trac_svn.sh calls separate scripts to backup trac and svn repositories and place tgz files of the backups in /home/back/svn_backup and /home/back/trac_backup which are held for 14 days. rsync_back rsyncs www.osgeo.org/home/back with test.osgeo.net/home/back (the second PEER1 server)

= FDO Repository Merge =

To merge repositories you need to parse through the dumpfiles.

In the fdo merge all subprojects had the same directory strucuture, /trunk/Providers/ All dumpfiles had to be 'filtered' to just pull the /trunk/Providers/* and drop branches and tags.

SVNDUMPFILTER

$ svndumpfilter include --help Filter out nodes without given prefixes from dumpstream Usage: svndumpfilter include PATH_PREFIX Options: --drop-empty-revs		Remove revisions emptied by filtering --renumber-revs			Renumbe revisions left after filtering --preserve-revprops		Don't filter revision properties --quiet					Do not display filtering statistics

$ svndumpfilter exclude --help Filter out nodes with given prefixes from dumpstream Usage: svndumpfilter exclude PATH_PREFIX Options: --drop-empty-revs		Remove revisions emptied by filtering --renumber-revs			Renumbe revisions left after filtering --preserve-revprops		Don't filter revision properties --quiet					Do not display filtering statistics

This process followed for all repositories

merge fdogdal trunk

$ cat fdogdal.dmp | svndumpfilter include trunk/www/ > fdogdal-merge-trunk.www.dmp $ cat fdogdal.dmp | svndumpfilter include trunk/Providers/ > fdogdal-merge-trunk.Providers.GDAL.dmp

edit file and remove lines, Node-path: trunk/ ....  Node-action: add ....  PROPS-END

Node-path: trunk/www/ ....  Node-action: add ....  PROPS-END

Node-path: trunk/Providers/ ....  Node-action: add ....  PROPS-END

Rename index.html to fdogdal-index.html $ perl -pi.bak -e 's/^Node-path:\ trunk\/www\/index.html/Node-path:\ trunk\/www\/fdogdal-index.html/g' fdo-merge.trunk.dmp

$ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp