Revision as of 07:01, 12 December 2007

Description of current layout and installed software

Notes from Shawn on our PEER1 system:

OS Version: Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
For most elements of system i tried to stay with the default redhat locations and red hat el 4 rpms installed from peer 1's up2date repository. Reasoning, the servers are updated automatically against this repository and supported by PEER1 - reduce sysadmin load on keeping packages updated by using packages not available through PEER1 repository

Installed software using up2date

apache - httpd-2.0.52-28.ent.i386
postfix - postfix-2.2.10-1.RHEL4.2.i386
php - php-4.3.9-3.22PIDH.i386
python - python-2.3.4-14.3.i386
mailman - mailman-2.1.5.1-34.rhel4.5.i386

Red Hat EL 4 rpms installed manually (rpm -i)

MySQL-client-standard-5.0.27-0.rhel4.i386.rpm
MySQL-server-standard-5.0.27-0.rhel4.i386.rpm
MySQL-devel-standard-5.0.27-0.rhel4.i386.rpm
MySQL-shared-compat-5.0.27-0.rhel4.i386.rpm
clearsilver-0.10.1-1.2.el4.rf.i386.rpm
sqlite-2.8.16-1.2.el4.rf.i386.rpm
python-clearsilver-0.10.1-1.2.el4.rf.i386.rpm
python-sqlite-1.0.1-12.el4.rf.i386.rpm
subversion-1.4.3-0.1.el4.rf.i386.rpm
mod_dav_svn-1.4.3-0.1.el4.rf.i386.rpm

Not specific to Red Hat EL 4 rpms installed manually

MySQL-zrm-1.1.2-1.noarch.rpm

Source Install

drupal-4.7.4.tar.gz
phpldapadmin-0.9.8.3.tar.gz
trac-0.10.3.tar.gz

Paths to services directories

Apache root directory

   /var/www/html/

Subversion parent directory

   /var/www/svn/repos/

Trac parent directory

  /var/www/trac/

IP tables custom

All iptables rules must be written in /etc/sysconfig/iptables-custom

Current custom rules are:

/etc/sysconfig/iptables-custom/SSH

  *filter
  -A OUTPUT -p tcp --dport 22 -j ACCEPT
  COMMIT

/etc/sysconfig/iptables-custom/VERITAS

note: this file was placed here by PEER1

  *filter
  -A INPUT -s 10.0.48.0/24 -d 0/0 -p tcp -j ACCEPT
  -A OUTPUT -s 0/0 -d 10.0.48.0/24 -p tcp -j ACCEPT
  COMMIT

System login and maintenance procedures

Note: if change root password must let PEER1 know the new password for backup/restore and tickets. i would refer no root ssh login but, PEER1 admins need root access.

Apache

As the system is Red Hat use /sbin/services httpd start|stop|restart|configtest|reload instead of /usr/sbin/apchectl

the main Config file for apache is, /etc/httpd/conf/httpd.conf

additional config files are at /etc/httpd/conf.d/

ssl.conf
subversion.conf
trac.conf
virtual_host.conf
rewrite.conf
mailman.conf
trac.gdal.conf
php.conf
python.conf
perl.conf
phpldapadmin.conf - http auth and ssl directives for access to https://www.osgeo.org/ldapadmin
working.conf - http auth and ssl directives for access to https://www.osgeo.org/_ldap/ldap.php
webalizer.conf - http auth and ssl directives for access to https://www.osgeo.org/usage.php

Postfix

Postfix documentation

The postfix daemon can be be stared|stopped|restarted|reloaded with,

$ sudo /sbin/service postfix start|stop|restart|reload|abort|flush|check|status|condrestart

The main postfix config file is,

/etc/postfix/main.cfg

The aliases file is,

/etc/aliases

If aliases file is edited remember to run newaliases

 $ sudo /usr/bin/newaliases
 $ sudo /sbin/service postfix reload

OpenLDAP

OpenLDAP Admin Guide

The ldap daemon can manipulated with,

$ sudo /sbin/service ldap start|stop|restart|

The main ldap config file is,

/etc/openldap/slapd.conf

Currently ldap structure is pretty basic. The purpose for keeping this structure simple is to allow for a more complex structure to be evolved as ldap becomes increasingly integrated into the full osgeo systems structure.

Loading an ldif file into an ldap directory

  $ ldapadd -a -W -x -D "cn=Manager,dc=osgeo,dc=org" -f fileName.ldif

Accessing the ldap directory through phpldapadmin

access an ldapadmin interface at https://www.osgeo.org/ldapadmin

Access to this interface is restricted to the admin group

users can be added to the ou=people,dc=osgeo,dc=org once users are added they can then be added to cn=project,ou=svn,dc=osgeo,dc=org

login as cn=Manager,dc=osgeo,dc=org

expand dc=osgeo,dc=org tree
expand the ou=people tree
click on 'create new entry here'
choose 'custom' and then 'proceed'
- RDN is in the form of uid=username
- ObjectClass is InetOrgPerson
- click proceed
  - cn is common name in form of firstname lastname
  - sn is surname - lastname
  - no optionals are required
- click 'create object'
  - on this page click 'add new attribute'
    - choose mail
      - enter email address
      - click add
    - on this page click 'add new attribute'
      - choose userPassord
      - enter password as an md5
      - click add

this adds the new user to ou=people

to add users to the cn=project,ou=svn group

expand the ou=svn tree
click on cn=project
- under member click 'add value'
- add user to this in same form as you and howard are listed
- or can click on the 'folder with magnifying glasss' to select a user

that's it.

ldap structure

dc=osgeo,dc=org

- cn=Manager

- ou=people

- - Separate entity for each user

uid=login,ou=people,dc=osgeo,dc=org

objectClass=inetOrgPerson

cn=firstName lastName

sn=lastName

uid=login

mail=email@address

userPassword={md5}YPTyViiMKhiuWKEmFUOKLA==

- ou=projects,dc=osgeo,dc=org
objectClass=organizationalUnit

ou=project

description=separate entity for each osgeo project with list of members

- - Separate entity for each project group
cn=admin,ou=projects,dc=osgeo,dc=org

objectClass=groupOfNames

cn=admin

description=osgeo sysadmin group

- ou=svn
objectClass=organizationalUnit

ou=svn

description=separate entity for each repository with list of members with commit rights

- - separate entity for each svn group
cn=fdo,ou=svn,dc=osgeo,dc=org

objectClass=groupOfNames

cn=fdo

member= dn of member

cn=gdal,ou=svn,dc=osgeo,dc=org

objectClass=groupOfNames

cn=gdal

member= dn of member

cn=mapguide,ou=svn,dc=osgeo,dc=org

objectClass=groupOfNames

cn=mapguide

member= dn of member

cn=mapbender,ou=svn,dc=osgeo,dc=org

objectClass=groupOfNames

cn=mapbender

member= dn of member

Example ldif file

<code>
   version: 1

   dn: dc=osgeo,dc=org
   objectClass: dcObject
   objectClass: organization
   description: OSGeo ldap dit
   o: OSGeo
   dc: osgeo

   dn: cn=Manager,dc=osgeo,dc=org
   objectClass: organizationalRole
   cn: Manager

   dn: ou=people,dc=osgeo,dc=org
   ou: people
   description: all users of osgeo
   objectClass: organizationalUnit

   dn: uid=jsmith,ou=people,dc=osgeo,dc=org
   objectClass: inetOrgPerson
   uid: jsmith
   cn: Jon Smith
   sn: Smith 
   givenName: Jon
   mail: jsmith@somewhere.com
   userPassword: {md5}5Or4zfzGqo3jh/6iIUgKcA==

   dn: uid=jbrown,ou=people,dc=osgeo,dc=org
   objectClass: inetOrgPerson
   uid: jbrown
   cn: Jane Brown
   sn: Brown
   givenName: Jane 
   mail: jbrown@someotherplace.com
   userPassword: {md5}1iWhTyvkK2m4Uuar+Dp/IA==

   dn: ou=projects,dc=osgeo,dc=org
   ou: projects
   description: separate entity for each osgeo project with list of members
   objectClass: organizationalUnit

   dn: cn=admin,ou=projects,dc=osgeo,dc=org
   cn: admin
   description: osgeo sysadmin group
   objectClass: groupOfNames
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: ou=svn,dc=osgeo,dc=org
   ou: svn
   description: separate entity for for each repository.list of members with commit rights
   objectClass: organizationalUnit

   dn: cn=fdo,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: fdo
   member: uid=jsmith,ou=people,dc=osgeo,dc=org
   member: uid=jbrown,ou=people,dc=osgeo,dc=org

   dn: cn=gdal,ou=svn,dc=osgeo,dc=org
   cn: gdal
   objectClass: groupOfNames
   objectClass: top
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: cn=mapbender,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: mapbender
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: cn=mapguide,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: mapguide
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
</code>

Subversion

See Subversion for details on subversion configuration.

Trac

Trac Instances

Mailman Maintenance

See SAC:Mailing Lists.

Backups

See SAC:Backups.

DNS

See SAC DNS Registry.

FDO Repository Merge

To merge repositories you need to parse through the dumpfiles.

In the fdo merge all subprojects had the same directory strucuture, /trunk/Providers/ All dumpfiles had to be 'filtered' to just pull the /trunk/Providers/* and drop branches and tags.

SVNDUMPFILTER

$ svndumpfilter include --help

Filter out nodes without given prefixes from dumpstream

Usage: svndumpfilter include PATH_PREFIX Options: --drop-empty-revs Remove revisions emptied by filtering --renumber-revs Renumbe revisions left after filtering --preserve-revprops Don't filter revision properties --quiet Do not display filtering statistics

$ svndumpfilter exclude --help Filter out nodes with given prefixes from dumpstream Usage: svndumpfilter exclude PATH_PREFIX Options: --drop-empty-revs Remove revisions emptied by filtering --renumber-revs Renumbe revisions left after filtering --preserve-revprops Don't filter revision properties --quiet Do not display filtering statistics

This process followed for all repositories

merge fdogdal trunk

 $ cat fdogdal.dmp | svndumpfilter include trunk/www/ > fdogdal-merge-trunk.www.dmp
 $ cat fdogdal.dmp | svndumpfilter include trunk/Providers/ > fdogdal-merge-trunk.Providers.GDAL.dmp

edit file and remove lines,

  Node-path: trunk/
  ....
  Node-action: add
  ....
  PROPS-END

  Node-path: trunk/www/
  ....
  Node-action: add
  ....
  PROPS-END

  Node-path: trunk/Providers/
  ....
  Node-action: add
  ....
  PROPS-END

Rename index.html to fdogdal-index.html

 $ perl -pi.bak -e 's/^Node-path:\ trunk\/www\/index.html/Node-path:\ trunk\/www\/fdogdal-index.html/g' fdo-merge.trunk.dmp

 $ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp

@@ Line 311: / Line 311: @@
 = Backups =
-=== Daily tape backup ===
+See [[SAC:Backups]].
-* PEER1 is doing daily tape backup of entire system
-=== Other backups ===
-==== Mysql ====
-MySQL is being backed up using [http://mysqlbackup.zmanda.com MySQL-zrm]
-==== current cron jobs for backups ====
-/etc/cron.d/backup.cron
-   #### Backup cron jobs
-   # min hour day month dayofweek user command
-   #
-   # Daily rsync to /home/back
-   # /etc
-10 * * * root /usr/bin/rsync -a --delete /etc/ /home/back/etc/
-   # /var/www/html
-10 * * * root /usr/bin/rsync -a --delete /var/www/html/ /home/back/html/
-   # Every 3 hours
-   # /var/lib/mailman
-*/3 * * * root /usr/bin/rsync -a --delete /var/lib/mailman/ /home/back/mailman/
-   # /etc/mysql-zrm
-*/3 * * * root /usr/bin/rsync -a --delete /etc/mysql-zrm/ /home/back/etc/mysql-zrm/
-   # /var/lib/mysql-zrm
-*/3 * * * root /usr/bin/rsync -a --delete /var/lib/mysql-zrm/ /home/back/mysql-zrm/
-   # call backup_trac_svn.sh to backup
-   # subversion /var/www/svn/repos
-   # trac /var/www/trac
-1 * * * root /root/scripts/backup_svn_full.sh
-*/3 * * * root /root/scripts/backup_svn_incremental.sh
-*/3 * * * root /root/scripts/trac_backup.pl
-   # once a day rsync /home/back to osgeo2.osgeo.net
-22 * * * root /root/scripts/rsync_back.pl
-backup script are in /root/scripts
-backup_svn_incremental.sh runs every 3 hours and only dumps what has changed in the repository in the last three hours to /home/back/svn_backup/project/incremental
-backup-svn_full.sh runs once a day at 1 am and does a 'svnadmin hotcopy' of the repository in /home/back/svn_backup/project/full.  14 copies of the repository are kept.
-When adding a new svn repository, make sure to edit both the _full and _incremental scripts to add backups for your projects.  You will also have to create the /home/back/svn_backup/project/{full,incremental} directories or the scripts will not work.
-'''rsync_back''' rsyncs www.osgeo.org/home/back with test.osgeo.net/home/back (the second PEER1 server)
 = DNS =

Difference between revisions of "Migration Documentation"