|
|
(13 intermediate revisions by 4 users not shown) |
Line 89: |
Line 89: |
| *working.conf - http auth and ssl directives for access to https://www.osgeo.org/_ldap/ldap.php | | *working.conf - http auth and ssl directives for access to https://www.osgeo.org/_ldap/ldap.php |
| *webalizer.conf - http auth and ssl directives for access to https://www.osgeo.org/usage.php | | *webalizer.conf - http auth and ssl directives for access to https://www.osgeo.org/usage.php |
− |
| |
− | = OpenLDAP =
| |
− | [http://www.openldap.org/doc/admin23/ OpenLDAP Admin Guide]
| |
| | | |
− | The ldap daemon can be be stared|stopped|restarted with,
| + | = Postfix = |
− | $ sudo /sbin/service ldap start|stop|restart
| + | [http://www.postfix.org/documentation.html Postfix documentation] |
| | | |
− | The main ldap config file is, | + | The postfix daemon can be be stared|stopped|restarted|reloaded with, |
− | /etc/openldap/slapd.conf | + | $ sudo /sbin/service postfix start|stop|restart|reload|abort|flush|check|status|condrestart |
| | | |
− | Currently ldap structure is pretty basic. The purpose for keeping this structure simple is to allow for a more complex structure to be evolved as ldap becomes increasingly integrated into the full osgeo systems structure.
| + | The main postfix config file is, |
| + | /etc/postfix/main.cfg |
| | | |
− | ''' Loading an ldif file into an ldap directory '''
| + | The aliases file is, |
− | $ ldapadd -a -W -x -D "cn=Manager,dc=osgeo,dc=org" -f fileName.ldif
| + | /etc/aliases |
| | | |
− | ''' Accessing the ldap directory through phpldapadmin '''
| + | If aliases file is edited remember to run newaliases |
| + | $ sudo /usr/bin/newaliases |
| + | $ sudo /sbin/service postfix reload |
| | | |
− | access an ldapadmin interface at https://www.osgeo.org/ldapadmin
| + | = LDAP = |
| | | |
− | Access to this interface is restricted to the admin group
| + | See [[SAC:LDAP]] |
− | | |
− | users can be added to the ou=people,dc=osgeo,dc=org
| |
− | once users are added they can then be added to
| |
− | cn=project,ou=svn,dc=osgeo,dc=org
| |
− | | |
− | login as cn=Manager,dc=osgeo,dc=org
| |
− | | |
− | *expand dc=osgeo,dc=org tree
| |
− | *expand the ou=people tree
| |
− | *click on 'create new entry here'
| |
− | *choose 'custom' and then 'proceed'
| |
− | **RDN is in the form of uid=username
| |
− | **ObjectClass is InetOrgPerson
| |
− | **click proceed
| |
− | ***cn is common name in form of firstname lastname
| |
− | ***sn is surname - lastname
| |
− | ***no optionals are required
| |
− | **click 'create object'
| |
− | ***on this page click 'add new attribute'
| |
− | ****choose mail
| |
− | *****enter email address
| |
− | *****click add
| |
− | ****on this page click 'add new attribute'
| |
− | *****choose userPassord
| |
− | *****enter password as an md5
| |
− | *****click add
| |
− | | |
− | this adds the new user to ou=people
| |
− | | |
− | to add users to the cn=project,ou=svn group
| |
− | *expand the ou=svn tree
| |
− | *click on cn=project
| |
− | **under member click 'add value'
| |
− | **add user to this in same form as you and howard are listed
| |
− | **or can click on the 'folder with magnifying glasss' to select a user
| |
− | | |
− | that's it.
| |
− | | |
− | === ldap structure ===
| |
− | *dc=osgeo,dc=org
| |
− | | |
− | **cn=Manager
| |
− | | |
− | **ou=people
| |
− | | |
− | ***Separate entity for each user
| |
− | | |
− | *:uid=login,ou=people,dc=osgeo,dc=org
| |
− | *:objectClass=inetOrgPerson
| |
− | *:cn=firstName lastName
| |
− | *:sn=lastName
| |
− | *:uid=login
| |
− | *:mail=email@address
| |
− | *:userPassword={md5}YPTyViiMKhiuWKEmFUOKLA==
| |
− | | |
− | **ou=projects,dc=osgeo,dc=org
| |
− | *:objectClass=organizationalUnit
| |
− | *:ou=project
| |
− | *:description=separate entity for each osgeo project with list of members
| |
− | | |
− | ***Separate entity for each project group
| |
− | *:cn=admin,ou=projects,dc=osgeo,dc=org
| |
− | *:objectClass=groupOfNames
| |
− | *:cn=admin
| |
− | *:description=osgeo sysadmin group
| |
− | | |
− | **ou=svn
| |
− | *:objectClass=organizationalUnit
| |
− | *:ou=svn
| |
− | *:description=separate entity for each repository with list of members with commit rights
| |
− | | |
− | ***separate entity for each svn group
| |
− | *:cn=fdo,ou=svn,dc=osgeo,dc=org
| |
− | *:objectClass=groupOfNames
| |
− | *:cn=fdo
| |
− | *:member= dn of member
| |
− | | |
− | *:cn=gdal,ou=svn,dc=osgeo,dc=org
| |
− | *:objectClass=groupOfNames
| |
− | *:cn=gdal
| |
− | *:member= dn of member
| |
− | | |
− | *:cn=mapguide,ou=svn,dc=osgeo,dc=org
| |
− | *:objectClass=groupOfNames
| |
− | *:cn=mapguide
| |
− | *:member= dn of member
| |
− | | |
− | *:cn=mapbender,ou=svn,dc=osgeo,dc=org
| |
− | *:objectClass=groupOfNames
| |
− | *:cn=mapbender
| |
− | *:member= dn of member
| |
− | | |
− | | |
− | '''Example ldif file'''
| |
− | <pre><code>
| |
− | version: 1
| |
− | | |
− | dn: dc=osgeo,dc=org
| |
− | objectClass: dcObject
| |
− | objectClass: organization
| |
− | description: OSGeo ldap dit
| |
− | o: OSGeo
| |
− | dc: osgeo
| |
− | | |
− | dn: cn=Manager,dc=osgeo,dc=org
| |
− | objectClass: organizationalRole
| |
− | cn: Manager
| |
− | | |
− | dn: ou=people,dc=osgeo,dc=org
| |
− | ou: people
| |
− | description: all users of osgeo
| |
− | objectClass: organizationalUnit
| |
− | | |
− | dn: uid=jsmith,ou=people,dc=osgeo,dc=org
| |
− | objectClass: inetOrgPerson
| |
− | uid: jsmith
| |
− | cn: Jon Smith
| |
− | sn: Smith
| |
− | givenName: Jon
| |
− | mail: jsmith@somewhere.com
| |
− | userPassword: {md5}5Or4zfzGqo3jh/6iIUgKcA==
| |
− | | |
− | dn: uid=jbrown,ou=people,dc=osgeo,dc=org
| |
− | objectClass: inetOrgPerson
| |
− | uid: jbrown
| |
− | cn: Jane Brown
| |
− | sn: Brown
| |
− | givenName: Jane
| |
− | mail: jbrown@someotherplace.com
| |
− | userPassword: {md5}1iWhTyvkK2m4Uuar+Dp/IA==
| |
− | | |
− | dn: ou=projects,dc=osgeo,dc=org
| |
− | ou: projects
| |
− | description: separate entity for each osgeo project with list of members
| |
− | objectClass: organizationalUnit
| |
− | | |
− | dn: cn=admin,ou=projects,dc=osgeo,dc=org
| |
− | cn: admin
| |
− | description: osgeo sysadmin group
| |
− | objectClass: groupOfNames
| |
− | member: uid=jbrown,ou=people,dc=osgeo,dc=org
| |
− | member: uid=jsmith,ou=people,dc=osgeo,dc=org
| |
− | | |
− | dn: ou=svn,dc=osgeo,dc=org
| |
− | ou: svn
| |
− | description: separate entity for for each repository.list of members with commit rights
| |
− | objectClass: organizationalUnit
| |
− | | |
− | dn: cn=fdo,ou=svn,dc=osgeo,dc=org
| |
− | objectClass: groupOfNames
| |
− | cn: fdo
| |
− | member: uid=jsmith,ou=people,dc=osgeo,dc=org
| |
− | member: uid=jbrown,ou=people,dc=osgeo,dc=org
| |
− | | |
− | dn: cn=gdal,ou=svn,dc=osgeo,dc=org
| |
− | cn: gdal
| |
− | objectClass: groupOfNames
| |
− | objectClass: top
| |
− | member: uid=jbrown,ou=people,dc=osgeo,dc=org
| |
− | member: uid=jsmith,ou=people,dc=osgeo,dc=org
| |
− | | |
− | dn: cn=mapbender,ou=svn,dc=osgeo,dc=org
| |
− | objectClass: groupOfNames
| |
− | cn: mapbender
| |
− | member: uid=jsmith,ou=people,dc=osgeo,dc=org
| |
− | | |
− | dn: cn=mapguide,ou=svn,dc=osgeo,dc=org
| |
− | objectClass: groupOfNames
| |
− | cn: mapguide
| |
− | member: uid=jbrown,ou=people,dc=osgeo,dc=org
| |
− | </code></pre>
| |
| | | |
| = Subversion = | | = Subversion = |
− | [http://svnbook.red-bean.com/nightly/en/index.html Subversion Book]
| |
− | === Creating a repository ===
| |
− | $ sudo mkdir /var/www/svn/repos/<repo_name>
| |
− | $ sudo svnadmin /var/www/svn/repos/<repo_name>
| |
− | $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>
| |
| | | |
− | === Loading a repository from a dumpfile ===
| + | See [[Subversion]] for details on subversion configuration. |
− | $ sudo svnadmin load /var/www/svn/repos/<repo_name> < /path/to/dumpfile
| |
− | $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>
| |
| | | |
− | === Dumping a repository === | + | = Trac = |
− | $ sudo svnadmin dump /var/www/svn/repos/<repo_name> > dumpfile
| |
| | | |
− | === Recovering / unlocking repository ===
| + | [[Trac Instances]] |
− | * subversion may lock if user ctl-c during checkout or checkout is interupted with an apache restart
| |
− | * Need policy on who to contact and who can run 'svnadmin recover' as
| |
− | priviledged access is needed (may need to stop/start apache to drop
| |
− | requests to repository before recover)
| |
| | | |
− | Subversion was upgraded to version 1.4.3 (30 Jan. 2007) to reduce the repository locking problem if ctl-c used to end a checkout.
| + | = Mailman Maintenance = |
| | | |
− | If the repository locks the following command should be used to recover the repository:
| + | See [[SAC:Mailing Lists]]. |
− | | |
− | $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
| |
− | $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>
| |
− | | |
− | In most cases this will work in the odd case that you are waiting a very long time for the command to run then apache may have to be restarted to drop anything accessing the repository and preventing the repository from being recovered
| |
− | | |
− | $ sudo /sbin/service httpd restart
| |
− | $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
| |
− | $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>
| |
− | | |
− | = TRAC =
| |
− | [http://trac.edgewall.org/wiki/TracGuide Trac Guide] | |
− | === Creating a trac instance ===
| |
− | $ sudo trac-admin /var/www/trac/<proj_name> initenv
| |
− | $ sudo chown -R apache:apache /var/www/trac/<proj_name>
| |
− | | |
− | === Hotcopy a trac instance ===
| |
− | $ sudo trac-admin /var/www/html/trac/<proj_name> hotcopy /path/to/copy/trac/to
| |
− | | |
− | = Mailman Maintenance =
| |
− | === create a new mailing list ===
| |
− | $sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd
| |
− | === renaming a list ===
| |
− | * create new list
| |
− | $sudo /usr/lib/mailman/bin/newlist listname admins@email lists_passwd
| |
− | * move original lists archive to newlists archive location
| |
− | $ sudo cp /var/lib/mailman/oldlist/oldlist.mbox/oldlist.mbox \
| |
− | /var/lib/mailman/newlist/newlist.mbox/
| |
− | * create archive
| |
− | $ sudo /usr/lib/mailman/bin/arch --wipe newlist
| |
− | * export subscribers from old list regular and digest members
| |
− | $ sudo /usr/lib/mailman/bin/list_members -r listname > listname-regular.txt
| |
− | $ sudo /usr/lib/mailman/bin/list_members -d listname > listname-digest.txt
| |
− | * import subscribers into new list
| |
− | $ sudo /usr/lib/mailman/bin/add_members --regular-members=listname-regular.txt --welcome-msg=y listname
| |
− | $ sudo /usr/lib/mailman/bin/add_members --digest-members=listname-digest.txt --welcome-msg=y listname
| |
− | * remove old list
| |
− | $ sudo /usr/lib/mailman/bin/rmlist listname
| |
− | * update aliases and check that proper permissions are set
| |
− | $ sudo /usr/lib/mailman/bin/genaliases
| |
− | $ sudo /usr/lib/mailman/bin/check_perms -f
| |
| | | |
| = Backups = | | = Backups = |
| | | |
− | === Daily tape backup ===
| + | See [[SAC:Backups]]. |
− | * PEER1 is doing daily tape backup of entire system
| |
− | | |
− | === Other backups ===
| |
− | ==== Mysql ====
| |
− | MySQL is being backed up using [http://mysqlbackup.zmanda.com MySQL-zrm]
| |
− | ==== current cron jobs for backups ====
| |
− | /etc/cron.d/backup.cron
| |
− | | |
− | #### Backup cron jobs
| |
− | # min hour day month dayofweek user command
| |
− | #
| |
− | # Daily rsync to /home/back
| |
− | # /etc
| |
− | 05 10 * * * root /usr/bin/rsync -a --delete /etc/ /home/back/etc/
| |
− | # /var/www/html
| |
− | 08 10 * * * root /usr/bin/rsync -a --delete /var/www/html/ /home/back/html/
| |
− | # Every 3 hours
| |
− | # /var/lib/mailman
| |
− | 20 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mailman/ /home/back/mailman/
| |
− | # /etc/mysql-zrm
| |
− | 33 */3 * * * root /usr/bin/rsync -a --delete /etc/mysql-zrm/ /home/back/etc/mysql-zrm/
| |
− | # /var/lib/mysql-zrm
| |
− | 34 */3 * * * root /usr/bin/rsync -a --delete /var/lib/mysql-zrm/ /home/back/mysql-zrm/
| |
− | # call backup_trac_svn.sh to backup
| |
− | # subversion /var/www/svn/repos
| |
− | # trac /var/www/trac
| |
− | 45 */3 * * * root /root/scripts/backup_trac_svn.sh
| |
− | # once a day rsync /home/back to osgeo2.osgeo.net
| |
− | 0 22 * * * root /root/scripts/rsync_back.pl
| |
− | | |
− | backup script are in /root/scripts
| |
− | | |
− | '''backup_trac_svn.sh''' calls separate scripts to backup trac and svn repositories and place tgz files of the backups in /home/back/svn_backup and /home/back/trac_backup which are held for 14 days.
| |
− | '''rsync_back''' rsyncs www.osgeo.org/home/back with test.osgeo.net/home/back (the second PEER1 server)
| |
| | | |
| = DNS = | | = DNS = |
− | dns is administered via [https://www.pairnic.com/login.html PairNIC]
| |
| | | |
− | | + | See [[SAC DNS Registry]]. |
− | The zone file for the osgeo.org dns was mirrored from CN's zonefile. There is some work here to get rid of * (wildcard) entry in dns.
| |
− | @ IN A 66.223.95.242
| |
− | www IN A 66.223.95.242
| |
− | mapguide IN A 66.223.95.245
| |
− | find.geodata IN A 198.202.74.216
| |
− | mapbender2 IN A 66.223.95.245
| |
− | dev.geodata IN A 198.202.74.216
| |
− | wiki IN A 85.214.35.116
| |
− | svn IN A 66.223.95.242
| |
− | geodata IN A 66.223.95.242
| |
− | *.wiki IN A 85.214.35.116
| |
− | access.geodata IN A 198.202.74.216
| |
− | buildbot IN A 198.202.74.219
| |
− | fdo2 IN A 66.223.95.245
| |
− | www.communitymapbuilder IN A 63.246.7.187
| |
− | * IN A 66.223.95.242
| |
− | community IN A 66.223.95.242
| |
− | download IN A 198.202.74.219
| |
− | testbed IN A 140.211.166.123
| |
− | lists IN A 66.223.95.242
| |
− | mapguide2 IN A 66.223.95.245
| |
− | *.community IN A 140.211.166.123
| |
− | mapbender IN A 66.223.95.245
| |
− | mail IN A 66.223.95.242
| |
− | public.geodata IN A 198.202.74.216
| |
− | communitymapbuilder IN A 63.246.7.187
| |
− | fdo IN A 66.223.95.245
| |
| | | |
| = FDO Repository Merge = | | = FDO Repository Merge = |
Line 476: |
Line 190: |
| | | |
| $ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp | | $ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp |
| + | |
| + | [[Category:Infrastructure]] |
Description of current layout and installed software
Notes from Shawn on our PEER1 system:
- OS Version: Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
- For most elements of system i tried to stay with the default redhat locations and red hat el 4 rpms installed from peer 1's up2date repository. Reasoning, the servers are updated automatically against this repository and supported by PEER1 - reduce sysadmin load on keeping packages updated by using packages not available through PEER1 repository
Installed software using up2date
- apache - httpd-2.0.52-28.ent.i386
- postfix - postfix-2.2.10-1.RHEL4.2.i386
- php - php-4.3.9-3.22PIDH.i386
- python - python-2.3.4-14.3.i386
- mailman - mailman-2.1.5.1-34.rhel4.5.i386
Red Hat EL 4 rpms installed manually (rpm -i)
- MySQL-client-standard-5.0.27-0.rhel4.i386.rpm
- MySQL-server-standard-5.0.27-0.rhel4.i386.rpm
- MySQL-devel-standard-5.0.27-0.rhel4.i386.rpm
- MySQL-shared-compat-5.0.27-0.rhel4.i386.rpm
- clearsilver-0.10.1-1.2.el4.rf.i386.rpm
- sqlite-2.8.16-1.2.el4.rf.i386.rpm
- python-clearsilver-0.10.1-1.2.el4.rf.i386.rpm
- python-sqlite-1.0.1-12.el4.rf.i386.rpm
- subversion-1.4.3-0.1.el4.rf.i386.rpm
- mod_dav_svn-1.4.3-0.1.el4.rf.i386.rpm
Not specific to Red Hat EL 4 rpms installed manually
- MySQL-zrm-1.1.2-1.noarch.rpm
Source Install
- drupal-4.7.4.tar.gz
- phpldapadmin-0.9.8.3.tar.gz
- trac-0.10.3.tar.gz
Paths to services directories
Apache root directory
/var/www/html/
Subversion parent directory
/var/www/svn/repos/
Trac parent directory
/var/www/trac/
IP tables custom
All iptables rules must be written in /etc/sysconfig/iptables-custom
Current custom rules are:
/etc/sysconfig/iptables-custom/SSH
*filter
-A OUTPUT -p tcp --dport 22 -j ACCEPT
COMMIT
/etc/sysconfig/iptables-custom/VERITAS
note: this file was placed here by PEER1
*filter
-A INPUT -s 10.0.48.0/24 -d 0/0 -p tcp -j ACCEPT
-A OUTPUT -s 0/0 -d 10.0.48.0/24 -p tcp -j ACCEPT
COMMIT
System login and maintenance procedures
Note: if change root password must let PEER1 know the new password for backup/restore and tickets. i would refer no root ssh login but, PEER1 admins need root access.
Apache
As the system is Red Hat use /sbin/services httpd start|stop|restart|configtest|reload instead of /usr/sbin/apchectl
the main Config file for apache is,
/etc/httpd/conf/httpd.conf
additional config files are at
/etc/httpd/conf.d/
Postfix
Postfix documentation
The postfix daemon can be be stared|stopped|restarted|reloaded with,
$ sudo /sbin/service postfix start|stop|restart|reload|abort|flush|check|status|condrestart
The main postfix config file is,
/etc/postfix/main.cfg
The aliases file is,
/etc/aliases
If aliases file is edited remember to run newaliases
$ sudo /usr/bin/newaliases
$ sudo /sbin/service postfix reload
LDAP
See SAC:LDAP
Subversion
See Subversion for details on subversion configuration.
Trac
Trac Instances
Mailman Maintenance
See SAC:Mailing Lists.
Backups
See SAC:Backups.
DNS
See SAC DNS Registry.
FDO Repository Merge
To merge repositories you need to parse through the dumpfiles.
In the fdo merge all subprojects had the same directory strucuture,
/trunk/Providers/
All dumpfiles had to be 'filtered' to just pull the /trunk/Providers/* and drop branches and tags.
SVNDUMPFILTER
$ svndumpfilter include --help
Filter out nodes without given prefixes from dumpstream
Usage: svndumpfilter include PATH_PREFIX
Options:
--drop-empty-revs Remove revisions emptied by filtering
--renumber-revs Renumbe revisions left after filtering
--preserve-revprops Don't filter revision properties
--quiet Do not display filtering statistics
$ svndumpfilter exclude --help
Filter out nodes with given prefixes from dumpstream
Usage: svndumpfilter exclude PATH_PREFIX
Options:
--drop-empty-revs Remove revisions emptied by filtering
--renumber-revs Renumbe revisions left after filtering
--preserve-revprops Don't filter revision properties
--quiet Do not display filtering statistics
This process followed for all repositories
merge fdogdal trunk
$ cat fdogdal.dmp | svndumpfilter include trunk/www/ > fdogdal-merge-trunk.www.dmp
$ cat fdogdal.dmp | svndumpfilter include trunk/Providers/ > fdogdal-merge-trunk.Providers.GDAL.dmp
edit file and remove lines,
Node-path: trunk/
....
Node-action: add
....
PROPS-END
Node-path: trunk/www/
....
Node-action: add
....
PROPS-END
Node-path: trunk/Providers/
....
Node-action: add
....
PROPS-END
Rename index.html to fdogdal-index.html
$ perl -pi.bak -e 's/^Node-path:\ trunk\/www\/index.html/Node-path:\ trunk\/www\/fdogdal-index.html/g' fdo-merge.trunk.dmp
$ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp