Difference between revisions of "Migration Documentation"

From OSGeo
Jump to navigation Jump to search
Line 319: Line 319:
 
To edit an individual userid use ldap_user_edit.py. If you add ?userid=''osgeo_userid'' you can edit someone elses ldap entry as long as you are in the admins group.  
 
To edit an individual userid use ldap_user_edit.py. If you add ?userid=''osgeo_userid'' you can edit someone elses ldap entry as long as you are in the admins group.  
  
−
   https://www.osgeo.org/cgi-bin/auth/ldap_user_edit.py
+
   https://www.osgeo.org/cgi-bin/auth/ldap_edit_user.py
−
   https://www.osgeo.org/cgi-bin/auth/ldap_user_edit.py?userid=osgeotest123
+
   https://www.osgeo.org/cgi-bin/auth/ldap_edit_user.py?userid=osgeotest123
  
 
= Subversion =
 
= Subversion =

Revision as of 13:14, 2 February 2008

Description of current layout and installed software

Notes from Shawn on our PEER1 system:

  • OS Version: Red Hat Enterprise Linux ES release 4 (Nahant Update 4)
  • For most elements of system i tried to stay with the default redhat locations and red hat el 4 rpms installed from peer 1's up2date repository. Reasoning, the servers are updated automatically against this repository and supported by PEER1 - reduce sysadmin load on keeping packages updated by using packages not available through PEER1 repository

Installed software using up2date

  • apache - httpd-2.0.52-28.ent.i386
  • postfix - postfix-2.2.10-1.RHEL4.2.i386
  • php - php-4.3.9-3.22PIDH.i386
  • python - python-2.3.4-14.3.i386
  • mailman - mailman-2.1.5.1-34.rhel4.5.i386

Red Hat EL 4 rpms installed manually (rpm -i)

  • MySQL-client-standard-5.0.27-0.rhel4.i386.rpm
  • MySQL-server-standard-5.0.27-0.rhel4.i386.rpm
  • MySQL-devel-standard-5.0.27-0.rhel4.i386.rpm
  • MySQL-shared-compat-5.0.27-0.rhel4.i386.rpm
  • clearsilver-0.10.1-1.2.el4.rf.i386.rpm
  • sqlite-2.8.16-1.2.el4.rf.i386.rpm
  • python-clearsilver-0.10.1-1.2.el4.rf.i386.rpm
  • python-sqlite-1.0.1-12.el4.rf.i386.rpm
  • subversion-1.4.3-0.1.el4.rf.i386.rpm
  • mod_dav_svn-1.4.3-0.1.el4.rf.i386.rpm

Not specific to Red Hat EL 4 rpms installed manually

  • MySQL-zrm-1.1.2-1.noarch.rpm

Source Install

  • drupal-4.7.4.tar.gz
  • phpldapadmin-0.9.8.3.tar.gz
  • trac-0.10.3.tar.gz

Paths to services directories

Apache root directory 

   /var/www/html/

Subversion parent directory 

   /var/www/svn/repos/

Trac parent directory 

  /var/www/trac/

IP tables custom

All iptables rules must be written in /etc/sysconfig/iptables-custom

Current custom rules are:

/etc/sysconfig/iptables-custom/SSH 

  *filter
  -A OUTPUT -p tcp --dport 22 -j ACCEPT
  COMMIT

/etc/sysconfig/iptables-custom/VERITAS

note: this file was placed here by PEER1 

  *filter
  -A INPUT -s 10.0.48.0/24 -d 0/0 -p tcp -j ACCEPT
  -A OUTPUT -s 0/0 -d 10.0.48.0/24 -p tcp -j ACCEPT
  COMMIT

System login and maintenance procedures

Note: if change root password must let PEER1 know the new password for backup/restore and tickets. i would refer no root ssh login but, PEER1 admins need root access.

Apache

As the system is Red Hat use /sbin/services httpd start|stop|restart|configtest|reload instead of /usr/sbin/apchectl

the main Config file for apache is, /etc/httpd/conf/httpd.conf

additional config files are at /etc/httpd/conf.d/

Postfix

Postfix documentation

The postfix daemon can be be stared|stopped|restarted|reloaded with, 

$ sudo /sbin/service postfix start|stop|restart|reload|abort|flush|check|status|condrestart

The main postfix config file is, 

/etc/postfix/main.cfg

The aliases file is, 

/etc/aliases

If aliases file is edited remember to run newaliases 

 $ sudo /usr/bin/newaliases
 $ sudo /sbin/service postfix reload

LDAP

OpenLDAP

OpenLDAP Admin Guide

The ldap daemon can manipulated with, 

$ sudo /sbin/service ldap start|stop|restart|

The main ldap config file is, 

/etc/openldap/slapd.conf

Currently ldap structure is pretty basic. The purpose for keeping this structure simple is to allow for a more complex structure to be evolved as ldap becomes increasingly integrated into the full osgeo systems structure.

Loading an ldif file into an ldap directory 

  $ ldapadd -a -W -x -D "cn=Manager,dc=osgeo,dc=org" -f fileName.ldif

Accessing the ldap directory through phpldapadmin

access an ldapadmin interface at https://www.osgeo.org/ldapadmin

Access to this interface is restricted to the admin group

users can be added to the ou=people,dc=osgeo,dc=org once users are added they can then be added to cn=project,ou=svn,dc=osgeo,dc=org

login as cn=Manager,dc=osgeo,dc=org

  • expand dc=osgeo,dc=org tree
  • expand the ou=people tree
  • click on 'create new entry here'
  • choose 'custom' and then 'proceed'
    • RDN is in the form of uid=username
    • ObjectClass is InetOrgPerson
    • click proceed
      • cn is common name in form of firstname lastname
      • sn is surname - lastname
      • no optionals are required
    • click 'create object'
      • on this page click 'add new attribute'
        • choose mail
          • enter email address
          • click add
        • on this page click 'add new attribute'
          • choose userPassord
          • enter password as an md5
          • click add

this adds the new user to ou=people

to add users to the cn=project,ou=svn group

  • expand the ou=svn tree
  • click on cn=project
    • under member click 'add value'
    • add user to this in same form as you and howard are listed
    • or can click on the 'folder with magnifying glasss' to select a user

that's it.

ldap structure

  • dc=osgeo,dc=org
    • cn=Manager
    • ou=people
      • Separate entity for each user
  • uid=login,ou=people,dc=osgeo,dc=org
    objectClass=inetOrgPerson
    cn=firstName lastName
    sn=lastName
    uid=login
    mail=email@address
    userPassword={md5}YPTyViiMKhiuWKEmFUOKLA==
    • ou=projects,dc=osgeo,dc=org
    objectClass=organizationalUnit
    ou=project
    description=separate entity for each osgeo project with list of members
      • Separate entity for each project group
    cn=admin,ou=projects,dc=osgeo,dc=org
    objectClass=groupOfNames
    cn=admin
    description=osgeo sysadmin group
    • ou=svn
    objectClass=organizationalUnit
    ou=svn
    description=separate entity for each repository with list of members with commit rights
      • separate entity for each svn group
    cn=fdo,ou=svn,dc=osgeo,dc=org
    objectClass=groupOfNames
    cn=fdo
    member= dn of member
  • cn=gdal,ou=svn,dc=osgeo,dc=org
    objectClass=groupOfNames
    cn=gdal
    member= dn of member
  • cn=mapguide,ou=svn,dc=osgeo,dc=org
    objectClass=groupOfNames
    cn=mapguide
    member= dn of member
  • cn=mapbender,ou=svn,dc=osgeo,dc=org
    objectClass=groupOfNames
    cn=mapbender
    member= dn of member


Example ldif file 

<code>
   version: 1

   dn: dc=osgeo,dc=org
   objectClass: dcObject
   objectClass: organization
   description: OSGeo ldap dit
   o: OSGeo
   dc: osgeo

   dn: cn=Manager,dc=osgeo,dc=org
   objectClass: organizationalRole
   cn: Manager

   dn: ou=people,dc=osgeo,dc=org
   ou: people
   description: all users of osgeo
   objectClass: organizationalUnit

   dn: uid=jsmith,ou=people,dc=osgeo,dc=org
   objectClass: inetOrgPerson
   uid: jsmith
   cn: Jon Smith
   sn: Smith 
   givenName: Jon
   mail: jsmith@somewhere.com
   userPassword: {md5}5Or4zfzGqo3jh/6iIUgKcA==

   dn: uid=jbrown,ou=people,dc=osgeo,dc=org
   objectClass: inetOrgPerson
   uid: jbrown
   cn: Jane Brown
   sn: Brown
   givenName: Jane 
   mail: jbrown@someotherplace.com
   userPassword: {md5}1iWhTyvkK2m4Uuar+Dp/IA==

   dn: ou=projects,dc=osgeo,dc=org
   ou: projects
   description: separate entity for each osgeo project with list of members
   objectClass: organizationalUnit

   dn: cn=admin,ou=projects,dc=osgeo,dc=org
   cn: admin
   description: osgeo sysadmin group
   objectClass: groupOfNames
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: ou=svn,dc=osgeo,dc=org
   ou: svn
   description: separate entity for for each repository.list of members with commit rights
   objectClass: organizationalUnit

   dn: cn=fdo,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: fdo
   member: uid=jsmith,ou=people,dc=osgeo,dc=org
   member: uid=jbrown,ou=people,dc=osgeo,dc=org

   dn: cn=gdal,ou=svn,dc=osgeo,dc=org
   cn: gdal
   objectClass: groupOfNames
   objectClass: top
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: cn=mapbender,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: mapbender
   member: uid=jsmith,ou=people,dc=osgeo,dc=org

   dn: cn=mapguide,ou=svn,dc=osgeo,dc=org
   objectClass: groupOfNames
   cn: mapguide
   member: uid=jbrown,ou=people,dc=osgeo,dc=org
</code>

LDAP Tools

Some public notes on OSGeo userids available at: 

 http://www.osgeo.org/osgeo_userid

Administrators can login to this special ldap search tool, and will see email address, and will have a link to edit the LDAP entries. You need to be in the cn=admin,ou=projects listing. 

 https://www.osgeo.org/cgi-bin/auth/ldap_web_search.py

SVN and other groups can be administered with the group editor. You need to either be in the group being viewed/modified or in the cn=admin,ou=projects group in order to edit a group.

Edit GDAL commit list: 

 https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=gdal

Edit Admins list: 

 https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=admin&ou=projects

To edit an individual userid use ldap_user_edit.py. If you add ?userid=osgeo_userid you can edit someone elses ldap entry as long as you are in the admins group. 

 https://www.osgeo.org/cgi-bin/auth/ldap_edit_user.py
 https://www.osgeo.org/cgi-bin/auth/ldap_edit_user.py?userid=osgeotest123

Subversion

See Subversion for details on subversion configuration.

Trac

Trac Instances

Mailman Maintenance

See SAC:Mailing Lists.

Backups

See SAC:Backups.

DNS

See SAC DNS Registry.

FDO Repository Merge

To merge repositories you need to parse through the dumpfiles.

In the fdo merge all subprojects had the same directory strucuture, /trunk/Providers/ All dumpfiles had to be 'filtered' to just pull the /trunk/Providers/* and drop branches and tags.


SVNDUMPFILTER

$ svndumpfilter include --help 

Filter out nodes without given prefixes from dumpstream

Usage: svndumpfilter include PATH_PREFIX Options: --drop-empty-revs Remove revisions emptied by filtering --renumber-revs Renumbe revisions left after filtering --preserve-revprops Don't filter revision properties --quiet Do not display filtering statistics

$ svndumpfilter exclude --help Filter out nodes with given prefixes from dumpstream Usage: svndumpfilter exclude PATH_PREFIX Options: --drop-empty-revs Remove revisions emptied by filtering --renumber-revs Renumbe revisions left after filtering --preserve-revprops Don't filter revision properties --quiet Do not display filtering statistics


This process followed for all repositories

merge fdogdal trunk 

 $ cat fdogdal.dmp | svndumpfilter include trunk/www/ > fdogdal-merge-trunk.www.dmp
 $ cat fdogdal.dmp | svndumpfilter include trunk/Providers/ > fdogdal-merge-trunk.Providers.GDAL.dmp

edit file and remove lines, 

  Node-path: trunk/
  ....
  Node-action: add
  ....
  PROPS-END

  Node-path: trunk/www/
  ....
  Node-action: add
  ....
  PROPS-END

  Node-path: trunk/Providers/
  ....
  Node-action: add
  ....
  PROPS-END

Rename index.html to fdogdal-index.html 

 $ perl -pi.bak -e 's/^Node-path:\ trunk\/www\/index.html/Node-path:\ trunk\/www\/fdogdal-index.html/g' fdo-merge.trunk.dmp

 $ svnadmin load /var/www/svn/repos/fdocore < fdogdal-merge.trunk.dmp
Retrieved from "http://wiki.osgeo.org/w/index.php?title=Migration_Documentation&oldid=22352"