Subversion

Intro

OSGeo maintains a set of subversion services, running at http://svn.osgeo.org/ (the tracsvn VM at OSU OSL).

Subversion access is authenticated based on OSGeo Userids and each project has a committer list also maintained in LDAP. Excellent general information on subversion is available from the Subversion Book.

Current Repositories

• Full list: http://svn.osgeo.org/
• http://svn.osgeo.org/fdo - FDO (also a bunch of old repositories exist for FDO)
• http://svn.osgeo.org/gdal - GDAL (now at https://github.com/osgeo/gdal)
• http://svn.osgeo.org/geotools - GeoTools ( now at https://github.com/geotools/geotools )
• http://svn.osgeo.org/grass - GRASS GIS
• http://svn.osgeo.org/mapbender - Mapbender
• http://svn.osgeo.org/mapguide - MapGuide
• http://svn.osgeo.org/moss4g - MOSS4g
• http://svn.osgeo.org/osgeo - OSGeo
• http://svn.osgeo.org/ossim - OSSIM
• http://svn.osgeo.org/geos - GEOS
• http://svn.osgeo.org/mapserver - MapServer (now at https://github.com/mapserver/mapserver )
• http://svn.osgeo.org/qgis - QGIS (now at https://github.com/qgis/QGIS )
• http://svn.osgeo.org/fusion - Fusion
• http://svn.osgeo.org/osgeo4w - OSGeo4W
• http://svn.osgeo.org/postgis - PostGIS
• http://svn.osgeo.org/metacrs - MetaCRS
• http://svn.osgeo.org/geoinformatica/ - Geoinformatica

Administration

Frank Warmerdam (warmerdam), Howard Butler (hobu), and Chris Schmidt (crschmidt) are the primary contacts for Subversion administration issues. Requests can be submitted via the SAC Trac.

Add/Remove Committer

Any member of a subversion committer group can add and remove members to the commiter group at an url of the following form, with the project name substituted for gdal. OSGeo Userids has details on finding or creating OSGeo userids.

 https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=gdal

Creating a repository

Create the repository on the main server using the create_svn_repo.sh script on tracsvn.osgeo.org

  $ cd /osgeo/tools
  $ sudo ./create_svn_repo.sh project_name

Create the authorization group for the svn repository by using the tool

  /osgeo/tools/create_ldap_group.py group_name admin_username

This will prompt you for the manager password.

Then restart apache gracefully:

 $ sudo /etc/init.d/apache2 reload

Notify hobu/mloskot about adding a mirror of the newly created repository to http://svnmirror.osgeo.org

Loading a repository from a dumpfile

By root on main server:

  $ sudo svnadmin load /var/www/svn/repos/<repo_name> < /path/to/dumpfile
  $ sudo chown -R www-data:www-data /var/www/svn/repos/<repo_name>

Dumping a repository

By root on main server:

  $ sudo svnadmin dump /var/www/svn/repos/<repo_name> > dumpfile

Recovering / unlocking repository

• subversion may lock if user ctl-c during checkout or checkout is interupted with an apache restart
• Need policy on who to contact and who can run 'svnadmin recover' as

priviledged access is needed (may need to stop/start apache to drop requests to repository before recover)

Subversion was upgraded to version 1.4.3 (30 Jan. 2007) to reduce the repository locking problem if ctl-c used to end a checkout.

If the repository locks the following command should be used to recover the repository:

  $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
  $ sudo chown -R www-data:www-data /var/www/svn/repos/<repo_name>

In most cases this will work in the odd case that you are waiting a very long time for the command to run then apache may have to be restarted to drop anything accessing the repository and preventing the repository from being recovered

  $ sudo /sbin/service httpd restart
  $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
  $ sudo chown -R www-data:www-data /var/www/svn/repos/<repo_name>

Authz Rules

By default we use one LDAP "group" for an entire subversion repository. Everyone in that group has commit priveledges on the whole repository. However, some project desire to have subgroups. For instance a core commiter group with update on the trunk source tree, and a broader contributor group that has commit access in a subarea, such as /sandbox. We accomplish this using an "authz" file to hold the rules for the respository, and multiple LDAP groups.

A project wishing this will need to prepare an authz template file looking something like:

[groups]
committers = @@gdal@@
sandbox = @@gdal_sandbox@@

[/]
* = r
@committers = rw

[/sandbox]
* = r
@committers = rw
@sandbox = rw

This file should be submitted to SAC via a Trac ticket for placement on the server, and for creation of any corresponding LDAP groups. Thereafter any successful update via https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=project will result in the authz being generated from the corresponding authz.tmpl.

Administration Notes

The authz.tmpl files need to be named exactly authz.tmpl and place in the corresponding project repository directory, /var/www/svn/repos/gdal for instance. Make sure the file is owned by www-date.www-data.

The corresponding /etc/httpd/conf.d/subversion/project.conf file should look something like the following. Note the removal of <LimitExcept> stuff, and reference to any specific ldap group as well as the AuthzSVNAccessFile.

    <Location /gdal>
      DAV svn
      SVNPath /var/www/svn/repos/gdal
      AuthzSVNAccessFile /var/www/svn/repos/gdal/authz
      Include conf.d/ldap_auth_url.inc
      Satisfy Any
      Require valid-user
    </Location>

The LDAP groups should be created as groupOfNames nodes below the ou=svn element using PHPLDAPAdmin. Make sure the membership is pre-initialized with an administrator for the project so they can do future updates themselves. Try to keep the names of the groups clearly related to the base project. eg. gdal, gdal_sandbox, gdal_documentors.

Notes

The script /var/www/svn/site-local/ldap_update_authz.py script is run by an os.system() call out of ldap_group.py on any successful group update. It will scan for /var/www/svn/repos/*/authz.tmpl.

There are lots of things that can go wrong in the script which has poor error checking, and it might only manifest as some or all of the authz file no longer getting regenerated. So when an authz.tmpl file is put in place, or updated do a test update and verify that all authz files are being updated (check timestamps).

Anyone (on osgeo1) can run ldap_update_authz.py, but if authz files are created by someone that apache.apache can't delete, problems may ensue. If this occurs, the following as root may help.

 cd /var/www/svn
 chown www-data.www-data */*/authz*

SVN Mirror

OSGeo uses "svnsync" to mirror SVN repositories to ensure we have a live SVN backup with full history. Currently a mirror is maintained at Frank's system (iweb.gdal.org) but it is also possible for others to setup their own mirrors. To mirror all repositories you can do the following steps:

 % mkdir osgeo-svn-mirror
 % cd osgeo-svn-mirror
 % wget http://svn.osgeo.org/osgeo-svn-mirror.sh
 % chmod a+x osgeo-svn-mirror.sh
 % ./osgeo-svn-mirror.sh

The above will run for a long time creating and mirroring repos in osgeo-svn-mirror/repos. To keep them up to date setup a cronjob to run the osgeo-svn-mirror.sh script regularly.

Currently the mirrored svn at iweb can be examined, but not updated using urls of the form:

 svn://iweb.gdal.org/home/warmerda/osgeo-svn-mirror/repos/<project>

So for instance, gdal could be extracted like:

 svn checkout svn://localhost/home/warmerda/osgeo-svn-mirror/repos/gdal/trunk/gdal gdal-local

TODO: add details on how to setup apache so these repositories will just work.

TODO: make sure that svnmirror.osgeo.org points to a live svn mirror, possibly at iweb.

Note: SVN mirrors do *not* capture special per-repository configurations like authz files.

See also: svnsync.txt - useful general info on svnsync.