Subversion

From OSGeo
Jump to navigation Jump to search

Intro

OSGeo maintains a set of subversion services, currently hosted on the primary Peer1 server at the virtual host http://svn.osgeo.org/. Subversion access is authenticated based on OSGeo Userids and each project has a committer list also maintained in LDAP. Excellent general information on subversion is available from the Subversion Book.

Current Repositories

Administration

Howard Butler (hobu), and Shawn Barnes (sbarnes) are the primary contacts for Subversion administration issues. Requests can be submitted via the SAC Trac.

Add/Remove Committer

Any member of a subversion committer group can add and remove members to the commiter group at an url of the following form, with the project name substituted for gdal. OSGeo Userids has details on finding or creating OSGeo userids. 

 https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=gdal

Creating a repository

Create the repository on the main server using the create_svn_repo.sh script on tracsvn.osgeo.org 

  $ cd /osgeo/tools
  $ sudo ./create_svn_repo.sh project_name

Create the authorization group for the svn repository by using the tool 

  /osgeo/tools/create_ldap_group.py group_name admin_username

This will prompt you for the manager password.

Then restart apache gracefully: 

 $ sudo /etc/init.d/apache2 reload

Notify hobu/mloskot about adding a mirror of the newly created repository to http://svnmirror.osgeo.org

Loading a repository from a dumpfile

By root on main server: 

  $ sudo svnadmin load /var/www/svn/repos/<repo_name> < /path/to/dumpfile
  $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>

Dumping a repository

By root on main server: 

  $ sudo svnadmin dump /var/www/svn/repos/<repo_name> > dumpfile

Recovering / unlocking repository

  • subversion may lock if user ctl-c during checkout or checkout is interupted with an apache restart
  • Need policy on who to contact and who can run 'svnadmin recover' as

priviledged access is needed (may need to stop/start apache to drop requests to repository before recover)

Subversion was upgraded to version 1.4.3 (30 Jan. 2007) to reduce the repository locking problem if ctl-c used to end a checkout.

If the repository locks the following command should be used to recover the repository: 

  $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
  $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>

In most cases this will work in the odd case that you are waiting a very long time for the command to run then apache may have to be restarted to drop anything accessing the repository and preventing the repository from being recovered 

  $ sudo /sbin/service httpd restart
  $ sudo svnadmin --wait recover /var/www/svn/repos/<repo_name>
  $ sudo chown -R apache:apache /var/www/svn/repos/<repo_name>

Authz Rules

By default we use one LDAP "group" for an entire subversion repository. Everyone in that group has commit priveledges on the whole repository. However, some project desire to have subgroups. For instance a core commiter group with update on the trunk source tree, and a broader contributor group that has commit access in a subarea, such as /sandbox. We accomplish this using an "authz" file to hold the rules for the respository, and multiple LDAP groups.

A project wishing this will need to prepare an authz template file looking something like: 

[groups]
committers = @@gdal@@
sandbox = @@gdal_sandbox@@

[/]
* = r
@committers = rw

[/sandbox]
* = r
@committers = rw
@sandbox = rw

This file should be submitted to SAC via a Trac ticket for placement on the server, and for creation of any corresponding LDAP groups. Thereafter any successful update via https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=project will result in the authz being generated from the corresponding authz.tmpl.

Administration Notes

The authz.tmpl files need to be named exactly authz.tmpl and place in the corresponding project repository directory, /var/www/svn/repos/gdal for instance. Make sure the file is owned by www-date.www-data.

The corresponding /etc/httpd/conf.d/subversion/project.conf file should look something like the following. Note the removal of <LimitExcept> stuff, and reference to any specific ldap group as well as the AuthzSVNAccessFile. 

    <Location /gdal>
      DAV svn
      SVNPath /var/www/svn/repos/gdal
      AuthzSVNAccessFile /var/www/svn/repos/gdal/authz
      Include conf.d/ldap_auth_url.inc
      Satisfy Any
      Require valid-user
    </Location>

The LDAP groups should be created as groupOfNames nodes below the ou=svn element using PHPLDAPAdmin. Make sure the membership is pre-initialized with an administrator for the project so they can do future updates themselves. Try to keep the names of the groups clearly related to the base project. eg. gdal, gdal_sandbox, gdal_documentors.

Notes

The script /var/www/svn/site-local/ldap_update_authz.py script is run by an os.system() call out of ldap_group.py on any successful group update. It will scan for /var/www/svn/repos/*/authz.tmpl.

There are lots of things that can go wrong in the script which has poor error checking, and it might only manifest as some or all of the authz file no longer getting regenerated. So when an authz.tmpl file is put in place, or updated do a test update and verify that all authz files are being updated (check timestamps).

Anyone (on osgeo1) can run ldap_update_authz.py, but if authz files are created by someone that apache.apache can't delete, problems may ensue. If this occurs, the following as root may help. 

 cd /var/www/svn
 chown apache.apache */*/authz*

SVN Mirror

The OSGeo SVN mirror is maintained at http://svnmirror.osgeo.org This machine is located at TelaScience, but it is not a blade and not part of the normal cadre of machines that OSGeo typically uses of TelaScience's. The mirror is Subversion 1.5.6 svnsync-based. When new repositories are created, a new mirror repository must also be created if it is wished that it is backed up, this is normally done by Howard Butler. Nothing prevents individual users or projects from maintaining their own backups with svnsync.

The mirror itself is located in /state/partition1/osgeo_svn_mirror There is a in /etc/cron.hourly/svnmirror that runs the syncs.sh script in /state/partition1/osgeo_svn_mirror to do the repository synchronization.

To create a new mirror:

1) Make a new .conf file in /etc/httpd/conf.d/subversion/ for the repository (use an existing one as a template)

2) Initialize the repo 

  svnadmin init /state/partition1/osgeo_svn_mirror/repos/newrepo

3) restart apache 

  sudo /etc/init.d/httpd restart

4) Copy the pre-revprop-change hook script from an existing repo into the new repo 

  sudo cp /state/partition1/osgeo_svn_mirror/repos/gdal/hooks/pre-revprop-change /state/partition1/osgeo_svn_mirror/repos/newrepo/hooks/pre-revprop-change

5) init the svnsync (password is empty \'\' without the backslashes) 

  svnsync init --username svnsync --password  http://svnmirror.osgeo.org/newrepo http://svn.osgeo.org/newrepo

6) Add it to the repositories list in /state/partition1/osgeo_svn_mirror/syncs.sh

7) Run the syncs.sh script 

  sudo ./syncs.sh
Retrieved from "http://wiki.osgeo.org/w/index.php?title=Subversion&oldid=55966"