There is a need for a "download server" for OSGeo that provides for bulk downloads of software.

We are hesitant to handle this on our primary server hosted by Peer1 because of the risk of exceeding our bandwidth allotment and getting charged a lot extra. For that reason the download server is being hosted on a telascience blade. Telascience has extensive bandwidth and is not charging OSGeo for it.

Configuration

For now the 198.202.74.218 blade is being used as a download server.

It is known as "download.osgeo.org", and is configured to respond to that.
It is also known as "ftp.remotesensing.org", for historical reasons (actually this is still running on .219).
The downloadable tree is found in /osgeo/download on the server.
/osgeo/download is also cross mounted on the .219 server which is where a variety of project scripts run for generating nightly tar files, and such.
The virtual host declaration is found in /etc/httpd/conf.d/sites/download.osgeo.org.conf.
Logs are in /etc/httpd/logs/download_access_log and /etc/httpd/logs/download_error_log.

Access Log Viewer (awstats)

To maintain/view the Apache access logs of download server, awstats has been installed on the server. Logs can be seen by visiting http://download.osgeo.org/logs/ The details of awstats installation are

awstats version 6.6-1 installed using rpm
application root dir /usr/local/awstats/
location of cgi executable /usr/local/awstats/wwwroot/cgi-bin/awstats.pl
site configuration file /etc/awstats/awstats.www.download.osgeo.org.conf (create such file for any other host to be added in future)
Data dir (dir where stats data is stored) /osgeo/download/logs
Updation is done daily 1am (server time) using a cronjob by executing /osgeo/scripts/update_logs.sh (to add more sites in future, add entry to this script)

Management

It is intended that each interested project should have a directory under /osgeo/download with appropriate permissions so a project member can manage the subdirectories and files. Scp, or sftp can be used to bring files onto the server. Someone from each project will need a shell account on the server.

Any administrator can create new project directories, and chown them to a project representative.

Contact Frank Warmerdam for overall configuration questions or about problems.

End Users

End users should be referred to downloads similarly to:

http://download.osgeo.org/gdal/gdal-1.3.2.tar.gz

Directory indexing is left on so requesting a directory will give a file list.

FTP Service

There is FTP service configured on the download.osgeo.org blade. The FTP access is served by vsftpd - secure and fastest FTP server for UNIX-like systems.

NOTE: ~~ftp service still handle by .219 server off the NFS mounted /osgeo/download tree. Migration to download.osgeo.org addressed in http://trac.osgeo.org/osgeo/ticket/230~~

The vsftpd daemon configuration includes as minimal set of options turned on as possible:

anonymous only access
root directory: /osgeo/download
no access for local users
no write, mkdir or upload privileges
session owned by non-privileged user (there has been created a dedicated local user)
custom banner message: Welcome to OSGeo Foundation FTP service.
maximum number of connected clients: 50
maximum number of connections per IP address: 2
user and group identifiers in are hidden from directories listings, and replaced with ftp

Controlling FTP Service

In order to control the vsftpd server, you need to:

belong to sudoers
login in to the download.osgeo.org host using SSH client.

Here you can see how to issue two basic commands:

Start

$ sudo /etc/init.d/vsftpd start

Stop

$ sudo /etc/init.d/vsftpd stop

Restart

$ sudo /etc/init.d/vsftpd restart

Rsync

The /osgeo/download area is exported for anonymous rsync read access as a module named "download". The following can be used to pull a local copy of the gdal data tree for instance:

rsync -av download.osgeo.org::download/gdal/data .

This is controlled by the rsync configuration file at /etc/rsyncd.conf. /etc/xinetd.d/rsync was also updated to set set disable=yes to disable=no in the hopes the daemon will restart when the blade is restarted (not tested).

Backup

Nightly backups of the /osgeo/download tree are managed using rsync to the /home/other_backups/download directory on the secondary osgeo server (test.osgeo.net). This is done by a cron job running as user back on osgeo2 (see /home/other_backups for details).

Issues

There is only 10GB of free disk space on this blade, and in general the telascience blades have rather limited disk space, so these systems offer limited scalability for the amount of software available.

Download Server

Contents