Revision as of 00:20, 28 September 2011

There is a need for a "download server" for OSGeo that provides for bulk downloads of software.

Configuration

The OSU OSL "Download" VM is being used (140.211.15.132).

It is known as "download.osgeo.org", and is configured to respond to that.
It is also known as "ftp.remotesensing.org", for historical reasons.
The upload.osgeo.org DNS name also points to it.
The downloadable tree is found in /osgeo/download on the server.
The virtual host declaration is found in /etc/apache2/sites-available/download2.osgeo.org.conf.
Logs are in /var/log/apache2/download_access_log and /var/log/apache2/download_error_log.

Access Log Viewer (awstats)

awstats has not been reenabled yet on the new VM

To maintain/view the Apache access logs of download server, the debian awstats package has been installed on the server. with site configuration in /etc/awstats/awstats.download.osgeo.org.conf. It's updated every 4 hours by /etc/cron.d/awstats. See them now at [1]

Management

It is intended that each interested project should have a directory under /osgeo/download with appropriate permissions so a project member can manage the subdirectories and files. Scp, or sftp can be used to bring files onto the server. Someone from each project will need their LDAP OSGeo Userid shell enabled, for ssh/scp access.

Any administrator can create new project directories, and chown them to a project representative.

Contact Frank Warmerdam for overall configuration questions or about problems.

End Users

End users should be referred to downloads similarly to:

http://download.osgeo.org/gdal/gdal-1.3.2.tar.gz

Directory indexing is left on so requesting a directory will give a file list.

FTP Service

The ftp service is not yet re-enabled.

There is FTP service configured on the download.osgeo.org blade. The FTP access is served by vsftpd - secure and fastest FTP server for UNIX-like systems.

The vsftpd daemon configuration is the default one provided with Debian and is suitable for anonymous ftp only.

root directory: /osgeo/download
no access for local users
no write, mkdir or upload privileges

Controlling FTP Service

In order to control the vsftpd server, you need to:

belong to sudoers
login in to the download.osgeo.org host using SSH client.

Here you can see how to issue two basic commands:

Start

$ sudo /etc/init.d/vsftpd start

Stop

$ sudo /etc/init.d/vsftpd stop

Restart

$ sudo /etc/init.d/vsftpd restart

Rsync

Rsync support is not yet re-enabled

The /osgeo/download area is exported for anonymous rsync read access as a module named "download". The following can be used to pull a local copy of the gdal data tree for instance:

rsync -av download.osgeo.org::download/gdal/data .

This is controlled by the rsync configuration file at /etc/rsyncd.conf. It is possible that rsyncd will not automatically restart on reboot in which case "rsyncd --daemon" as root may be necessary to start it.

Backup

This is not yet re-enabled.

Nightly backups of the /osgeo/download tree are managed using rsync to the /osgeo/backup/rsync/download.osgeo.org /home/other_backups/download directory backup.osgeo.org by a nightly cronjob, and this mirror is available at the url download2.osgeo.org if needed.

There is also an rsync to iweb.gdal.org (Frank's personal server in Montreal) in case of extreme emergency, and this is found at iweb.gdal.org/download.

@@ Line 1: / Line 1: @@
-'''Status: The old download server has failed and not been restored.  It is being re-established at OSU OSL.  Details in: http://trac.osgeo.org/osgeo/ticket/798'''
+There is a need for a "download server" for OSGeo that provides for bulk downloads of software.
-There is a need for a "download server" for OSGeo that provides for bulk downloads of software.
-We are hesitant to handle this on our primary server hosted by Peer1 because of the risk of exceeding our bandwidth allotment and getting charged a lot extra.  For that reason the download server is being hosted on a telascience blade.  Telascience has extensive bandwidth and is not charging OSGeo for it.
 = Configuration =
-For now the 198.202.74.218 blade is being used as a download server.
+The OSU OSL "Download" VM is being used (140.211.15.132).
 * It is known as "download.osgeo.org", and is configured to respond to that.
@@ Line 14: / Line 9: @@
 * The upload.osgeo.org DNS name also points to it.
 * The downloadable tree is found in /osgeo/download on the server.
-* The virtual host declaration is found in /etc/apache2/sites-available/download.osgeo.org.conf.
+* The virtual host declaration is found in /etc/apache2/sites-available/download2.osgeo.org.conf.
-* Logs are in /var/log/apache2/download_access.log and /var/log/apache2/download_error.log.
+* Logs are in /var/log/apache2/download_access_log and /var/log/apache2/download_error_log.
 = Access Log Viewer (awstats) =
+'''awstats has not been reenabled yet on the new VM'''
 To maintain/view the Apache access logs of download server, the debian [http://awstats.sf.net/ awstats] package has been installed on the server.
@@ Line 39: / Line 36: @@
 = FTP Service =
+'''The ftp service is not yet re-enabled.'''
 There is FTP service configured on the '''download.osgeo.org''' blade. The FTP access is served by [http://vsftpd.beasts.org/ vsftpd] - secure and fastest FTP server for UNIX-like systems.
@@ Line 69: / Line 68: @@
 = Rsync =
+'''Rsync support is not yet re-enabled'''
 The /osgeo/download area is exported for anonymous rsync read access as a module named "download".  The following can be used to pull a local copy of the gdal data tree for instance:
@@ Line 77: / Line 78: @@
 = Backup =
+'''This is not yet re-enabled.'''
 Nightly backups of the /osgeo/download tree are managed using rsync to the /osgeo/backup/rsync/download.osgeo.org

Difference between revisions of "Download Server"