Difference between revisions of "Download Server"
|Line 13:||Line 13:|
'''TODO: The rest of this page is obsolete and covers the old download.osgeo.osuosl.org
'''TODO: The rest of this page is obsolete and covers the old download.osgeo.osuosl.org . download.osgeo.org is now running under nginx as of May 2019'''
= Access Log Viewer (awstats) =
= Access Log Viewer (awstats) =
Revision as of 06:29, 15 September 2019
There is a need for a "download server" for OSGeo that provides for bulk downloads of software.
The download container on osgeo7, ssh via download.osgeo.org (requires SSH key) Refer to SAC_Service_Status Download Container
- It is known as "download.osgeo.org", and is configured to respond to that.
- The upload.osgeo.org DNS name also points to it.
- The downloadable tree is found in /osgeo/download on the server.
- /osgeo/download/download6 sub tree is nfs mounted from osgeo6 (currently osgeo4w and qgis in /osgeo/download point to a corresponding directory in download6) -- TODO: Is this still true?
- The virtual host declaration is found in /etc/nginx/sites-available
- Logs are in /var/log/nginx
TODO: The rest of this page is obsolete and covers the old download.osgeo.osuosl.org setup. download.osgeo.org is now running under nginx as of May 2019
Access Log Viewer (awstats)
To maintain/view the Apache access logs of download server, the debian awstats package has been installed on the server. with site configuration in /etc/awstats/awstats.download.osgeo.org.conf. It's updated every 4 hours by /etc/cron.d/awstats. See them now at 
It is intended that each interested project should have a directory under /osgeo/download with appropriate permissions so a project member can manage the subdirectories and files. Scp, or sftp can be used to bring files onto the server. Someone from each project will need their LDAP OSGeo Userid shell enabled, for ssh/scp access.
Any administrator can create new project directories, and chown them to a project representative.
Contact Frank Warmerdam for overall configuration questions or about problems.
End users should be referred to downloads similarly to:
Directory indexing is left on so requesting a directory will give a file list.
There is FTP service configured on the download.osgeo.org blade. The FTP access is served by vsftpd - secure and fastest FTP server for UNIX-like systems.
The vsftpd daemon configuration is the default one provided with Debian and is suitable for anonymous ftp only.
- root directory: /osgeo/download
- no access for local users
- no write, mkdir or upload privileges
Controlling FTP Service
In order to control the vsftpd server, you need to:
- belong to sudoers
- login in to the download.osgeo.org host using SSH client.
Here you can see how to issue two basic commands:
$ sudo /etc/init.d/vsftpd start
$ sudo /etc/init.d/vsftpd stop
$ sudo /etc/init.d/vsftpd restart
The /osgeo/download area is exported for anonymous rsync read access as a module named "download". The following can be used to pull a local copy of the gdal data tree for instance:
rsync -av download.osgeo.org::download/gdal/data .
This is controlled by the rsync configuration file at /etc/rsyncd.conf. It is possible that rsyncd will not automatically restart on reboot in which case "rsyncd --daemon" as root may be necessary to start it.
Some projects using Maven as their build tool require webdav in order to work. Here's what's required to get that setup:
- Enable relevant apache modules
sudo a2enmod dav sudo a2enmod dav_fs sudo a2enmod ldap sudo a2enmod authnz_ldap
- Add configuration for project specific location and LDAP group filtering
- Add configuration for apache ldap auth
In the vhost config (examples from svn server)
In the project.conf (example from tracsvn server
Nightly backups of the /osgeo/download tree are managed using rsync to /mirror/rsync/download.osgeo.org on backup.osgeo.org by root's crontab. /home/other_backups/download directory backup.osgeo.org by a nightly cronjob, and this mirror is available at the url download2.osgeo.org if needed (NOTE: that rsync does not delete file on backup.osgeo.org, if they were removed from download.osgeo.org).