Live GIS Add Project

What gets into OSGeoLive?

Priority for selecting OSGeoLive projects to package is:

1. OSGeo Projects
2. OSGeo Projects in incubation
3. Stable GeoSpatial Open Source Software being presented in workshops, tutorials, or presentations at FOSS4G
4. Other Stable GeoSpatial Open Source Software with establish project communities

Which version to use?

• Stable, always! LiveGIS users are usually seeing OSGeo newbies and will be turned off by any software bugs.

Project Champion

Each project needs at least one champion who will liaise between the OSGeoLive team and the project. The champion is responsible for ensuring someone updates scripts and documentation to the appropriate version for each release, and ensures the application is tested.

Getting started

• Recommended: Subscribe to the Live Demo email list to discuss issues.

• Optional: Install the latest Arramamgong / GIS virtual machine as per: Live GIS Disc Quick Start.
• Write a shell script which installs and configures your stable package into the current Live GIS virtual machine (which will usually be the same as installing on Xubuntu or Ubuntu).

All install scripts for building the LiveGIS disc shall be licensed as LGPL.

• Add the script to subversion, in https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/bin/ , as per http://wiki.osgeo.org/wiki/Live_GIS_Disc#Subversion

If you are unfamiliar with subversion then you can send the script to the OSGeo live-demo mailing list as an attachment and we'll see that it makes it in.

• Notify the LiveGIS team to test your script, and will ensure to reference your script from main.sh.

The first think you will want to do is check out a copy of the existing build scripts using Subversion:

svn co https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk osgeo_live/trunk/

Example scripts

You can view example scripts at:

https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/bin/

The script may be as simple as: install_mapserver.sh, which just

apt-get install mapserver

Projects that haven't been packaged for Ubuntu yet are slightly more complicated:

The uDig package is a good example: install_udig.sh

• 3rd Party Repositories: Create an <app>.list file with your repository in sources.list.d/ in the svn, in your script copy the file to the local machine like

cp -f ../sources.list/ubuntugis.list /etc/apt/sources.list.d/

don't forget to add the repository key like so

apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 68436DDF

Packaging conventions

• sudo: Assume the script is to be executed as root. This means that sudo is only needed if you need to execute a command as e.g. the database user.

This means that ~/ refers to the directory /home/root/, so you should use ~user/ to reference /home/user/.

Preferred method is to put the USER_NAME and USER_HOME variables at the top, so these can be easily changed in all scripts if needed.

USER_NAME="user"
USER_HOME="/home/$USER_NAME"
cp package_readme.txt "$USER_HOME"

• Called twice: Assume that the script will be executed multiple times, (as people may re-run the install process to get it right).

Make sure that nothing is corrupted if the script runs a second time.

• No Prompts: Try not to have any interactive steps which prompt the user for an answer, as we want the scripts to be run automatically. In particular, try running the script a second time, and see if you get prompted to overwrite files.

• /tmp: Copy all downloads into /tmp/build_<package>/

Don't delete the /tmp/build_<package> directory afterwards. If the script is re-run, then time and bandwidth is saved by not having to download large files again.

• Icons: Where appropriate, create an icon on the desktop which loads the application. The file should be called <packagename>.desktop and placed in /home/user/Desktop/ for further processing. The icons will be sorted and the Geospatial menu will be auto-generated from here.

Install <packagename>.png icon files to the /usr/share/icons/ directory.

see install_qgis.sh for an example

• Browser bookmarks: If the application is launched from the browser, then include the application in Firefox bookmarks.

(Todo: How do we do this?)

• wget: When downloading large files using wget, use the "-c" continue command, as it will not re-download files if they already exist, and will continue if one is partially downloaded:

wget -c package.tar.gz

Do not use -c if there is a chance the file of that name will change on the server. Add "--progress=dot:mega" to the wget command line if the file is at all large.

Tiny files should use wget's -nv (non-verbose) flag.

• mkdir: Use -p (create parents too) option when creating a directory, as it doesn't complain about creating the directory a second time.

mkdir -p <dir>

• Binary files: Do not put large (>5mb) binary files into the SVN repository without asking first. Due to SVN's preservation of history, once they are added they can never be removed, and forever-more clog up the backend database. Subversion is designed to be a source code management tool, not a data warehouse. There is space for large static data files (e.g. sample data) at http://download.osgeo.org/livedvd/data/. Just ask on the mailing list if you'd like to house something there.

.deb Packaging

While many packages are installed from a bash script, ideally, applications should be packaged in UbuntuGIS or DebianGIS repositories, or in a Launchpad PPA (e.g. available from the UbuntuGIS repository). Next would be packaged into .debs from your own Ubuntu repository. An official Debian package could be used, but we'd have to rebuild it from source (this takes time but is fairly easy; usually these are already sucked in Ubuntu automatically so it isn't an issue).

If you must build from source, building a .deb package and installing that is preferred to "make install". If you do need to "make install", please set the path prefix to /usr/local/ and ensure that your package does not introduce namespace conflicts.

In practice a number of scripts employ a hybrid approach, with as much as possible installed via official packages, and the occasional plugin compiled from source.

A base set of build-tools are already included, but if you install a bunch of -dev packages to build your app, make sure that you clean them all up after you are done.

Writing docs

Definition document

Each project is to have a distinct one or two sentences defining the project, as a html snippet inside a <li>tag</li>. There is a template here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/template_definition.html.

The debian package title should provide a suitable description for .deb packaged projects. Provided by:

dpkg -s $PACKAGE | grep '^Description:' | cut -f2- -d' '

Save the file here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/descriptions/<project>_definition.html

These files will be later assembled into the master disc contents help page.

Description document

(how best to do this is the subject of ongoing discussion, check on the mailing list)

Each project is to also have up to one page description about how to use the project on the live DVD, saved in the .odt format (which can be edited with Open Office or AbiWord). See example here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/template_description.odt

Once complete, save your docs into <project>_description.odt, and email the Live-demo list to ensure it is reviewed by the community. This file will be converted to html format as part of the build process.

You may include links to locally installed or remote pdf or html manuals, but keep in mind that there is a good chance that the user will not have internet access.

Save the file here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/descriptions/project_definition.odt

Licence document

Each project should create one table row for each licence offered by the project. A template for the file to be produced is here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/template_licence.html

Save the file here: https://svn.osgeo.org/osgeo/livedvd/gisvm/trunk/doc/descriptions/project_licence.html

Please also include the software license info in the package_description file.

Add to the master project list

Once your application is added in SVN please make sure the information about it is up to date on the status table at: http://spreadsheets.google.com/ccc?key=0Al9zh8DjmU_RdGIzd0VLLTBpQVJuNVlHMlBWSDhKLXc&hl=en_GB

Testing procedure

Provide test steps for your application as per the Live GIS Disc Testing wiki page.

Once the application has been included into a release candidate, ensure the test steps are executed and work on the release candidate. Write up the results as per the Live GIS Disc Testing wiki page.

Packaging considerations

The criteria used to select applications for the LiveGIS follow.

Technical limitations

Space considerations

• DVD: approx 4.2gb compressed
• USB stick: For a 4gb persistent USB stick with some room left over for user-workspace, 3.2-3.5gb compressed. (This pretty much works out to the DVD version minus the preloaded Mac and Windows installers)
• Virtual Machines: theoretically unlimited, but in practice 20gb uncompressed

So smaller apps (<30mb) of lesser priority may easily slip in under the ire radar, but an external project that wants to use a 25% of the available disc space would be the first to be compromised away.

Memory considerations

Both Virtual Machines, and a LiveDVD images are likely to be constrained by limited memory. So to reduce memory usage. Disk image size is not of major concern, as we can just distribute less data.

The following principles should be followed.

• Do not start applications upon power up. (Ie, don't start deamons, allow users to start them instead).
• Set up examples which, by default, don't depend on other applications. Less applications open, means less memory. Ie, Have GeoServer access a shapefile instead of PostGIS.
• Try to avoid scenarios which write data to disk, as disk space in the Live DVD is stored in RAM, and is not cleared afterward.

File permissions and filesystem user IDs can get garbled during the ISO build process. Therefore a special method is used to ensure that data files which must be read-write can be accessed.

To install a writable data directory or files, change its group to "users" and set the file(s) as group-writable:

adduser $USER_NAME users
chmod -R g+w /usr/local/share/<packagename>/data/
chown -R root.users /usr/local/share/<packagename>/data/

and symlink into the user's home directory if necessary.

Setting files to have universal write permission is strongly discouraged.

Directory Structure

1. /tmp Temporary files (e.g., downloaded archives) go into /tmp. Please create a separate folder for your project.
2. /usr/lib application are usually installed into /usr/lib
3. /usr/bin things that get executed by the user such as startup scripts or links to them should go into /usr/bin
4. /etc/init.d startup/shutdown scripts for services (e.g., postgres, apache, tomcat) are stored in /etc/init.d
5. /usr/local/share sample data and documentation goes into /usr/local/share
6. /etc config files are stored in /etc
7. /home/user user specific config files or working directories can go into /home/user/. However, keep files in /home/user as small as possible as this folder is loaded into memory in the Live DVD. Symlinks into /usr/local can be useful here.

• You are strongly encouraged to use /usr/local/ and /var/local/ for storing non-packaged content.