Difference between revisions of "Geodata processing at telascience.org"

Latest revision as of 06:11, 27 April 2008

Background

As one objective of OSGeo, we want to generate/update/enhance existing free geodata for the community. This comprises re-packaging, data storage in convenient formats, tiling and other.

We are kindly supported by http://telascience.org (available resources).

Aim/Vision

The idea of the entire geodata processing initiative is to deliver quality datasets based on available public sources. User shall be enabled to download enriched datasets in common GIS formats. It is hoped that a mechanism can be implemented to deliver spatial subsets on the fly. Also different levels of generalization are desired (e.g., world map at different scales/vector "resolution").

Besides common GIS formats, these improved geodata shall also be delivered as OGC Web Services (WMS, WFS etc).

Status of geodata.telascience.org

Participants

Markus Neteler
Frank Warmerdam
Martin Spott

Installed Software

I have installed the following software packages on geodata.telascience.org:

geos 2.2.3
proj 4.5.0 - SPEC file
ogdi-CVS (17 August 2006) - compiled with external proj, expat, zlib
GDAL 1.3.2 - SPEC file
GRASS 6.2.0 - SPEC file
GDAL-OGR-GRASS plugin - SPEC file
OSSIM (19 August 2006) - default compilation
FWTools 1.0.5 - in /usr/local/FWTools-1.0.5, use "fwtools" alias to add to path

Source code:

/home/neteler/software/<software>
if taken from CVS, anon-CVS was used
configuration is saved in each subdir as "conf_<software>.sh"

Binaries:

all binaries are installed into /usr/local/*
/usr/local/lib was added to /etc/ld.so.conf

Apache

Apache is running with a default configuration. The web site contents goes in /var/www/html. After a machine restart it may be necessary to restart apache with "apachectl restart" as root/sudo.

Geodata processing

data are stored on geodata.telascience.org in /geodata/<user>
only root/sudo can add further directories in /geodata/ (FrankW, markusN, hobu, ...)

Data: VMAP0 1:1Mio World Vector Data

/geodata/markus/vmap0/
eur, sas, soa are from 2000, noa from 2002
status: downloaded
TODO:
- convert to SHAPE
- tile into smaller pieces
- cleanup attributes
- remove annoying grid lines (dissolve in GRASS)
- enrich country maps with population data (see here for inspiration, I have their SQL code somewhere)
Predefined subset (focusing on landuse data) already part of the Geodata Repository (Martin).

Data: GSHHS World Costline Vector Data

(Global Self-consistent Hierarchical High-resolution Shoreline)

/geodata/markus/gshhs1.5/
from Sept. 2004
status: downloaded (~~original server disappeared~~, but MN had a copy)
- Original server is now here, versions newer than 1.3 are licensed as GPL:

http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.5/

- Closed Polygon Shapefiles of the GSHHS 1.6 coastlines contributed by the Custom Scenery Project and available for download here (primary site):

ftp://ftp.ihg.uni-duisburg.de/GIS/GISData/GSHHS/

TODO:
- tile into smaller pieces
- cleanup/add attributes
GSHHS 1.6 already part of the Geodata Repository (Martin).

Lake data is not very accurate. Fortunately the OpenStreetMap.org wiki lists some other alternatives:

http://wiki.openstreetmap.org/index.php/Potential_Datasources#Shoreline_database

That is a very nice page to explore!

- Remember - in case of doubt - to take claims made by OSM with a grain of salt. Certain "Not Invented Here"-attitude has been reported. Actually a highly accurate, complete and truly 'open' (TM) global shoreline dataset has not been spotted so far. Remember as well that shorelines that are derived from satellite images (either automagically or manually) usually feature certain tidal errors by design.

Data: Gazetteer data World from geonames.org

/geodata/markus/geonames.org/
- database dump in dump_17aug2006/
- docs in docs/
- fetch.sh script in the dump/ directory for future downloads
status: downloaded
TODO:
- convert to SHAPE?
- ~~insert into PostGIS?~~ - see docs/geonames_dump_into_postgresql.txt
Already part of the Geodata Repository (Martin).

@@ Line 4: / Line 4: @@
 We are kindly supported by http://telascience.org (available [[SAC Service Status|resources]]).
+== Aim/Vision ==
+The idea of the entire geodata processing initiative is to deliver quality datasets based on available public sources. User shall be enabled to download enriched datasets in common GIS formats. It is hoped that a mechanism can be implemented to deliver spatial subsets on the fly. Also different levels of generalization are desired (e.g., world map at different scales/vector "resolution").
+Besides common GIS formats, these improved geodata shall also be delivered as OGC Web Services (WMS, WFS etc).
 == Status of geodata.telascience.org ==
@@ Line 11: / Line 17: @@
 * [[User:Neteler|Markus Neteler]]
 * Frank Warmerdam
-* Martin Spott
+* [[User:Mspott|Martin Spott]]
 === Installed Software ===
@@ Line 55: / Line 61: @@
 ** remove annoying grid lines (dissolve in GRASS)
 ** enrich country maps with population data (see [http://www.intevation.de/pipermail/freegis-list/2004-July/001784.html here] for inspiration, I have their SQL code somewhere)
-* There is now also a [http://wiki.osgeo.org/index.php/Geodata_Repository#VMap0_-_and_some_other_vector_data PostGIS installation of VMAP0] on the xxx telascience machine (define access possibilities here)
+* Predefined subset (focusing on landuse data) already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
 ==== Data: GSHHS World Costline Vector Data ====
+(''Global Self-consistent Hierarchical High-resolution Shoreline'')
 * /geodata/markus/gshhs1.5/
 * from Sept. 2004
-* status: downloaded (original server disappeared, but MN had a copy)
+* status: downloaded (<strike>original server disappeared</strike>, but MN had a copy)
+** Original server is now here, versions newer than 1.3 are licensed as GPL:
+::http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.5/
+** Closed Polygon Shapefiles of the GSHHS 1.6 coastlines contributed by the [http://www.custom-scenery.org/ Custom Scenery Project] and available for download here (primary site):
+::ftp://ftp.ihg.uni-duisburg.de/GIS/GISData/GSHHS/
 * TODO:
-** convert to SHAPE
 ** tile into smaller pieces
 ** cleanup/add attributes
+* GSHHS 1.6 already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
+* Lake data is not very accurate. Fortunately the OpenStreetMap.org wiki lists some other alternatives:
+:http://wiki.openstreetmap.org/index.php/Potential_Datasources#Shoreline_database
+: That is a very nice page to explore!
+** Remember - in case of doubt - to take claims made by OSM with a grain of salt. Certain "Not Invented Here"-attitude has been reported. Actually a highly accurate, complete '''and''' truly 'open' (TM) global shoreline dataset has not been spotted so far. Remember as well that shorelines that are derived from satellite images (either automagically or manually) usually feature certain tidal errors by design.
 ==== Data: Gazetteer data World from geonames.org ====
@@ Line 76: / Line 92: @@
 * TODO:
 ** convert to SHAPE?
-** insert into PostGIS? - see docs/geonames_dump_into_postgresql.txt
+** <strike>insert into PostGIS?</strike> - see docs/geonames_dump_into_postgresql.txt
+* Already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
+[[Category:Public Geospatial Data Committee]]