Difference between revisions of "Geodata processing at telascience.org"

From OSGeo
Jump to navigation Jump to search
m (Tidy inter-Wiki links)
 
(2 intermediate revisions by the same user not shown)
Line 17: Line 17:
 
* [[User:Neteler|Markus Neteler]]
 
* [[User:Neteler|Markus Neteler]]
 
* Frank Warmerdam
 
* Frank Warmerdam
−
* Martin Spott
+
* [[User:Mspott|Martin Spott]]
  
 
=== Installed Software ===
 
=== Installed Software ===
Line 61: Line 61:
 
** remove annoying grid lines (dissolve in GRASS)
 
** remove annoying grid lines (dissolve in GRASS)
 
** enrich country maps with population data (see [http://www.intevation.de/pipermail/freegis-list/2004-July/001784.html here] for inspiration, I have their SQL code somewhere)
 
** enrich country maps with population data (see [http://www.intevation.de/pipermail/freegis-list/2004-July/001784.html here] for inspiration, I have their SQL code somewhere)
−
* Already in [[Geodata_Repository#PostGIS_serving_vector_data]] (Martin).
+
* Predefined subset (focusing on landuse data) already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
  
 
==== Data: GSHHS World Costline Vector Data ====
 
==== Data: GSHHS World Costline Vector Data ====
Line 71: Line 71:
 
** Original server is now here, versions newer than 1.3 are licensed as GPL:
 
** Original server is now here, versions newer than 1.3 are licensed as GPL:
 
::http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.5/
 
::http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.5/
 +
** Closed Polygon Shapefiles of the GSHHS 1.6 coastlines contributed by the [http://www.custom-scenery.org/ Custom Scenery Project] and available for download here (primary site):
 +
::ftp://ftp.ihg.uni-duisburg.de/GIS/GISData/GSHHS/
 
* TODO:
 
* TODO:
−
** <strike>convert to SHAPE</strike> - find them on the NOAA site in the shapefiles/ dir, or I've (Hamish) got a copy.
 
−
::: If you really want to load in original data, you can use GRASS 5.4's v.in.gshhs module.
 
 
** tile into smaller pieces
 
** tile into smaller pieces
 
** cleanup/add attributes
 
** cleanup/add attributes
−
* Already in [[Geodata_Repository#PostGIS_serving_vector_data]] (Martin).
+
* GSHHS 1.6 already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
  
−
* The OpenStreetMap.org wiki claims that the lake data is not very good. Fortunately they list some other alternatives:
+
* Lake data is not very accurate. Fortunately the OpenStreetMap.org wiki lists some other alternatives:
 
:http://wiki.openstreetmap.org/index.php/Potential_Datasources#Shoreline_database
 
:http://wiki.openstreetmap.org/index.php/Potential_Datasources#Shoreline_database
 
: That is a very nice page to explore!
 
: That is a very nice page to explore!
Line 92: Line 92:
 
* TODO:
 
* TODO:
 
** convert to SHAPE?
 
** convert to SHAPE?
−
** insert into PostGIS? - see docs/geonames_dump_into_postgresql.txt
+
** <strike>insert into PostGIS?</strike> - see docs/geonames_dump_into_postgresql.txt
−
* Already in [[Geodata_Repository#PostGIS_serving_vector_data]] (Martin).
+
* Already part of the [[Geodata_Repository#PostGIS_serving_vector_data|Geodata Repository]] (Martin).
  
 
[[Category:Public Geospatial Data Committee]]
 
[[Category:Public Geospatial Data Committee]]

Latest revision as of 06:11, 27 April 2008

Background

As one objective of OSGeo, we want to generate/update/enhance existing free geodata for the community. This comprises re-packaging, data storage in convenient formats, tiling and other.

We are kindly supported by http://telascience.org (available resources).

Aim/Vision

The idea of the entire geodata processing initiative is to deliver quality datasets based on available public sources. User shall be enabled to download enriched datasets in common GIS formats. It is hoped that a mechanism can be implemented to deliver spatial subsets on the fly. Also different levels of generalization are desired (e.g., world map at different scales/vector "resolution").

Besides common GIS formats, these improved geodata shall also be delivered as OGC Web Services (WMS, WFS etc).

Status of geodata.telascience.org

Participants

Installed Software

I have installed the following software packages on geodata.telascience.org:

  • geos 2.2.3
  • proj 4.5.0 - SPEC file
  • ogdi-CVS (17 August 2006) - compiled with external proj, expat, zlib
  • GDAL 1.3.2 - SPEC file
  • GRASS 6.2.0 - SPEC file
  • GDAL-OGR-GRASS plugin - SPEC file
  • OSSIM (19 August 2006) - default compilation
  • FWTools 1.0.5 - in /usr/local/FWTools-1.0.5, use "fwtools" alias to add to path

Source code:

  • /home/neteler/software/<software>
  • if taken from CVS, anon-CVS was used
  • configuration is saved in each subdir as "conf_<software>.sh"

Binaries:

  • all binaries are installed into /usr/local/*
  • /usr/local/lib was added to /etc/ld.so.conf

Apache

Apache is running with a default configuration. The web site contents goes in /var/www/html. After a machine restart it may be necessary to restart apache with "apachectl restart" as root/sudo.

Geodata processing

  • data are stored on geodata.telascience.org in /geodata/<user>
  • only root/sudo can add further directories in /geodata/ (FrankW, markusN, hobu, ...)

Data: VMAP0 1:1Mio World Vector Data

  • /geodata/markus/vmap0/
  • eur, sas, soa are from 2000, noa from 2002
  • status: downloaded
  • TODO:
    • convert to SHAPE
    • tile into smaller pieces
    • cleanup attributes
    • remove annoying grid lines (dissolve in GRASS)
    • enrich country maps with population data (see here for inspiration, I have their SQL code somewhere)
  • Predefined subset (focusing on landuse data) already part of the Geodata Repository (Martin).

Data: GSHHS World Costline Vector Data

(Global Self-consistent Hierarchical High-resolution Shoreline)

  • /geodata/markus/gshhs1.5/
  • from Sept. 2004
  • status: downloaded (original server disappeared, but MN had a copy)
    • Original server is now here, versions newer than 1.3 are licensed as GPL:
http://www.ngdc.noaa.gov/mgg/shorelines/data/gshhs/version1.5/
    • Closed Polygon Shapefiles of the GSHHS 1.6 coastlines contributed by the Custom Scenery Project and available for download here (primary site):
ftp://ftp.ihg.uni-duisburg.de/GIS/GISData/GSHHS/
  • TODO:
    • tile into smaller pieces
    • cleanup/add attributes
  • GSHHS 1.6 already part of the Geodata Repository (Martin).
  • Lake data is not very accurate. Fortunately the OpenStreetMap.org wiki lists some other alternatives:
http://wiki.openstreetmap.org/index.php/Potential_Datasources#Shoreline_database
That is a very nice page to explore!
    • Remember - in case of doubt - to take claims made by OSM with a grain of salt. Certain "Not Invented Here"-attitude has been reported. Actually a highly accurate, complete and truly 'open' (TM) global shoreline dataset has not been spotted so far. Remember as well that shorelines that are derived from satellite images (either automagically or manually) usually feature certain tidal errors by design.

Data: Gazetteer data World from geonames.org

  • /geodata/markus/geonames.org/
    • database dump in dump_17aug2006/
    • docs in docs/
    • fetch.sh script in the dump/ directory for future downloads
  • status: downloaded
  • TODO:
    • convert to SHAPE?
    • insert into PostGIS? - see docs/geonames_dump_into_postgresql.txt
  • Already part of the Geodata Repository (Martin).
Retrieved from "http://wiki.osgeo.org/w/index.php?title=Geodata_processing_at_telascience.org&oldid=24586"