Difference between revisions of "Geodata Repository"

From OSGeo
Jump to navigation Jump to search
(link to Education_and_Curriculum_Committee)
m (Mspott is now Martin)
 
(200 intermediate revisions by 11 users not shown)
Line 1: Line 1:
−
= Why should OSGeo be running a geodata repository? =
+
''The notes on the [[Talk:Geodata_Repository]] Talk page for this page describe the background to this effort''
  
−
* Without data, Open Source Geospatial Software isn't much use.
+
A full list of suggestions for public domain data sets that are nice-to-haves is maintained at [[Geodata Discovery Working Group]].
  
−
* OSGeo has been generously offered use of the facilities at Telascience - http://osgeo.telascience.org/ - which is already mirroring a lot of public domain sources and can provide hosting to a 'stack' of sample apps, which good data collections will support.
+
= Getting involved =
  
−
* The [[Geodata Discovery Working Group]] needs a good selection of data sources to motivate discussion of indexing and search; the [[Geodata Packaging Working Group]] needs high-quality data and metadata to produce [[Education_and_Curriculum_Committee|educational]] packages from.  
+
The geodata repository has a dedicated blade server at telascience - cf [[SAC Service Status]]. Right now this is geodata.telascience.org ; we need to get an osgeo.org domain pointed to it. The plan is to have something very much up, running and demoable by FOSS4G.
  
−
* This is a service OSGeo can usefully offer to anyone who wants a safe home for their data in the future.
+
== How you can help ==
 +
 
 +
* Offer feedback on the [[Geodata Metadata Requirements]] - if you have a dataset you would like to contribute, does this model express it adequately?
 +
* Write or contribute an example [[Simple Catalog Interface]]
 +
* If you would like to install software and/or data on the repository machine, please talk to the good people at [[SAC]] or visit #osgeo and #telascience on irc.freenode.net to find out how to get an account and get started
 +
 
 +
== Who is involved now ==
 +
 
 +
* [[User:JoWalsh|Jo Walsh]] is working on the backend data store and machine level metadata interfaces (also presenting this material at [[OSGeo at FOSS4G2006|FOSS4G 2006]])
 +
* Chris Schmidt is helping with an OpenLayers-based browsing interface
 +
* Schuyler Erle is helping with web service / application glue
 +
* Sean Gillies' OWSLib is, half-beknownst to him, a big part of all these applications - see the OWSLib API user stories at http://trac.gispython.org/projects/PCL/wiki/OwsCapabilitiesUserStory
 +
* Markus Neteler is contributing data processing code and resources as part of the GRASS data packaging effort
 +
* Norm Vine is providing a point of contact for administering the system and general reality-checking to all involved.
 +
* [[User:Martin|Martin Spott]] maintains the content of the [[Geodata_Repository#PostGIS_serving_vector_data|PostGIS-repository]] on 'zuluviz'.
 +
 
 +
= Interface Design =
 +
 
 +
In order to be useful to people, a geodata repository needs a way for users to become quickly + easily informed as to what kind of data they're getting: metadata about the data, and the quality/quantity of data involved.
 +
 
 +
For Raster images, one way to do this is to include an extent-wide image of the data -- http://openlayers.org/gallery/ has screenshots which show some of what I mean where, although more directed towards applications. By having a whole-extent screenshot overview, users can quickly and easily see what they're getting. Additionally, assuming this data is to be made available as WMS, setting up an OpenLayers instance to allow users to browse the rasters and see at a more detailed level would  be beneficial.
 +
 
 +
Vector data/attribute data would also need to be described, in text or some other way. http://freemap.in/world/ is an example of a browsable map which displays attribute data -- clicking on a country fills the sidebar with data. This interface was set up in about 20 minutes, and given decent map files, this kind of set up could be largely automated, allowing users to (again) get an overview of the data they're looking at before they download it or set it up on their own servers.
  
 
= Data sources =
 
= Data sources =
  
−
These are suggestions that have been offered to the group so far.  
+
Sources of public geodata for initial setup in a repository.
 +
 
 +
== PostGIS serving vector data ==
 +
 
 +
There's a [http://www.postgis.org/ PostGIS] server (thanks to John Graham for hosting this site at [http://www.telascience.org/ TelaScience]) which stores different flavours of vector data; also known as "Landcover-DB". Current datasets include: VMap0, VMap1, AptNav, GSHHS, PGS, SWBD, MGRS, TIGER, StatsCan, OSM, GeoNames, CountryCodes and FGSODB - see below for further explanation.
 +
 
 +
=== Access - how to get to it !?===
  
−
== Well-known sources of publically collected and maintained geodata ==
+
* [http://mapserver.flightgear.org/ Graphical MapServer interface]
 +
* [http://mapserver.flightgear.org/download.psp Download Shapefiles]
 +
* [[Geodata_Repository#WMS_.2B_WFS_.2B_TileCache_services|WMS + WFS + TileCache services - see below]]
  
−
See list maintained at [[Geodata Discovery Working Group]]
+
    The web mapping colour schema has been applied in accordance with the [http://etc-lusi.eionet.europa.eu/CLC2000/classes/ Corine Land Cover] project.
  
−
== Lesser known or less widely distributed sources of publically collected geodata ==
+
* Direct read access to the database is available to users of the 'geodata.telascience.org' system. Retrieval in SQL syntax as "SELECT asText(wkb_geometry) FROM <Tablename> WHERE <Column> LIKE '<Keyword>'".
 +
* Collaborative editing via [http://www.featureserver.org/ FeatureServer] is being prepared.
 +
* Dump into Shapefiles provided occasionally or upon request; regular schedule possible if required.
 +
* Other data access upon request, depending on the purpose and use.
  
−
* Local government departments which are putting geodata into the public domain (MassGIS, Manitoba, MetroGIS)
+
'''CAVEAT''' We regret that, due to technical reasons, direct PostGIS database access is currently unavailable.
  
−
== Privately contributed / collectively constructed geodata ==
+
=== On Offer ! ===
  
−
* [http://openstreetmap.org/ OpenStreetmap] makes data available through a [http://wiki.openstreetmap.org/index.php/REST HTTP based XML interface]. They've also been getting requests for shapefile and GML output of their data.
+
Short explanation of available datasets (to be extended - the number of datasets as well as their explanation  ;-):
  
−
= No data without metadata! =
+
{|border="1" cellpadding="4" cellspacing="0"
 +
!Name
 +
!Description
 +
!# of layers
 +
|-
 +
|VMap0
 +
|Selected subsets of [http://www.mapability.com/info/vmap0_intro.html Vector Smart Map Level 0] polygons, lines and points, starting with a selection that has proven to be useful for creating [http://www.flightgear.org/ FlightGear] Scenery from it. Added a 'geonameid' column for joining urban areas with GeoNames (see below). Current details explained at the [http://www.custom-scenery.org/Background.262.0.html World Custom Scenery Project], will get synced some day.
 +
|33 [[LandcoverDB_VMap0_Detail|(DETAIL)]]
 +
|-
 +
|VMap1
 +
|First attempt of a selection that would be "nice to have" for FlightGear from [http://www.mapability.com/info/vmap1_intro.html Vector Smart Map Level 1] - and certainly for other purposes as well. Added a 'geonameid' column for joining urban areas with GeoNames. Details similar to VMap0.
 +
|58 [[LandcoverDB_VMap1_Detail|(DETAIL)]]
 +
|-
 +
|AptNav
 +
|Geometric average of runway center locations plus runway/taxiway shapes as used by the FlightGear and X-Plane flight simulators; data taken from [http://www.x-plane.org/home/robinp/ Robin Peel's Airport Database]. Locations converted to OGC-style POINT geometries. Use 'icao' column for searching.
 +
* This import is currently tied to the state of the FlightGear 1.0.0 Base Package release.
 +
|1 [[LandcoverDB_AptNav_Detail|(DETAIL)]]
 +
|-
 +
|GSHHS
 +
|[http://www.soest.hawaii.edu/wessel/gshhs/gshhs.html Global Self-consistent, Hierarchical, High-resolution Shoreline Database] 1.6 shorelines.
 +
|4 [[LandcoverDB_GSHHS_Detail|(DETAIL)]]
 +
|-
 +
|PGS
 +
|[http://www.nga.mil/portal/site/nga01/index.jsp?epi-content=GENERIC&itemID=9328fbd8dcc4a010VgnVCMServer3c02010aRCRD&beanID=1629630080&viewID=Article NGA Prototype Global Shoreline].
 +
|1 [[LandcoverDB_PGS_Detail|(DETAIL)]]
 +
|-
 +
|SWBD
 +
|[http://edc.usgs.gov/products/elevation/swbd.html SRTMv2] Water Body Data.
 +
|1 [[LandcoverDB_SWBD_Detail|(DETAIL)]]
 +
|-
 +
|MGRS
 +
|Military Grid Reference System, alias UTMREF.
 +
|1 [[LandcoverDB_MGRS_Detail|(DETAIL)]]
 +
|-
 +
|TIGER
 +
|[http://www.census.gov/geo/www/tiger/ Topologically Integrated Geographic Encoding and Referencing system] line data. Roads, railroads and water/stream line data from the 2006se release, water body and landmark polygons from 2005fe (thanks to Chris Holmes at [http://www.openplans.org/ The Open Planning Project] for providing pre-processed data).
 +
|6 [[LandcoverDB_TIGER_Detail|(DETAIL)]]
 +
|-
 +
|StatsCan
 +
|Line data of the [http://www.statcan.ca/ Statistics Canada] [http://www.statcan.ca/bsolc/english/bsolc?catno=92-500-XWE 2006 Road Network File].
 +
|1 [[LandcoverDB_StatsCan_Detail|(DETAIL)]]
 +
|-
 +
|OSM
 +
|[http://www.openstreetmap.org/ OpenStreetMap] Import of the [http://planet.openstreetmap.org/ planet dump]. Split up into 23 different road-, railroad- and stream-layers; schema taken from [http://wiki.openstreetmap.org/index.php/Map_Features#Highway Highway Map Features].
 +
* This import is done manually - typically the weekend after a new planet dump is being made available.
 +
|6 [[LandcoverDB_OSM_Detail|(DETAIL)]]
 +
|-
 +
|GeoNames
 +
|Complete content of the "allCountries" export table from the [http://www.geonames.org/about.html Geonames.org geographical database] (as of 2008-08-08). Locations converted to OGC-style POINT geometries. Added a 'pplkey' column for searchable classification of size for populated places [1-7]; schema proposed by Markus Neteler:
 +
* continental scale (>= 1:50 million): >= 1 million inhabitants
 +
* multi-national scale (>= 1:10 million): 500000-1 million inhab.
 +
* country scale (>= 1:1 million): 100000-499999 inhab.
 +
* regional scale (>= 1:500000): 50000-99999 inhab.
 +
* city scale (>= 1:50000): 10000-49999 inhab.
 +
* local scale: < 10000
 +
|6 [[LandcoverDB_GeoNames_Detail|(DETAIL)]]
 +
|-
 +
|CountryCodes
 +
|Translation table for country codes as proposed [http://www.intevation.de/pipermail/freegis-list/2004-July/001784.html here] (thanks to Silke Reimer for preparing the table).
 +
|1
 +
|-
 +
|FGSODB
 +
| This is the primary location of the [http://scenemodels.flightgear.org/models.php FlightGear Scenery Models Repository]; models consist of AC3D geometries, RGB/PNG textures and in some cases animations that are defined in XML wrappers. Locations in OGC-style POINT geometries.
 +
|1 [[LandcoverDB_FGSODB_Detail|(DETAIL)]]
 +
|-
 +
|Custom Scenery
 +
|Landuse data at selected areas [http://mapserver.flightgear.org/map/?lon=9.20438&lat=47.63982&zoom=11 (example)] has been auto-classified from Landsat7-images and converted into suitable polygons at the [http://www.custom-scenery.org/Research-Deve.274.0.html World Custom Scenery Project].
 +
In contrast to the datasets which are listed above, this chapter is the place for our future development. It consists of VMap0 data as a basis and is being improved either by "home grown" landcover data or by imports of "free" (TM) datasets.
 +
|61 [[LandcoverDB_CS_Detail|(DETAIL)]]
 +
|}
 +
Many thanks to Norman Vine for running the HCRA, the "Human Communications Relay Agent"  :-)
  
−
An important factor for telascience is to have better metadata management facilities, and this is something that OSGeo's collective intelligence can really help with. During their [http://katrina.telascience.org/ data support efforts for Katrina hurricane relief] data was being collected in "a very ad-hoc fashion". To be sure that OSGeo has the rights to redistribute data that it collects, a base level of metadata indicating contact details and license terms has to be established.  
+
=== Procedures .... ===
  
−
* [[Geodata Metadata Requirements]] - a baseline for what would be asked for / expected of data sets in an OSGeo repository
+
==== How the data gets in ====
 +
Many datasets are readable with the GDAL/OGR(/OGDI) toolbox. Notably these are the
 +
* VMap datasets,
 +
* GSHHS,
 +
* PGS,
 +
* SWBD and
 +
* MGRS.
 +
 
 +
The 'ogr2ogr' command is used here, hidden behind a somewhat complex contruct of shell scripts which automates the whole thing. Writing to the DB is accomplished by the PostGIS driver provided by 'ogr2ogr'.
 +
 
 +
Some other datasets are automatically or manually (for those sets which are not expected to change often) transformed into OGC-compilant SQL scripts and run through the respective SQL monitor.
 +
 
 +
Few datasets are being imported directly into the DB with Perl/DBI. AptNav, for example, is parsed from a text file with a home-grown parser in Perl and written to the database table with DBI.
 +
 
 +
Everything that is meant to represent a geometry is stored in the DB using OCG-compilant geometry types POLYGON, LINESTRING and POINT. A POINT, for example, is written into the DB as
 +
 
 +
  INSERT INTO <Tablename> ([...]) VALUES ([...] PointFromText('POINT(<Lon> <Lat>)', 4326) [...])
 +
 
 +
The data are stored internally in the respective, geospatially searchable geometry data type.
 +
 
 +
=== Timeline ===
 +
 
 +
==== Phase 1 ====
 +
 
 +
* Import data from many different sources and shape it into a unified format - first results available (see above).
 +
* Retrieve exact locations of major river dams and waterfalls (keywords: St. Lawrence, Niagara, Bosporus, Gibraltar, ....).
 +
* Build joins between VMap0/1 urban areas and GeoNames populated places (via names and geographic vincinity).
 +
 
 +
==== Phase 2 ====
 +
 
 +
* Generation of static per-country and per-region shapefiles; distribution via HTTP and via [http://geotorrent.org/browse.php geotorrent.org].
 +
 
 +
==== Phase 3 ====
 +
 
 +
* Design and implement a storage-/data-model for road data that is capable of serving the needs of [http://www.openstreetmap.org/ OpenStreetMap] while remaining conformant to OGC-standards. Merge the ideas explained in the [http://www.remote.org/frederik/tmp/towards-a-new-data-model-for-osm.pdf OSM New Data Model] paper as well as Schuler's [http://freemap.in/~sderle/pg_osm/ OSM on PostGIS] initiative - in other words: Try "squaring the circle"  ;-)
 +
* Now that OpenStreetMap has (also) imported TIGER, convince OSM to imcorporate an equivalent of VMap0 at places that are currently not covered, finally to create a global road network of maximum detail and accuracy.
 +
* Merge landcover- and stream-layers from VMap1, TIGER/OSM and Landsat7-classification into the foundation of VMap0 to create a global landcover dataset of maximum detail and accuracy.
 +
 
 +
==== Sequence undefined ====
 +
 
 +
* Add proper metadata according to OSGeo metadata recommendation/standard.
 +
* Find suitable desktop client to [http://www.featureserver.org/ FeatureServer].
 +
* Add polygon data that's been automagically retrieved from Landsat7 images at the [http://www.custom-scenery.org/Research-Deve.274.0.html World Custom Scenery Project] - procedure available.
 +
* Add polygon data that's been manually digitized from Landsat7 images.
 +
 
 +
==== What you can do ====
 +
 
 +
* Help is very much appreciated to add a reasonable colour schema. Work is currently on its way to convert the map display over to the CORINE [http://www.corine.dfd.dlr.de/media/download/clc_lut_en.pdf colour values] but this schema doesn't cover all our needs.
 +
 
 +
==== DONE ====
 +
 
 +
* Generation of shapefiles on-the-fly for user-defined region and layer(s). Use 'pgsql2shp' and consider the 2 GByte-per-file limit: [http://mapserver.flightgear.org/download.psp Landcover Database Shapefile Download]
 +
* Rebuild TIGER layers from 2006se release according to [http://docs.codehaus.org/display/GEOSDOC/Loading+TIGER+data these instructions] - mostly finished.
 +
* Limit the "viewing angle" of WMS/WFS-services in order to save the DB-servers' life ....  Solved by setting MAXSCALE for some layers.
 +
 
 +
=== Old status ===
 +
 
 +
* see [[Geodata processing at telascience.org#Data:_VMAP0_1:1Mio_World_Vector_Data]]
 +
 
 +
== Blue Marble NG ==
 +
 
 +
* Mapserver as WMS
 +
 
 +
=== Status ===
 +
* By bittorrent: http://data.freemap.in/view/DataSet/9
 +
 
 +
== SRTM ==
 +
 
 +
=== Status ===
 +
* SRTM30 data is available now by WMS/WCS: http://data.freemap.in/view/DataSet/11
 +
 
 +
== Landsat-7 ==
 +
 
 +
* Mapserver as WMS
 +
* Telascience already serving the raw data - http://onearth.telascience.org/
 +
 
 +
=== Status ===
 +
 
 +
* Waiting for disk space to finish unpacking the raw data on zuluviz.sdsu.edu
 +
* Interim plan is to write a simple WMS wrapper script that generates a GDAL VRT to assemble composites on the fly
  
−
== Common metadata standards ==
+
== Seamless imagery ==
  
−
US Federal Geographic Data Committee metadata stipulations are the baseline. Working agreement is that what we should collect, should be at least FGDC compliant.
+
=== Status ===
 +
* Create mechanism to have multi-resolution seamless "best of" imagery server [[seamlessimagery|Multi-resolution seamless imagery]]
  
−
* [http://www.fgdc.gov/metadata/ FGDC metadata overview].
+
== WMS + WFS + TileCache services ==
−
* [http://dcs.dlese.org/preview/admin/view.do?id=TEST-000-000-000-008 GEON metadata taxonomy]
 
  
 +
Many layers (see list above) are served as WMS/WFS at http://mapserver.flightgear.org
  
 +
    WMS: http://mapserver.flightgear.org/ms?Service=WMS&Version=1.1.1&request=GetCapabilities
 +
   
 +
    WFS: http://mapserver.flightgear.org/ms?Service=WFS&Version=1.0.0&request=GetCapabilities
 +
   
 +
    TileCache: http://mapserver.flightgear.org/tc (EPSG:900913 !!)
  
−
== Potentially useful toolsets ==
+
== TelaScience / OSGeo / FlightGear Landcover Database Shapefile Download ==
  
−
* http://www.dlese.org/libdev/dcs_overview.html
+
See http://mapserver.flightgear.org/download.psp
−
* Any open and free implementations providing a WCS interface?
 
  
−
== Data+metadata transmission standards ==
+
== City of Vienna, Austria ==
  
−
* OGC Web Catalog Services
+
[http://data.wien.gv.at OGD Wien] offers data under CC-BY. This data can be imported to [[OSM]]. Supported interface standard include:
 +
* WMS
 +
* WFS GML
 +
* JSON
 +
* KML geplant
 +
Contact on IRC at #ogdwien.
  
−
== Related efforts ==
+
= Metadata =
  
 +
* [[Geodata Metadata Requirements]] - a baseline for what would be asked for / expected of data sets in an OSGeo repository
 +
* A simple metadata-in-GeoRSS webservice that is also as FGDC compliant as possible/desirable.
 +
* [[Simple Catalog Interface]]
 +
 +
= See Also =
 +
 +
* OSGeo's [[Geodata Discovery Working Group]] - half-dormant
 +
* http://grasswiki.osgeo.org/wiki/Global_datasets
 
* [http://www.mapdex.org/search/ MapDex] offers a metadata search facility
 
* [http://www.mapdex.org/search/ MapDex] offers a metadata search facility
 
* [http://www.geonames.org/ GeoNames] is a gazetteer and naming collection
 
* [http://www.geonames.org/ GeoNames] is a gazetteer and naming collection
 
* http://www.fao.org/geonetwork/srv/en/main.search
 
* http://www.fao.org/geonetwork/srv/en/main.search
−
* http://geoportail-geoportal.ainc-inac.gc.ca/metaindex_e.asp  
+
* http://geoportail-geoportal.ainc-inac.gc.ca/metaindex_e.asp
−
 
+
* http://en.wikipedia.org/wiki/List_of_towns
−
--------
+
* http://mapas.mma.gov.br/i3geo/menutemas/servicoswms.php WMS servers collection in RSS format
 +
* http://mapas.mma.gov.br/i3geo/wscliente.htm Web service client
  
−
Initiated after the [[Geodata Committee Meeting 20060329|second Public Geodata Committee meeting]].
+
[[Category:Public Geospatial Data Committee]]
 +
[[Category:Infrastructure]]

Latest revision as of 03:44, 5 April 2021

The notes on the Talk:Geodata_Repository Talk page for this page describe the background to this effort

A full list of suggestions for public domain data sets that are nice-to-haves is maintained at Geodata Discovery Working Group.

Getting involved

The geodata repository has a dedicated blade server at telascience - cf SAC Service Status. Right now this is geodata.telascience.org ; we need to get an osgeo.org domain pointed to it. The plan is to have something very much up, running and demoable by FOSS4G.

How you can help

  • Offer feedback on the Geodata Metadata Requirements - if you have a dataset you would like to contribute, does this model express it adequately?
  • Write or contribute an example Simple Catalog Interface
  • If you would like to install software and/or data on the repository machine, please talk to the good people at SAC or visit #osgeo and #telascience on irc.freenode.net to find out how to get an account and get started

Who is involved now

  • Jo Walsh is working on the backend data store and machine level metadata interfaces (also presenting this material at FOSS4G 2006)
  • Chris Schmidt is helping with an OpenLayers-based browsing interface
  • Schuyler Erle is helping with web service / application glue
  • Sean Gillies' OWSLib is, half-beknownst to him, a big part of all these applications - see the OWSLib API user stories at http://trac.gispython.org/projects/PCL/wiki/OwsCapabilitiesUserStory
  • Markus Neteler is contributing data processing code and resources as part of the GRASS data packaging effort
  • Norm Vine is providing a point of contact for administering the system and general reality-checking to all involved.
  • Martin Spott maintains the content of the PostGIS-repository on 'zuluviz'.

Interface Design

In order to be useful to people, a geodata repository needs a way for users to become quickly + easily informed as to what kind of data they're getting: metadata about the data, and the quality/quantity of data involved.

For Raster images, one way to do this is to include an extent-wide image of the data -- http://openlayers.org/gallery/ has screenshots which show some of what I mean where, although more directed towards applications. By having a whole-extent screenshot overview, users can quickly and easily see what they're getting. Additionally, assuming this data is to be made available as WMS, setting up an OpenLayers instance to allow users to browse the rasters and see at a more detailed level would be beneficial.

Vector data/attribute data would also need to be described, in text or some other way. http://freemap.in/world/ is an example of a browsable map which displays attribute data -- clicking on a country fills the sidebar with data. This interface was set up in about 20 minutes, and given decent map files, this kind of set up could be largely automated, allowing users to (again) get an overview of the data they're looking at before they download it or set it up on their own servers.

Data sources

Sources of public geodata for initial setup in a repository.

PostGIS serving vector data

There's a PostGIS server (thanks to John Graham for hosting this site at TelaScience) which stores different flavours of vector data; also known as "Landcover-DB". Current datasets include: VMap0, VMap1, AptNav, GSHHS, PGS, SWBD, MGRS, TIGER, StatsCan, OSM, GeoNames, CountryCodes and FGSODB - see below for further explanation.

Access - how to get to it !?

   The web mapping colour schema has been applied in accordance with the Corine Land Cover project.
  • Direct read access to the database is available to users of the 'geodata.telascience.org' system. Retrieval in SQL syntax as "SELECT asText(wkb_geometry) FROM <Tablename> WHERE <Column> LIKE '<Keyword>'".
  • Collaborative editing via FeatureServer is being prepared.
  • Dump into Shapefiles provided occasionally or upon request; regular schedule possible if required.
  • Other data access upon request, depending on the purpose and use.

CAVEAT We regret that, due to technical reasons, direct PostGIS database access is currently unavailable.

On Offer !

Short explanation of available datasets (to be extended - the number of datasets as well as their explanation ;-):

Name Description # of layers
VMap0 Selected subsets of Vector Smart Map Level 0 polygons, lines and points, starting with a selection that has proven to be useful for creating FlightGear Scenery from it. Added a 'geonameid' column for joining urban areas with GeoNames (see below). Current details explained at the World Custom Scenery Project, will get synced some day. 33 (DETAIL)
VMap1 First attempt of a selection that would be "nice to have" for FlightGear from Vector Smart Map Level 1 - and certainly for other purposes as well. Added a 'geonameid' column for joining urban areas with GeoNames. Details similar to VMap0. 58 (DETAIL)
AptNav Geometric average of runway center locations plus runway/taxiway shapes as used by the FlightGear and X-Plane flight simulators; data taken from Robin Peel's Airport Database. Locations converted to OGC-style POINT geometries. Use 'icao' column for searching.
  • This import is currently tied to the state of the FlightGear 1.0.0 Base Package release.
 1 (DETAIL)
GSHHS Global Self-consistent, Hierarchical, High-resolution Shoreline Database 1.6 shorelines. 4 (DETAIL)
PGS NGA Prototype Global Shoreline. 1 (DETAIL)
SWBD SRTMv2 Water Body Data. 1 (DETAIL)
MGRS Military Grid Reference System, alias UTMREF. 1 (DETAIL)
TIGER Topologically Integrated Geographic Encoding and Referencing system line data. Roads, railroads and water/stream line data from the 2006se release, water body and landmark polygons from 2005fe (thanks to Chris Holmes at The Open Planning Project for providing pre-processed data). 6 (DETAIL)
StatsCan Line data of the Statistics Canada 2006 Road Network File. 1 (DETAIL)
OSM OpenStreetMap Import of the planet dump. Split up into 23 different road-, railroad- and stream-layers; schema taken from Highway Map Features.
  • This import is done manually - typically the weekend after a new planet dump is being made available.
 6 (DETAIL)
GeoNames Complete content of the "allCountries" export table from the Geonames.org geographical database (as of 2008-08-08). Locations converted to OGC-style POINT geometries. Added a 'pplkey' column for searchable classification of size for populated places [1-7]; schema proposed by Markus Neteler:
  • continental scale (>= 1:50 million): >= 1 million inhabitants
  • multi-national scale (>= 1:10 million): 500000-1 million inhab.
  • country scale (>= 1:1 million): 100000-499999 inhab.
  • regional scale (>= 1:500000): 50000-99999 inhab.
  • city scale (>= 1:50000): 10000-49999 inhab.
  • local scale: < 10000
 6 (DETAIL)
CountryCodes Translation table for country codes as proposed here (thanks to Silke Reimer for preparing the table). 1
FGSODB This is the primary location of the FlightGear Scenery Models Repository; models consist of AC3D geometries, RGB/PNG textures and in some cases animations that are defined in XML wrappers. Locations in OGC-style POINT geometries. 1 (DETAIL)
Custom Scenery Landuse data at selected areas (example) has been auto-classified from Landsat7-images and converted into suitable polygons at the World Custom Scenery Project.

In contrast to the datasets which are listed above, this chapter is the place for our future development. It consists of VMap0 data as a basis and is being improved either by "home grown" landcover data or by imports of "free" (TM) datasets.

61 (DETAIL)

Many thanks to Norman Vine for running the HCRA, the "Human Communications Relay Agent" :-)

Procedures ....

How the data gets in

Many datasets are readable with the GDAL/OGR(/OGDI) toolbox. Notably these are the

  • VMap datasets,
  • GSHHS,
  • PGS,
  • SWBD and
  • MGRS.

The 'ogr2ogr' command is used here, hidden behind a somewhat complex contruct of shell scripts which automates the whole thing. Writing to the DB is accomplished by the PostGIS driver provided by 'ogr2ogr'.

Some other datasets are automatically or manually (for those sets which are not expected to change often) transformed into OGC-compilant SQL scripts and run through the respective SQL monitor.

Few datasets are being imported directly into the DB with Perl/DBI. AptNav, for example, is parsed from a text file with a home-grown parser in Perl and written to the database table with DBI.

Everything that is meant to represent a geometry is stored in the DB using OCG-compilant geometry types POLYGON, LINESTRING and POINT. A POINT, for example, is written into the DB as 

 INSERT INTO <Tablename> ([...]) VALUES ([...] PointFromText('POINT(<Lon> <Lat>)', 4326) [...])

The data are stored internally in the respective, geospatially searchable geometry data type.

Timeline

Phase 1

  • Import data from many different sources and shape it into a unified format - first results available (see above).
  • Retrieve exact locations of major river dams and waterfalls (keywords: St. Lawrence, Niagara, Bosporus, Gibraltar, ....).
  • Build joins between VMap0/1 urban areas and GeoNames populated places (via names and geographic vincinity).

Phase 2

  • Generation of static per-country and per-region shapefiles; distribution via HTTP and via geotorrent.org.

Phase 3

  • Design and implement a storage-/data-model for road data that is capable of serving the needs of OpenStreetMap while remaining conformant to OGC-standards. Merge the ideas explained in the OSM New Data Model paper as well as Schuler's OSM on PostGIS initiative - in other words: Try "squaring the circle" ;-)
  • Now that OpenStreetMap has (also) imported TIGER, convince OSM to imcorporate an equivalent of VMap0 at places that are currently not covered, finally to create a global road network of maximum detail and accuracy.
  • Merge landcover- and stream-layers from VMap1, TIGER/OSM and Landsat7-classification into the foundation of VMap0 to create a global landcover dataset of maximum detail and accuracy.

Sequence undefined

  • Add proper metadata according to OSGeo metadata recommendation/standard.
  • Find suitable desktop client to FeatureServer.
  • Add polygon data that's been automagically retrieved from Landsat7 images at the World Custom Scenery Project - procedure available.
  • Add polygon data that's been manually digitized from Landsat7 images.

What you can do

  • Help is very much appreciated to add a reasonable colour schema. Work is currently on its way to convert the map display over to the CORINE colour values but this schema doesn't cover all our needs.

DONE

  • Generation of shapefiles on-the-fly for user-defined region and layer(s). Use 'pgsql2shp' and consider the 2 GByte-per-file limit: Landcover Database Shapefile Download
  • Rebuild TIGER layers from 2006se release according to these instructions - mostly finished.
  • Limit the "viewing angle" of WMS/WFS-services in order to save the DB-servers' life .... Solved by setting MAXSCALE for some layers.

Old status

Blue Marble NG

  • Mapserver as WMS

Status

SRTM

Status

Landsat-7

Status

  • Waiting for disk space to finish unpacking the raw data on zuluviz.sdsu.edu
  • Interim plan is to write a simple WMS wrapper script that generates a GDAL VRT to assemble composites on the fly

Seamless imagery

Status

WMS + WFS + TileCache services

Many layers (see list above) are served as WMS/WFS at http://mapserver.flightgear.org 

   WMS: http://mapserver.flightgear.org/ms?Service=WMS&Version=1.1.1&request=GetCapabilities
   
   WFS: http://mapserver.flightgear.org/ms?Service=WFS&Version=1.0.0&request=GetCapabilities
   
   TileCache: http://mapserver.flightgear.org/tc (EPSG:900913 !!)

TelaScience / OSGeo / FlightGear Landcover Database Shapefile Download

See http://mapserver.flightgear.org/download.psp

City of Vienna, Austria

OGD Wien offers data under CC-BY. This data can be imported to OSM. Supported interface standard include:

  • WMS
  • WFS GML
  • JSON
  • KML geplant

Contact on IRC at #ogdwien.

Metadata

See Also

Retrieved from "http://wiki.osgeo.org/w/index.php?title=Geodata_Repository&oldid=126433"