Difference between revisions of "Geodata Metadata Requirements"

From OSGeo
Jump to navigation Jump to search
 
(66 intermediate revisions by 8 users not shown)
Line 12: Line 12:
 
* [[Simple Catalog Interface]]  
 
* [[Simple Catalog Interface]]  
 
* [[Geometa Engine]]
 
* [[Geometa Engine]]
 +
* [[ebRIM]]
  
 
= Draft Metadata model =
 
= Draft Metadata model =
 +
 +
Attempting to abstract the minimal metadata model below ([[#Information model for metadata exchange|DClite4G]]) out into [[Geodata Metadata Model]] (see there), which serves for a metadata management tool. This is what OSGeo Geodata Committee participants have identified as their core needs for metadata.
  
 
+
It was generated from an [http://frot.org/osgeo/geometadata.rdf RDF model]. This picks an arbitrary namespace for an OWL schema that maps to most, if not all, of the FGDC mandatory properties and provides some extra ones.  
[http://frot.org/osgeo/geometadata.png Graph illustrating a basic metadata model] generated from an [http://frot.org/osgeo/geometadata.rdf RDF model] of what OSGeo Geodata Committee participants have identified as their core needs for metadata.
 
 
 
This picks an arbitrary namespace for an OWL schema that maps to most, if not all, of the FGDC mandatory properties and provides some extra ones.  
 
 
 
[[Image:Graph0.png]]
 
  
 
== Data Set ==
 
== Data Set ==
  
=== title ===
+
This is an draft to define a metadata information model for a OSGeo catalogue. [[Geodata_Metadata_Requirements#Information_model_for_metadata_exchange|Below]] there is another minimal metadata information model targeted for metadata exchange and harvesting (name proposed: DClite4G).  
 
 
Title of the data set. Corresponds to Dublin Core '''title'''
 
 
 
=== description ===
 
 
 
Text description of the data set. Corresponds to Dublin Core '''description''' element.
 
 
 
=== originator ===
 
 
 
== Person ==
 
 
 
A person responsible for publication of the data set - name and contact email address. These properties are well-defined in the [http://xmlns.com/foaf/0.1/ FOAF] vocabulary.
 
 
 
== Organization ==
 
 
 
A organization responsible for publication of the data set - name and contact email address. These properties are well-defined in the [http://xmlns.com/foaf/0.1/ FOAF] vocabulary.
 
 
 
== Spatial Data Organization ==
 
 
 
Vector, Raster or Point data, as described in FGDC.
 
(cf http://biology.usgs.gov/fgdc.metadata/version2/sdorg.htm )
 
 
 
=== datasource ===
 
 
 
URL from which the data can be downloaded via different protocols
 
 
 
==== WFS ====
 
 
 
For Vector data in GML
 
 
 
==== WMS ====
 
 
 
For Raster data described in GML
 
 
 
==== File at HTTP URL ====
 
 
 
==== BitTorrent ====
 
 
 
URL of bittorrent .torrent tracker file.
 
 
 
==== Other Web API ====
 
 
 
For example, OpenStreetmap API ( http://wiki.openstreetmap.org/index.php/REST )
 
 
 
== License information ==
 
 
 
Emphasis on public geographic data licenses: PGL, possible LPGL, Public Domain, Creative Commons-type licenses. These can be represented by URLs.
 
 
 
== Publication date ==
 
 
 
Corresponds to Dublin Core '''date''': ISO compliant date of publication.
 
 
 
== timespan ==
 
 
 
== Time Period ==
 
 
 
=== start date and end date ===
 
 
 
=== single date ===
 
 
 
=== extents ===
 
 
 
== Spatial Domain ==
 
 
 
A lot of this can be inferred either using GDAL/OGR or collected from a WMS/WFS GetCapabilities. It would be nice to bypass human error on collecting this kind of metadata.
 
 
 
=== bounding coordinates ===
 
 
 
FGDC specifies north, east, west, south bounding co-ordinates.
 
It doesn't specify a projection in which these should be described.
 
For reasons of simplicity it could make sense to require these be in WGS84 (EPSG:4236) - for the same reasons [http://georss.org/ GeoRSS] decided to mandate WGS84, rather than complicate matters by dictating that people also specify an SRS.
 
 
 
=== Projection (Raster, Vector, Coverage) ===
 
 
 
Original projection of the data (reference to an ?)
 
 
 
Horizontal and vertical datum;
 
 
 
Horizontal and vertical units.
 
 
 
=== Resolution (Raster,Coverage) ===
 
 
 
(property of DataSet)
 
 
 
e.g. map units per pixel where map units are defined by SRS
 
 
 
can be different in horizontal / vertical axes e.g. non square pixels
 
 
 
=== Colour Depth (Raster) ===
 
 
 
8/16/24 bit etc - this is useful rather than required
 
 
 
=== Transparency (Raster) ===
 
 
 
=== Scale (Vector) ===
 
 
 
Map scale at which vectors are considered accurate
 
 
 
Quantified as a fractional/dimensionless number - 'inches per inch' - on a scale between 1 and 0 - or inverse scale such as 1:50000 - and we would want to store this in a consistent way.
 
 
 
== Layers ==
 
 
 
DataSet has multiple Layers
 
 
 
=== Name ===
 
 
 
=== Description ===
 
 
 
=== Extent ===
 
 
 
can be non-rectangular
 
 
 
=== Scale Hinting ===
 
 
 
minscale / maxscale - cf resolution and scale - are these actually properties of layers and not really of data sets? (eg data set contains multiple layers - will they be in any way likely to contain different scale properties?)
 
 
 
=== Optional extra properties ===
 
  
== Taxonomy/Ontology ==
+
* Title: Title of the data set. Corresponds to Dublin Core '''title'''
 
+
* Description: Text description of the data set. Corresponds to Dublin Core '''description''' element.
Currently undecided; would be good to refer this to current well-known thesauri for data themes.
+
* Originator
 +
** Person: A person responsible for publication of the data set - name and contact email address. These properties are well-defined in the [http://xmlns.com/foaf/0.1/ FOAF] vocabulary.
 +
** Organization: A organization responsible for publication of the data set - name and contact email address. These properties are well-defined in the [http://xmlns.com/foaf/0.1/ FOAF] vocabulary.
 +
* Spatial Data Organization: Vector, Raster or Point data, as described in FGDC. (cf http://biology.usgs.gov/fgdc.metadata/version2/sdorg.htm )
 +
* Datasource: URL from which the data can be downloaded via different protocols
 +
** WFS: For Vector data in GML
 +
** File at HTTP URL: For Raster data described in GML
 +
** BitTorrent: URL of bittorrent .torrent tracker file.
 +
** Other Web API: For example, OpenStreetmap API ( http://wiki.openstreetmap.org/index.php/REST )
 +
* License information: Emphasis on public geographic data licenses: PGL, possible LPGL, Public Domain, Creative Commons-type licenses. These can be represented by URLs.
 +
* Publication date: Corresponds to Dublin Core '''date''': ISO compliant date of publication.
 +
* Timespan
 +
* Time Period
 +
** start date and end date
 +
** single date
 +
** Extents
 +
* Spatial Domain: A lot of this can be inferred either using GDAL/OGR or collected from a WMS/WFS GetCapabilities. It would be nice to bypass human error on collecting this kind of metadata.
 +
** bounding coordinates: FGDC specifies north, east, west, south bounding co-ordinates. It doesn't specify a projection in which these should be described. For reasons of simplicity it could make sense to require these be in WGS84 (EPSG:4236) - for the same reasons [http://georss.org/ GeoRSS] decided to mandate WGS84, rather than complicate matters by dictating that people also specify an SRS.
 +
** Projection (Raster, Vector, Coverage): Original projection of the data (reference to an ?)
 +
** Horizontal and vertical datum;
 +
** Horizontal and vertical units.
 +
** Resolution (Raster,Coverage): (property of DataSet). e.g. map units per pixel where map units are defined by SRS. can be different in horizontal / vertical axes e.g. non square pixels
 +
** Colour Depth (Raster): 8/16/24 bit etc - this is useful rather than required
 +
** Transparency (Raster)
 +
** Scale (Vector):Map scale at which vectors are considered accurate. Quantified as a fractional/dimensionless number - 'inches per inch' - on a scale between 1 and 0 - or inverse scale such as 1:50000 - and we would want to store this in a consistent way.
 +
* Layers: DataSet has multiple Layers
 +
** Name
 +
** Description
 +
** Extent: can be non-rectangular
 +
** Scale Hinting: minscale / maxscale - cf resolution and scale - are these actually properties of layers and not really of data sets? (eg data set contains multiple layers - will they be in any way likely to contain different scale properties?)
 +
** Optional extra properties
 +
* Taxonomy/Ontology: Currently undecided; would be good to refer this to current well-known thesauri for data themes.
  
 
= Discovery =  
 
= Discovery =  
Line 158: Line 70:
 
* ability to harvest and classify public and private resources
 
* ability to harvest and classify public and private resources
 
* ability to establish and maintain user/group/role based authentication
 
* ability to establish and maintain user/group/role based authentication
* ability to provide a RESTful authentication mechansim
+
* ability to provide a [[RESTful]] authentication mechansim
  
 
Find:
 
Find:
Line 167: Line 79:
 
Bind:
 
Bind:
 
* ability to perform discovery operations with spatial, aspatial and temporal predicates
 
* ability to perform discovery operations with spatial, aspatial and temporal predicates
* ability to provide a RESTful request API
+
* ability to provide a [[RESTful]] request API
 
* ability to provide responses in XML
 
* ability to provide responses in XML
 
* ability to expose resource/service metadata in a manner which facilitates dynamic connection to a resource/service
 
* ability to expose resource/service metadata in a manner which facilitates dynamic connection to a resource/service
  
=== Information model for metadata exchange ===
+
== Webcrawlers perspective or: How to boost your geodata? ==
 +
 
 +
Geoinformation content needs to be published and get disseminated somehow before it is being found by users. Following are some thoughts to help discovery/search services/brokers and their webcrawlers/harvesters/aggregators to do a better job. So the main question here is: What can content owners do to promote their information. It's mainly registration, declaration and citation. -- [[User:Sfkeller|Stefan]] 11:25, 2 October 2006 (CEST) (http://tinyurl.com/ghhb2)
 +
 
 +
* Self-registration through content provider
 +
** submit URL (like DNS)
 +
** Do a 'ping' (like RSS)
 +
** register through UDDI (if SOAP is 'unavoidable'...)
 +
* Self-declaration:
 +
** Machine-readable only ('invisible'):
 +
*** (Geo) Sitemaps Protocol including GeoRSS and [http://en.wikipedia.org/wiki/Robots_Exclusion_Standard robots.txt] (see sitemaps.org)
 +
*** GeoRSS feed of feeds which contains GeoRSS encoded metadata records ('RSS of RSS')
 +
*** OPML (natural to RSS, but more complicated than just 'RSS of RSS')
 +
*** HTML link: <nowiki><link rel="alternate" type="application/rss+xml" title="RSS" href="http://rss.news.yahoo.com/rss/tech"></nowiki>
 +
** Visible/human readable:
 +
*** Feed icons
 +
*** Chicklets
 +
* Citation through others:
 +
** XML link or relationship inside metadata record (see also the [http://www.openarchives.org/OAI/2.0/guidelines-friends.htm 'friends' container in OAI-PMH] metadata set header)
 +
** HTML link pointing to Webpage (suboptimal)
 +
 
 +
Note: See also "RSS auto-discovery": e.g. '"Blog Optimization", SES San Jose (aug. 2006): 1. Submit your feeds... Or: "Pimp My Blog in 8 Steps" (sept. 2006): (...) 3. Sign up for feedburner.com, (...) 8. Enable auto discovery (feed icon, chicklet).'
 +
 
 +
== Information model for metadata exchange ==
  
This is the http://tinyurl.com/kfkyv model (:->).
+
This is the 'Dublin Core lite for Geo' (DClite4G) model (http://tinyurl.com/kfkyv).
  
 
Some design considerations:
 
Some design considerations:
Line 189: Line 124:
  
  
==== Minimal Information Model ====
+
==== Dublin Core lite for Geo (DClite4G) ====
  
* Attempting to abstract this out into [[Geodata Metadata Model]]
+
The following are notes on initial development.  [[DCLite4G]] contains details of the finalised model which now has an XML namespace at [http://dclite4g.xmlns.com/ dclite4g.xmlns.com], intended for use in RDF documents though also usable in the context of Atom or KML.
* Aligned and rearranged to mode after some discussions on osgeodata-list and geotools-devel-list. Next steps: first consensus on approach/fields, then consensus about which encoding (DC, RDF or?). - [[User:Sfkeller|Stefan]] 08:33, 26 September 2006 (CEST)  
+
 +
* Aligned and rearranged after some discussions on osgeodata-list and geotools-devel-list. Next steps: first consensus on approach/fields, then consensus about which encoding (DC, RDF or GeoRSS?). - [[User:Sfkeller|Stefan]] 08:33, 26 September 2006 (CEST)  
  
[[Image:DClite4G.png|center|Diagram of minimal metadata information model 'Dublin Core lite for Geo' (DClite4G)]]
+
;Figure: Analysis UML class diagram of minimal metadata information model 'Dublin Core lite for Geo' (DClite4G) which consists of a single entity set (green); two entities/instances/records are shown, a 'dataset' (left, grey) and a 'service' (right, red).
 +
[[Image:DClite4G.png|center|Class diagram of 'Dublin Core lite for Geo' (DClite4G)]]
  
 
'''Dublin Core lite for Geo''' (DClite4G) (Mandatory subset of DC elements plus georesource relationships, some XML content exept dc:identifier may be null/empty):
 
'''Dublin Core lite for Geo''' (DClite4G) (Mandatory subset of DC elements plus georesource relationships, some XML content exept dc:identifier may be null/empty):
 
{| border="1" cellspacing="0" cellpadding="10" style="border-collapse:collapse;"  
 
{| border="1" cellspacing="0" cellpadding="10" style="border-collapse:collapse;"  
 
|-
 
|-
|'''Attr. name'''|| '''Cardinality''' || '''Attr. type''' || '''Explanation''' || '''Poss. to autom.?''' || '''Status'''
+
|'''Attr. name'''|| '''Card.''' || '''Attr. type''' || '''Explanation''' || '''Equal to iso19115/128?''' || '''Status'''
 
|-
 
|-
|dc:identifier  || [1]    || URI    || An unambiguous reference to the resource within a given context. Used for GetRecord and in dc:relation elements and subtypes. (GeoTools uses for GetCapabilities requests for type 'service' and 'jdbc:...'; see also [http://www.openarchives.org/OAI/2.0/guidelines-oai-identifier.htm OAI-PMH]) (Clarification needed: Each metadata provider can define its context?) || System generated || Ok?
+
|dc:identifier  || [1]    || URI    || A unique identifier which unambigiously identifies an item within a given context (i.e. a catalogue/repository, a model and a language) used for extracting and referencing metadata. Note that still several records can co-exist with the same dc:identifier either in different sets implementing different metadata models or because of multiple instances of records using different languages in title, etc. Note also that this identifier is not necessary the identifier of a resource. To access the resource associated with a metadata record, one must use dct:hasPart and dclite4g:OnlineResource (see also [http://www.openarchives.org/OAI/2.0/guidelines-oai-identifier.htm OAI-PMH] and GeoTools) || iso19128:Identifier, iso19115: CI_Citation.identifier, else up to (meta-)data manager || Ok?
 
|-
 
|-
|dc:title        || [0..1] || string (XML encoded) || A name given to the resource (regardless e.g. of naming restrictions of file systems). If a layer or file name exist in addition to a title this name can be wrapped in square brackets (tbd! proposal) and concatenated at end of this title string. || GetCapabilities || Ok
+
|dc:title        || [0..1] || string (XML encoded) || A name given to the resource (regardless e.g. of naming restrictions of file systems). If a layer or file name exist in addition to a title this can be appended with // (tbd! proposal) and concatenated at end of this title string. || iso19128:title, iso19115: CI_Citation.title || Ok
 
|-
 
|-
|dc:description  || [0..1] || URI or string (XML encoded)|| A description of the resource. Either a URI reference to a human readable resource or a string. In case of dc:type 'service' (OWS) its the abstract (string) from of GetCapabilities document (GeoTools ServiceInfo has two fields: abstract and description...?) (Why should we use 'dct:abstract' as ISO 19115 does?) || GetCapabilities || Ok
+
|dct:abstract    || [0..1] || URI or string (XML encoded)|| A description of the resource. Either a URI reference to a human readable resource or a string. In case of dc:type 'service' (OWS) its the abstract (string) from of GetCapabilities document (GeoTools ServiceInfo has two fields: abstract and description...?) Subtype of dc:description; we use this as WMS, csw:record and ISO 19115 are using it. || iso19128:abstract, iso19115: MD_DataIdentification.abstract || Ok
 
|-
 
|-
|dc:type        || [0..1] || string (ASCII) || The nature or genre of the content of the resource. 'dataset' or 'service'! else document, text, image, sound, etc. (see DC-Terms). Could be more specific but the more unlikely to agree here: 'geodata', 'vector', 'raster', 'grid'?) or 'data access service' or 'WPS' (= Web Processing Service; services which have no specific georesource they are responsible for) or documents like SLD, WMC etc. (GeoTools distinquishes GeoResourceInfo and ServiceInfo and somehow by querying 'getGeoResource().resolve(GridCoverage.class, null)') || Can be derived; (GetCapabilities) || Ok?
+
|dc:type        || [0..1] || enumeration || The nature or genre of the content of the resource. 'dataset' or 'service' else 'document, 'text', 'image', 'sound', etc. (see [http://dublincore.org/documents/dcmi-type-vocabulary/ DCMI type (controlled) vocabulary]). (Not yet defined for Web Processing Service which have no specific georesource they are responsible for. What about special documents like OGC:SLD, OGC:WMC etc.?) || if WxS/OWS then 'service' else 'dataset' or another media type, up to (meta-)data manager || Ok
 
|-
 
|-
|dc:format      || [0..1] || URI    || A machine readable reference to an xml schema namespace URI, mime type, internet media type which contains the either the schema of the information model of the 'original' resource (dc:type 'dataset') or a description of the service (dc:type 'service') as key-value pairs (so that multiple schemas are possible, like in namespaces syntax) || (GetCapabilities) || Ok
+
|dc:format      || [0..1] || URI    || A machine readable reference to either a [http://www.iana.org/assignments/media-types/ internet media (mime) type (recommended)] or a an xml schema namespace URI linking to an namespace authority. In case of dc:type='service' multiple schemas are to expect: insert them comma-separated? || 19128:AuthorityURL, iso19115: AuthorityURL || Ok
 
|-
 
|-
|dct:spatial    || [0..1] || dcmiBox:Box with CRS || Subtype of dc:coverage. (CRS=WGS84; other possible values offered by dct: Point and TGN) || Boundary/BBox from resource; GetCapabilities || Ok
+
|dct:spatial    || [0..1] || dcmiBox:Box with CRS || Subtype of dc:coverage. (CRS=WGS84; other possible values offered by dct: Point and TGN) || iso19128:EX_BoundingBox, iso19115: EX_BoundingBox || Ok
 
|-
 
|-
 
|dct:modified    || [0..1] || date    || Date of last (published) change of resource (see [http://www.w3.org/TR/NOTE-datetime W3C Encoding rules]). Subtype of dc:date || Timestamp of resource; GetCapabilities || Ok
 
|dct:modified    || [0..1] || date    || Date of last (published) change of resource (see [http://www.w3.org/TR/NOTE-datetime W3C Encoding rules]). Subtype of dc:date || Timestamp of resource; GetCapabilities || Ok
 
|-
 
|-
|dc:subject      || [0..1] || string (ASCII) || A keyword list; e.g. ISO 19115 classification list. (clarification needed: separated by comma and/or semicolon?) || GetCapabilities || Ok?
+
|dc:subject      || [0..1] || string (ASCII) || A keyword list; e.g. ISO 19115 classification list. (clarification needed: separated by comma) || KeywordList, iso19115: MD_TopicCategoryCode || Ok
|-
 
|dclite4g:onLineSrc || [0..*] || URI  || (or use cld:isAccessedVia?) if type 'service' then this is the baseURL to it (similar to iso19115). If dc:type 'dataset'. Either a dc:identifier which points to a metadata record of type 'service' or a full path to e.g. ftp:// host.com/path/filename. Subtype of dc:relation; see [http://dublincore.org/groups/collections/collection-application-profile/2006-08-24/#colcldisAccessedVia here]. || Can be derived || Ok?
 
 
|-
 
|-
|dct:hasPart    || [0..*] || URI     || Only applicable if type 'service'. A dc:identifier which points to a metadata record of dc:type 'dataset' for which it is 'responsible' for. Subtype of dc:relation. || Can be derived || Ok?
+
|dclite4g:onlineResource || [0..*] || URI   || (formerly called onLineSrc) If dc:type='service' then this is the baseURL to it. If dc:type='dataset' it's either a dc:identifier which points to a metadata record of type 'service' or a dataset URL e.g. uri:ftp:// host.com/path/filename. Subtype of dc:relation. Note that in some implementations this seems redundant to dc:identifier but in fact this is the real access point of the resource. || if dc:type='dataset' given by provider, if dc:type='service' iso19128: OnlineResource (can be the same as described in dct:hasPart (dataSetURI) of a service metadata record) || Ok?
 
|-
 
|-
|dc:source      || [0..1] || URI or string (XML encoded) || Description from which the present resource is derived, lineage information. Either an URI reference to a human readable resource or a string. (Note: Server base URLs and file URIs are handled elsewhere) || Up to data owner || Ok
+
|dct:hasPart    || [0..*] || URI     || Either a URI to a dataset or a dc:identifier (which points to a metadata record of dc:type='dataset'). Only applicable if type 'service'; N/A for type 'dataset'. Subtype of dc:relation. Note that in some implementations this seems redundant to dc:identifier but in fact this is the real access point the resource associated with a metadata record. || iso19128: DataURL, iso19115: MD_metadata.dataSetURI || Ok?
|}
 
 
 
 
 
'''Remaining (optional) DC elements''':  
 
{| border="1" cellspacing="0" cellpadding="10" style="border-collapse:collapse;"
 
 
|-
 
|-
|'''Attr. name'''|| '''Cardinality''' || '''Attr. type''' || '''Explanation''' || '''Poss. to autom.?''' || '''Status'''
+
|dc:source      || [0..1] || URI or string (XML encoded) || Description from which the present resource is derived, lineage information. Either an URI reference to a human readable resource or a string. (Note: Server base URLs and file URIs are handled elsewhere) || Up to (meta-)data owner || Ok
 
|-
 
|-
|dc:publisher    || [0..1] || string (XML encoded) || Civic Address: if type service derived from GetCapabilities ('ContactInformation') else flattened KML style (= xAL) (still awaiting consensus in other OGC specs. or GeoRSS) || Up to data owner, GetCapabilities || Ok?
+
|dc:publisher    || [0..1] || string (XML encoded) || Civic Address: if type service derived from GetCapabilities ('ContactInformation') else flattened KML style (= xAL) (still awaiting consensus in other OGC specs. or GeoRSS) || Title element of 19128:Attribution; OrganisationName element of 19115:CI_ResponsibleParty || Ok  
 
|-
 
|-
|dc:language    || [0..1] || enum    || RFC 1766 (ISO 639, followed optionally by country ISO 3166) ||  (could be guessed from title/description?) || Ok?
+
|dc:language    || [0..1] || enumeration || RFC 1766 (ISO 639, followed optionally by country ISO 3166). Max. Cardinality is intentionally 1 which for the sake of simplicity means that translations of a metadata record in different languages is recorded in several metadata instances and not in multiple dc:language, dc:title or dct:abstract occurrences. ||  Up to (meta-)data manager, WxS spec. do foresee this || Ok
 
|-
 
|-
|dc:rights      || [0..1] || URI or string (XML encoded) || License information about the resource. Probably also disclaimers. Either a URI reference to a human readable resource or a string. Machine readable license is an issue though: [http://backend.userland.com/creativeCommonsRssModule This] could be an starting point. || Up to data owner, GetCapabilities || Ok?
+
|dc:rights      || [0..1] || URI or string (XML encoded) || License information link about the resource. Probably also (link to) disclaimers. Either an URI reference to a human readable resource or a string (machine readable licenses is an issue[http://backend.userland.com/creativeCommonsRssModule this] could be an starting point. || Up to (meta-)data manager; iso19115:???|| Ok
 
|-
 
|-
|dc:relation    || [0..*] || URI    || A reference to a related resource. These are some useful subtypes. One usage could be a machine readable reference to other metadata providers in order to let discover other (meta) data providers. See also dc:hasPart and iso19115:onLineSrc above. Note that OAI-PMH has such a relationship called '[http://www.openarchives.org/OAI/2.0/guidelines-friends.htm friends]' but on the metadata collection/set level. || Up to data owner || Ok?
+
|dc:relation    || [0..*] || URI    || A reference to a related resource. DClite4G uses following subtypes: dc:hasPart and iso19115: OnlineResource (see above). If using DC unqualified dc:relation could be used. This could be a machine readable reference to other metadata providers in order to let discover other (meta) data providers. Note that OAI-PMH has such a relationship called '[http://www.openarchives.org/OAI/2.0/guidelines-friends.htm friends]' but on the metadata collection/set level. ------------------------------------------------------------------------------- || Placeholder; should be refined in a specific model; meaning up to (meta-)data owner || Ok?
 
|}
 
|}
  
Legend: 'Poss. to autom.' means Possible to automate
+
Legend: 'Equal to' means possible to derive from iso19128 (= WxS GetCapabilities).
  
 
Remarks:
 
Remarks:
 
* General:
 
* General:
** DC attributes/properties left as they are...: Audience; Contributor; Creator.
+
** DC attributes/properties left as they are: dc:Audience; dc:Contributor; dc:Creator.
** All attributes/properties have at most cardinality 1 except iso19115:onLineSrc and dct:hasPart (and dc:relation from complementary part of dclite4g).
+
** All attributes/properties have at most cardinality 1 except iso19115: OnlineResource and dct:hasPart (and dc:relation from complementary part of DClite4G).
** Depending on the modeling approach, even these elements can become cardinality 1. NOTE that datasets (geodata resources) and services (data access services) in principle have a many-to-many relationship: Here a geodata resource (dc:type dataset) can have many iso19115:onLineSrc elements and a dc:type data_access_service has only one dc:description which can be a GetCapabilities document.
+
** Depending on the modeling approach, even these elements can become cardinality 1. NOTE that datasets (geodata resources) and services (data access services) in principle have a many-to-many relationship: Here a geodata resource (dc:type dataset) can have many iso19115: OnlineResource elements and a dc:type data_access_service has only one dc:description which can be a GetCapabilities document.
** No additinal DC attributes/properties required; few them needed to be specialized (see dct:...); ** See for some general explanations about [http://cicharvest.grainger.uiuc.edu/qualifieddc.asp dc/dct here].
+
** No additinal DC attributes/properties required; few of them need to be specialized (see dct:...);
** Some attributes/properties still need unambiguous meaning (see tbd.).  
+
** See for some general explanations about [http://cicharvest.grainger.uiuc.edu/qualifieddc.asp dc/dct here].
 
** Assume metadata (as opposite to many geodata sets) is always free and open information, like [http://creativecommons.org/licenses/by-sa/2.5/ Creative Commons Share Alike]  
 
** Assume metadata (as opposite to many geodata sets) is always free and open information, like [http://creativecommons.org/licenses/by-sa/2.5/ Creative Commons Share Alike]  
 
** An encoding still has to be discussed (see following example). need schemaLocation in OSGeo!?
 
** An encoding still has to be discussed (see following example). need schemaLocation in OSGeo!?
 
* Details:
 
* Details:
 +
** GetCapabilities adds following attributes (not yet modeled here): Fees, ScaleHint and Style.
 
** Sorted out or highly disputed (non-DC) elements: fees, scalehint, harvestinterval.
 
** Sorted out or highly disputed (non-DC) elements: fees, scalehint, harvestinterval.
 
** dct:modified and dct:spatial can be sync'ed from dataset.
 
** dct:modified and dct:spatial can be sync'ed from dataset.
Line 256: Line 187:
 
** Attribute 'publisher': Carl mentioned such a structure [http://lists.eogeo.org/pipermail/georss/2006-May/000514.html here] which includes StreetAddress, addressee, primaryAddressNumber, streetName, city, state, zipCode, countryCode (like in KML and behind Google geocoding service!?)
 
** Attribute 'publisher': Carl mentioned such a structure [http://lists.eogeo.org/pipermail/georss/2006-May/000514.html here] which includes StreetAddress, addressee, primaryAddressNumber, streetName, city, state, zipCode, countryCode (like in KML and behind Google geocoding service!?)
 
** Keywords is included in attribute 'dc:subject'; I think people have a hard time to agree on an enumerated list (see the success of folksonomy).
 
** Keywords is included in attribute 'dc:subject'; I think people have a hard time to agree on an enumerated list (see the success of folksonomy).
** GetCapabilities adds following attributes (not yet modeled here): Fees, ScaleHint and Style.
+
* Note that OAI-PMH...  
* Note that OAM-PMH...  
 
 
** puts a XML envelope around this metadata and adds a header containing two attributes: 'identifier' to identify an metadata record and 'datestamp' as date of last (published) change of metadata record.
 
** puts a XML envelope around this metadata and adds a header containing two attributes: 'identifier' to identify an metadata record and 'datestamp' as date of last (published) change of metadata record.
 
** requires to define a name for metadata sets. Let's don't care about this yet.
 
** requires to define a name for metadata sets. Let's don't care about this yet.
  
==== Example ====
+
==== Examples ====
 +
 
 +
Some examples of '''DClite4G''' instances/records:
 +
(legend: 'literal' is a constant, // is a comment) (http://tinyurl.com/eaaaj)
 +
 
 +
'''A web mapping service instance example derived from GetCapabilities''': Mapping OGC:WxS GetCapabilities to a service instance. Can also be called a 'data access point' (Note GetCapabilities finally needs to be delivered by some service owner!):
 +
 
 +
  dc:identifier      = identifier of the metadata record (ev. a machine readable URI)
 +
  dc:title          = WxS Service/Title
 +
  dct:abstract      = WxS Service/Abstract
 +
  dc:type            = 'service'
 +
  dc:format          = namespace to OGC:WxS schema.xsd
 +
  dct:spatial        = Root BoundingBox // from Capabilities XML
 +
  dct:modified      = timestamp // e.g. from HTTP header or updateSequence
 +
  dc:subject        = WxS Service/KeywordList
 +
  dclite4g:onlineResource = baseURI of WxS // seems redundant to id but is the real link to the service
 +
  dct:hasPart        = a dc:identifier which points to each Layer element
 +
  dct:hasPart        = another dc:identifier, etc.
 +
  dc:source          = null // N/A. Note: Not meant as an onlineResource
 +
  dc:publisher      = WxS Service ContactInformation/Organization
 +
  dc:language        = maybe HTTP header for lang (ISO3166 code), soon supported by WMS
 +
  dc:rights          = WxS Services Fees/AccessConstraints
 +
  dc:relation        = definition up to metadata provider
 +
 
 +
'''A dataset example derived from GetCapabilities''': Mapping OGC:WxS GetCapabilities to a dataset/georesource/data access point (Note: There is no hasPart relationship in datasets):
 +
 
 +
  dc:identifier      = identifier of the metadata record (ev. a machine readable URI)
 +
  dc:title          = WxS Layer/Title
 +
  dct:abstract      = WxS Layer/Abstract
 +
  dc:type            = 'dataset'
 +
  dc:format          = namespace to format
 +
  dct:spatial        = BoundingBox from Layer/BoundingBox
 +
  dct:modified      = timestamp from HTTP header maybe or updateSequence
 +
  dc:subject        = Layer/KeywordList
 +
  dclite4g:onlineResource = a baseURI of WxS (GetMap/GetFeatures) // seems redundant to id. but is the real 'data access point'
 +
  dclite4g:onlineResource = another baseURI of WxS // another dataset binding
 +
  dc:source          = null // Note: Not meant as an onlineResource
 +
  dc:publisher      = from Service if available?
 +
  dc:language        = from Service if available?
 +
  dc:rights          = from Service if available?
 +
  dc:relation        = definition up to metadata provider
 +
 
 +
'''A dataset example delivered by a dataset owner''':
 +
 
 +
  dc:identifier      = a data access point metadata defined by dataset owners context
 +
  dc:title          = entered by dataset owner
 +
  dct:abstract      = entered by dataset owner
 +
  dc:type            = 'dataset'
 +
  dc:format          = namespace to format entered by dataset owner
 +
  dct:spatial        = BoundingBox from data warehouse
 +
  dct:modified      = file timestamp from data warehouse
 +
  dc:subject        = keyword list entered by dataset owner
 +
  dclite4g:onlineResource = a http URI entered by dataset owner // dataset binding
 +
  dclite4g:onlineResource = a baseURI of a WxS entered by dataset owner // another binding
 +
  dc:source          = entered by dataset owner // Note: Not online resource
 +
  dc:publisher      = entered by dataset owner
 +
  dc:language        = entered by dataset owner (or derived?)
 +
  dc:rights          = entered by dataset owner
 +
  dc:relation        = definition up to metadata provider
 +
 
 +
'''A dataset example with OAI-PMH XML encoding in DClite4G format''':
 +
 
 
Notes:  
 
Notes:  
 
* Example values are only for explanation purposes and purely fictive.
 
* Example values are only for explanation purposes and purely fictive.
Line 268: Line 259:
 
* Took 'dclite4g' as envelope name.
 
* Took 'dclite4g' as envelope name.
  
 +
<pre>
 
   <dclite4g:qualifieddc  
 
   <dclite4g:qualifieddc  
 
     xmlns:dclite4g="http://www.osgeo.org/schemas/dclite4g/0.01"  
 
     xmlns:dclite4g="http://www.osgeo.org/schemas/dclite4g/0.01"  
Line 275: Line 267:
 
     xsi:schemaLocation="http://www.osgeo.org/schemas/dclite4g/ dclite4g.xsd">
 
     xsi:schemaLocation="http://www.osgeo.org/schemas/dclite4g/ dclite4g.xsd">
  
     <dc:identifier>f264-77d2-09ce-aa39-f0f0</dc:identifier>
+
     <dc:identifier>www.osgeo.org/geodata/:f264-77d2-09ce-aa39-f0f0</dc:identifier>
 
     <dc:title>National Elevation Mapping Service for Texas</dc:title>
 
     <dc:title>National Elevation Mapping Service for Texas</dc:title>
     <dc:description>Elevation data collected for the National Elevation  
+
     <dct:abstract>Elevation data collected for the National Elevation  
    Dataset (NED).</dc:description>
+
                  Dataset (NED).</dct:abstract>
 
     <dc:type>dataset</dc:type>
 
     <dc:type>dataset</dc:type>
     <dc:format>...uri to the schema of the information model (xsd, realxng, schematron, ili, ...)</dc:format>
+
     <dc:format>...uri to the schema of the information model  
 +
              (xsd, realxng, schematron, ili, ...)</dc:format>
 
     <dct:spatial>
 
     <dct:spatial>
 
       <Box projection="EPSG:4326" name="Geographic">
 
       <Box projection="EPSG:4326" name="Geographic">
Line 291: Line 284:
 
     <dct:modified>2004-03-01</dct:modified>
 
     <dct:modified>2004-03-01</dct:modified>
 
     <dc:subject>Elevation, Hypsography, and Contours</dc:subject>
 
     <dc:subject>Elevation, Hypsography, and Contours</dc:subject>
     <dclite4g:onLineSrc>uri:http://www.osgeo.org/geodata/ned_grid_georss.xml</dclite4g:onLineSrc>
+
     <dclite4g:onlineResource>uri:http://www.osgeo.org/geodata/ned_grid_georss.xml</dclite4g:onlineResource>
     <dclite4g:onLineSrc>uri:http://www.osgeo.org/services/wms/</dclite4g:onLineSrc>
+
     <dclite4g:onlineResource>uri:http://www.osgeo.org/services/wms/</dclite4g:onlineResource>
     <dclite4g:onLineSrc>uri:http://www.osgeo.org/geodata/ned_grid.shp</dclite4g:onLineSrc>
+
     <dclite4g:onlineResource>uri:http://www.osgeo.org/geodata/ned_grid.shp</dclite4g:onlineResource>
 
     <dc:source>Lineage: Based on 30m horizontal and 15m vertical accuracy.</dc:source>
 
     <dc:source>Lineage: Based on 30m horizontal and 15m vertical accuracy.</dc:source>
  
Line 300: Line 293:
 
     <dc:language>en</dc:language>
 
     <dc:language>en</dc:language>
 
     <dc:rights>uri:http://www.usgs.gov/pubprod/</dc:rights>
 
     <dc:rights>uri:http://www.usgs.gov/pubprod/</dc:rights>
    <dc:relation>g264-77d2-09ce-aa39-g0g0</dc:relation>
 
 
   </dclite4g:qualifieddc>
 
   </dclite4g:qualifieddc>
 +
</pre>
 +
 +
'''Same record example as before but with unqualified DC encoding''':
 +
 +
Note that this unqualified DC record can be seen as a mapping from DClite4G by using it's well defined semantics and content 'encoding'. See explanations above to understand the semantics of these DC-elements:
 +
 +
  ...
 +
  <dc:identifier>www.osgeo.org/geodata/:f264-77d2-09ce-aa39-f0f0</dc:identifier>
 +
  <dc:title>National Elevation Mapping Service for Texas</dc:title>
 +
  <dc:description>Elevation data collected for the National Elevation
 +
                  Dataset (NED).</dc:description>
 +
  <dc:type>dataset</dc:type>
 +
  <dc:format>...uri to the schema of the information model
 +
            (xsd, realxng, schematron, ili, ...)</dc:format>
 +
  <dc:coverage>34.353 -96.223 28.229 -108.44</dc:coverage>
 +
  <dc:date>2004-03-01</dc:date>
 +
  <dc:subject>Elevation, Hypsography, and Contours</dc:subject>
 +
  <dc:relation>uri:http://www.osgeo.org/services/wms/</dc:relation>
 +
  <dc:source>Lineage: Based on 30m horizontal and 15m vertical accuracy.</dc:source>
 +
  ...
  
 
== Other Relevant Info ==
 
== Other Relevant Info ==
Line 307: Line 319:
 
* [http://www.gis.hsr.ch/wiki/OSGeodata OSGeodata on GISpunkt Wiki] - These pages are about the search of an open, lean and mean "protocol for the incremental exchange of metadata about geographic resources between systems". Profiled specifications like WFS or OAI-PMH are currently on our short list. Delving into 'Open Archives Initiative Protocol for Metadata Harvesting' (OAI-PMH) is strongly encouraged. It's a low barrier interoperability specification based around metadata harvesting model, it's stable (subsequent revisions are backwards compatible) and uses unqualified Dublin Core as default metadata information model; there exist open source tools (like OAICat) and it has been adopted among others by Google and Yahoo! but it's not a search protocol.
 
* [http://www.gis.hsr.ch/wiki/OSGeodata OSGeodata on GISpunkt Wiki] - These pages are about the search of an open, lean and mean "protocol for the incremental exchange of metadata about geographic resources between systems". Profiled specifications like WFS or OAI-PMH are currently on our short list. Delving into 'Open Archives Initiative Protocol for Metadata Harvesting' (OAI-PMH) is strongly encouraged. It's a low barrier interoperability specification based around metadata harvesting model, it's stable (subsequent revisions are backwards compatible) and uses unqualified Dublin Core as default metadata information model; there exist open source tools (like OAICat) and it has been adopted among others by Google and Yahoo! but it's not a search protocol.
 
* See here a [http://www.gis.hsr.ch/wiki/OAI-PMH comparison between CSW, WFS and OAI-PMH].
 
* See here a [http://www.gis.hsr.ch/wiki/OAI-PMH comparison between CSW, WFS and OAI-PMH].
 +
 +
== Guidelines for a minimal OAI-PMH implementation ==
 +
 +
OAI-PMH means ''Open Archives Initiative Protocol for Metadata Harvesting''. For an introduction to OAI-PMH 2 see [http://www.gis.hsr.ch/wiki/OAI-PMH here].
 +
 +
This is a draft implementation guideline for a minimal OAI-PMH implementation for geospatial resources which contains following five steps:
 +
 +
* '''1. Join the community''': Register to the 'OAI Implementers list' at the [http://www.openarchives.org/ OAI Homepage]
 +
* '''2. Read the spec.''': [http://www.openarchives.org/OAI/openarchivesprotocol.html The Harvesting Protocol (version 2.0) specification] together with [http://www.openarchives.org/OAI/2.0/guidelines.htm Implementation Guidelines]
 +
* '''3. Look at existing [http://www.openarchives.org/tools/tools.html tools]'''.
 +
* '''4. Implement protocol with DC model following these recommendations and test it''': [http://www.purl.org/NET/oai_explorer The official OAI Repository Explorer].
 +
* '''5. Extend it''' later eventually with specific metadata models like ISO19139 or DClite4G and respective XML formats.
 +
 +
Following are more specific guidelines for a minimal OAI-PMH implementation of a so called 'data provider' using only the mandatory 'unqualified' Dublin Core (DC):
 +
 +
* Only '''three''' operations (verbs) are needed: Identify, ListMetadataFormats and ListRecords.
 +
* Following operations are not required (initially): ListIdentifiers, ListSets, GetRecord.
 +
* No incremental harvesting (resumption process for ListXxx operations with more than 1000 records)
 +
* No compression as defined in the OAI-PMH spec. (compression at lower http level still possible)
 +
* Date granularity may be 'day' not seconds (YYYY-MM-DD)
 +
* Keeping track of deleted record may not be supported (deletedRecord=no)
 +
* Mandatory DC supported as data model is sufficient for a start but with specific semantics (e.g. coverage, relation) (see also example below):
 +
** dc:description contains dct:abstract
 +
** dc:coverage contains bounding box encoding as defined in http://georss.org/simple.html#Box
 +
** dc:date means in fact dct:modified
 +
** dc:relation is filled in with dclite4g:onLineSrc. If dc:type='service' dct:hasPart can be derived from GetCapabilities.
 +
 +
=== OAI implementations ===
 +
 +
OAI with Dublin Core:
 +
* 'Geo-Metadatabase' (GMDB), open source (GPL, PHP)
 +
* GeoShop, infoGrips GmbH, Zurich, Switzerland (C, Java)
 +
* [http://www.osgeo.org/geonetwork GeoNetwork opensource] (GPL, Java)
  
 
= References =
 
= References =
Line 322: Line 367:
 
* [http://en.wikipedia.org/wiki/Web_Feature_Service WFS]
 
* [http://en.wikipedia.org/wiki/Web_Feature_Service WFS]
  
* [http://geoapi.sourceforge.net/stable/site/index.html GeoAPI] contains an implementation of ISO 19115
+
* [http://geoapi.sourceforge.net/ GeoAPI] contains an implementation of ISO 19115
  
 
== RDF ==
 
== RDF ==
Line 364: Line 409:
 
* [[Geodata Repository]]
 
* [[Geodata Repository]]
 
* [[Geodata processing at telascience.org]]
 
* [[Geodata processing at telascience.org]]
 +
* [http://lists.osgeo.org/mailman/listinfo/geodata OSGeo-Geodata Mailing list] (discussion archives, please join the list)
  
 
= Comments =
 
= Comments =
  
 
* I would like to propose an additional element for the metadata model--data source (or lineage).  If the data is derived from some other data, we should be able to backtrack and look at its parent/s.  "Lineage" is a conditional element in FGDC but I think it's important enough that we should include it in our model. I suppose this can also be included in the Description but wouldn't it be nice to have this included as a required element? This is useful when checking for errors/consistency.  -Perry
 
* I would like to propose an additional element for the metadata model--data source (or lineage).  If the data is derived from some other data, we should be able to backtrack and look at its parent/s.  "Lineage" is a conditional element in FGDC but I think it's important enough that we should include it in our model. I suppose this can also be included in the Description but wouldn't it be nice to have this included as a required element? This is useful when checking for errors/consistency.  -Perry
 +
 +
* I find it difficult to believe that these "requirements" can be useful without switching their basis from FGDC to ISO-19115.  I have posted more extensive comments as Discussion.  Please read and comment. -- Bruce
 +
 +
[[Category:Public Geospatial Data Committee]]
 +
 +
[[Category:Standards]]

Latest revision as of 14:18, 5 July 2009

One goal of the Public Geospatial Data Project is to offer, in the future, a repository of reusable public geographic data that can support open source geospatial software projects, both inside and outside the foundation.

One big requirement for a potential Geodata Repository is that there be a well-defined baseline for metadata. This can be seen as a quality assurance effort - data won't be accepted without a certain amount of metadata.

The Federal Geographic Data Committee metadata standard emphasises conformance, but doesn't emphasise exchangeability / reusability. FDGC is standard for "Spatial Data Infrastucture" efforts, but doesn't have much of a "geospatial web" orientation.

There are some properties in addition to FGDC which it would be really useful to have - different distribution channels like WFS, bittorrent which have come into existence since FGDC was originally defined. For many elements, FGDC asks for full-text descriptions. More structure in descriptions would help with automating discovery or re-use.

This is perpetual work in progress. See also:

Draft Metadata model

Attempting to abstract the minimal metadata model below (DClite4G) out into Geodata Metadata Model (see there), which serves for a metadata management tool. This is what OSGeo Geodata Committee participants have identified as their core needs for metadata.

It was generated from an RDF model. This picks an arbitrary namespace for an OWL schema that maps to most, if not all, of the FGDC mandatory properties and provides some extra ones.

Data Set

This is an draft to define a metadata information model for a OSGeo catalogue. Below there is another minimal metadata information model targeted for metadata exchange and harvesting (name proposed: DClite4G).

  • Title: Title of the data set. Corresponds to Dublin Core title
  • Description: Text description of the data set. Corresponds to Dublin Core description element.
  • Originator
    • Person: A person responsible for publication of the data set - name and contact email address. These properties are well-defined in the FOAF vocabulary.
    • Organization: A organization responsible for publication of the data set - name and contact email address. These properties are well-defined in the FOAF vocabulary.
  • Spatial Data Organization: Vector, Raster or Point data, as described in FGDC. (cf http://biology.usgs.gov/fgdc.metadata/version2/sdorg.htm )
  • Datasource: URL from which the data can be downloaded via different protocols
    • WFS: For Vector data in GML
    • File at HTTP URL: For Raster data described in GML
    • BitTorrent: URL of bittorrent .torrent tracker file.
    • Other Web API: For example, OpenStreetmap API ( http://wiki.openstreetmap.org/index.php/REST )
  • License information: Emphasis on public geographic data licenses: PGL, possible LPGL, Public Domain, Creative Commons-type licenses. These can be represented by URLs.
  • Publication date: Corresponds to Dublin Core date: ISO compliant date of publication.
  • Timespan
  • Time Period
    • start date and end date
    • single date
    • Extents
  • Spatial Domain: A lot of this can be inferred either using GDAL/OGR or collected from a WMS/WFS GetCapabilities. It would be nice to bypass human error on collecting this kind of metadata.
    • bounding coordinates: FGDC specifies north, east, west, south bounding co-ordinates. It doesn't specify a projection in which these should be described. For reasons of simplicity it could make sense to require these be in WGS84 (EPSG:4236) - for the same reasons GeoRSS decided to mandate WGS84, rather than complicate matters by dictating that people also specify an SRS.
    • Projection (Raster, Vector, Coverage): Original projection of the data (reference to an ?)
    • Horizontal and vertical datum;
    • Horizontal and vertical units.
    • Resolution (Raster,Coverage): (property of DataSet). e.g. map units per pixel where map units are defined by SRS. can be different in horizontal / vertical axes e.g. non square pixels
    • Colour Depth (Raster): 8/16/24 bit etc - this is useful rather than required
    • Transparency (Raster)
    • Scale (Vector):Map scale at which vectors are considered accurate. Quantified as a fractional/dimensionless number - 'inches per inch' - on a scale between 1 and 0 - or inverse scale such as 1:50000 - and we would want to store this in a consistent way.
  • Layers: DataSet has multiple Layers
    • Name
    • Description
    • Extent: can be non-rectangular
    • Scale Hinting: minscale / maxscale - cf resolution and scale - are these actually properties of layers and not really of data sets? (eg data set contains multiple layers - will they be in any way likely to contain different scale properties?)
    • Optional extra properties
  • Taxonomy/Ontology: Currently undecided; would be good to refer this to current well-known thesauri for data themes.

Discovery

Requirements

A discovery resource is essential to expose resultant metadata as per this document. Below are requirements:

Publish:

  • ability to publish/register a web service
  • ability to publish/register a static resource
  • ability to harvest and classify public and private resources
  • ability to establish and maintain user/group/role based authentication
  • ability to provide a RESTful authentication mechansim

Find:

  • ability to discover the existence of a web service
  • ability to discover the existence of a resource which is available via web services (i.e WMS layer, WFS feature type)
  • ability to discover the existence of a static resource (dataset, document, etc.)

Bind:

  • ability to perform discovery operations with spatial, aspatial and temporal predicates
  • ability to provide a RESTful request API
  • ability to provide responses in XML
  • ability to expose resource/service metadata in a manner which facilitates dynamic connection to a resource/service

Webcrawlers perspective or: How to boost your geodata?

Geoinformation content needs to be published and get disseminated somehow before it is being found by users. Following are some thoughts to help discovery/search services/brokers and their webcrawlers/harvesters/aggregators to do a better job. So the main question here is: What can content owners do to promote their information. It's mainly registration, declaration and citation. -- Stefan 11:25, 2 October 2006 (CEST) (http://tinyurl.com/ghhb2)

  • Self-registration through content provider
    • submit URL (like DNS)
    • Do a 'ping' (like RSS)
    • register through UDDI (if SOAP is 'unavoidable'...)
  • Self-declaration:
    • Machine-readable only ('invisible'):
      • (Geo) Sitemaps Protocol including GeoRSS and robots.txt (see sitemaps.org)
      • GeoRSS feed of feeds which contains GeoRSS encoded metadata records ('RSS of RSS')
      • OPML (natural to RSS, but more complicated than just 'RSS of RSS')
      • HTML link: <link rel="alternate" type="application/rss+xml" title="RSS" href="http://rss.news.yahoo.com/rss/tech">
    • Visible/human readable:
      • Feed icons
      • Chicklets
  • Citation through others:
    • XML link or relationship inside metadata record (see also the 'friends' container in OAI-PMH metadata set header)
    • HTML link pointing to Webpage (suboptimal)

Note: See also "RSS auto-discovery": e.g. '"Blog Optimization", SES San Jose (aug. 2006): 1. Submit your feeds... Or: "Pimp My Blog in 8 Steps" (sept. 2006): (...) 3. Sign up for feedburner.com, (...) 8. Enable auto discovery (feed icon, chicklet).'

Information model for metadata exchange

This is the 'Dublin Core lite for Geo' (DClite4G) model (http://tinyurl.com/kfkyv).

Some design considerations:

  • This is a minimal metadata information model regarding to a metadata exchange protocol for harvesting (e.g. no filter nor GML implementation needed) and according to the ideas about a Simple Catalog Interface/protocol.
  • Based on Dublin Core (DC) and Catalogue Services Specification 2.0.1, OGC 04-021r3, p.22.
  • Dublin Core need refined semantics of some properties/attributes.
  • Have had hard times with the abundance use of namespaces. This is because DC specs and other XML 'practices' specialize properties/attribute types instead of specializing whole classes.
  • All properties/attributes have cardinality [0..1] except for identifiers (which are mandatory) and for those attributes which are really needed (as unbounded) for automation!
  • Take all information one can in an automated manner, e.g. from data set resource.

Details:

  • Services are included in attribute 'format' in the sense that WMS, etc. are just protocol bindings to geodata. Real well known services on it's own like filter or label placement services have a place there too. They could be still detected by challenging them with GetCatabilites (taken from OWS/WxS).
  • Indicating of quality of service could be a nice task for search service provider; no need to add it as attribute
  • Relationships between features is part of schema metadata: How to handle this...?


Dublin Core lite for Geo (DClite4G)

The following are notes on initial development. DCLite4G contains details of the finalised model which now has an XML namespace at dclite4g.xmlns.com, intended for use in RDF documents though also usable in the context of Atom or KML.

  • Aligned and rearranged after some discussions on osgeodata-list and geotools-devel-list. Next steps: first consensus on approach/fields, then consensus about which encoding (DC, RDF or GeoRSS?). - Stefan 08:33, 26 September 2006 (CEST)
Figure
Analysis UML class diagram of minimal metadata information model 'Dublin Core lite for Geo' (DClite4G) which consists of a single entity set (green); two entities/instances/records are shown, a 'dataset' (left, grey) and a 'service' (right, red).
Class diagram of 'Dublin Core lite for Geo' (DClite4G)

Dublin Core lite for Geo (DClite4G) (Mandatory subset of DC elements plus georesource relationships, some XML content exept dc:identifier may be null/empty):

Attr. name Card. Attr. type Explanation Equal to iso19115/128? Status
dc:identifier [1] URI A unique identifier which unambigiously identifies an item within a given context (i.e. a catalogue/repository, a model and a language) used for extracting and referencing metadata. Note that still several records can co-exist with the same dc:identifier either in different sets implementing different metadata models or because of multiple instances of records using different languages in title, etc. Note also that this identifier is not necessary the identifier of a resource. To access the resource associated with a metadata record, one must use dct:hasPart and dclite4g:OnlineResource (see also OAI-PMH and GeoTools) iso19128:Identifier, iso19115: CI_Citation.identifier, else up to (meta-)data manager Ok?
dc:title [0..1] string (XML encoded) A name given to the resource (regardless e.g. of naming restrictions of file systems). If a layer or file name exist in addition to a title this can be appended with // (tbd! proposal) and concatenated at end of this title string. iso19128:title, iso19115: CI_Citation.title Ok
dct:abstract [0..1] URI or string (XML encoded) A description of the resource. Either a URI reference to a human readable resource or a string. In case of dc:type 'service' (OWS) its the abstract (string) from of GetCapabilities document (GeoTools ServiceInfo has two fields: abstract and description...?) Subtype of dc:description; we use this as WMS, csw:record and ISO 19115 are using it. iso19128:abstract, iso19115: MD_DataIdentification.abstract Ok
dc:type [0..1] enumeration The nature or genre of the content of the resource. 'dataset' or 'service' else 'document, 'text', 'image', 'sound', etc. (see DCMI type (controlled) vocabulary). (Not yet defined for Web Processing Service which have no specific georesource they are responsible for. What about special documents like OGC:SLD, OGC:WMC etc.?) if WxS/OWS then 'service' else 'dataset' or another media type, up to (meta-)data manager Ok
dc:format [0..1] URI A machine readable reference to either a internet media (mime) type (recommended) or a an xml schema namespace URI linking to an namespace authority. In case of dc:type='service' multiple schemas are to expect: insert them comma-separated? 19128:AuthorityURL, iso19115: AuthorityURL Ok
dct:spatial [0..1] dcmiBox:Box with CRS Subtype of dc:coverage. (CRS=WGS84; other possible values offered by dct: Point and TGN) iso19128:EX_BoundingBox, iso19115: EX_BoundingBox Ok
dct:modified [0..1] date Date of last (published) change of resource (see W3C Encoding rules). Subtype of dc:date Timestamp of resource; GetCapabilities Ok
dc:subject [0..1] string (ASCII) A keyword list; e.g. ISO 19115 classification list. (clarification needed: separated by comma) KeywordList, iso19115: MD_TopicCategoryCode Ok
dclite4g:onlineResource [0..*] URI (formerly called onLineSrc) If dc:type='service' then this is the baseURL to it. If dc:type='dataset' it's either a dc:identifier which points to a metadata record of type 'service' or a dataset URL e.g. uri:ftp:// host.com/path/filename. Subtype of dc:relation. Note that in some implementations this seems redundant to dc:identifier but in fact this is the real access point of the resource. if dc:type='dataset' given by provider, if dc:type='service' iso19128: OnlineResource (can be the same as described in dct:hasPart (dataSetURI) of a service metadata record) Ok?
dct:hasPart [0..*] URI Either a URI to a dataset or a dc:identifier (which points to a metadata record of dc:type='dataset'). Only applicable if type 'service'; N/A for type 'dataset'. Subtype of dc:relation. Note that in some implementations this seems redundant to dc:identifier but in fact this is the real access point the resource associated with a metadata record. iso19128: DataURL, iso19115: MD_metadata.dataSetURI Ok?
dc:source [0..1] URI or string (XML encoded) Description from which the present resource is derived, lineage information. Either an URI reference to a human readable resource or a string. (Note: Server base URLs and file URIs are handled elsewhere) Up to (meta-)data owner Ok
dc:publisher [0..1] string (XML encoded) Civic Address: if type service derived from GetCapabilities ('ContactInformation') else flattened KML style (= xAL) (still awaiting consensus in other OGC specs. or GeoRSS) Title element of 19128:Attribution; OrganisationName element of 19115:CI_ResponsibleParty Ok
dc:language [0..1] enumeration RFC 1766 (ISO 639, followed optionally by country ISO 3166). Max. Cardinality is intentionally 1 which for the sake of simplicity means that translations of a metadata record in different languages is recorded in several metadata instances and not in multiple dc:language, dc:title or dct:abstract occurrences. Up to (meta-)data manager, WxS spec. do foresee this Ok
dc:rights [0..1] URI or string (XML encoded) License information link about the resource. Probably also (link to) disclaimers. Either an URI reference to a human readable resource or a string (machine readable licenses is an issue; this could be an starting point. Up to (meta-)data manager; iso19115:??? Ok
dc:relation [0..*] URI A reference to a related resource. DClite4G uses following subtypes: dc:hasPart and iso19115: OnlineResource (see above). If using DC unqualified dc:relation could be used. This could be a machine readable reference to other metadata providers in order to let discover other (meta) data providers. Note that OAI-PMH has such a relationship called 'friends' but on the metadata collection/set level. ------------------------------------------------------------------------------- Placeholder; should be refined in a specific model; meaning up to (meta-)data owner Ok?

Legend: 'Equal to' means possible to derive from iso19128 (= WxS GetCapabilities).

Remarks:

  • General:
    • DC attributes/properties left as they are: dc:Audience; dc:Contributor; dc:Creator.
    • All attributes/properties have at most cardinality 1 except iso19115: OnlineResource and dct:hasPart (and dc:relation from complementary part of DClite4G).
    • Depending on the modeling approach, even these elements can become cardinality 1. NOTE that datasets (geodata resources) and services (data access services) in principle have a many-to-many relationship: Here a geodata resource (dc:type dataset) can have many iso19115: OnlineResource elements and a dc:type data_access_service has only one dc:description which can be a GetCapabilities document.
    • No additinal DC attributes/properties required; few of them need to be specialized (see dct:...);
    • See for some general explanations about dc/dct here.
    • Assume metadata (as opposite to many geodata sets) is always free and open information, like Creative Commons Share Alike
    • An encoding still has to be discussed (see following example). need schemaLocation in OSGeo!?
  • Details:
    • GetCapabilities adds following attributes (not yet modeled here): Fees, ScaleHint and Style.
    • Sorted out or highly disputed (non-DC) elements: fees, scalehint, harvestinterval.
    • dct:modified and dct:spatial can be sync'ed from dataset.
    • Attribute 'relation': This was'nt discussed yet. Simply helps harvesters to discover more (meta) data providers.
    • Attribute 'publisher': Carl mentioned such a structure here which includes StreetAddress, addressee, primaryAddressNumber, streetName, city, state, zipCode, countryCode (like in KML and behind Google geocoding service!?)
    • Keywords is included in attribute 'dc:subject'; I think people have a hard time to agree on an enumerated list (see the success of folksonomy).
  • Note that OAI-PMH...
    • puts a XML envelope around this metadata and adds a header containing two attributes: 'identifier' to identify an metadata record and 'datestamp' as date of last (published) change of metadata record.
    • requires to define a name for metadata sets. Let's don't care about this yet.

Examples

Some examples of DClite4G instances/records: (legend: 'literal' is a constant, // is a comment) (http://tinyurl.com/eaaaj)

A web mapping service instance example derived from GetCapabilities: Mapping OGC:WxS GetCapabilities to a service instance. Can also be called a 'data access point' (Note GetCapabilities finally needs to be delivered by some service owner!):

 dc:identifier      = identifier of the metadata record (ev. a machine readable URI) 
 dc:title           = WxS Service/Title
 dct:abstract       = WxS Service/Abstract
 dc:type            = 'service'
 dc:format          = namespace to OGC:WxS schema.xsd
 dct:spatial        = Root BoundingBox // from Capabilities XML
 dct:modified       = timestamp // e.g. from HTTP header or updateSequence
 dc:subject         = WxS Service/KeywordList
 dclite4g:onlineResource = baseURI of WxS // seems redundant to id but is the real link to the service
 dct:hasPart        = a dc:identifier which points to each Layer element
 dct:hasPart        = another dc:identifier, etc.
 dc:source          = null // N/A. Note: Not meant as an onlineResource
 dc:publisher       = WxS Service ContactInformation/Organization
 dc:language        = maybe HTTP header for lang (ISO3166 code), soon supported by WMS
 dc:rights          = WxS Services Fees/AccessConstraints
 dc:relation        = definition up to metadata provider

A dataset example derived from GetCapabilities: Mapping OGC:WxS GetCapabilities to a dataset/georesource/data access point (Note: There is no hasPart relationship in datasets):

 dc:identifier      = identifier of the metadata record (ev. a machine readable URI) 
 dc:title           = WxS Layer/Title
 dct:abstract       = WxS Layer/Abstract
 dc:type            = 'dataset' 
 dc:format          = namespace to format
 dct:spatial        = BoundingBox from Layer/BoundingBox
 dct:modified       = timestamp from HTTP header maybe or updateSequence
 dc:subject         = Layer/KeywordList
 dclite4g:onlineResource = a baseURI of WxS (GetMap/GetFeatures) // seems redundant to id. but is the real 'data access point'
 dclite4g:onlineResource = another baseURI of WxS // another dataset binding
 dc:source          = null // Note: Not meant as an onlineResource
 dc:publisher       = from Service if available?
 dc:language        = from Service if available?
 dc:rights          = from Service if available?
 dc:relation        = definition up to metadata provider

A dataset example delivered by a dataset owner:

 dc:identifier      = a data access point metadata defined by dataset owners context
 dc:title           = entered by dataset owner
 dct:abstract       = entered by dataset owner
 dc:type            = 'dataset' 
 dc:format          = namespace to format entered by dataset owner
 dct:spatial        = BoundingBox from data warehouse
 dct:modified       = file timestamp from data warehouse
 dc:subject         = keyword list entered by dataset owner
 dclite4g:onlineResource = a http URI entered by dataset owner // dataset binding
 dclite4g:onlineResource = a baseURI of a WxS entered by dataset owner // another binding
 dc:source          = entered by dataset owner // Note: Not online resource
 dc:publisher       = entered by dataset owner
 dc:language        = entered by dataset owner (or derived?)
 dc:rights          = entered by dataset owner
 dc:relation        = definition up to metadata provider

A dataset example with OAI-PMH XML encoding in DClite4G format:

Notes:

  • Example values are only for explanation purposes and purely fictive.
  • XML Schema (= geometadc.xsd? or dclite4g.xsd?) still tbd.
  • This record is not yet validated!
  • Took 'dclite4g' as envelope name.
  <dclite4g:qualifieddc 
    xmlns:dclite4g="http://www.osgeo.org/schemas/dclite4g/0.01" 
    xmlns:dc="http://purl.org/dc/elements/1.1/" 
    xmlns:dct="http://purl.org/dc/terms/" 
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
    xsi:schemaLocation="http://www.osgeo.org/schemas/dclite4g/ dclite4g.xsd">

    <dc:identifier>www.osgeo.org/geodata/:f264-77d2-09ce-aa39-f0f0</dc:identifier>
    <dc:title>National Elevation Mapping Service for Texas</dc:title>
    <dct:abstract>Elevation data collected for the National Elevation 
                  Dataset (NED).</dct:abstract>
    <dc:type>dataset</dc:type>
    <dc:format>...uri to the schema of the information model 
               (xsd, realxng, schematron, ili, ...)</dc:format>
    <dct:spatial>
      <Box projection="EPSG:4326" name="Geographic">
        <northlimit>34.353</northlimit>
        <eastlimit>-96.223</eastlimit>
        <southlimit>28.229</southlimit>
        <westlimit>-108.44</westlimit>
      </Box>
    </dct:spatial>
    <dct:modified>2004-03-01</dct:modified>
    <dc:subject>Elevation, Hypsography, and Contours</dc:subject>
    <dclite4g:onlineResource>uri:http://www.osgeo.org/geodata/ned_grid_georss.xml</dclite4g:onlineResource>
    <dclite4g:onlineResource>uri:http://www.osgeo.org/services/wms/</dclite4g:onlineResource>
    <dclite4g:onlineResource>uri:http://www.osgeo.org/geodata/ned_grid.shp</dclite4g:onlineResource>
    <dc:source>Lineage: Based on 30m horizontal and 15m vertical accuracy.</dc:source>

    <!-- part of DC but not part of DClite4G -->
    <dc:publisher>U.S. Geological Survey</dc:publisher>
    <dc:language>en</dc:language>
    <dc:rights>uri:http://www.usgs.gov/pubprod/</dc:rights>
  </dclite4g:qualifieddc>

Same record example as before but with unqualified DC encoding:

Note that this unqualified DC record can be seen as a mapping from DClite4G by using it's well defined semantics and content 'encoding'. See explanations above to understand the semantics of these DC-elements:

 ...
 <dc:identifier>www.osgeo.org/geodata/:f264-77d2-09ce-aa39-f0f0</dc:identifier>
 <dc:title>National Elevation Mapping Service for Texas</dc:title>
 <dc:description>Elevation data collected for the National Elevation 
                 Dataset (NED).</dc:description>
 <dc:type>dataset</dc:type>
 <dc:format>...uri to the schema of the information model 
            (xsd, realxng, schematron, ili, ...)</dc:format>
 <dc:coverage>34.353 -96.223 28.229 -108.44</dc:coverage>
 <dc:date>2004-03-01</dc:date>
 <dc:subject>Elevation, Hypsography, and Contours</dc:subject>
 <dc:relation>uri:http://www.osgeo.org/services/wms/</dc:relation>
 <dc:source>Lineage: Based on 30m horizontal and 15m vertical accuracy.</dc:source>
 ...

Other Relevant Info

  • Simple_Catalog_Interface
  • OSGeodata on GISpunkt Wiki - These pages are about the search of an open, lean and mean "protocol for the incremental exchange of metadata about geographic resources between systems". Profiled specifications like WFS or OAI-PMH are currently on our short list. Delving into 'Open Archives Initiative Protocol for Metadata Harvesting' (OAI-PMH) is strongly encouraged. It's a low barrier interoperability specification based around metadata harvesting model, it's stable (subsequent revisions are backwards compatible) and uses unqualified Dublin Core as default metadata information model; there exist open source tools (like OAICat) and it has been adopted among others by Google and Yahoo! but it's not a search protocol.
  • See here a comparison between CSW, WFS and OAI-PMH.

Guidelines for a minimal OAI-PMH implementation

OAI-PMH means Open Archives Initiative Protocol for Metadata Harvesting. For an introduction to OAI-PMH 2 see here.

This is a draft implementation guideline for a minimal OAI-PMH implementation for geospatial resources which contains following five steps:

Following are more specific guidelines for a minimal OAI-PMH implementation of a so called 'data provider' using only the mandatory 'unqualified' Dublin Core (DC):

  • Only three operations (verbs) are needed: Identify, ListMetadataFormats and ListRecords.
  • Following operations are not required (initially): ListIdentifiers, ListSets, GetRecord.
  • No incremental harvesting (resumption process for ListXxx operations with more than 1000 records)
  • No compression as defined in the OAI-PMH spec. (compression at lower http level still possible)
  • Date granularity may be 'day' not seconds (YYYY-MM-DD)
  • Keeping track of deleted record may not be supported (deletedRecord=no)
  • Mandatory DC supported as data model is sufficient for a start but with specific semantics (e.g. coverage, relation) (see also example below):
    • dc:description contains dct:abstract
    • dc:coverage contains bounding box encoding as defined in http://georss.org/simple.html#Box
    • dc:date means in fact dct:modified
    • dc:relation is filled in with dclite4g:onLineSrc. If dc:type='service' dct:hasPart can be derived from GetCapabilities.

OAI implementations

OAI with Dublin Core:

  • 'Geo-Metadatabase' (GMDB), open source (GPL, PHP)
  • GeoShop, infoGrips GmbH, Zurich, Switzerland (C, Java)
  • GeoNetwork opensource (GPL, Java)

References

Geospatial

  • GeoAPI contains an implementation of ISO 19115

RDF

  • FOAF metadata model for people and organisations
  • DOAP metadata model for open source software projects and code repositories

Notes

metadata isn't an easy task. The balance between completeness and people simply ignoring to generate it...

I wish I had had a prexisting plan of how to index and search for the data sets on extent and 'type' that we were adding

From Geodata Packaging Working Group:

    • Specifications of a data set
      • Creator
      • Date
      • License
      • Data Type
      • Topic
      • Spatial Extent
      • Coordinate System/Projection
      • Target Scale/Precision
      • Attribute Data

See Also

Comments

  • I would like to propose an additional element for the metadata model--data source (or lineage). If the data is derived from some other data, we should be able to backtrack and look at its parent/s. "Lineage" is a conditional element in FGDC but I think it's important enough that we should include it in our model. I suppose this can also be included in the Description but wouldn't it be nice to have this included as a required element? This is useful when checking for errors/consistency. -Perry
  • I find it difficult to believe that these "requirements" can be useful without switching their basis from FGDC to ISO-19115. I have posted more extensive comments as Discussion. Please read and comment. -- Bruce