Geoservices REST API

From OSGeo
Jump to navigation Jump to search

This wiki page aims to collate community concerns related to the adoption of the "Geoservices REST API" document as a standard of the Open Geospatial Consortium (OGC). The page is being collaboratively edited, targeting completion by Wednesday 15 May 2013, after which it will be delivered by the board of the OSGeo Foundation (OSGeo) to the OGC and OGC voting members.

Cover Letter from the OSGeo Board

Please don't edit this "Cover Letter" statement, which has been approved by the OSGeo Board.

The board of the Open Source Geospatial Foundation (OSGeo) is presenting this letter to the OGC. It highlights concerns about the "GeoServices REST API" from many people within the OSGeo community. As always, if there is anything the OSGeo board can do to help, then please let us know.

Signed: Jeff McKenna (OSGeo president), Peter Batty, Jáchym Čepický, Michael Gerlek, Anne Ghisla, Mark Lucas, Daniel Morissette, Cameron Shorter, Frank Warmerdam

Open Letter to OGC and voting members

Please don't edit this "Open Letter" statement, comments and discussion should go in the "Further Concerns" section below.

May 2013

We, the undersigned, have concerns that approving the "Geoservices REST API" as an OGC standard, would have detrimental impacts on interoperability within the spatial industry.

We strongly urge that the proposed "Geoservices REST API", as it stands in May 2013, be rejected as an OGC standard.

People have listed different reasons for concern. They are described below.

Signed

Please add your name here if you agree with the above statement. Include name, work title (if appropriate), very brief title/involvement in OSGeo if appropriate. (Link to OSGeo profile if appropriate). You may sign as a group, such as the Project Steering Committee of XXX project if you wish, or as Your Name on behalf of YYY company.

  1. Cameron Shorter, Geospatial Solutions Director at LISAsoft, core contributor & coordinator of OSGeo-Live, OSGeo Board member
  2. Mark Lucas, Founding member and board of directors for OSGeo foundation, Prinicipal Scientist for RadiantBlue Technologies Inc.
  3. Stephen Woodbridge, Director of iMaptools.com, Contributor and/or PSC of Mapserver, pgRouting, PAGC, and PostGIS
  4. Even Rouault, Geospatial developer, OSGeo Charter Member, core contributor and PSC member of GDAL/OGR, contributor of Mapserver, PROJ.4, libgeotiff, shapelib, libtiff
  5. Gerhard Triebnig, Managing Director at EOX IT Services GmbH
  6. Brent Wood, Environmental Information Delivery Programme Leader, NIWA, New Zealand. OGC member, Aust/NZ OSGEO chapter member, NZOSS Council member
  7. Stephan Meissl, CTO at EOX IT Services GmbH, contributor to Mapserver, PSC chair of EOxServer
  8. Jeroen Ticheler, Director of GeoCat, project founder and PSC chair of GeoNetwork opensource
  9. Just van den Broecke, Director at Just Objects, contributor to Heron Mapping Client, secretary of OSGeo Dutch Local Chapter, member at OpenGeoGroep
  10. Milo van der Linden, member at OpenGeoGroep
  11. Landon Blake, GIS Department Manager/Land Surveyor at KSN, OSGeo California Chapter Board Representative.
  12. Daniel Morissette, President at Mapgears, core contributor and PSC member of Mapserver and GDAL/OGR. Former OGC TC member and involved in the implementation of several OGC WxS services specs in MapServer.
  13. Bob Basques, GIS Systems Developer at the City of Saint Paul, MN. Public Works GIS (GISmo), Technical Director at SharedGeo, OSGeo Charter Member, OSGeo TCMUG local chapter member, Co-founder and PSC member of GeoMoose project.
  14. Pedro-Juan Ferrer Matoses, PM at Omnium Strategic Intelligence, Spain, OSGeo Charter Member, OSGeo Spanish Local Chapter Liaison officer.
  15. Bevan Rudge, Director Lucion Limited, IT Advisor at Conservation Strategy Fund, Esri client
  16. María Arias de Reyna, software engineer at GeoCat, Spain, member of OSGeo Spanish Local Chapter.
  17. Anne Ghisla, OSGeo Board Member, Italy, member of OSGeo Italian Local Chapter.
  18. Micho Garcia, Freelance and member of geomati.co, Spain, member of Spanish Local Chapter
  19. Margherita Di Leo, OSGeo Charter Member, Post-doctoral researcher at the European Commission, JRC, Italy
  20. Jorge Sanz, GIS Consultant at Prodevelop, OSGeo Charter Member, OSGeo Spanish Local Chapter Member, Spain
  21. Pablo Sanxiao, CTO and co-founder at iCarto, OSGeo Spanish Local Chapter Member, Spain
  22. Frank Steggink, GIS software developer at Vicrea, The Netherlands, member of the Dutch Local Chapter
  23. Olivier Courtin, Oslandia co-founder, core contributor or/and PSC member of Mapserver and PostGIS. OGC TC member.
  24. Wladimir Szczerban, OSGeo Spanish Local Chapter Member, Spain
  25. Anita Graser, GIS specialist with AIT Austrian Institute of Technology, OSGeo Charter member and QGIS team member.
  26. Volker Mische, geospatial software engineer, creator of GeoCouch
  27. Iván Sánchez, OSGeo Spanish Local Chapter Member, head of OpenStreetMap Spain, OpenStreetMap Foundation member, Humanitarian OpenStreetMap Team member, Spanish SDI working group member
  28. Gabriel Carrión, Strategy Manager at gvSIG association
  29. Sandro Santilli, OSGeo Charter Member, PostGIS and GEOS PSC member and core hacker.
  30. Javier Diaz, member of Geoinquietos Bs As [1], member of the Organizing Committee FOSS4G Bs As 2013 [2]
  31. Jo Cook, Consultant at Astun Technology, former Director of OSGeo, Charter Member, founder of UK Local Chapter, Deputy Chair of FOSS4G 2013
  32. Francisco José Peñarrubia, CTO and co-founder at SCOLAB. Members of gvSIG Association
  33. Shanmugam Ganeshkumar, Director of GeoICON, member OSGeo Malaysia Chapter
  34. Barry Rowlingson, Senior Researcher, Lancaster University and Software Sustainability Institute Fellow
  35. Stefan Keller, University of Applied Sciences, Rapperswil (Switzerland), Member of Swiss OSM (SOSM) and QGIS association and of organizing committees of pgConf.DE and FOSSGIS 2013, and member of eCH (e-government standards of Switzerland)
  36. Andrew Bailey, OSGeo member, Astun Technology
  37. Suchith Anand, OSGeo Charter member, OSGeo Education member, FOSS4G 2013 LOC member
  38. Carlos Krefft, GIS software developer at CSTARS - University of Miami, OGC and OSGeo Member.
  39. Stefano Costa, OSGeo member, GFOSS.it member and former board member, Ministero per i Beni e le Attività Culturali (Italy)
  40. Peter Baumann, Jacobs University, OGC member, WCS.SWG chair, editor of 10+ specs (disclaimer: this is an expression of my personal opinion and not in any way endorsed by OGC)
  41. Peter Batty, CTO of Geospatial Division at Ubisense, OSGeo board member, former CTO of Intergraph and GE Smallworld, Technical Committee member of OGC in its formative years c 1995-97
  42. Barend Köbben, OSGEO Chartered Member, OSGeo.nl Dutch chapter treasurer, Senior Lecturer at ITC-University of Twente
  43. Paolo Cavallini, Faunalia, OSGeo member, GFOSS.it member and former president, QGIS-PSC
  44. FRans Thamura, Indonesia, OSGeo Indonesia, organizer]
  45. Sanghee Shin, Founder and CEO of Gaia3D, OSGeo Charter Member, Representative of OSGeo Korean Chapter, Chairman of Open Source GIS Alliance Korea
  46. Benni Purwonegoro,Indonesia, IT-Spatial Engineer @ Geospatial Information Agency .
  47. Jachym Cepicky, Czech Republic, member of OSGeo Board of Directors
  48. Pat Cappelaere, Vightel Corporation
  49. Jürgen Fischer, norBIT GmbH, QGIS core developer
  50. Maria Antonia Brovelli, OSGeo Charter member, OSGeo Education member, GIS Professor and Vice Rector for the Como Campus at Politecnico di Milano, Italy
  51. Nacho Varela, GIS Consultant, OSGeo Spanish Local Chapter Member, Spain
  52. Vasile Craciunescu, OSGeo Charter member, OSGeo Romania Local Chapter Leader, Researcher at Romanian National Meteorological Administration, Romania
  53. Abbas Abdul Wahab, Asst. Director, Federal Department of Town & Country Planning, Peninsular Malaysia
  54. Roy Braam, Software Engineer @ | B3Partners
  55. Peteris Bruns, Latvia, GIS Consultant & Software Engineer @ | SunGIS
  56. Giovanni Manghi, Portugal, Faunalia, OSGeo member, OSGeo-Portugal
  57. Hugo Martins, UK, Lutra Consulting, WebGIS Developer, OSGeo-Portugal Member
  58. Saber Razmjooei, UK, Lutra Consulting, Co-Founder
  59. Peter Wells, UK, Lutra Consulting, Co-Founder
  60. Sidney Gijzen, The Netherlands, Researcher GIS @ Alterra, Wageningen UR
  61. Miles Fidelman, US, Principal, Protocol Technologies Group, LLC
  62. Puneet Kishor, OSGeo Charter Member; Geology, Univ. of Wisconsin-Madison; Creative Commons
  63. António José Silva, Portugal, GIS Consultant, OSGeo-Portugal Member
  64. AndreMano, Portugal, Natural History Society - GIS Department, OSGeo-Portugal Member
  65. Mauricio Miranda, Argentina, OSGeo Charter Member, OSGeo Spanish Local Chapter Board Member
  66. Paulo Machado, Portugal, Software Engineer @ PT Inovação
  67. Alvaro Anguix, Spain, General Manager at gvSIG Association
  68. Santiago Higuera, CEO at Mercatorlab, OSGeo Spanish Local Chapter Board Member, Spain
  69. Alan Boudreault, Developer at Mapgears, contributor to Mapserver and GDAL/OGR.

Summary

Please don't edit this section - it is now locked to ensure editorial review. You may send comments to Cameron Shorter AT gmail .com

The document titled "GeoServices REST API" is currently, in May 2013, being considered to be included as one of the Open Geospatial Consortium (OGC) standards. The vote to accept the document as a standard is unusually contentious; the controversy is the cause of this page.

The document was previously released for public comment and can be found on the request for public comment page (though public comment has been closed for now).

The document attempts to standardize a suite of web services such as a service which provides map images, a service which provides geospatial feature data, and a service which performs geospatial processing. The standard focuses on interactions via a defined hierarchy of URLs and using predominantly a particular set of JSON schemas for the exchange of geospatial data.


The adoption of the document as an OGC Standard is contentious for a wide variety of reasons including:

  • the process through which the document was developed which is thought to lack sufficient flexibility to respond to input from various stakeholders,
  • the focus of the document on 'REST' and 'API' which is seen as not matching the ideas others have for these concepts,
  • the names of the standard and of the services which are seen as potentially confusing,
  • the functionality of the new services which are considered to duplicate that of existing services already standardized by the OGC such as WMS, WFS, WCS, and WPS,
  • the addition of a new set of services based on new URL patterns and new JSON exchange formats which is seen as duplicating the efforts of other working groups bringing similar ideas to the updates of existing OGC services,
  • the re-introduction in the new services of previously resolved interoperability issues which is seen as failing to build on the existing knowledge and experience,
  • the use of the particular JSON schemas which are seen as having little industry acceptance and are incompatible with other widely used schemas, and
  • the lack of implementation diversity which is thought to give the vendor of the one complete implementation an unusual commercial advantage on top of the vendor's already dominant position in the domain.

These issues have potential impacts on the use of 'Open Standards' by governments and companies, on the interoperability of software interacting with standards compliant OGC services, on the costs to developers and users of standards compliant software, on the understanding of 'Open Standards' by the public at large, and, possibly, on the reputation of the OGC as a champion of interoperability.

In particular there are concerns by some that adoption of the standard will likely result in a combination of the following:

  1. The cost to application developers, systems integrators, testers and sponsors to support all relevant OGC standards will be substantially increased.
  2. Consequently, organisations and/or applications may choose to only support one standard, or only support one standard fully.
  3. Sponsors (such as governments) who require compliance with OGC standards will discover that applications don't communicate together, due to applications supporting different OGC standards that essentially do the same thing.
  4. This will result in a diminished importance of OGC, as the "OGC standards" stamp of approval will not equate interoperability.
  5. After a while, in order to solve interoperability issues, a respected international organisation or program will likely take the initiative to mandate one standard as the preferred standard for all agencies to follow. To date, the OGC has provided this leadership.
  6. One standard taking prominance over the other will likely lead to the other being neglected or deprecated, resulting in many OGC compliant systems becoming legacy systems in the process. This should be considered an undesirable outcome for a standards organisation.

(The public comments on the standard should eventually be released as a publication of the OGC and may already have been. This will require further investigation.)


In response to these issues, the authors of the Geoservices REST API document have stated that:

  • the process of the OGC has been followed completely,
  • the specification actually is RESTful and does define an API,
  • the name, due to the controversy, may be open for modification
  • the OGC does not forbid duplication of service functionality, already has duplication between the W*S and the S*S (sensor) family of standards, should not block progress in the name of 'one true way', and harmonization between the services can be considered in the future,
  • the JSON format exists and functions, and
  • there are alternative implementations for some of these services.

The authors also stress that the existence of a large user base shows the service is useful, and that the standardization of the services at the OGC may encourage new implementations.

(The formal response documents prepared by the working group are in the process of being released to the public, as of early May 2013. When they are released, links to those responses will hopefully be presented here as well.)

Positions on the vote

Please don't edit this section - it is now locked to ensure editorial review. You may send comments to Cameron Shorter AT gmail .com

The discussion raises a number of issues, many based upon complex technical concepts and implications. This makes it difficult for voting OGC members considering whether to support "Geoservices REST API" as a standard. The following provides one analysis of the positions on the vote, aimed to simplify and summarize key points. However, it does not necessarily represent the opinions held by all signatories above.

The pros for accepting the "Geoservices REST API" document as an OGC standard
  • The OGC should be in the business of developing good standards, not in choosing which standards should be implemented.
  • The proposers of the document want to make a standard and have followed all the rules of the OGC. The work of any such group of members deserves serious, good faith consideration.
  • The need for an integrated suite of services using simple data, which is addressed (partially) by the document, is real. The proposed document is pushing the OGC on this issue.
  • The proposed document could be useful to a number of people.
  • The proposed document is not significantly more broken than the existing standards of the OGC. As one author of standards notes:
"I know how totally impossible it is to write a good standard, so the weaknesses in the existing document seem more acceptable."
The cons
  • The OGC is in the position of recommending interoperable standards for geospatial services. The proposed document is not good enough, not widely enough implemented, and not publicly supported enough, to be considered at the same level as existing standards.
  • Adopting a standard implies a desire to maintain the standard, but OGC's desire to support this approach by has been questioned by some. In particular, the lack of collaboration and willingness to accept recommendations from the community on this version of the "Geoservices REST API" document bodes ill for the future.
  • The overlap in functionality between the proposed services and the existing services, notably with the ongoing work to modularize the existing services, is almost 100 percent. However, compatibility is low.
  • There is already a published document: http://www.esri.com/library/whitepapers/pdfs/geoservices-rest-spec.pdf so there is no need for the document to be adopted as an OGC Standard merely for interoperability with the ESRI implementation.
  • The document, as a new, separate effort, repeats mistakes which were made and since solved by the other services.
  • The document focuses on the past (notably with backwards compatibility and use of only GET/POST) not on the future.
  • The document needs a comprehensive editorial rewrite. (see at end)
A conclusion

Both simple answers are bad.

A simple acceptance of the standard would introduce a new set of 'OGC approved' open services. The OGC approval might enable governments to buy a XXXX-new-name-here-XXXX solution instead of a W*S or a S*S solution. The path forwards towards harmonizing the services is unclear. Fixing this document in addition to fixing the W*S services will be a pain.

Simply rejecting the solution would be bad for the OGC. It would place the OGC in the position of picking winners and losers in the standards business. It would mean that the OGC is stuck on the project of fixing the W*S standards to meet some nebulous future functionality without having any path to get there. It would discourage innovation and progress.

Is there any third way?

Well, actually, there is a different way of thinking of the issue. Overall, there appears to be a shared desire for an integrated suite of geospatial services, originally focused on a simple data model, built on the exchange of well defined resources in simple formats including JSON, accessible and usable using the core HTTP verbs, and discoverable through following HTML links and patterns of URL paths. The hope is that such a suite can be designed based on the best expertise of the OGC, can be widely supported by the community, and can be implemented and tested by multiple groups. Neither the proposed document, nor the current services meet this vision. So the work, ultimately, is on improving all the services at the OGC, first to modularize them, then to enable simple implementations, and finally to link those implementations into a functional suite. Since this is the work that is already happening, perhaps the vote is an unfortunate distraction and the productive way forwards merely to redouble the efforts to create the next versions of the standards.

Issues with the document

Please don't edit this section - it is now locked to ensure editorial review. You may send comments to Cameron Shorter AT gmail .com

Beyond the controversy described above, there are issues with the Geoservices REST API document itself. Even if the standard deserves support, these issues could be considered blockers to the adoption of the current, May 2013, document.

The critique is incomplete because it quickly falls into a full editorial review of the text, something which takes a lot of time and effort and is beyond the scope and intent of this Open Letter.

The critique can be found at: http://wiki.osgeo.org/wiki/Geoservices_REST_API_critique.

Note that some of these critiques hold the document to OGC's current, standards writing guidelines. The OGC has been striving to develop better standards so new standards must meet higher requirements than past standards. The lack of clarity in the proposed document is not substantially worse than many published standards but ought to be resolved in new standards.

Further Concerns

--- DRAFT ____

Please add concerns not addressed above as bullet points below. Try to be concise, and try not to repeat concepts which have been covered above. Points you have previously added, which are now covered above should be removed. Where appropriate, link to external web pages (such as email achieves)

Political Concerns

  • Adopting the standard will expose the OGC to a strong suspicion of acting as a rubber stamp organization under ESRI weight, and will be detrimental to its recognized position as a reference organization for geospatial standards.
  • It is a dubious practice that a standardization organisation promotes competing standards, without explicitely obsoleting (or at least recommending) some of them. How is a newcomer to the industry supposed to select the appropriate standard if several ones share the same scope : WFS or GeoServices REST API Feature Service, WMS or GeoServices REST API Map Service, etc. ?

Commercial Concerns

  • Promoting standards from an existing implementation made by a single vendor leads to an obvious bias in competition.
  • Supporting multiple overlapping standards greatly reduces usability while it increases complexity and cost of development and maintenance.
  • Many SME's have invested in supporting existing OGC standards in their products. They will be forced to choose the standards they support (and can explain), resulting in decreased interoperability, confusion and frustration for clients.
  • Confusing customers with new, overlapping OGC standards will lower the credibility of companies and of OGC, reducing business opportunities.

Technical Concerns

  • The Geoservices REST API overlaps in large proportion with existing OGC standards such as WMS, WCS, WFS, WMTS, CSW, with no effort made to reconcile with those standards.
  • The standardization of WKT for Spatial reference systems is unfortunately currently quite weak in OGC standards. Geoservices REST API is tied with ESRI's version of WKT, which is not properly specified in the Geoservices REST API documents, and is known to be incompatible with other OGC documents, which will lead to a larger confusion. See the following comment for more details on this issue.
  • The Geoservices REST API is not particularly RESTful - it's a thinly disguised service call, not an address space for RESTful objects that can be operated on.
  • At least as far as "imagery" is concerned, OGC standards arguably are substantially more mature, powerful, flexible, and modular then the ESRI "Geoservice REST API" Part 6 (and some design principles suggest that scalability may be hampered as well):
    • data model:
      • the ESRI "Geoservice REST API" appears constrained to 2-D imagery, plus optional time stamps. OGC has established a unified coverage model which fully supports n-D spatio-temporal data. It allows use and exchange of coverages between different services, such as WCS, WCPS, WPS, and SWE.
      • OGC coverages support both regular and irregular grids; the ESRI "Geoservice REST API" supports only regular grids, more specifically: only rectified grids with quadrilateral pixels.
      • the ESRI "Geoservice REST API" lacks support for temporal data; it only offers timestamps, measured in milliseconds; this is inconvenient for users and immediately excludes, e.g., geological dates. OGC has established uniform handling of horizontal, vertical, and temporal coordinate reference systems (CRSs), following a deep consensus process with GIS science and backwards compatible with EPSG. The ESRI "Geoservice REST API" specific way of handling coordinates is not known to support this, thereby excluding appropriate timeseries handling in remote sensing, air traffic, MetOcean, etc.
      • OGC coverages provide a concise, versatile model for supporting different binary formats; the ESRI "Geoservice REST API" supports only very few selected 2-D formats, excluding, e.g., JPEG2000, NetCDF, HDF, etc.
      • the ESRI "Geoservice REST API" lacks a clear model of their data structures, it can be deduced only implicitly from the operation mechanics.
    • service model:
      • The ESRI "Geoservice REST API" Part 6 lacks conciseness, thereby opening up ways for implementations that are not interoperable. For developers of alternative implementations this may mean they have to acquire ESRI licenses for finding out the intended behavior.
      • Functionality in the ESRI "Geoservice REST API" appears randomly chosen, with no clear concept visible; this burdens implementers while still leaving holes of functionality. For example, this functionality appears restricted to mapping applications and does not easily extend into other domains.
      • It has been said that the ESRI "Geoservice REST API" can be seen as a "wrapper around OGC W*S" services. This is not true for WCS (and WCPS), at least: the ESRI "Geoservice REST API" Part 6 is too poor in functionality and too different in mechanics to accomplish this.
    • In summary, the ESRI "Geoservice REST API" Imaging part is at a technological level where WCS departed from some 5 years ago. Inconciseness of the specification at large will make it difficult for third parties to come up with interoperable implementations. The components making up the ESRI "Geoservice REST API" provide natural blocks assignable to the matching SWGs. As for Part 6 of the ESRI "Geoservice REST API", if to become a standard it needs to be discussed in the WCS.SWG for harmonization, clarification, and improvement.

Methodological Concerns

  1. No public response (nor private to the authors of the comments) has been made to the various comments sent on the OGC Requests mailing list in [July 2012] and [August 2012] during the 30 day public comments period.
  2. The Geoservices REST API can not be amended (other than editorial changes in the specification document), because of a requirement of backward compatibility with ESRI implementation. Consequently, the standard is unlikely to improve, or its evolution will be only lead by ESRI.
  3. OGC standards normally require interoperability experiments and a richer process to ratify a standard such as this one. No explanation has been forthcoming as to why a simplified process is appropriate in this case.
  4. ESRI/OGC have specifically rejected requests to openly share their justifications document being selectively sent to OGC voters.

A response (from Adrian Custer):

  1. The OGC is working to release those documents to the public as of 9 May 2013.
  2. Work on the next version of the document can take any perspective acceptable to those working on the document. (The OGC has no method in place in case two groups want to evolve the document in two different directions.)
  3. That is not true, the OGC does not require interoperability experiments or other processes.

Further Reading

Please add links to referenced documents, related news stories or blog posts here.

Retrieved from "http://wiki.osgeo.org/w/index.php?title=Geoservices_REST_API&oldid=71025"