Geoservices REST API critique

A Critique of the proposed "GeoServices REST API" document

The document "GeoServices REST API" has proposed for adoption as a standard of the Open Geospatial Consortium (OGC). This has generated extensive controversy, discussed on the Geoservices_REST_API page.

Beyond the controversy described on that page, there are issues with the standard document itself. Even if the standard deserves support, these issues could be considered blockers to the adoption of the current, May 2013, document.

This critique is incomplete because it quickly falls into a full editorial review of the text, something which takes a lot of time and effort.

Note that some of these critiques hold the document to the current, higher standard of the OGC. The OGC has been striving to develop better standards so new standards must meet higher requirements than past standards. The lack of clarity in the proposed document is not substantially worse than many published standards.

The name

The document currently uses a name that is confusing in all discussions on the matter, both internally at the OGC and externally.

(Postnote 9 May 2013: the Standards Working Group proposing the document agreed to accept proposals for new names; if they can find a good alternative, they may adopt it.)

The design

The document does not show an overall, coherent design to the service suite. This is mostly because the design reflects a suite that [initially] evolved piecemeal at ESRI rather than one designed from the start to meet the needs of a broad swath of users. This is also due because the experts at the OGC did not contribute to an improved design for a multitude of reasons.

The design of the service which does exist is focused on yesteryear. As an apology for the existing design, section~6.2.2 of part 1, Core says that the only HTTP verbs used are GET and PUT because "the API was originally developed several years ago" but that's okay because it enables support for "Microsoft Silverlight" and "Adobe Flex" two standards which are dying rapidly. In 2013, a new standard ought to be focused on the next 10 years not the past ten.

The writing

The document is poorly written.

The document is poorly introduced

The document should start with a discussion of the overall design of the suite of services and what each individual service provides within that suite. This would make it easy for a user to understand the focus of the suite. (The note between requirements~3 and requirement~4, stating that the spec makes no requirements on clients, is really part of the scope of the document and should be front and center.)

The document instead starts with a whole section on REST.

1. Scope

The GeoServices REST API provides a standard way for web clients to communicate with geographic information system (GIS) servers based on Representational State Transfer (REST) principles.

The scope claims that the document identifies resources and a way to use "structured URLs" to exchange those resources between clients and services. If that were true, I would expect

(1) a list of resources at least as an example, and

(2) an example of the structure and structuring principles of the URLs.

Section~6 stats this process. Section~6.1 introduces the concept of a root URL for each service and mixes in that that root URL is also a catalog and that the hierarchy offers resources. This all happens in a single, overly compact paragraph. This section needs an editorial rewrite for clarity separating

(1) the end point,

(2) the catalog,

(3) the URL hierarchy,

(4) the resources, and

(5) interaction of clients with the URLs through the exchange (get/post) of the resources to the different URLs.

Unfortunately, section~6 then decays into a meandering discussion of REST which mentions the buzzword a lot but does not discuss the aspects of REST design that have been met by the suite and those that have not. If REST is a form of design which offers certain benefits, OGC standards should be discussing the benefits achieved not the buzz word. To what extent are the resources cachable and how do clients and servers agree on that? To what extent are the endpoints and actions on those endpoints discoverable, and how? Those are the issues of RESTful design.

Section~6.3 discusses Resources but immediately then states:

Each resource has a unique URL. Resources MAY support parameters.

which is a discussion of the URL hierarchy. (And 'Resources' do not 'support parameters'! Services may 'handle requests containing parameters' if that is what is meant.) Then the standard introduces a new idea 'Controller Resources' which are able to 'edit' and 'query'!? A web search does not reveal any formal discussion of 'Controller Resource' so this new type of resource probably deserves some kind of formal introduction.

Section~6.3.3 is not really about resources but the fact that the URL hierarchy is not discoverable from the Resources themselves.

Section~6.3.4 does give a list of resources, but since 'image' is not part of it but will be used by the Service for Map Images, I surmise that the list is of the things 'we have designed a JSON representation for' rather than a list of 'the resources identified and exchanged in the service suite' which is what I expect in an introductory design.

This introduction therefore needs a substantial editorial rewrite to be effective in its goal of explaining what the suite of standards actually does.

This criticism is not limited to the core document.

The Service for Maps document also starts without setting out the five core elements (end pt, catalog?, URL hierarchy, resources, and interactions).

Section~6 of the Service for Maps part, in the middle of the second paragraph, introduces a new concept, with capitalized letters:

The Map Service Root resource includes a tables property ...

without giving us any clue what this is. It "provides basic information" (perhaps it is a data structure?) but it also "supports several operations" (maybe it's a magical 'Controller Resource'). To me, this just reads as 'the authors of the spec are confused and imprecise in their written language'.

Then, in the figure of 'Resources,' there is actually a resource called 'All layers and tables'. Sorry, but this reader needs some real help to understand what that resource might be.

The document requirements are poor

The very first requirement, Requirement~1 of the Core document, states:

Req 1 If a request uses the HTTP GET method, the processing of the request by the service SHALL be safe and idempotent as specified by RFC 2616, 9.1.

This is a *ridiculous* requirement, completely untestable, and explicitly what the Mod Spec should be preventing with its attempt to require all injunctive language to be accompanied by formal tests. The formal test indeed shows that the requirement is untestable.

Inspect the documentation of the implementation to identify, if requests specified in the GeoServices REST API standard that support the HTTP GET method, are all safe and idempotent, i.e. free of side- effects, as specified in HTTP (RFC 2616, section 9.1). Alternatively, ask the developer of the the service.

Note that it is not possible to run automated tests against a service to verify conformance with the requirement (and IETF provides no test for conformance with HTTP either). In the absence of such tests, a statement of the developers of the service is sufficient indication that the requirement was addressed.

So essentially this requirement is met if the developers say 'yeah, we thought about it.' In other words, requirement 1 of the whole shebang is a recommendation. Contrast this with requirement 2 which has a nice test defined for it and is a good, solid requirement.

Again, broadening the critique to the Services for Maps, its first requirement states:

Req 1 The Map Service Root resource SHALL accept requests that conform to the URI template in Table and use any HTTP method identified in the same table.

but the test is then completely irrelevant. At the protocol level, all web services 'shall accept' request messages, so we are not discussing that. A functional test for this 'shall accept' is to construct all valid requests and see that the service never returns an exception to those requests. Even worse, the first test is a catch all 'please test the service here' test. This is *not* a well written specification of the tests for the injunctions of the Service for Maps.

The requirements need a complete, systematic review to see that they work, they are testable, and have good tests.

The document has numerous other issues which need resolution

• There is no discussion of the overlap of the f= parameter and of the HTTP format header. What happens if they conflict?
• The lack of full integration of the four core HTTP verbs, notably DELETE and PUT, exists merely for backwards compatibility not for forwards progress.
• Requirement~3 blocks forwards progress and experimentation with no explanation as to its importance.
• This goes on and on. This is the work for an EDITOR. I am not interested in doing that work.

The Technical Approach

• At least as far as "imagery" is concerned, OGC standards arguably are substantially more mature, powerful, flexible, and modular then the ESRI "Geoservice REST API" Part 6 (and some design principles suggest that scalability may be hampered as well):
  • data model:
    • the ESRI "Geoservice REST API" appears constrained to 2-D imagery, plus optional time stamps. OGC has established a unified coverage model which fully supports n-D spatio-temporal data. It allows use and exchange of coverages between different services, such as WCS, WCPS, WPS, and SWE.
    • OGC coverages support both regular and irregular grids; the ESRI "Geoservice REST API" supports only regular grids, more specifically: only rectified grids with quadrilateral pixels.
    • the ESRI "Geoservice REST API" lacks support for temporal data; it only offers timestamps, measured in milliseconds; this is inconvenient for users and immediately excludes, e.g., geological dates. OGC has established uniform handling of horizontal, vertical, and temporal coordinate reference systems (CRSs), following a deep consensus process with GIS science and backwards compatible with EPSG. The ESRI "Geoservice REST API" specific way of handling coordinates is not known to support this, thereby excluding appropriate timeseries handling in remote sensing, air traffic, MetOcean, etc.
    • OGC coverages provide a concise, versatile model for supporting different binary formats; the ESRI "Geoservice REST API" supports only very few selected 2-D formats, excluding, e.g., JPEG2000, NetCDF, HDF, etc.
    • the ESRI "Geoservice REST API" lacks a clear model of their data structures, it can be deduced only implicitly from the operation mechanics.
  • service model:
    • The ESRI "Geoservice REST API" Part 6 lacks conciseness, thereby opening up ways for implementations that are not interoperable. For developers of alternative implementations this may mean they have to acquire ESRI licenses for finding out the intended behavior.
    • Functionality in the ESRI "Geoservice REST API" appears randomly chosen, with no clear concept visible; this burdens implementers while still leaving holes of functionality. For example, this functionality appears restricted to mapping applications and does not easily extend into other domains.
    • It has been said that the ESRI "Geoservice REST API" can be seen as a "wrapper around OGC W*S" services. This is not true for WCS (and WCPS), at least: the ESRI "Geoservice REST API" Part 6 is too poor in functionality and too different in mechanics to accomplish this.
  • In summary, the ESRI "Geoservice REST API" Imaging part is at a technological level where WCS departed from some 5 years ago. Inconciseness of the specification at large will make it difficult for third parties to come up with interoperable implementations. The components making up the ESRI "Geoservice REST API" provide natural blocks assignable to the matching SWGs. As for Part 6 of the ESRI "Geoservice REST API", if to become a standard it needs to be discussed in the WCS.SWG for harmonization, clarification, and improvement.

WKT

• Geoservices REST API is tied with ESRI's version of WKT, which is not properly specified in the Geoservices REST API documents, and is known to be incompatible with other OGC documents, which will lead to a larger confusion. See the following comment for more details on this issue.