Geospatial data support in OData

This is a strawman proposal. Please challenge it in the OData mailing list.

OData supports geospatial data types as a new set of primitives. They can be used just like any other primitives - passed in URLs as literals, as types and values for properties, projected in $select, and so on. Like other primitives, there is a set of canonical functions that can be used with them.

The only restriction, as compared to other primitives, is that geospatial types may not be used as entity keys (see below).

The rest of this spec goes into more detail about the geospatial type system that we support, how geospatial types are represented in $metadata, how their values are represented in Atom and JSON payloads, how they are represented in URLs, and what canonical functions are defined for them.

Modeling

Primitive Types

Our type system is firmly rooted in the OGC Simple Features geometry type system. We diverge from their type system in only four ways.

Figure 1: The OGC Simple Features Type Hierarchy

First, we expose a subset of the type system and a subset of the operations. For details, see the sections below.

Second, the OGC type system is defined for only 2-dimensional geospatial data. We extend the definition of a position to be able to handle a larger number of dimensions. In particular, we handle 2d, 3dz, 3dm, and 4d geospatial data. See the section on Coordinate Reference Systems (CRS) for more information.

Third, the OGC type system ised for flat-earth geospatial data (termed geometrical data hereafter). OGC does not define a system that handles round-earth geospatial data (termed geographical data). Thus, we duplicate the OGC type system to make a parallel set of types for geographic data.

We refer to the geographical vs geometrical distinction as the topology of the type. It describes the shape of the space that includes that value.

Some minor changes in representation are necessary because geographic data is in a bounded surface (the spheroid), while geometric data is in an infinite surface (the plane). This shows up, for example, in the definition of a Polygon. We make as few changes as possible; see below for details. Even when we make changes, we follow prior art wherever possible.

Finally, like other primitives in OData, the geospatial primitives do not use inheritance and are immutable. The lack of inheritance, and the subsetting of the OGC type system, give us a difficulty with representing some data. We resolve this with a union type that behaves much like the OGC's base class. See the Union types section below for more details.

Coordinate Reference Systems

Although we support many Coordinate Reference Systems (CRS), there are several limitations (as compared to the OGC standard):

• We only support named CRSated by an SRID. This should be an official SRID. In particular, we don't support custom CRS defined in the metadata, as does GML.
  • Thus, some data will be inexpressible. For example, there are hydrodynamics readings data represented in a coordinate system where each point has coordinates [lat, long, depth, time, pressure, temperature]. This lets them do all sorts of cool analysis (eg, spatial queries across a surface defined in terms of the temperature and time axes), but is beyond the scope of OData.
  • There are also some standard coordinate systems that don't have codes. So we couldn't represent those. Ex: some common systems in New Zealand & northern Europe.
• The CRS is part of the static type of a property. Even if that property is of the base type, that property is always in the same CRS for all instances.
  • The CRS is static under projection. The above holds even between results of a projection.
  • There is a single "undefined" SRID value. This allows a service to explicitly state that the CRS varies on a per-instance basis.
• Geometric primitives with different CRS are not type-compatible under filter, group, sort, any action, or in any other way. If you want to filter an entity by a point defined in Stateplane, you have to send us literals in Stateplane. We will not transform WGS84 to Stateplane for you.
  • There are client-side libraries that can do some coordinate transforms for you.
  • Servers could expose coordinate transform functions as non-OGC canonical function extensions. See below for details.

Nominally, the Geometry/Geography type distinction is redundant with the CRS. Each CRS is inherently either round-earth or flat-earth. However, we are not going to automatically resolve this. Our implementation will not know which CRS match which model type. The modeler will have to specify both the type & the CRS.

There is a useful default CRS for Geography (round earth) data: WGS84. We will use that default if none is provided.

The default CRS for Geometry (flat earth) data is SRID 0. This represents an arbitrary flat plane with unit-less dimensions.

The Point types - Edm.Point and Edm.GeometricPoint

"Point" is defined as per the OGC. Roughly, it consists of a single position in the underlying topology and CRS. Edm.Point is used for points in the round-earth (geographic) topology. Edm.GeometricPoint is a point in a flat-earth (geometric) topology.

These primitives are used for properties with a static point type. All entities of this type will have a point value for this property.

Example properties that would be of type point or geometric point include a user's current location or the location of a bus stop.

The LineString types - Edm.LineString and Edm.GeometricLineString

"LineString" is defined as per the OGC. Roughly, it consists of a set of positions with linear interpolation between those positions, all in the same topology and CRS, and represents a path. Edm.LineString is used for geographic LineStrings; Edm.GeometricLineString is used for geometric ones.

These primitives are used for properties with a static path type. Example properties would be the path for a bus route entity, or the path that I followed on my jog this morning (stored in a Run entity).

The Polygon types - Edm.Polygon and Edm.GeometricPolygon

"Polygon" is defined as per the OGC. Roughly, it consists of a single bounded area which may contain holes. It is represented using a set of LineStrings that follow specific rules. These rules differ for geometric and geographic topologies.

These primitives are used for properties with a static single-polygon type. Examples include the area enclosed in a single census tract, or the area reachable by driving for a given amount of time from a given initial point.

Some things that people think of as polygons, such as the boundaries of states, are not actually polygons. For example, the state of Hawaii includes several islands, each of which is a full bounded polygon. Thus, the state as a whole cannot be represented as a single polygon. It is a Multipolygon, and can be represented in OData only with the base types.

The base types - Edm.Geography and Edm.Geometry

The base type represents geospatial data of an undefined type. It might vary per entity. For example, one entity might hold a point, while another holds a multi-linestring. It can hold any of the types in the OGC hierarchy that have the correct topology and CRS.

Although core OData does not support any functions on the base type, a particular implementation can support operations via extensions (see below). In core OData, you can read & write properties that have the base types, though you cannot usefully filter or order by them.

The base type is also used for dynamic properties on open types. Because these properties lack metadata, the server cannot state a more specific type. The representation for a dynamic property MUST contain the CRS and topology for that instance, so that the client knows how to use it.

Therefore, spatial dynamic properties cannot be used in $filter, $orderby, and the like without extensions. The base type does not expose any canonical functions, and spatial dynamic properties are always the base type.

Edm.Geography represents any value in a geographic topology and given CRS. Edm.Geometry represents any value in a geometric topology and given CRS.

Each instance of the base type has a specific type that matches an instantiable type from the OGC hierarchy. The representation for an instance makes clear the actual type of that instance.

Thus, there are no instances of the base type. It is simply a way for the $metadata to state that the actual data can vary per entity, and the client should look there.

Spatial Properties on Entities

Zero or more properties in an entity can have a spatial type. The spatial types are regular primitives. All the standard rules apply. In particular, they cannot be shredded under projection. This means that you cannot, for example, use $select to try to pull out the first control position of a LineString as a Point.

For open types, the dynamic properties will all effectively be of the union type. You can tell the specific type for any given instance, just as for the union type. However, there is no static type info available. This means that dynamic properties need to include the CRS & topology.

Spatial-Primary Entities (Features)

This is a non-goal. We do not think we need these as an intrinsic. We believe that we can model this with a pass-through service using vocabularies.

Communicating

Metadata

We define new types: Edm.Geography, Edm.Geometry, Edm.Point, Edm.GeometricPoint, Edm.Polygon, Edm.GeometricPolygon. Each of them has a facet that is the CRS, called "coordinate_system".

Entities in Atom

Unknown.

To spark discussion, and because it is perhaps the best of a bad lot, the strawman proposal is to use the same GML profile as Sql Server uses. This is an admittedly hacked-down simplification of full GML, but seems to match the domain reasonably well.

Here are several other options, and some of the problems with each:

GeoRSS only supports some of the types.

Full GML supports way too much - and is complex because of it.

KML ised for spatial data that may contain embedded non-spatial data. This allows creating data that you can't then use OData to easily query. We would prefer that people use full OData entities to express this metadata, so that it can be used by clients that do not support geospatial data.

Another option would be an extended WKT. This isn't XML. This isn't a problem for us, but may annoy other implementers(?). More critically, WKT doesn't support 3d or 4d positions. We need those in order to support full save & load for existing data. The various extensions all disagree on how to extend for additional dimensions. I'd prefer not to bet on any one WKT implementation, so that we can wait for another standards body to pick the right one.

PostGIS does not seem to have a native XML format. They use their EWKT.

Finally, there's the SqlServer GML profile. It is a valid GML profile, and isn't quite as much more than we need as is full GML. I resist it mostly because it is a Microsoft format. Of course, if there is no universal format, then perhaps a Microsoft one is as good as we can get.

Entities in JSON

We will use GeoJSON. Technically, since GeoJSON ised to support entire geometry trees, we are only using a subset of GeoJSON. We do not allow the use of types "Feature" or "FeatureCollection." Use entities to correlate a geospatial type with metadata.

Furthermore, "type" SHOULD be ordered first in the GeoJSON object, followed by coordinates, then the optional properties.

This looks like:

{ "d" : {
    "results": [
      {
        "__metadata": {  
            "uri": "https://services.odata.org/Foursquare.svc/Users('Neco447')",  
          "type": "Foursquare.User"
        },
        "ID": "Neco447",
        "Name": "Neco Fogworthy",
        "FavoriteLocation": {
            "type": "Point",
          "coordinates": [-127.892345987345, 45.598345897]
        },
        "LastKnownLocation": {
          "type": "Point",
          "coordinates": [-127.892345987345, 45.598345897],
          "crs": {
              "type": "name",
              "properties": {
                  "name": "urn:ogc:def:crs:OGC:1.3:CRS84"
              }
          },
          "bbox": [-180.0, -90.0, 180.0, 90.0]
        }
      },  
      { /* another User Entry */ }
  ],
  "__count": "2",
} }

Dynamic Properties

Geospatial values in dynamic properties are represented exactly as they would be for static properties, with one exception: the CRS is required. The client will not be able to examine metadata to find this value, so the value must specify it.

Querying

Geospatial Literals in URIs

There are at least 3 common extensions to WKT (PostGIS, ESRI, and Sql Server), but all use the same extension to include an SRID. As such, they all use the same representation for values with 2d coordinates. Here are some examples:

/Stores$filter=Category/Name eq "coffee" and distanceto(Location, POINT(-127.89734578345 45.234534534)) lt 900.0

/Stores$filter=Category/Name eq "coffee" and intersects(Location, SRID=12345;POLYGON((-127.89734578345 45.234534534,-127.89734578345 45.234534534,-127.89734578345 45.234534534,-127.89734578345 45.234534534)))

/Me/Friends$filter=distance_to(PlannedLocations, SRID=12345;POINT(-127.89734578345 45.234534534) lt 900.0 and PlannedTime eq datetime'2011-12-12T13:36:00'

If OData were to support 3dm, then that last one could be exposed and used something like one of (depending on which standard we go with):

PostGIS: /Me/Friends$filter=distance_to(PlannedLocations, SRID=12345;POINTM(-127.89734578345 45.234534534 2011.98453) lt 900.0

ESRI: /Me/Friends$filter=distance_to(PlannedLocations, SRID=12345;POINT M (-127.89734578345 45.234534534 2011.98453) lt 900.0

Sql Server: /Me/Friends$filter=distance_to(PlannedLocations, SRID=12345;POINT(-127.89734578345 45.234534534 NULL 2011.98453) lt 900.0

/Stores$filter=Category/Name eq "coffee" and intersects(Location, {"type":"polygon", "coordinates":[[[-127.892345987345, 45.598345897], -127.892345987345, 45.598345897], [-127.892345987345, 45.598345897], [-127.892345987345, 45.598345897]]]})

Not usable everywhere

Geospatial primitives MAY NOT be compared using lt, eq, or similar comparison operators.

Geospatial primitives MAY NOT be used as keys.

Geospatial primitives MAY NOT be used as part of an entity's ETag.

Distance Literals in URLs

Some queries, such as the coffee shop search above, need to represent a distance.

Distance is represented the same in the two topologies, but interpreted differently. In each case, it is represented as a float scalar. The units are interpreted by the topology and coordinate system for the property with which it is compared or calculated.

Because a plane is uniform, we can simply define distances in geometric coordinates to be in terms of that coordinate system's units. This works as long as each axis uses the same unit for its coordinates, which is the general case.

Geographic topologies are not necessarily uniform. The distance between longitude -125 and -124 is not the same at all points on the globe. It goes to 0 at the poles. Similarly, the distance between 30 and 31 degrees of latitude is not the same as the distance between 30 and 31 degrees of longitude (at least, not everywhere). Thus, the underlying coordinate system measures position well, but does not work for describing a distance.

For this reason, each geographic CRS also defines a unit that will be used for distances. For most CRSs, this is meters. However, some use US feet, Indian feet, German meters, or other units. In order to determine the meaning of a distance scalar, the developer must read the reference for the CRS involved.

New Canonical Functions

Each of these canonical functions is defined on certain geospatial types. Thus, each geospatial primitive type has a set of corresponding canonical functions. An OData implementation that supports a given geospatial primitive type SHOULD support using the corresponding canonical functions in $filter. It MAY support using the corresponding canonical functions in $orderby.

distance

Distance is a canonical function defined between points. It returns a distance, as defined above. The two arguments must use the same topology & CRS. The distance is measured in that topology. Distance is one of the corresponding functions for points. Distance is defined as equivalent to the OGC ST_Distance method for their overlapping domain, with equivalent semantics for geographical points.

intersects

Intersects identifies whether a point is contained within the enclosed space of a polygon. Both arguments must be of the same topology & CRS. It returns a Boolean value. Intersects is a corresponding function for any implementation that includes both points and polygons. Intersects is equivalent to OGC's ST_Intersects in their area of overlap, extended with the same semantics for geographic data.

length

Length returns the total path length of a linestring. It returns a distance, as defined above. Length is a corresponding function for linestrings. Length is equivalent to the OGC ST_Length operation for geometric linestrings, and is extended with equivalent semantics to geographic data.

All other OGC functions

We don't support these, because we want to make it easier to stand up a server that is not backed by a database. Some are very hard to implement, especially in geographic coordinates.

A provider that is capable of handling OGC Simple Features functions MAY expose those as Functions on the appropriate geospatial primitives (using the new Function support).

We are reserving a namespace, "Geo," for these standard functions. If the function matches a function specified in Simple Features, you SHOULD place it in this namespace. If the function does not meet the OGC spec, you MUST NOT place it in this namespace. Future versions of the OData spec may define more canonical functions in this namespace. The namespace is reserved to allow exactly these types of extensions without breaking existing implementations.

In the SQL version of the Simple Features standard, the function names all start with ST_ as a way to provide namespacing. Because OData has real namespaces, it does not need this pseudo-namespace. Thus, the name SHOULD NOT include the ST_ when placed in the Geo namespace. Similarly, the name SHOULD be translated to lowercase, to match other canonical functions in OData. For example, OGC's ST_Buffer would be exposed in OData as Geo.buffer. This is similar to the Simple Features implementation on CORBA.

All other geospatial functions

Any other geospatial operations MAY be exposed by using Functions. These functions are not defined in any way by this portion of the spec. See the section on Functions for more information, including namespacing issues.

Examples

Find coffee shops near me

/Stores$filter=/Category/Name eq "coffee" and distanceto(/Location, POINT(-127.89734578345 45.234534534)) lt 0.5&$orderby=distanceto(/Location, POINT-127.89734578345 45.234534534)&$top=3

Find the nearest 3 coffee shops, by drive time

This is not directly supported by OData. However, it can be handled by an extension. For example:

/Stores$filter=/Category/Name eq "coffee"&$orderby=MyNamespace.driving_time_to(POINT(-127.89734578345 45.234534534, Location)&$top=3

Note that while distanceto is symmetric in its args, MyNamespace.driving_time_to might not be. For example, it might take one-way streets into account. This would be up to the data service that is defining the function.

Compute distance along routes

/Me/Runs$orderby=length(Route) desc&$top=15

Find all houses close enough to work

For this example, let's assume that there's one OData service that can tell you the drive time polygons around a point (via a service operation). There's another OData service that can search for houses. You want to mash them up to find you houses in your price range from which you can get to work in 15 minutes.

First,

/DriveTime(to=POINT(-127.89734578345 45.234534534), maximum_duration=time'0:15:00')

returns

{ "d" : {
  "results": [
    {
      "__metadata": { 
         "uri": "https://services.odata.org/DriveTime(to=POINT(-127.89734578345 45.234534534), maximum_duration=time'0:15:00')", 
         "type": "Edm.Polygon"
      },
      "type": "Polygon",
      "coordinates": [[[-127.0234534534, 45.089734578345], [-127.0234534534, 45.389734578345], [-127.3234534534, 45.389734578345], [-127.3234534534, 45.089734578345], [-127.0234534534, 45.089734578345]],
[[-127.1234534534, 45.189734578345], [-127.1234534534, 45.289734578345], [-127.2234534534, 45.289734578345], [-127.2234534534, 45.189734578345], [-127.1234534534, 45.289734578345]]]
    }
  ],
  "__count": "1",
} }

Then, you'd send the actual search query to the second endpoint:

/Houses$filter=Price gt 50000 and Price lt 250000 and intersects(Location, POLYGON((-127.0234534534 45.089734578345,-127.0234534534 45.389734578345,-127.3234534534 45.389734578345,-127.3234534534 45.089734578345,-127.0234534534 45.089734578345),(-127.1234534534 45.189734578345,-127.1234534534 45.289734578345,-127.2234534534 45.289734578345,-127.2234534534 45.189734578345,-127.1234534534 45.289734578345)))

/Houses$filter=Price gt 50000 and Price lt 250000 and intersects(Location, Ref("http://drivetime.services.odata.org/DriveTime(to=POINT(-127.89734578345 45.234534534), maximum_duration=time'0:15:00')"))

GET /Houses?$filter=Price gt 50000 and Price lt 250000 and intersects(Location, @drive_time_polygon)&orderby=distanceto(@microsoft)&$hash=HASHCODE
Host: www.example.com
X-ODATA-QUERY: {
  "microsoft": SRID=1234;POINT(-127.2034534534 45.209734578345),
  "drive_time_polygon": POLYGON((-127.0234534534 45.089734578345,-127.0234534534 45.389734578345,-127.3234534534 45.389734578345,-127.3234534534 45.089734578345,-127.0234534534 45.089734578345),(-127.1234534534 45.189734578345,-127.1234534534 45.289734578345,-127.2234534534 45.289734578345,-127.2234534534 45.189734578345,-127.1234534534 45.289734578345)),
}