Queryable OData Metadata · OData - the Best Way to REST

Today if you browse an OData service there are two ways to learn about it: Service Documents and $metadata.

The question is this: Would it be useful to add more metadata capabilities to the protocol?

Service Documents

You can look at the 'Atom Service Document', available from the root of the service, which gives you the titles and urls for each of the service's feeds.

That's it though.

The service document doesn't tell you anything about the shape of the entries exposed by that feed or anything about relationships between the feeds.

$metadata

This is where the $metadata comes in. It returns an EDMX document that contains a complete description of the feeds, types, properties, relationships exposed by the service in EDM.

Most OData client libraries use this information to drive the generation of client-side classes to represent server types and aid programmability.

There are some limitations with $metadata though:

It is all or nothing. Lots of metadata means a big document.
It forces the server to prepare metadata for every type in the system, forcing an up front, rather than on demand model upon the service.
It isn't queryable. So if you want to find Types that have an Address property you have to retrieve the whole EDMX and search the xml, yourself.

Queryable $metadata

To address these issues one thing we've been considering - I even have a prototype - is extending $metadata so that if becomes just another OData Service. This time exposing the metadata of the service rather than its data.

You could think of this as Reflection for OData.

So this:

GET http://localhost:999/dataservice/$metadata/

Would return the service document for a metadata service used to interrogate the data service:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<service xml:base="http://localhost:999/dataservice/$metadata/" 
         xmlns:atom="https://www.w3.org/2005/Atom" 
         xmlns:app="https://www.w3.org/2007/app" 
         xmlns="https://www.w3.org/2007/app">
  <workspace>
    <atom:title>Default</atom:title>
    <collection href="ResourceTypes">
      <atom:title>ResourceTypes</atom:title>
    </collection
    <collection href="ResourceProperties">
      <atom:title>ResourceProperties</atom:title>
    </collection>
    <collection href="ResourceSets">
      <atom:title>ResourceSets</atom:title>
    </collection>
  </workspace>
</service>

As you can see this metadata service has a feed containing information about the 'feeds exposed by the data service' (aka ResourceSets), a feed of types (aka ResourceTypes) and a feed of properties (aka ResourceProperties).

Now you can ask questions about specific types using the standard OData query conventions.

For example to get information for the Product type you could do this:

GET http://localhost:999/dataservice/$metadata/ResourceTypes('Namespace.Product')

And it would return something like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://localhost:999/dataservice/$metadata/" 
       xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
       xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
       xmlns="https://www.w3.org/2005/Atom">
  <id>http://localhost:999/dataservice/$metadata/ResourceTypes('Namespace.Product')</id>
  <title type="text"></title>
  <updated>2010-04-21T17:34:41Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="ResourceType" href="ResourceTypes('Namespace.Product')" />
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/BaseType" 
        type="application/atom+xml;type=entry" 
        title="BaseType" 
        href="ResourceTypes('Namespace.Product')/BaseType" />
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Properties" 
        type="application/atom+xml;type=feed" 
        title="Properties" 
        href="ResourceTypes('Namespace.Product')/Properties" />
  <category term="System.Data.Services.Providers.ResourceType" 
        scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:FullName>Namespace.Product</d:FullName>
      <d:Name>Product</d:Name>
      <d:Namespace>Namespace</d:Namespace>
      <d:InstanceType>OData.WebApp.Product</d:InstanceType>
      <d:IsAbstract m:type="Edm.Boolean">false</d:IsAbstract>
      <d:IsMediaLinkEntry m:type="Edm.Boolean">false</d:IsMediaLinkEntry>
      <d:IsOpenType m:type="Edm.Boolean">false</d:IsOpenType>
      <d:ResourceTypeKind>EntityType</d:ResourceTypeKind>
    </m:properties>
  </content>
</entry>

Which as you can see is essentially just information about the Type - in OData format - with links to where you can learn more about its Properties and BaseType.

Learning about the shape of a feed on the fly

In fact the power of OData becomes apparent when you consider this scenario.

Imagine you are about to query this feed:

http://localhost:999/dataservice/Products

But you want to know the meaning of what you will get back.

Okay…

First you can get the metadata for the feed.

http://localhost:999/dataservice/$metadata/ResourceSets('Products')

Now from the ResourceSet you can navigate to its ResourceType

http://localhost:999/dataservice/$metadata/ResourceSets('Products')/ResourceType

And if you want to know about its properties, you can get them at the same time too using expand:

GET http://localhost:999/dataservice/$metadata/ResourceSets('Products')/ResourceType/?$expand=Properties

This url should return something like this:

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://localhost:999/dataservice/$metadata/" 
       xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
       xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
       xmlns="https://www.w3.org/2005/Atom">
  <id>http://localhost:999/dataservice/$metadata/ResourceTypes('Namespace.Product')</id>>
  <title type="text"></title>
  <updated>2010-04-21T17:51:09Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="ResourceType" href="ResourceTypes('Namespace.Product')" />
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/BaseType" 
        type="application/atom+xml;type=entry" 
        title="BaseType" 
        href="ResourceTypes('Namespace.Product')/BaseType" />
  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Properties" 
        type="application/atom+xml;type=feed" 
        title="Properties" 
        href="ResourceTypes('Namespace.Product')/Properties">
    <m:inline>
      <feed>
        <title type="text">Properties</title>
        <id>http://localhost:999/dataservice/$metadata/ResourceTypes('Namespace.Product')/Properties</id>
        <updated>2010-04-21T17:51:09Z</updated>
        <link rel="self" title="Properties" href="ResourceTypes('Namespace.Product')/Properties" />
        <entry>
          <id>http://localhost:999/dataservice/$metadata/ResourceProperties('Namespace.Product.ProdKey')</id>
          <title type="text"></title>
          <updated>2010-04-21T17:51:09Z</updated>
          <author>
            <name />
          </author>
          <link rel="edit" title="ResourceProperty" href="ResourceProperties('Namespace.Product.ProdKey')" />
          <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ResourceType" 
                type="application/atom+xml;type=entry" 
                title="ResourceType" 
                href="ResourceProperties('Namespace.Product.ProdKey')/ResourceType" />
          <category term="System.Data.Services.Providers.ResourceProperty" 
                scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
          <content type="application/xml">
            <m:properties>
              <d:FullName>Namespace.Product.ProdKey</d:FullName>
              <d:Name>ProdKey</d:Name>
              <d:IsKey m:type="Edm.Boolean">true</d:IsKey>
              <d:IsETag m:type="Edm.Boolean">false</d:IsETag>
              <d:IsComplexType m:type="Edm.Boolean">false</d:IsComplexType>
              <d:IsPrimitive m:type="Edm.Boolean">true</d:IsPrimitive>
              <d:IsReference m:type="Edm.Boolean">false</d:IsReference>
              <d:IsSetReference m:type="Edm.Boolean">false</d:IsSetReference>
              <d:ResourceTypeName>Edm.Int32</d:ResourceTypeName>
              <d:MimeType m:null="true" />
            </m:properties>
          </content>
        </entry>
        <entry>
          <id>http://localhost:999/dataservice/$metadata/ResourceProperties('Namespace.Product.Name')</id>
          <title type="text"></title>
          <updated>2010-04-21T17:51:09Z</updated>
          <author>
            <name />
          </author>
          <link rel="edit" title="ResourceProperty" href="ResourceProperties('Namespace.Product.Name')" />
          <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ResourceType" 
                type="application/atom+xml;type=entry" 
                title="ResourceType" 
                href="ResourceProperties('Namespace.Product.Name')/ResourceType" />
          <category term="System.Data.Services.Providers.ResourceProperty" 
                scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
          <content type="application/xml">
            <m:properties>
              <d:FullName>Namespace.Product.Name</d:FullName>
              <d:Name>Name</d:Name>
              <d:IsKey m:type="Edm.Boolean">false</d:IsKey>
              <d:IsETag m:type="Edm.Boolean">false</d:IsETag>
              <d:IsComplexType m:type="Edm.Boolean">false</d:IsComplexType>
              <d:IsPrimitive m:type="Edm.Boolean">true</d:IsPrimitive>
              <d:IsReference m:type="Edm.Boolean">false</d:IsReference>
              <d:IsSetReference m:type="Edm.Boolean">false</d:IsSetReference>
              <d:ResourceTypeName>Edm.String</d:ResourceTypeName>
              <d:MimeType m:null="true" />
            </m:properties>
          </content>
        </entry>
        <entry>
          <id>http://localhost:999/dataservice/$metadata/ResourceProperties('Namespace.Product.Price')</id>
          <title type="text"></title>
          <updated>2010-04-21T17:51:09Z</updated>
          <author>
            <name />
          </author>
          <link rel="edit" title="ResourceProperty" href="ResourceProperties('Namespace.Product.Price')" />
          <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ResourceType" 
                type="application/atom+xml;type=entry" 
                title="ResourceType" 
                href="ResourceProperties('Namespace.Product.Price')/ResourceType" />
          <category term="System.Data.Services.Providers.ResourceProperty" 
               scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
          <content type="application/xml">
            <m:properties>
              <d:FullName>Namespace.Product.Price</d:FullName>
              <d:Name>Price</d:Name>
              <d:IsKey m:type="Edm.Boolean">false</d:IsKey>
              <d:IsETag m:type="Edm.Boolean">false</d:IsETag>
              <d:IsComplexType m:type="Edm.Boolean">false</d:IsComplexType>
              <d:IsPrimitive m:type="Edm.Boolean">true</d:IsPrimitive>
              <d:IsReference m:type="Edm.Boolean">false</d:IsReference>
              <d:IsSetReference m:type="Edm.Boolean">false</d:IsSetReference>
              <d:ResourceTypeName>Edm.Decimal</d:ResourceTypeName>
              <d:MimeType m:null="true" />
            </m:properties>
          </content>
        </entry>
        <entry>
          <id>http://localhost:999/dataservice/$metadata/ResourceProperties('Namespace.Product.Category')</id>
          <title type="text"></title>
          <updated>2010-04-21T17:51:09Z</updated>
          <author>
            <name />
          </author>
          <link rel="edit" title="ResourceProperty" href="ResourceProperties('Namespace.Product.Category')" />
          <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/ResourceType" 
                type="application/atom+xml;type=entry" 
                title="ResourceType" 
                href="ResourceProperties('Namespace.Product.Category')/ResourceType" />
          <category term="System.Data.Services.Providers.ResourceProperty" 
                scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
          <content type="application/xml">
            <m:properties>
              <d:FullName>Namespace.Product.Category</d:FullName>
              <d:Name>Category</d:Name>
              <d:IsKey m:type="Edm.Boolean">false</d:IsKey>
              <d:IsETag m:type="Edm.Boolean">false</d:IsETag>
              <d:IsComplexType m:type="Edm.Boolean">false</d:IsComplexType>
              <d:IsPrimitive m:type="Edm.Boolean">false</d:IsPrimitive>
              <d:IsReference m:type="Edm.Boolean">true</d:IsReference>
              <d:IsSetReference m:type="Edm.Boolean">false</d:IsSetReference>
              <d:ResourceTypeName>Namespace.Category</d:ResourceTypeName>
              <d:MimeType m:null="true" />
            </m:properties>
          </content>
        </entry>
      </feed>
    </m:inline>
  </link>
  <category term="System.Data.Services.Providers.ResourceType" 
            scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:FullName>Namespace.Product</d:FullName>
      <d:Name>Product</d:Name>
      <d:Namespace>Namespace</d:Namespace>
      <d:InstanceType>OData.WebApp.Product</d:InstanceType>
      <d:IsAbstract m:type="Edm.Boolean">false</d:IsAbstract>
      <d:IsMediaLinkEntry m:type="Edm.Boolean">false</d:IsMediaLinkEntry>
      <d:IsOpenType m:type="Edm.Boolean">false</d:IsOpenType>
      <d:ResourceTypeKind>EntityType</d:ResourceTypeKind>
    </m:properties>
  </content>
</entry>
  </content>
</entry>

This result contains all the information you would normally get in an EDM <EntityType> definition including the properties, but it is targeted and specific, and the result is in OData format too.

And if that is too verbose for you - you could use content-type negotiation to get the results in JSON format.

Cool huh?

Changing the way <link @rel> works?

Recently Michael Hausenblas wrote an interesting post comparing OData and Linked Data. In that post he lamented that unlike RDF in OData predicate links do not dereference.

Huh? What does that mean?

Well if you look at an <entry> and one of its links

<entry>
  <id>http://localhost:999/dataservice/Products(1)</id>
  <title type="text"></title>
  <updated>2010-04-21T19:47:28Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title=  <link rel="http://schemas.microsoft.com/ado/2007/08/dataservices/related/Category" 
        type="application/atom+xml;type=entry" 
        title="Category" 
        href="Products(1)/Category" />
   …
</entry>

If you try to map the Category Link into RDF terms (subject-predicate-object) this is what you'll get:

A 'Product (http://localhost:999/dataservice/Products(1) )' has a 'Category Property (http://schemas.microsoft.com/ado/2007/08/dataservices/related/Category)' pointing to 'a Category (Products(1)/Category)'

Unfortunately even though the Category Property predicate has a URL, http://schemas.microsoft.com/ado/2007/08/dataservices/related/Category, it returns a 404, i.e. it is not dereference-able.

RDF on the other hand allows predicates to have dereferencable urls.

Queryable metadata to the rescue?

If however you had queryable metadata you could choose an @rel (predicate url) that queries the metadata to get more information about the Product.Category property, something like this:

<link rel="http://localhost:999/dataservice/$metadata/ResourceProperties('Namespace.Product.Category')"
      type="application/atom+xml;type=entry" 
      title="Category" 
      href="Products(1)/Category" href="Products(1)/Category" />

And now you have something that is essentially isomorphic with RDF.

Summary

As you can see we have some thoughts about how to extend the OData protocol to enrich the types of metadata discovery and queries that are possible.

But as always we are keen to hear your thoughts.

Do you think queryable metadata is interesting?
And if so do you think making @rel dereference-able is important?

Please let us know what you think.

Alex James