Falcor and GraphQL: Querying JSON APIs (Part Five of Three)

What's the best way to query a REST API which returns JSON? I look at two popular libraries - Netflix's Falcor and Facebook's GraphQL - which aim to overcome problems with API performance and "chattiness".

Querying JSON

XML (and related standards such as XSLT and XQuery) benefit from the power of XPath for selecting and querying XML. However, JSON has no direct equivalent to XPath. (Although there are a lot of projects which have named them selves [jJ][Pp]ath!)

I still like the approach taken by JSONiq- it is essentially XQuery for JSON. However, in this post, I want to talk about two libraries - Falcor and GraphQL - which address the problem in a somewhat different way: how to get just the JSON you want from an API?

2009APR101606 by Peter Renshaw
https://flic.kr/p/6dYAsw

Trade Offs: Speed and Complexity

When you write a client for a typical REST API, you have to confront two basic problems: performance and complexity. On the one hand, if a REST API contains more data than you need, then you're paying a penalty for every unnecessary byte (being transferred over the network and parsed by your code). On the other hand, if a given API response doesn't have everything you need, then you will need to make follow-up calls, which adds complexity and, of course, more latency as you choreograph the back-and-forth.

The designer of the REST API should try to anticipate likely uses, so that they can provide just the right information, in the right ways. And, as I've previously recommended, it is a good idea to build in support for full or partial API responses. However, part of what is exciting about APIs is that they unlock innovation. So, if your API is a success, you will - by definition - have hard-to-anticipate uses of your design.

Sit! by Craig Sunter
https://flic.kr/p/rZ2tyS

Sitting in the Middle

Rather than rely on the REST API perfectly fitting your needs (or supporting a powerful query language) why not have an adapter which sits in-between your client and the REST API? Both Netflix's Falcor and Facebook's GraphQL take this approach: they are each implemented as servers which you configure to turn the REST API you have to work with into one that you want to work with. They differ somewhat in their philosophy and power, however.

Falcor

Falcor - All of the Data in One Giant Model

Netflix has open-sourced their Falcor library, which they use to power their UIs. At the time of writing, it is still in "Developer Preview", however, many people outside of Netflix are using Falcor. You can try out the demo Falcor application or read the Falcor documentation for more details.

Falcor adds some capabilities to the standard JSON model - such as "virtual JSON models" and a "JSON Graph" - to make it easier to cache data on the client side. Using Falcor, you can

• eliminate the need for multiple HTTP requests to get all the data you need
• cache the data locally for better performance
• deal with data using graphs, which are more flexible than the standard tree-model used in JSON
• adapt JSON or non-JSON APIs into a JSON model customized for your application

Falcor is a server-side Javascript library run within a nodejs server. You construct a Falcor data model and define how each component maps to the actual APIs you need to use via "paths". Your application then interacts with the Falcor data model you've defined, while the Falcor server takes care of interacting with the APIs to get you the data you need, including handling caching for greater performance - particularly when you have multiple instances of your application querying a single data model.

A nice overview of working with Falcor is provided by Auth0. And you can find a lot more documentation on the Falcor website.

GraphQL

GraphQL - a Schema and Resolve Functions

Facebook has open-sourced their GraphQL library, which they developed to power their mobile and web apps. At the time of writing, Facebook has released a working draft of the GraphQL spec and a reference implementation in Javascript. They have also created an implementation you can actually download and use. Various people have started to build GraphQL tools and implementations, including GraphQL support in Python (one of my favourite languages). Check out the GraphQL documentation for more details.

As a GraphQL client, you send the server a query, which defines what data you want back. For example

{
  user(id: "1") {
    name
  }
}

Which says "give me back the name of the user who has an id=1".

On the GraphQL server side, you need to configure the schema and the resolve functions. The schema defines the data model which may be fetched from the server. The resolve functions map the fields in the schema into the backend services. A GraphQL resolve function therefore contains whatever code is necessary to fetch and transform data from a backend service - such as a REST API, a MongoDB or a SQL RDBMS - into the form promised by the schema.

There's a nice overview of working with GraphQL on RisingStack. And you can find Facebook's full documentation on GraphQL.

"Choice" by Jeremy Brooks
https://flic.kr/p/nyPkd2

Which One Should You Choose?

Falcor is somewhat simpler to learn than GraphQL. In part, this is because GraphQL is more powerful - in particular it has a much a more sophisticated query capability. Both libraries have been implemented in Javascript, but only GraphQL is designed to be implemented in other languages, too.

Finally, it is worth considering whether you want to adopt either one at all: the REST architecture (when implemented correctly) has tremendous support for caching and scalability. So, rather than abandon a REST API altogether, consider whether you have the option of instead tuning it to perform better (tip: look at the granularity of the resources you've defined).

Designing JSON

This is part of my occasional series on designing and working with JSON:

• Part 1: An approach to designing JSON schema
• Part 2: JSON tools and standards
• Part 3: Lessons learnt from the JSON schema I've worked on
• Part 4: Heroku's HTTP API design guide

This post - the fifth in the trilogy - picks up on a topic I discussed in Part 3 - Lessons Learnt - how to select and query the JSON you get back from an API.

CQL - A Web Friendly Query Language with Metadata Support?

I've been thinking about query languages recently.

As part of my experimentation with MarkLogic, I've been playing with XQUERY, the query language for XML. I find it to be able to do everything I need it to do. I haven't invested enough time and effort to fully grok it (in the way that I did with XSLT last year). However, the fact that XQUERY and XSLT share a common addressing model (XPATH) is a huge win for me. At least I can start *somewhere* with XQUERY. But I can't find an XQUERY equivalent to dpawson's excellent XSL Frequently Asked Questions. I suppose I need to go Old Skool and actually read one of the two books on XQUERY that I bought years ago.

Another query language that I have looked at but have done little more than think about is SPARQL, the query language for RDF. Just like XQUERY (and clearly modeled on the query language category killer SQL) you essentially compose a query document that you post to a service and get back a results document in return (XQUERY lets you construct results in XML, SPARQL lets you construct results as RDF or as variable bindings, as I understand it). There are increasing opportunities to play with SPARQL endpoints, so I suspect that I will have the opportunity to actually work with SPARQL at some point.

However, it is difficult to discuss search without mentioning Google. One of the nice things about Google is that you can compose your searches as URI's. And you can supply additional fields to modify your search parameters using standard URI conventions. I was looking for something that might work in this appealingly simple way but give you access to the full power of searching metadata fields. I didn't want to have to invent my own query syntax.

I looked first at A9's Open Search. This is an attempt to allow search engines to publish a profile of search syntax and formats that they support. Initially very focused on keyword search, there is now a set of draft extensions that are mainly aimed at extended the *results* of a query with additional namespaced information. Although there is some limited support to allow a search to indicate which parameters it supports. So, the Open Search approach seems quite nice, but it doesn't quite hit the sweet spot I was looking for of being able to mix free text search with fielded search, all wrapped up in a RESTful interface.

Then, I came across SRU (Search/Retrieval via URL) and specifically CQL (now the "Contextual Query Language", but formerly known as the "Common Query Language"). It seems that SRU and CQL grew out of efforts to create a fully "of the web" successor to the pre-web Z39.50 library search and retrieve protocol. The SRU part makes it all RESTful (there's an SRW protocol for those of you who prefer to be SOAPy). And the CQL syntax lets you specify fielded search using a nice, extensible mechanism. For the cherry on top, you can return different types of XML (my favourite meta language).

The only thing I am struggling to find is much evidence of widespread adoption or even open source implementation. The Library of Congress are hosting the SRU/CQL pages, so I assume they have adopted it, at least in part. Some other library type organizations (such as COPAC) have SRU/SRW services, alongside their Z39.50 interfaces. And there is some evidence of attempts to somehow bring OpenSearch and SRU/CQL together, apparently by Nature.

Anyone suggestions as to where else I should look?