I recently co-presented a talk at DjangoCon and EuroPython (with Nate Aune) and there was a lot of interest, so I thought I’d write an article where I can go into more detail about the specifics of good API design for mobile. (You can see the slides for that talk at http://www.scribd.com/doc/57369015/iPhone-Python-Love-Affair.)
Most of the code samples are based on an app Nate and I helped build at MusicHackDay in NYC. In this article I’ll be using different examples, but if you’re curious you can download the code for the Valentunes Django web app here: https://github.com/natea/valentunes and the iPhone code here: https://github.com/jazztpt/Valentunes_iPhone.
I was just at a great talk by Raymond Hettinger (Python core developer) on API design, and he said something that caught my ear: if you control the client, provide the simplest interface that meets client needs, but if you don’t know what the client(s) might request, allow them to pass in parameters to customize it themselves.
To satisfy both of these, I’m going to divide this article into three pieces:
1. The Problem
I’ll describe a standard REST API and the specific needs of mobile, and why the two don’t play well together.
2. Custom Mobile API Design
Recommendations for building an API to a mobile app you (or your customer) controls
3. Generic Mobile API Design
Recommendations for building an API for n third-party mobile clients
1. The Problem
An API is a way to get information from your website to something else, in this case, to a mobile phone. Most web frameworks provide an automatic way to create a REST API: a generic interface that should work for all clients. Developers will ask, isn’t REST the best API? Don’t I just want one API for everything? What makes mobile different from other clients?
But before we get to those questions, let’s cover REST APIs and what they are.
Your website has data in your database that other sites or apps might want. You want to reveal your data, but not give access to your database. In order to do this, you create specific HTTP requests that clients can ping that respond with a text representation of the data.
People often use a set of technical and vague statements to define REST. Much more useful, I think, is to understand how REST APIs are implemented in popular frameworks like Django and Rails. That’s the choice you’re making as a developer: use the ready-to-go implementation, or code your own. This article should give you an understanding of why it’s better to code your own API for mobile instead of using the standard REST API.
A typical REST API exposes the smallest number of methods necessary to create, read, update, and delete objects that exist in your database. This table covers the calls provided by a standard REST API:
RESTful Web Service HTTP methods
|Collection URI, such as http://example.com/resources/||Create a new entry in the collection. The new entry’s URL is assigned automatically and is usually returned by the operation.||Retrieve a List the URIs and perhaps other details of the collection’s members.||Update by Replacing the entire collection with another collection.||Delete the entire collection.|
|Element URI, such as http://example.com/resources/ef7d-xj36p||Treat the addressed member as a collection in its own right and create a new entry in it.||Retrieve a representation of the addressed member of the collection, expressed in an appropriate Internet media type.||Update the addressed member of the collection.||Delete the addressed member of the collection.|
While there are others, I’ll use JSON (http://www.json.org/) for these representations. Let’s use an example to look at some of these calls.
Our site is a social blogging website. There are many topic-driven blogs that any user can post to. So our database has three tables: Users, Blogs, and Articles. (In the below code I’m assuming user authentication, so the author is assumed to be the logged in user.)
To create a new article:
This should return a way to reach the newly created article. This can be a unique identifier, the url to the item, or an id with a url mapping. I’ll talk about the difference between these later.
To see the food blog (if the food blog id is 12):
This returns a representation of that blog:
Now if the client wants to see more information about each article in that blog it has to visit those urls. And for mobile, this becomes a problem.
Why Mobile is Different
Mobile users are unwilling to wait. Smartphone owners use each app for a very limited time, so any wait period is a larger portion of their app usage than it is on the web. And wait times on mobile are longer by nature because:
- connections are often slow, spotty, or non-existent, and
- mobile devices are not as powerful at accessing the database or performing calculations.
So in our example, If there are 10 articles in the food blog, and the mobile device wants to show a table of those articles, it will have to ping the API 11 times — once for the blog information, and once for each article. This is fine for an HTTP client that has a reliable connection to the internet, but the mobile app user might make this request just before entering a tunnel and only get the first three requests before losing connectivity. Providing incomplete/inaccurate data can be worse than not providing data at all — if you tell the user they are offline, they know to make their request again later, but if you show them a blog with only two articles, they might think their article was never sent, or that this blog doesn’t have any interesting articles.
For mobile, you always want to deliver the least amount of data necessary, but all in one call.
Mobile needs your API to:
- Return only the requested format
- Return hierarchical data
- Accept arrays of data
- Return pre-calculated data or data that doesn’t exist on device
There are also some other concerns you might have with mobile that aren’t related to the device itself. Because the web came first, it’s common to base a new mobile app on an existing web-based business, and you may be contracting out the mobile development. On iPhone, you have an extra constraint: the App Store, which might take two weeks or more to approve your app. Both of these lead to slower development cycles on the mobile side than on the web side. You want to be able to change some things from the server instead of having everything baked into the app itself. So in addition to tailoring your API to mobile needs, here are ways to gain some amount of control over the device:
- Action-Specific Error Codes
2. Custom Mobile API Design
Here I’m only talking about the case where you or your client controls the mobile app.
Return only the requested format
Please, if your client is requesting JSON, don’t ever return HTML. Even 500 errors should be returned via JSON.
Return Hierarchical Data
In general, mobile apps need hierarchical data. So for our mobile app, which displays a table of article titles and authors within each blog, the above GET request should return this:
Notice that we are not returning all fields of each article; the text field could be very large, and we don’t want to download entire blog posts that the user may not ever see.
Accept Arrays of Data
To create a new blog post, the mobile device sends a POST request to /article. But the user may have created many blog posts while they had no connection. So we want to be able to send an array of articles. And once we’re sending an array, we may as well always accept an array (so to create one blog post, send an array with only one item.
Our new POST will accept an array of articles:
Notice that the device sent in a guid (globally unique identifier). When sending an array of data, the device needs to know which ones were correctly saved to the database on the server. The device sends a guid for each item, and now the server can respond with that guid. Remember that the device is marking locally stored items to be sent, and won’t mark them as saved to the server (and not to resend) until it receives this return from the POST method:
Return Pre-Calculated Data or Data that Doesn’t Exist on Device
Here is where things get complicated. There are plenty of things you won’t store on the device at all, and you need a way to get that data to the device. The classic example is leaderboards.
Let’s say that we want to encourage our users to post great articles. We’ll keep a count of how often each article is opened as a rough rating system. So on our mobile device, we will present a leaderboard where you can see a list of users sorted by their total number of page hits.
How do we get this to the phone in one call? The device stores as little data as possible, so in most mobile apps there isn’t even a users table — the device only stores information for a single logged-in user.
In this case, we have to provide a custom API call. Here’s one that would work for this example:
Action-Specific Error Codes
One way you can exercise some control over the app is through error codes. Certain HTTP codes are fine as they are: 200 for updating, 201 for creating a new object, and some others — pick the ones you like. But there are other codes that are not very helpful, especially 500 (application error). In your API doc you should specify what error codes you will be sending back and how they should be interpreted. Here is an example:
This kind of error code API lets you update your error messages without having to submit a new app to the store. It also helps keep a consistent messaging system across mobile platforms.
3. Generic Mobile API Design
I don’t believe mobile API design is ready for a fully-generic interface protocol like REST. The space is still quite new and things are changing fast. I’ll make a couple of recommendations for a fully-generic protocol, and other recommendations for making a custom API for your app that is as generic as possible for multiple third-party clients.
Return only the requested format (fully-generic)
I must admit that I am baffled by the fact that all of the websites that I have worked with (including Django, Rails, ASP.net, etc) do not default to JSON when the Accept header says JSON. This should be fixed in the framework/API library/package.
Return ids and url mappings instead of only one of these two
Until now, for simplicity’s sake, I have presented only urls to identify objects:
Jacob Kaplan Moss suggests that if you need the id of the object (which a mobile device might need, depending on the circumstances), you should present the id and a url mapping. So the best generic solution would provide this instead:
Accept arrays of data (fully-generic)
All REST APIs should accept arrays of objects and return an array of ids with a mapping.
As described above, all resource POSTs would accept arrays of objects. But the return should have the id and url mapping as well as the guid. So the generic response for POST should be:
Return hierarchical data
The simplest way to generify your GET requests is to allow the client to add a depth parameter. You can do this in your url or as a json parameter — Dan Fairs makes an argument for putting it in the accept header. This allows the client to determine how many levels of objects will be returned by each request. This is a good fully-generic answer for now.
But doing this doesn’t solve the whole problem, because as you saw in the example, you don’t always want to send all of the sub-objects’ fields. Tom Christie is rewriting the serializer for Django, and I’m basing this recommendation loosely on his work. If you want a truly generic mobile API, you would at minimum allow your client to set both a depth level and to request specific fields. The default could be to return only the id & url mapping, with a “fields” parameter that includes a comma-separated string of the requested fields of that object. Not terribly elegant, but it would be more generic.
Return pre-calculated data
Here is where we leave the fully-generic world. If you want a relatively generic API for this, my advice is to think about it in terms of what uses you want to encourage. If you want to encourage mobile apps to use social networking, have a leader board call that returns an array of user dictionaries with any data they could need — points, awards, whatever your app has that could be compared.
In the example, we returned only the total hits for the leader board. To generify it, we could return other comparable info like how many articles that user has posted or across how many different blogs they have contributed to. Here is a more generic API:
Think about what database tables exist on the server that may not exist on device, and what information the mobile app might need related to them. And when one mobile app asks for a new API call, ask them about the use case and think generically before creating it.