Friday, December 01, 2017

On API Design and Simplicity




Most of my experience has been with software written in C, so this is primarily for the C language. I never thought this post would be so long, but once I started writing, stuff kept coming up. Its a long read: not how the API documentation should be ;-)

For a software module, its API is the gateway for using the module. It could often be a make-or-break for the module. I feel very strongly about this, so here are some of my experiences with designing APIs.


1. It needs attention

In early part of our career, we don’t care much about the API we expose. The first functions that we write become the API. How about changing this API, someone asks. Meh, it gets the job done, we say, we are too set in our ways using them as they are.
I think making a simple and intuitive API plays a big role in the success of any project. Think Arduino, for example. Every additional line required must be questioned (do you really need to include all of those header files?), every parameter argued.
The API just doesn’t show up. Someone has been obsessing over it. Iteratively polishing and refining it.
I wished I realised this earlier. The API design needs careful attention.


2. Punishing 90% of the users

As developers, we are proud of the ingenious ways in which we implemented an advanced feature, rightly so. And we want to world to know about the ingenuity. After all, what good is an engineer if she is not proud of her deliverables.
We stretch our API to accommodate these advanced features. And soon, this starts to interfere with the simplicity of the API. Turns out, most users don’t even bother about that feature. But by making it a core part of our API experience, we end up punishing these majority of users.
The users of our API now have to use some arcane API, thanks to that advanced, but-not-often-used feature.
What we should be doing instead is to focus on those 90% of the users. Make sure we make their lives easy.


3. Incremental Education

A filesystem allows us to read and write data from a file, all by using only the calls open(), read(), write() and close(). Internally, it is has to manage caching, directory lookups, optimising the block device driver reads/writes etc. But the user doesn’t necessarily have to worry about all of it.
Most APIs expose abstractions. And abstractions end up being leaky. Of course, there will be some users of the API that want to control that internal setting. They would want to control blocking/non-blocking behaviour, lock parts of the file or duplicate file descriptors.
But they don’t have to worry about learning those things, until they have a need for it.
This is what I mean by incremental education. The primary API is simple. And then there are additional APIs for the advanced features. You don’t need to know them, until you need them.
The obvious question that pops-up is, is it always possible (and wise) to abstract out the configurability and tunability that our module provides? One way to do it is to provide reasonable defaults.


4. Reasonable Defaults

We generally write software modules that implements mechanism without locking-in any policies within them, letting the users of the API choose that policy. This is the right approach.
However, it helps to have simpler APIs where default policies are chosen for the user. These policies are the default versions that we think 90% of our users should be ok with. And then there are the advanced APIs to override these policies.
For example, the UNIX filesystem API assumes the reasonable defaults that the read/write operation should be in the blocking-mode and caching should be used.
So the reasonable defaults do the Right Thing and are safe.
Defining what reasonable defaults are, can be quite tricky to begin with. But over a few iterations through our customers, the direction usually becomes quite clear.


5. “Convenience” Layers

As you may have realised in all of this, we want to make it as simple as possible, without having to lose as much flexibility as possible.
The model that, I thought, worked best is where we have multiple layers of convenience APIs.
The topmost layer provides the simplest API to the user as described above.
A user that needs a different behaviour, can peel these layers and use the layers below. She will have to learn a bit more, since the lower layer APIs are more flexible (and hence not as simple).
What needs to be ensured in this case, though, is a clear map of the various software layers and their dependencies on each other. As long as this block diagram is clear and easily available, a user can choose the layer at which they wish to operate for a given functionality.


6. Knowing Your Users

You might have noted that quite a number of methods above, mention ‘iteration’ or ‘as required by majority of our users’. Developing the module in isolation is hard indeed.
While we start off by a certain set of users in our minds to begin with, the user profile changes. It helps to stay in close quarters with the users, understand their current problems, and importantly, close the feedback loop and evolve our API.
This is quite obvious in most open-source projects, but becomes a hurdle in other projects with multiple intermediate layers of support involved.


7. Longevity

And finally, since we are talking about iteration, what about backward compatibility? Software will continue to evolve, impacting the API one way or the other.
Users are typically understanding of API changes as long as we explain the rationale for the API change. And if it makes the life of 90% of the users easy, more power to you.
As folks get into production though, there will be a resistance on the part of the users to change. You gotta bite the bullet and maintain that backward compatibility with the older not-so-simple API.
But the process of evolution and simplification continues, after all, we will have many more users in the future than we’ve had so far, wouldn’t we?


8. Patterns

I thought I was done talking, until it occurred to me that I am missing this point. If you’ve stayed so long, one bonus section for you :-)
Patterns convey information even before having to read the entire details. lock-unlock, open-close, read-write can quickly convey to a developer what might be happening.
Modelling an API around a well established pattern helps users to quickly associate and understand what the API intends to do.
The real danger though is an API that looks like a pattern but isn’t really.


Let me know what you think in the comments below…


Thanks to Amey Inamdar for his reviewing a draft of this. Thanks to pixabay and freeimages for the images.