Why you absolutely MUST write an API when you write your next app

I'm a bit of hacker. I've worked with serious software engineers, and I'm definitely not one of those, although I can be when it's required. They're the types that spend more time writing and talking about what they're going to code than they do actually coding it. The word engineer suits these types perfectly. A piece of software to them is constructed from piece by piece of pre-defined, pre-designed and pre-fabricated code. There is more art than science to what I do. I liken what I do to gardening more than building a bridge. To me, software is more of an organic thing and I start software the way I start in the garden. I pick up a shovel and start digging. Over time as plants grow (or die), the yard changes and you change with it. I'm not building bank software or landing planes. I'm building to an ever changing target, evolving my software to suit a need that is never well defined, never fully known. It's whatever I feel like making it.

The big disclaimer here is that sometimes I need to be an engineer. If I'm working for a client on a fixed price project, then obviously everything needs to be engineered to a certain goal. So I can work that way, I just prefer not to. My client work is structured and defined, my pet projects are the result of an ever evolving process of discovery and learning. Of course, that isn't to say you can forget about security, performance or reliability...but it means these things are just one part of an iterative process, not a project in themselves.

For the most part, not being constrained by things like "best practices" is a wonderful freedom. I think of something, and I start coding it. When I'm done, I might go back and tweak or even throw away the prototype and start again (kind of like how we're onto our second Japanese Maple, after the first one died). It's wonderful when you know nobody will ever look at your code, or discover your hacks, it allows you to solve problems now without worrying too much about later. It's like an author taking short cuts with facts, names and places instead of wasting time researching. It just gets in the way of writing the story. But there is a big problem with this type of development. What you gain in agility, you lose in portability. By coding for yourself, you're forgetting about everyone else. 

If you build a successful product, there is a fair chance that at some point you're going to need to get someone else involved. It may be as an employee joining your coding team, or showing a potential investor what you've been working on or it may be by releasing an API for others to integrate with your system. For the first time you're suddenly faced with that embarrassing feeling of your parents dropping in to visit...and it's the morning after a really really big party. It's at this point you suddenly regret all the shortcuts, the laziness, the hacks and the sambuca...oh Lord the sambuca!

That's where writing a public API from day 1 comes in.

You don't have to have your whole house neat, just the bits your parents are going to be seeing. Writing a public API as you develop your system forces you to keep a certain level of decency without constraining your ability to be agile elsewhere. By using your own API you also force yourself to follow the same rules that you expect others to follow when interfacing with your code. It prevents you taking shortcuts that you rightly wouldn't let anyone else take.

If you're an agile or "lean" developer, the one thing you never think about is what this app is going to need to do in the future. You're only ever worried about what it needs to do today. This means your code can evolve into a horrible flying spaghetti monster (if you believe in such things). What makes sense today, may not seem like such a good idea tomorrow. By having a well defined core API you have at least one part of your system (hopefully the important bit) that is well documented, well considered and well written. It's something akin to maintaining a tidy formal lounge whilst the rest of your house is being subjected to an ill-considered conga line.

When it eventually comes time to make your API public, it's already tested and known to work. You know what problems your users are likely to encounter, because you've already encountered them yourself. Best of all, you're not trying to shoehorn a heap of public points of access where they were never intended. Writing an API as you go means you solve a lot of problems before they ever happen...and all without having to think about it too much. The API is as flexible as you want it to be until the day you make it public.

As you grow your application, your API grows with it. The more reliable it becomes, the more likely you will turn to it rather than hacking in a new piece of code elsewhere. When new developers come on board they have something they can immediately recognise and understand. We're all terrible at documenting our code, but with a published API you all of a sudden are advertising what you're application does...you want it to be well documented. Your users won't stand for anything less. 

All of this is the main reason most developers recommend working on open source projects. Open source means everything you do is available for anyone to see. Hacks and cheats are found and highlighted by others, and hopefully you'll learn a better way in the process. You lose this benefit of peer review when you're working on your own private projects. Writing to your own API is the next best thing. Hacks look a whole lot worse when you have to document them for someone else.

13 responses
An API not only allows others to start working immediately, but it also allows for reusability. Personal projects that were learning tools can ultimately evolve into standard components that are incorporated into everyday work - even if that was not the original scope of the project. That helps you to develop a library of "go-to" components that can form the building blocks of your next "for-hire" job.
Amen! Writing a public API was the best thing I did for my last project!
I work exactly the same way in that - I "solve problems now without worrying too much about later". However, I don't fully agree with exposing an API from day one. The reason is that - most of the projects I work on while carrying out customer development - are 'prototypes', so the contract or API is changing rapidly depending on the customer usage. So, in my opinion, a contract in the prototyping (non-engineering) phase is only holding you back. Once you leave the prototyping phase, however, and move onto non-hacker engineering phase, I don't mind re-factoring/optimizing/and establishing APIs.

With regards to your comment about parents dropping by unannounced - I have had that happen to me several times - and I agree it is sometimes embarrassing. But, in my opinion, that's a small price to pay to quickly iterate on products & business models.

Thanks for sharing. Good to see there are others who balance the hacker & engineer in them.

This is a great idea. I often fall into trying to construct the modules of my code to fit together perfectly - but this care makes it harder to change. I think that writing to an API for unknown uses, by unknown users, will help keep the module general and independent. That is, it will encourage loose coupling in SE terms http://en.wikipedia.org/wiki/Coupling_(computer_science)

Incidentally, after a friend completed his PhD on unit testing, he thought that that was its main benefit: not the testing itself, but the indirect encouragement to write the 'units' with an API in mind, so they can be tested.

Keep writing about how to become a better hacker. You are great.
An "API" is just one form of encapsulation. And if encapsulation is good in "public" case, why wouldn't it be good internally as well?

You'll find that encapsulation is a great tool for thinking - a way to decompose a complex problem into manageable pieces and to differentiate between essential and incidental. And as a bonus, you'll get neatly modularized code that is flexible, testable and (perhaps most importantly) understandable.

not sure what you did...but I see two copies of your article slightly offset, so that it is completely unreadable. unless I 'select' the text... then it's fine. WTF?
Thought more time writing/talking than coding made me a slacker. You call me a Serious Engineer. Like!

As Brendan said, you can get past the agile YAGNI (you ain't gonna need it) principle by seeing the API as the stuff you want to call from your unit tests - then write your tests-firsts and you're doing agile test-driven development!

This write-up sounds a little naive to me. Thinking that design and documentation benefits only other people is a rookie mistake. Believe me - when the code you've personally written gets into the 100's of thousands of lines - you won't want to try to hold every little detail in your head - whether anybody else gets to ogle it or not. Organic is cool until you can't remember what you planted - and we all get to that point sooner or later. But to your credit - sounds like it's dawning on you now. Eventually you'll probably be doing API's for nearly everything you do, and raving about how much time and effort is saves you. Poof - you're a software engineer.
rt: Not sure what the problem is. What browser/OS you using?

Flip: There is a difference between simplification and naivety and I'm certainly no rookie. I've been doing web development for 15 years (yes really), and been a software engineer and architect for the last 10. I led a team of a dozen developers writing intranet software for companies like Shell Global and have developed several applications for one of the largest financial companies in the Australia. I can tell you I certainly didn't keep those code bases in my head, and I certainly didn't grow them organically. I've done my time in the trenches :)

In any case, we seem to be in violent agreement. The difference is you're speaking as a software engineer TO software engineers. I'm speaking as a software engineer to hackers. In my previously employ I would never suggest working this way, and as mentioned in the article, for many current projects I still wouldn't. Horses for courses as they say.

The point is that many developers (particular in the web world) will never do an ER diagram, never write use cases or test cases and would certainly never need write an SRS or FRD. They quite possibly don't know what a waterfall or spiral model is or how the USDP works, and they sure wouldn't implement it if they did.There is no point sprouting the benefits of these tools to people who won't use them. I'm providing an easy (and obvious) engineering solution that hackers will actually use.

Thinking on this further, we can go from the interface exposed by an object/unit within an ordinary program; to a webapp API (the kind of API you mean); to cloud component services (where the webapp is made of smaller object/unit-webapps with their own APIs); to those services on a multi-core processor.

Unlike an ordinary object API, a webapp API has shared-nothing/pure-message-passing distribution baked-in - which enables that effortless step to multi-core. It also means that, eg, if you want to refer to a same instance in different calls, you'll need to explicitly indicate it somehow (eg. pass-by-value, or an ID etc). It's coarse-grained distribution, in that these constraints only apply at unit-boundaries, which you can tailor to suit the problem you're solving. Within a unit, you can use whatever language you like, and code in whatever style/paradigm you like.

Well, don't look now but writing an API is the very first engineering step you could ever take, so all this "I'm not an engineer" boast is fairly unnecessary.

And by the way, been an engineer my self, I can tell you two things: first, you can be an agile and lean engineer. Second, been agile and lean is not about been short-sighted, but to be practical

1 visitor upvoted this post.