Documentation Driven Development

I’ve used the term “Documentation Driven Development” in conversation a few times over the years, and people usually seem to just get what I’m talking about. It’s like “test driven development”, and every bit as disciplined, but for docs instead of tests. I certainly did not invent this concept, and it’s very simple.

I also don’t think that many folks realize just how serious I am about this. And the repeated truism that bad documentation is the worst thing in open source seems to indicate that it’s not being evangelized enough.

Here’s the process in a nutshell. This is how I usually write JavaScript modules, but it’s also how I approach designing product changes at npm, from small tweaks to entire new product lines.

  1. Before anything else, write down what the thing is in I use markdown, but any portable plain-text format will do. This takes three parts:

    1. The name of the thing.
    2. The 1-sentence description of what the thing is.
    3. If necessary, a few paragraphs describing the thing.
    4. Example code demonstrating how to use the thing.

    Note: the thing doesn’t exist yet. You do this before it exists.

  2. Copy the example code into a test file. (Basically just do const t = require('tap') and throw some asserts around it.) For things bigger than a single JavaScript module, you can also copy the various points of the description into user story issues for QA or whatever process you use to verify that the thing you built is actually what the docs say it is.

  3. Now you are ready to write the code that does that thing. Whenever it seems like the usage example was wrong, update it. As it becomes more clear what it is and how it works, keep the docs in sync with the code and the tests. The goal is to maintain 100% test coverage and 100% documentation coverage.

  4. When you want to make a change to the thing, first update the docs and then write the code and the test. It doesn’t matter if the test comes along with the code or if you write the code and then the test, but the docs should absolutely be written before any implementation or test code.

If you ever find yourself thinking “It’s done, I just need to update the docs”, then you’ve violated this process by even having that thought. Not only is it not done, but you might’ve done the wrong thing.

It’s fine to write some code to try to figure out what the thing should be. But that is experimental scaffolding, and should not be thought of as anything other than useful garbage, like a sketch on a napkin. Maybe it can be turned into something enduring, but that starts with docs.

Writing clear prose about software is an unreasonably effective way to increase productivity by reducing both the frequency and cost of mistakes. Just as with good tests, it helps ensure that your program does what you think it does, and that you’ve thought it all the way through.

This unreasonable effectiveness of writing things down comes from the fact that a human brain’s working memory size is absurdly small. While it seems that we can hold a lot of things in our mind at once, it’s only by constantly swapping to long-term storage and ignoring increasing numbers of trees in order to see the forest. A written document does not have that problem. We can fully off-load significant chunks of thought with hardly any data-loss, allowing us to think slower and more carefully while still covering a huge semantic surface.

I don’t like using inline comments to generate interface documentation, for the simple reason that my sole focus must be on the user mental model when writing docs, and that should lead the creation of the thing. If I’m writing documentation and implementation at the same time (or even in the same format), I’m naturally inclined to create a “nice implementation”, which may or may not be an interface that meshes well with the user’s mental model.

The design of how a thing is used limits the implementations available, and vice versa, so whichever is done first will tend to limit the scope of the other. It can be a bit of extra work wrestling the code to fit into the constraints of a user’s mental model. But it’s work you only have to do once, as opposed to the ongoing slog of handling bugs and issues that result from trying to shape user expectations to fit an implementation.

4 Step Writing Process

Whether it’s an essay or a program, writing is not about what you put out, it’s about what you take in. The purpose of life is to learn, and that is why writing is important.

I don’t follow this process with everything, because it’s a bit slow. But for complicated subjects, it’s sometimes the only way to get something useful.

Step 1: Sketch

Write as quickly as possible, as dirty as necessary, giving no thought whatsoever to the quality of the output. Just get it out of your head and into the real medium. Make the code run. Put sentences on the page. Don’t worry if it’s ugly or slow or badly worded or whatever. That’s the point. Just the act of getting it out of your head is important and necessary.

Step 2: Discard and Rewrite

Don’t revise your rough draft. Throw it away. Maybe it had some good parts, fine. You can look at them, but you’re not allowed to copy and paste anything. Type it out again. You want it to filter through your brain and your hands so that every character is re-evaluated.

Go slow here. The goal is to make it as elegant as possible, to really carefully consider each part’s relationship to the whole, to enforce balance and symmetry and consistency of intent. This is where the vision takes shape.

The goal isn’t to end up with something clever. It’s to end up with something obvious. The solution should simply seem to grow out of the problem and attach to it, like its natural symbiotic twin. Reading your program or your essay should be easy, and should leave a reader with the understanding that you gained in your sketching and meditation.

Occasionally, you’ll learn something, and have to return to step 1. And that’s great.

Step 3: Test with Absurd Aggression

If it’s code, then find or write a test suite that goes over an absurd amount of edge cases. Try really hard to make your program break. And then make it pass every single test case. Write some more test cases to cover areas that the suite doesn’t.

If it’s text, then read it over and over. Scrutinize every word to make sure it means exactly what you want. Go to bed and wake up and read it again. Have other people read it and push them to be really critical. Ask them what they thought, and make notes of where it’s different from your intent.

Return to step 2 as necessary.

Step 4: Optimize

We always want to put this step earlier in the process, but that’s almost always a mistake. Certainly, you don’t want to be structuring your code in ways that are always going to be slow, but if you’re writing for elegance, that’s not likely to happen anyhow.

Profile your program, and find the parts of it that are slow. Make them fast, even if you lose a bit of beauty in the process.

If it’s an essay, try to remove everything that does not enforce the thesis. Any word that can be removed should be.

Keep returning to step 3 after any changes.

It’s never done. You just reach a point where you decide to study something different.