OSS, Risk, and Compliance

npmjs

I'm going to tell you a story.

There are no villains in this story. Just smart people doing their best, and unfortunately working at cross-purposes through no fault of their own.

The names and places have been changed, but it is a true story. I've heard this story a lot over the years in my role at npm.

Once Upon A Time...

Way back in the late 1900s, the once-successful ACME Corporation was falling behind. Their development of proprietary widgets (on a proprietary software stack) was unable to keep up with the competition, who were leveraging Open Source platforms at a much lower cost.

Many within ACME Corp wanted to adopt the OSS approach, but they were bound by a multitude of contracts and agreements with customers and the regulatory rules of the various countries in which ACME Corp operated.

ACME Corp was in a pickle. Over a barrel. Pickled in a barrel of mixed metaphors, one could say.

Accepting Open Source Software

Luckily, ACME Corp hit on a solution. They joined some of the foundations springing up to provide governance structures for popular OSS projects, and instituted a policy where any employee could use any Open Source code that they liked, provided it was submitted for review by their compliance team.

This allowed them to avoid projects that were abandoned, insecure, or published with an incompatible license. Using a simple form was all it took, their developers could deliver value using the most up to date methods and tools.

Life was good.

Then Life Changed

Shortly after the turn of the 21st century, a series of well-intended solutions to valid problems ended up causing new problems for ACME Corp. All solutions, in solving a problem, reveal new ones.

First, GitHub made it far easier for developers of Open Source to collaborate with one another. This allowed projects to become quite popular without any corporate or nonprofit backing.

Next, Node.js brought JavaScript out of the web browser. Prior to Node, plenty of Server-Side JS platforms had been hacked up as side projects, or funded projects of promising companies. But Node was the first to significantly benefit from GitHub's network effects.

The last piece of this puzzle was an early Node.js contributor, who'd been working in the SSJS space for a while, and decided to write a package manager. He'd seen the importance of package management as a development tool before, and had spent quite a bit of time thinking about how reducing friction makes great things happen.

Impacts

A simple module system and package methodology became ubiquitous. Suddenly JavaScript was easy to share and compose. Instead of JavaScript platforms having to include the kitchen sink, they could be lightweight toolkits with loose coupling between parts.

This reduction in friction enabled what came to be known as the "small modules" movement. A number of prolific Open Source enthusiasts began to conceive of a single file as the default unit of code sharing, instead of a branded platform backed by a foundation.

Meanwhile, back at ACME Corp...

With all this distributed sharing, instead of relying on 2 or 3 well-known OSS platforms with clear governance, web applications came to rely on an interconnected mesh of thousands of tiny modules, created by an unaffiliated horde of hundreds individual contributors.

At ACME Corp, the process has started to creak. Well, not "creak", exactly. More like "break". It's broken.

The compliance team insists on only using modules that pass review. Developers who do write hand-rolled scripts to catalog all of their dependencies for the requisition forms are laughed at.

"2305 modules? You've gotta be kidding me. Use less, or come back next year."

The best devs have moved on to companies with less stringent rules. New developers coming out of school don't even know how to create websites without npm and React and Babel and a zillion of these things.

Today, the battle lines are drawn within ACME Corp, forcing developers to rely on subterfuge. The cost of a security vulnerability or getting sued for violating a license can be in the millions. But failing to ship a website is an existential threat.

When compliance complains that the new continuous delivery system is circumventing their OSS rules, the CTO says "I know, I'm on it", and then quietly ignores it.

And they all lived happily ever after...?

I wish that this was pure fiction.

The approach to compliance in almost every industry has not kept up with the advances in Open Source Software practices. This is a pressing crisis facing some of the biggest software development teams in the world right now.

I believe this problem is solvable, but it is not adequately solved yet.

Most solutions ask an organization to choose between safety and efficiency; but inefficiency is never safe. The only valid approach is to reduce friction for development teams, while also helping compliance teams to do their job. This is the the only way to bring peace to the enterprise.

Documentation Driven Development

I’ve used the term “Documentation Driven Development” in conversation a few times over the years, and people usually seem to just get what I’m talking about. It’s like “test driven development”, and every bit as disciplined, but for docs instead of tests. I certainly did not invent this concept, and it’s very simple.

I also don’t think that many folks realize just how serious I am about this. And the repeated truism that bad documentation is the worst thing in open source seems to indicate that it’s not being evangelized enough.

Here’s the process in a nutshell. This is how I usually write JavaScript modules, but it’s also how I approach designing product changes at npm, from small tweaks to entire new product lines.

  1. Before anything else, write down what the thing is in README.md. I use markdown, but any portable plain-text format will do. This takes three parts:

    1. The name of the thing.
    2. The 1-sentence description of what the thing is.
    3. If necessary, a few paragraphs describing the thing.
    4. Example code demonstrating how to use the thing.

    Note: the thing doesn’t exist yet. You do this before it exists.

  2. Copy the example code into a test file. (Basically just do const t = require('tap') and throw some asserts around it.) For things bigger than a single JavaScript module, you can also copy the various points of the description into user story issues for QA or whatever process you use to verify that the thing you built is actually what the docs say it is.

  3. Now you are ready to write the code that does that thing. Whenever it seems like the usage example was wrong, update it. As it becomes more clear what it is and how it works, keep the docs in sync with the code and the tests. The goal is to maintain 100% test coverage and 100% documentation coverage.

  4. When you want to make a change to the thing, first update the docs and then write the code and the test. It doesn’t matter if the test comes along with the code or if you write the code and then the test, but the docs should absolutely be written before any implementation or test code.

If you ever find yourself thinking “It’s done, I just need to update the docs”, then you’ve violated this process by even having that thought. Not only is it not done, but you might’ve done the wrong thing.

It’s fine to write some code to try to figure out what the thing should be. But that is experimental scaffolding, and should not be thought of as anything other than useful garbage, like a sketch on a napkin. Maybe it can be turned into something enduring, but that starts with docs.

Writing clear prose about software is an unreasonably effective way to increase productivity by reducing both the frequency and cost of mistakes. Just as with good tests, it helps ensure that your program does what you think it does, and that you’ve thought it all the way through.

This unreasonable effectiveness of writing things down comes from the fact that a human brain’s working memory size is absurdly small. While it seems that we can hold a lot of things in our mind at once, it’s only by constantly swapping to long-term storage and ignoring increasing numbers of trees in order to see the forest. A written document does not have that problem. We can fully off-load significant chunks of thought with hardly any data-loss, allowing us to think slower and more carefully while still covering a huge semantic surface.

I don’t like using inline comments to generate interface documentation, for the simple reason that my sole focus must be on the user mental model when writing docs, and that should lead the creation of the thing. If I’m writing documentation and implementation at the same time (or even in the same format), I’m naturally inclined to create a “nice implementation”, which may or may not be an interface that meshes well with the user’s mental model.

The design of how a thing is used limits the implementations available, and vice versa, so whichever is done first will tend to limit the scope of the other. It can be a bit of extra work wrestling the code to fit into the constraints of a user’s mental model. But it’s work you only have to do once, as opposed to the ongoing slog of handling bugs and issues that result from trying to shape user expectations to fit an implementation.