OSS, Risk, and Compliance


I'm going to tell you a story.

There are no villains in this story. Just smart people doing their best, and unfortunately working at cross-purposes through no fault of their own.

The names and places have been changed, but it is a true story. I've heard this story a lot over the years in my role at npm.

Once Upon A Time...

Way back in the late 1900s, the once-successful ACME Corporation was falling behind. Their development of proprietary widgets (on a proprietary software stack) was unable to keep up with the competition, who were leveraging Open Source platforms at a much lower cost.

Many within ACME Corp wanted to adopt the OSS approach, but they were bound by a multitude of contracts and agreements with customers and the regulatory rules of the various countries in which ACME Corp operated.

ACME Corp was in a pickle. Over a barrel. Pickled in a barrel of mixed metaphors, one could say.

Accepting Open Source Software

Luckily, ACME Corp hit on a solution. They joined some of the foundations springing up to provide governance structures for popular OSS projects, and instituted a policy where any employee could use any Open Source code that they liked, provided it was submitted for review by their compliance team.

This allowed them to avoid projects that were abandoned, insecure, or published with an incompatible license. Using a simple form was all it took, their developers could deliver value using the most up to date methods and tools.

Life was good.

Then Life Changed

Shortly after the turn of the 21st century, a series of well-intended solutions to valid problems ended up causing new problems for ACME Corp. All solutions, in solving a problem, reveal new ones.

First, GitHub made it far easier for developers of Open Source to collaborate with one another. This allowed projects to become quite popular without any corporate or nonprofit backing.

Next, Node.js brought JavaScript out of the web browser. Prior to Node, plenty of Server-Side JS platforms had been hacked up as side projects, or funded projects of promising companies. But Node was the first to significantly benefit from GitHub's network effects.

The last piece of this puzzle was an early Node.js contributor, who'd been working in the SSJS space for a while, and decided to write a package manager. He'd seen the importance of package management as a development tool before, and had spent quite a bit of time thinking about how reducing friction makes great things happen.


A simple module system and package methodology became ubiquitous. Suddenly JavaScript was easy to share and compose. Instead of JavaScript platforms having to include the kitchen sink, they could be lightweight toolkits with loose coupling between parts.

This reduction in friction enabled what came to be known as the "small modules" movement. A number of prolific Open Source enthusiasts began to conceive of a single file as the default unit of code sharing, instead of a branded platform backed by a foundation.

Meanwhile, back at ACME Corp...

With all this distributed sharing, instead of relying on 2 or 3 well-known OSS platforms with clear governance, web applications came to rely on an interconnected mesh of thousands of tiny modules, created by an unaffiliated horde of hundreds individual contributors.

At ACME Corp, the process has started to creak. Well, not "creak", exactly. More like "break". It's broken.

The compliance team insists on only using modules that pass review. Developers who do write hand-rolled scripts to catalog all of their dependencies for the requisition forms are laughed at.

"2305 modules? You've gotta be kidding me. Use less, or come back next year."

The best devs have moved on to companies with less stringent rules. New developers coming out of school don't even know how to create websites without npm and React and Babel and a zillion of these things.

Today, the battle lines are drawn within ACME Corp, forcing developers to rely on subterfuge. The cost of a security vulnerability or getting sued for violating a license can be in the millions. But failing to ship a website is an existential threat.

When compliance complains that the new continuous delivery system is circumventing their OSS rules, the CTO says "I know, I'm on it", and then quietly ignores it.

And they all lived happily ever after...?

I wish that this was pure fiction.

The approach to compliance in almost every industry has not kept up with the advances in Open Source Software practices. This is a pressing crisis facing some of the biggest software development teams in the world right now.

I believe this problem is solvable, but it is not adequately solved yet.

Most solutions ask an organization to choose between safety and efficiency; but inefficiency is never safe. The only valid approach is to reduce friction for development teams, while also helping compliance teams to do their job. This is the the only way to bring peace to the enterprise.

Will Happen, Happening, Happened - Rebecca Sugar

Time is an illusion that helps things make sense
So we are always living in the present tense
It seems unforgiving when a good thing ends
But you and I will always be back then
You and I will always be back then
You and I will always be back then
You and I will always be back then

Singing: Will happen, happening, happened
Will happen, happening, happened
And will happen again and again
Cause you and I will always be back then
You and I will always be back then
Will happen, happening, happened
Will happen, happening, happened
And we'll happen again and again
'Cause you and I will always be back then

If there was some amazing force outside of time
  to take us back to where we were
And hang each moment up like pictures on the wall
Inside a billion tiny frames so we can see it
  all, all, all

It would look like, will happen, happening, happened
Will happen, happening happened
And there we are again and again
'Cause you and I will always be back then
You and I will always be back then
Will happen, happening, happened
Will happen, happening, happened
And there we are again and again
'Cause you and I will always be back then
You and I will always be back then
You and I will always be back then
You and I will always be back then
That's why
You and I will always be best friends

I know rationally that Adventure Time is over. I saw it coming, and heard it rumored, and heard the wails of anguish over it from friends, and consumed by some unconscious fear of my own mourning, I slowed down watching the show, and gradually fell further and further behind.

The ending is waiting for me in the future, bittersweet and beautiful.

It will happen. Until then, Adventure Time is still happening.

Plugin Published: gatsby-remark-tumble-media

Slight update on the port of this site to Gatsby. I published the tumblr-esque media handling bits as a standalone plugin. It's now available on npm.

If you try it out, please let me know! I'd love to see if anyone else finds it useful.

Porting to Gatsby

Last month, Tumblr announced that they'd no longer be allowing "Adult Content" on their platform. I found this distressing, not because I depend on Tumblr for pornography, but because it was clear that a hamfisted approach to automating the deletion of adult content would not go good.

I downloaded my blog content using a script I found, and started looking around for the best thing to port my blog to. I definitely wanted something that would let me write markdown and export static content. Blosxom looked promising, but it seemed pretty old, and I have no interest in learning Perl at this time.

Gatsby stood out as a serious contender. All the cool kids use React and Graphql these days, and it would let me both learn the new shape of web development, and leverage my JavaScript and Node.js skills.

My initial impressions so far:

  • React: basically an ideal component/data model, though the data handling was a bit of a confusing learning curve. (Gatsby and Graphql make this both better and worse in different ways.) JSX is an itchy sweater that I resent wearing. It's not hard to use, in fact, it's pretty easy, but the ceremony feels excessive, and switching between {xyz} in JSX and ${xyz} in template strings is annoying.
  • Gatsby: pretty effin rad. I like it a lot. Blends really nice dev-time hotswap experience with a powerful component model and an optimized static SSR build for production.

    The docs are well written, and it has them, but there are not nearly enough. As a result, they sometimes jump way too fast from "this is how you type codes into an editor" to "and then you use React components to..." The tutorials were frustrating at first because of being so novice, and then frustrating because they took a lot of know-how for granted.

    Gatsby feels like it's still a bit of a power tool for power users. When there's a paved path or a starter, and that path is marked, it's easy, but I found I ended up hacking through jungle a bunch of times. (Maybe that's what I was looking for? I did enjoy the experience!) There are a lot of paved paths, but they don't go everywhere, and they aren't all marked.

    That being said, it's extremely powerful, well thought-out, and fun to use. I believe this platform will mature very nicely as more docs and tutorials are added to fill in the blanks.

  • Graphql: confusing learning curve, still don't quite grok how it's doing stuff, but once the basics clicked, it's way easier to use than SQL, and I already get queries right on the first try more often than not. I can see why everyone's excited about it.
  • Cloudflare: happy with it as expected, but for one bump. You can't use them as a TLS terminator in front of a plaintext HTTP backend, which I guess sorta makes sense, but wasn't clear in their interface or docs.

The importer was fun to write, and pretty straightforward.

The media plugin goober thingie was originally only going to be used for photosets, but I ended up using it for all media types, so it's increasingly misnamed. I plan to publish it as a standalone plugin, though I'm not sure how many people would make use of it. I did actually try to make photosets work with flex-box and display:table, but in practice, nothing is as stable or reliable as actual <table> elements. I always got weird extra whitespace when turning divs into display:table elements.

I made them reactive by just making all the elements display:block. It's much more reliable to turn a td into a div than the other way around, apparently.

The template and CSS I built from scratch using my existing tumblr blog as a visual design. I went down a weird rabbit hole for a while because I didn't realize you were supposed to just import './foo.css' in a component in order to pull in the CSS. I was using Helmet to stuff a <style> tag in the header, and always getting a flash of unstyled content.

Overall, I'm happy with the result, and really glad that things like Gatsby exist :)

(via Understanding Your Partner’s Attachment Style: An Interview with Stan Tatkin)

A great conversation about attachment styles. Awareness is power.