Agile Scrum Sucks (but so do the alternatives)

Software development is not like manufacturing.

In manufacturing, you build to the same specifications over and over again. In software development, you create a new thing each time, and the specifications typically change along the way.

In manufacturing, 100 factories can generally produce 100 times as many things as 1 factory. Adding a new factory means that more things will be produced. 20 programmers are far less productive than 5. Adding another programmer often means that the project will be delayed even further.

In manufacturing, you can gradually tune your methods based on past experience, and become more efficient over time. In software development, past problems generally have little if anything to do with future problems. The best you can hope for is to find and learn very broad abstract efficiency boosters like “best practices.”

Manufacturing and software development are similar in what I think of as the “expert effect.” Generally speaking, for a given task and a given group of people, one of them will be best at that task. If that expert is given control of that task, then they will tend to do it well; if they are forced to take input from everyone else in the group who isn’t as good as them, then the task will tend to suffer, if only from the time required to evaluate the input and decide not to use it. Management is one such task, perhaps the most important one.

Most people I know seem to accept these premises. That is, unless you’re an Agile Scrum zealot. Then you may think that:

  1. Specifications change, but that’s OK, because it’s “iterative design”.
  2. You can predict how long something will take, and your predictions will get better over time.
  3. Tasks can be broken down based on the time required, and can be assigned to any “qualified” person.
  4. Programming experts will be happier if they do project management, and are qualified to do so.

This is all 100% bullshit. In the real world:

  1. Specifications change because it’s virtually impossible to design something that’s never been done if you can’t see it as you go along. The specs stop changing when the team puts their collective foot down to management and the customer, and that’s when “research” stops and “development” begins. The time at which this occurs is flexible, but if you’re targeting a fixed release date, it’d better be soon! If you don’t build in enough time for research, then the product will not be satisfactory.
  2. Your estimates suck. And they’ll keep sucking, no matter what you do. You’re probably around 10-15% accurate, and if you work your ass off, you can get up to a 20-25% accuracy rate. Estimates that aren’t even remotely accurate are a waste of everyone’s time.
  3. I’m not the only one, I’m sure, who finds that at least 60% of my planned tasks for any given sprint don’t get accomplished, either because a) they depend on something else that wasn’t evident at the planning meeting, b) they just weren’t as high priority as something that came up since the planning meeting, or c) my time estimates suck, and so I budgeted way less time than required for a few of my tasks.
  4. No one likes project planning meetings. But, since project managers typically aren’t programmers dealing with the software each day, they need our input. If you’re a product manager, trust me, your team will absolutely love you if you can get these chores down a half hour or less. Streamline every part of the meeting that requires their attention. Focus their time on what you actually need them to weigh in on. Have estimates prepared, and ask them to critique them. Wrapping it in silly games won’t help—that just makes it take longer. Programmers hate project management. So make them do as little of it as possible. Trust me, if they wanted to be project managers, they wouldn’t have degrees in computer science. Let them get back to work. There’s no worse blocker than having to stop coding for a meeting. Like torture or invasive surgery, it’d better be necessary, otherwise it’s all kinds of wrong. (The “5 levels of estimates” is the Dread Cthulhu of time-wasters.)
  5. Prototypes don’t pay the bills. If you create a bunch of dummy-functional UI screens, you haven’t created software, you’ve created a demo. A prototype is an important first step, a valuable communication tool. But you can’t sell a demo, which means you can’t create demos forever. Once the “research” phase is over, demos are a liability, because they tend to obscure missing functionality.

There are a few things that scrum gets right:

  1. When it comes to estimating, often the crowd is smarter than the individual. “Planning poker,” though overly goofy and time-consuming, does a pretty good job. But it takes forever, and while I’m waiting to show my card, I’m only thinking, I’d really rather be coding right now. I’m not a project manager (that’s part of the problem with Agile Scrum) but I know there simply must be a better way.
  2. A daily meeting is crucial if there’s more than one person on the team. It’s still best if everyone can physically see the people that they’re working with from where they sit, but that’s not always possible. Even in that case, forcing everyone to stand up for a few minutes each morning and tell everyone what they’re up to is a good way to make sure that the team doesn’t get sidetracked or splintered. It keeps programmers from “going dark” and coming back a week later with something that doesn’t mesh with the rest of the product.

It seems that some Scrum Masters are die-hard zealots, and others follow a more “pick and choose” approach. The Agile methodology explicitly encourages project managers to use what works, and customize the process to meet their needs.

In other words, Agile Scrum makes a lot of ridiculous assumptions, and suggests a set of practices—some harmless, some obvious, some useful, and others specifically counter-productive—and then says, customize to your team/product/environment. If your project fails or misses a deadline, well, you didn’t do it right, or the team wasn’t committed, or the management was unreasonable. Anything at all to avoid placing blame on the methodology.

Anyone else see the similarities to a faith healer?

Ultimately, a successful project that uses Agile Scrum is successful for the same reason that any other project is successful:

  1. The management managed, and got out of the workers’ way.
  2. The workers were capable of doing what they were called on to do, and happy doing it.
  3. The product vision was something the customer wanted
  4. No one went dark. (That goes for upper management, as well!)

On my current team at Yahoo!, we use Agile Scrum, at least in name. Ostensibly, Yahoo! officially endorses the use of Agile Scrum. In my experience, successful use of agile amounts to this:

No one likes to create crap. No one likes to be surprised with new requirements in the 11th hour. No one likes to be held responsible for something they’re no good at.

Everyone loves to be a part of a quality product. Everyone likes to have visibility into their own future. Everyone likes to be praised and respected for doing what they love.

By taking credit for the success of projects that use it, Agile Scrum glosses over these fundamental principles and tries to turn the team into a bunch of part-time project managers. Anything that deviates from the plan is derided as “cowboy coding” or “waterfall process.” Granted, these two paradigms do tend to standardize bad practices, and they do little to encourage communication, but they also admit important aspects of the software development world:

  1. Actual coding, typing characters into an editor, is best done alone without distractions. You think with your own brain only. All coding is, at some level, “cowboy coding.”
  2. If piece A depends on piece B, then it makes no sense to start working on A until B is done. It makes even less sense to QA a half-working early pre-alpha dev version of the product, or worse yet, file bugs against it. Of course it’s broken. Don’t distract your developers telling them what they already know.

Chances are wildly against getting any software product out on time or under budget. My heart goes out to project managers and executives who are tasked with controlling the chaos. I understand why many of them might grasp at ideologies that promise to solve the problems, but Scrum rarely helps the odds.

If you’re doing software development right, you’re probably doing Agile wrong.

code ecosystemsthe business