Last night I gave a talk entitled ‘Why writing software is nothing like building a house’ at LRUG’s August meetup. This is the text of the talk, which gives a brief overview of the topics covered in chapter 2 of my book.
Here’s how you go about building a house:
- Step 1: Decide on what you want to build
- Step 2: Build it
Now, I am simplifying slightly. But what’s amazing is that we’ve already diverged from how software development works. If you try this with software developers, they’ll look at you funny.
They’ll say: “Ooh, complete step 1 before you even start step 2? That sounds a bit waterfall to me.”
Or: “Ooh, decide what you want to build at the start? You can’t do that, that’s Big Design Up Front.”
We software developers have learnt to do things differently. If you put us in charge of building a house, the first thing we’d do is put up a cardboard box. Then we wouldn’t do anything else until we’d made the future owner of the house spend a night in the cardboard box, and then we’d have a review session to ask for their feedback.
Maybe if it rained that night they’d say they wanted something more waterproof, so we’d add a new feature to the backlog in the form of a tarpaulin on top. Or maybe they’d say they had nowhere to go to the bathroom so we’d dig a hole in the ground next to the box and call it an MVP toilet.
Then we’d stop work until we got more feedback. And we’d keep iterating like that until we had a house. Or at least, something that met the user’s requirements that may or may not conform to their preconceptions about what a ‘house’ really is. We certainly wouldn’t want to commit to a specific timescale for when that might happen by.
I’m making it all sound silly, but there’s a reason for that.
In software, Agile methods work better than traditional project management methods. And because of that, Agile methods – particularly SCRUM – are wildly popular among software developers. So much so that to us developers, iterative, incremental software development seems pretty normal.
But what I worry about is how all this looks to people who don’t have years of personal experience building software. Executives, entrepreneurs, non-technical project managers. For them, Agile software development often sounds just as silly as our Agile house experiment. They don’t see why they can’t use Gantt charts, why they can’t expect accurate estimates, why they can’t start with complete specs and designs and expect to end up in the right place. They don’t see the point of working iteratively. I’ve spent a lot of time thinking about, and trying to explain to non-technical people, why writing software is nothing like building a house. This is what I’ve come up with.
Unlike building a house, it’s impossible to know what you need when you start making software.
Even if you knew, it’s impossible to know how much work is required to make what you need.
And even if you knew that, it’s impossible to know how long that work would take a given team to do.
Let’s break that down:
The Imagination Problem
Let’s talk about imagination: So the thing about making a piece of software is that you’re never just making software. What you’re making is a new process that your users will act out, facilitated by software. When we imagine software, we’re also imagining what users will do with the software. And while it’s really easy to imagine a physical thing like a house, and you can build models of it, it’s much harder to do that for a process. When specs change, it’s not normally because we failed to consider some aspect of what the software would be like. It’s because we failed to consider some aspect of how the software was to be used.
In this regard, while writing software is nothing like building a house, it’s quite similar to building a hospital. The processes of domestic life are well understood and are pretty similar from house to house. BUT the processes of a hospital vary from hospital to hospital, and so need new consideration each time a new hospital is designed. And unsurprisingly therefore, hospital construction projects go disastrously wrong much more frequently than house-building projects.
The Estimation Problem
Let’s talk about estimates. Building a wall is building a wall. If you’ve built a wall before, you know how long it will take to build a comparable wall, because it’s an understood problem.
Writing software is about making unique things. Even though most software does kind of the same things overall (data in one end, data out the other), the result is achieved by novel combinations of lots of small components. It’s about solving lots and lots of small problems.
So when you ask developers to estimate a task you’re asking them: ‘How long will it take you to solve this problem?’, and the answer is normally: ‘Well, it depends on how hard the problem turns out to be – and I won’t know for sure until I’ve solved it’.
BUT all is not lost. This isn’t to say that software development is pure research of the sort that’s impossible to attach estimates to. Because there are lots of useful heuristics. In the end you find yourself saying: ‘Well, this problem feels like it has a particular level of difficulty to it, and prior experience suggests that problems with that feeling of difficulty take a particular amount of time to solve’. With a large enough margin for error, this inference-based estimation model can be a practical tool. We tend to call it ‘Story Points’. But it only works if you understand that it’s a prediction expressed as a single point on a probability curve that has a very long tail.
The Arithmetic Problem
Let’s talk about arithmetic: You can’t say: ‘Well, if it’ll take 5 developers 2 months to finish this, it’ll take 10 developers 1 month.”
In fact, when you take into account Brooks’s Law (“Adding manpower to a late project only serves to make it later”), it might take 10 developers more time than it would take 5 developers.
Why? It’s so counter-intuitive, isn’t it? Well, software development is about creating a conceptual model, a sort of imaginary landscape, populating it with imaginary entities and ensuring that they interact in consistent and meaningful ways. In that respect it’s more like getting a team of people to write a very complex choose-your-own adventure novel.
Imagine if your team of adventure writers is running behind schedule, and a deadline looms. Would adding another author help?
Well, no. That author wouldn’t know what was going on, so the original authors would have to stop and explain everything to them, and even then the new author would probably head off in a direction that didn’t quite chime with what the original authors wanted to do, so they’d have to spend ages vetting the new author’s contributions and re-orienting themselves as a group to make sure they were all on the same page.
More people does not make for less time.
So: The next time you’re at loggerheads with a non-coder about project planning and delivery, remind them:
- Software is about designing human processes, not static constructs
- Software is about solving lots of new small problems, not repeating understood processes
- Software is about building a complex conceptual model, not performing a series of independent tasks
And that’s why you can’t run a software project like a house-building project. If you still have difficulty communicating this idea with your non-technical colleagues, why not give them a copy of my book. It’s 200 pages of this kind of thing, developed specifically to explain software development to non-coders. If you disagree with anything you’ve just heard, let me know!