I recently pointed out to a (rather drunk) friend that successful processes are easier to follow than not follow. He remarked that I could usefully spend a career on the direct consequences of that statement. It occurred to me that, rather than give up code forever, I might usefully note down some of my thoughts on the subject. A note: I’ve been both a chief and an indian in various organizations, and have moved between the two roles more than once. This has given me a certain perspective on processes, both why they are important, and why they are a waste of time.
Why bother with a process at all?
Now, In our standard approval process, any developer can submit any release for approval. There’s no tracking of who looks after which projects, it’s just not necessary. There’s one exception to that, and it’s the one that proves the rule. DBAs were getting informed of proposed database changes too late in the day. By making DBAs the only ones who could submit database releases for approval, we ensured that they got an opportunity for review at an early stage.
In general terms, there are only three reasons to have a process:
- You need to pass the audit
- You need to capture some information
- Something goes wrong without it.
If your process isn’t satisfying one or more of these conditions, I’d reconsider why you’re bothering.
Audit Points
Dealing with audit points can be a pain. All to often, it feels like what you’re being asked to do is ridiculous. However, it’s well worth indulging in a bit of self-examination when going through these things. One firm I worked at actually had a pretty good track record on testing its systems. However, the auditor pointed out that we weren’t saving the test results anywhere. The more we thought about it, we came to the conclusion that it was important that not only did we test our systems, but that we could prove we had. So we addressed the audit point. Of course, the next year the same auditor came back and read our test documents. She said they weren’t standardized. We told her we didn’t care.
Again, this wasn’t a knee jerk reaction, we considered what standardized tests would buy us. The development team was ten-strong, and the systems we developed were quite heterogeneous. So the work was varied, the test requirements were varied, and the team was small enough that everyone could understand everyone else’s testing. There just wasn’t a benefit to changing our policy of “use your best judgement”, and the cost of writing a response that said we weren’t going to do anything about it was pretty small.
Finally, bear in mind that there is a massive amount audits don’t pick up. An auditor has a mental checkbox in her head of how the process should work. You know your business better than that. Now, no-one’s likely to tell the auditors about a serious problem the auditor didn’t directly ask about, but that doesn’t mean you don’t need to fix it. On good days, you can kill two birds with one stone. Look for creative ways to address audit points that address your own concerns. For instance, auditors are very keen on separation of duties. Most small departments can’t, for instance, employ a separate release manager. Most managers see this as a choice between either ignoring the audit point or seriously damaging their productivity. An automated release system can have the button pressed by anybody (you could get compliance to do it if your sense of humour ran that way…). Not only that, but the investment actually pays off: your project velocity goes way up.
Data Capture
In my experience, you need to be really careful with this one. There’s quite a few reasons this will go wrong, but one of the really basic ones is that people don’t like doing data capture. I’ll give an example, I used to work for a dotcom. Now, one of the most important figures that a commercial website has is conversion: the percentage of people that go from one step in the process to the next. It’s a number you monitor extremely carefully. The marketing department wanted to ask a question “How did you hear about us?”. Over the objections of other teams, they insisted that the question be made mandatory. You simply couldn’t use the site without answering this.
Conversion went down by 1%. 1% was £50,000 a month. The great thing about working for a professional dotcom is that you’ve got these numbers at your fingertips. I’m sure the marketing team found this information useful, but they weren’t about to convince anyone it was £50,000 worth of useful. Especially when you consider that you have no idea how many people who did answer the question actually gave the right answer. I’ll write more about data capture another time.
Something’s going wrong
This is the single best reason for putting a process in place. If there’s a developer who keeps checking in dud code, you’re going to want to institute code reviews pretty quickly. Your urgency is going to be lower if the build isn’t getting broken. Equally, if all the work in the team is getting done in reasonable time, you’re unlikely to be running prioritization meetings. Get overloaded, or under deliver, and that’s the shovel that will dig you out the hole. Agile processes are equally problem orientated: stand-up meetings are designed to improve communication, increase commitment and promote team cohesion.
Developers are, to a great extent, pretty hostile to processes (with the exception of those they originated). This is because, frankly, they’ve often cut their teeth in organizations with bad processes. But the more you subject processes to the same cost/benefit analysis that your CTO applies to everything, the more you’ll see what’s worth doing and what isn’t. This cuts both ways. Processes always introduce friction. A daily meeting can lose you an hour a day. This means that the benefit has be of the order of 16% additional productivity the rest of the time. (This is one of the reasons you’ve got to keep stand-up meetings short. At 15 minutes, the break-even point is 3.5%. I’m assuming a useful working day of 7.5 hours here.)
In other words, it’s not just enough that a process prevents something going wrong. It’s got to cost the company less to implement the process than to just live with the damage. A broken build once a year isn’t worth preventing, once a week that blocks a team of fifteen is. If you really want to change your organization, putting in decent controls on what’s important is one of the most productive things you can do. And the processes that don’t make the cut? I suggest a bonfire.
Colour Coding 