This post is a summary of where I’m at with thinking about the title question. I’m running an internal discussion on this topic later in the month.
In many software projects, coding skills are not essential to success. Most systems I’ve worked on involve simple, well-known functionality – sharing messages, automating workflows, making payments. These are all things people have been working on for decades but somehow projects still fail – despite advances in technology and methodologies. It’s not code that makes the difference.
For a while, I thought the issue was project resources – people trying to do things with too few people and too little time. But now I’ve come to think that the problems of maintaining a large code base for an organisation over years require skills very different to simply writing code. My thoughts on this have been focussed by reading Sean Goedecke’s essays, in particular Pure vs impure software development.
…pure engineering – is interested in solving a technical problem as perfectly as possible… The second kind – impure engineering – is interested in solving a real-world problem as efficiently as possible.
impure engineering is a brawl: you’re fighting decades of previous technical decisions, competing political views about how the product ought to work, consensus among your colleagues or the company at large, and in general much more incidental complexity.
This has led to me thinking about three questions:
- Why is enterprise software development so slow?
- What skills does it demand?
- How can we get better at it?
Why is enterprise software development so slow?
In a discussion about AI project success, Goedecke quoted the statistic that 84% of non-AI projects fail. Indeed, “The infamous 2015 CHAOS report found an 61% failure rate, going up to 98% for “large, complex projects”. Most IT projects fail.
One explanation Goedecke offers is the the wicked feature, the sort of addition to software that affects every other feature – for example, user types or different deployment options. As software grows, such changes accumulate making it harder to add new features. Companies can try to resist such changes but eventually give in to them to unlock new markets.
Incidentally, this is why the quality of big tech engineering is sometimes worse than you would expect. The cognitive load of operating in this environment is pretty intense. Delivering anything at all is really, really difficult. So really obvious balls sometimes get dropped because the engineer in question was trying to thread a path between ten other features.
What skills does it require?
One skill Goedecke has pointed out is the ability to explain large and subtle pieces of software, providing what he refers to as ‘technical clarity’: “In an organization, technical clarity is when non-technical decision makers have a good-enough practical understanding of what changes they can make to their software systems.”
Goedecke also sets out shipping as a distinct skill to writing code: “The default state of a project is to not ship: to be delayed indefinitely, cancelled, or to go out half-baked and burst into flames. Projects do not ship automatically once all the code has been written or all the Jira tickets closed. They ship because someone takes up the difficult and delicate job of shipping them.”
Some others I would add:
- Being able to track the work that needs doing.
- Effective asynchronous collaboration, particularly with external teams
- Running effective and efficient meetings
How do we get better at it?
I’ve only started thinking about this third question. My main concern is that most of the skills listed above are non-technical. I still think technical skills are important but, they are not sufficient. I’m looking forward to exploring this further.