The economics of software engineering - a short and definitive introduction
The economics of software engineering still seem to be the most misunderstood thing, even after 24 years of agile manifesto - and even among some technical people. So here we go: a short primer.
I recently watched this talk and have read this blog post. Inspired by some ideas, I decided to write down some brief thoughts on the economics of software development, and how to “manage” software development projects - and how not to.
This topic still seems to be the most misunderstood thing, even after 24 years of agile manifesto and even though software development is now at the heart of basically every bigger (and even smaller) business in most of the world. At the core of this misunderstanding lies the intangible nature of software, which makes it counterintuitive to grasp. Three specific aspects stand out, so let’s break them down for decision makers that never have developed software themselves:
“Manufacturing” software is cheap - so it can be done early and often
If we think about physical engineering disciplines like civil, mechanical or electrical, the design / engineering phase is concerned with designing the thing we want to make: bridges, machines, computers. But the outcome of this phase is usually a design document: CAD files, drawings, manufacturing specifications. The phase of building those things however happens during construction, manufacturing and assembly. That is a different phase, and requires different skills (production engineering, manufacturing or construction workers etc.).
Since construction, manufacturing and assembly is where the expensive machines, raw materials and energy come in, that is where the major cost blocks accrue, at scale. So what you want to make sure of is that you have a well-researched, validated and tested design that can actually be built, repeatedly. Relatively speaking, a bit more designing, engineering, testing and planning upfront is cheap compared to the amount of effort that goes into actually building physical things.
That is very different for software. The build process of software - compiling the code - is basically free and happens relatively quickly, within seconds or minutes. Running software requires a computer, which is relatively cheap as well. So it makes sense to do it very often, and as early as possible. Accordingly, basically all the cost of software is in the design/engineering stage - the complete opposite of physical engineering disciplines.
For intangible products the design process itself is the product (and expensive)
So it really makes sense to understand what makes the design / engineers stage unique. Software runs on computers, which are physical. But the software program consists of 0s and 1s stored in text files on such computers. It’s intangible. And unlike in physical disciplines, the result of design/engineering in software is not “a manufacturing specifications document” that some other manufacturing team will then make. The result is code, the written program itself.
So software engineering basically has all the characteristics of a service: It’s always unique and the process happens simultaneously with the outcome. Or rather: the process leads to the result in real time. You can't do “some software development” and then store it for later, like you can a product. It’s always use-case specific. You can only judge its success or quality during the process, or even only in hindsight.
In a sense developing software is like getting a haircut: hard to explicitly describe upfront in all the details, because you only roughly can articulate what you want (and you miss many details). It then kind of takes shape while it happens, gets adjusted based on real time feedback and can only be fully judged once it’s done (or not). And, as many of us know, it requires a bit of trust between you and your trusted hairdresser.
So the problem-solving and coding process itself is what creates value. Things like requirements documents or process drawings all might facilitate progress along the way. But they are not software design artifacts. Only intangible code (and tests) is. So that should be front-and-center. Code is what builds working software that customers use.
Software is complex - and upfront planning is inherently difficult
Software has thousands of interconnected parts, making it impossible to predict every detail upfront. Programs consist of many relationships - between pieces of code, within files, between files, even between frameworks or programs and databases. There are high level architectures, lower level modules, algorithms, and separate environments. (Tens of) Thousands of lines of code (or more) that depend on each other. All that is unique for every new problem space.
It’s therefore impossible to know upfront what that all entails, and what needs to be done. Have you ever tried to describe the haircut you want fully upfront, in written form? Good luck trying that! Designs will change all the time, based on real-time feedback, while they are being worked on - nowadays maybe even after a program has been shipped (SaaS, anyone?).
Two takeaways: Optimize for outsized customer value and for design flexibility
Software is cheap to build, planning is inherently difficult (changes are the norm, timelines are inherently fuzzy) and the value lies in the process of making it. So software can’t be managed like a production line, because it has 0 commonality. Software is as much art/craft as it is engineering. This leaves us with two things to take away, once and for all:
Outsized customer value
The economic upside of software has no cap, in theory. At scale, it’s zero-marginal cost, so your potential profits from one additional customer are not limited by any economic reason. Therefore your success in software stands and falls with defining what’s valuable for the customer. The moment you decide to build and ship irrelevant software, then all the process and planning and agility in the world won’t save you.
Design flexibility - enable future changes
Excessive upfront detail planning often becomes expensive waste, as software's complexity makes it hard to predict every requirement accurately. When real-world insights emerge, elaborate replanning can inflate costs further. The plan itself isn't the valuable part of software development—working code is. With major replans, significant portions of code may need to be reworked or discarded, adding to the expense.
Instead, you have to optimize your approach for flexibility. That is why agile development practices and principles like
Establishing a trusted relationship in the team (think: your doctor relationship needs trust)
Co-creating with the user
shipping small batches sooner (descope!)
seeking early feedback
testing your way to success (test-driven development)
Shipping / demoing in regular cadences
fast prototyping
designing systems for future changeability etc.
all make sense (and loads have been written about them in detail). They embrace the inherent uncertainty of the craft and make changes in the future easier, from the code level to the product level.
—-
So here is the tl;dr of this short piece:
You can’t project manage software development like you can project manage building a bridge or quality manage a manufacturing line. Just as you can’t project manage your visit at the hairdresser or your visit at the doctors. It’s inherently uncertain in the details - what needs to be done, how long it takes. So detailed upfront planning does not make sense to begin with, because the value is in the process, changes are so easy to make, and so often required. Therefore: Stop inflating the most expensive part of software development (which is the actual development) with rigid planning. Start managing it with the flexibility and realism it demands in 2025.