What is technical debt and why do I need to manage it?

Have you ever been in this situation before? You have a project and at first your programmers make fantastic progress. You’re delighted and add on a bunch of new features. This time, you notice that progress slows down and they start missing a deadline here and there. But you know they are giving your project the same attention as before and working their hardest, and they delivered so well before. So you chalk it up to bad luck and ignore it, and give them some more features to do. This time, they take even longer than before and you get a little more worried, but decide to press on. The process repeats itself until progress finally slows down to a snail’s pace. You end up receiving you later releases horribly late…or not at all. You end up spending much more time, effort and money than you planned and are unhappy with the result.

What happened to you? After all, your programmers weren’t goofing off or not putting in enough hours. What happened is that you probably were a victim of technical debt. If so, you aren't alone. Gartner group estimates that the cost of current global technical debt is $500 billion and will reach $1 trillion in five years!

What is technical debt? Every time your programmer creates a new feature for you, they must choose between two ways to create it:

1. The "short-term fast" way:
   This gets it done very quickly, but in a way that will be difficult for you/them to add features or fix bugs (perform code maintenance and enhancement) in the future.
2. The "short-term slow" way:
   This gets it done much slower, but in way that will make it much faster to perform code maintenance and enhancement in the future.

You might think the "short-term fast" way would always be the best. However it rarely is. The "short-term fast" way is actually the long-term slowest and most expensive way, because:

1. Only 10-20% of the money and time you spend on any feature will be from the first release. 80-90% will actually be for maintenance and enhancements.
2. The later in the process you wait to do work on something, the more exponentially expensive it becomes to do it.

So every time your programmer chooses the "short-term fast" way, it costs you more in the long run. That is what technical debt is. You will not see it as an end user when they release it to you, because your software works the same regardless of how they build it. But inevitably you’ll pay it back when you try to add to the software or have to fix bugs. And you will for it back with increased time, money, effort and missed deadlines.

So if the "short-term slow" way is so much better, why would programmers ever choose to incur technical debt? There are a few reasons (one good, and the others not):

1. Good reason:
   
   1. They understand that you need to get your first version to market as quickly as possible (perhaps to win funding). So they explain the choices to you and you choose to take on the technical debt for now. If you achieve your goal, you are fine with spending much of version 2 cleaning up the debt that you've accrued in version 1.
2. Bad reasons:
1. The programmer sees they are falling behind schedule and doesn’t want to miss the deadline so they take shortcuts to make up the time.
2. The programmer doesn't understand that there is a better way to implement a particular feature. So they unintentionally take on technical debt that they did not need to.
3. The programmer is too inexperienced to understand the trade-offs involved in technical debt and inadvertently takes it on without even considering it.

So how do you avoid this problem? Obviously you can’t rely on your programmer to announce, “I’m taking on technical debt, now!” when they themselves may not even realize it. And technical debt is very insidious because you can’t actually see it when you test the software (as an end-user). The software will look and works exactly the same way no matter how they build it (as long as they code it successfully). So testing won’t reveal it.

So what do you do? The way to determine when technical debt is occurring is called an inspection. If you are technical then you can do this yourself. If you are not (or are managing a project outside of your technical expertise) then you can hire a Tech Sherpa to do this for you. Doing so is like an investment and will save you far more than you spend.

Here’s what needs to be inspected:

1. Design/architecture inspections: After the design/architecture is completed, the documentation is reviewed. Technical debt is identified and either deemed an acceptable trade off or eliminated.
2. Code inspections: All code is reviewed (line by line) for technical debt. If you are using the spiral development method then this will occur right before or after each release to you. Again, technical debt is identified and either deemed an acceptable trade off or eliminated.

Managing technical debt does take time and effort. However it is an investment that pays back on itself many times over. On very small programs that you know will never grow into anything bigger, you may be able to safely ignore the issue of technical debt. But on medium to large sized projects, managing the debt is essential to completing the project on time and on budget.