Technical Debt vs Technical Bankruptcy
I had an interesting debate with an engineer colleague recently - on the topic of technical debt. In particular, he was framing almost everything in terms of “strategic is good - everything else is technical debt”, with “strategic” defined as delivering the highest quality code first time without any re-writes.
Before you completely dismiss his position, it’s worth adding that he worked in data engineering. Data pipelines don’t always suit the typical fast iteration style that engineers in other disciplines take for granted. A web application is often useful well before it has all the features on its backlog - but the output of a data pipeline is often unusable until it is finished and “correct”.
Nevertheless, I was surprised at the resistance to any consideration of iterative development, and it prompted me to think more deeply about the analogy that is the term “technical debt”.
Monthly Car Payments
Using credit to buy a car is fairly common occurrence. Not everyone loves it, but it’s a useful tool for many. Let’s think in detail about the choices we are making here.
Option 1: Save up £x and purchase the car for cash - this results in us paying the least amount for the car…but we have to wait until we’ve saved up before we can drive anywhere.
Option 2: Take a line of Credit. We pay y% more for the car over the next 5 years…but we get to drive the car for the next 5 years as well. This means that we have the benefits of car ownership immediately, and can spend the next 5 years driving to work, dropping the kids off at clubs and enjoying weekend drives to attractive destinations.
Paying a small(ish) premium for 5 years extra driving seems like a great deal. It’s why it’s so popular, but there is an obvious risk to this undertaking.
What if I lose my ability to complete the monthly payments?
Monthly Tech Debt Repayments
The tech equivalent to taking a car loan is to ship a feature early with promise to “tidy up” the code later.
This isn’t actually a big problem - as long as you have the resources to complete the tidy up. If a new feature is predicted to make the business $xK per day…do we really need to wait a fortnight while we complete the documentation? We can happily complete the documentation after the fact .. as long as we have the resource available to do it.
This might sound simple or obvious, but if it really was - we wouldn’t get into technical debt in the first place. There are some rcommon day-today business scenarios that can rapidly increase the tech debt of a project - often despite good intentions of all involved.
Be wary of who is paying the debt!
A pattern I often see is as follows:
Customer: “You’ve quoted 6 weeks for that feature, but I really want it earlier. Is there anything you can do?”
Engineer: (trying to be helpful)“I could ship after 3 weeks, but I would need to perform 2 hours a day of manual activities to keep the site / feature working”
Customer: “Sounds great! Let’s go for that!”
In this case the customer is happily taking on the debt because they are not paying for it - the engineer is! The poor engineer has now acquired 2 hours of manual activity per day…which naturally eats into their ability to finish the development properly.
This can be exacerbated if we’re working in the following environment.
Monday:
Customer: “Let’s go live. I understand that you will need to spend the next two weeks making the code more robust, and removing the need for manual activity”
Tuesday:
Customer: “I have a new urgent feature request. It is top priority”
This kind of scenario can be a natural consequence of a busy, competitive workplace, or a genuinely toxic environment, but it is a pattern to look out for.
Ultimately there is a quality/speed balance that is a natural fit for your organisation. Finding it is key to working well with your stakeholders.
Be particularly conscious of offering stakeholders options where you will be paying the tech debt on their behalf. I have genuinely worked in organisations where entire teams are dedicated to running semi-manual processes. In some case the concept of actually addressing the cause of the workarounds has stopped being discussed altogether.
Technical Bankruptcy
Technical Bankruptcy isn’t a particularly popular term - perhaps because it sounds so menacing, but it can be realistically applied to systems where the cost of addressing technical debt is greater than the budget allocated to support it.
Is my definition correct? We could just say that sometime the cost of replacing a legacy system gets particularly gigantic, but I like the emphasis on the support costs. A small team can happily run regular tech upgrades on (for example) a SQL Data Warehouse, but if they leave it too long, the complexity and risk can rapidly rise to unpalatable levels.
Two techniques when faced with Technical Bankruptcy
Tech debt in legacy systems can be a genuine drain on organisations. They constantly slow down or prevent critical change programs and inevitably entangle themselves into process and systems that spread accross the organisation.
In common with many of my articles, I’d like to look at this from a phycological angle.
Nobody wants to spend thousands of dollars just to end up where they started.
I.e. If it’s going to cost thousands of pounds to upgrade to the latest version of SQL Server…but no tangible business benefits will be achieved as a result. We’re ultimately paying of debt here, and paying off debt is no fun whatsoever.
Look for Technology Pivots
The world of technology happily presents us with regular transformational burst of new capabilities - think mobile, cloud and AI. This gives us an opportunity to build something with more capabilities instead of simply paying down our tech debt. I.e. spending thousands on upgrading a database server sucks - but switching to a platform with AI capabilities out-of-the-box sounds much more exciting!
New, not like-for-like
I recently had a project that involved migrating many analytics teams from their old system to a nice new shiny one. The teams were onboard because of the new capabilities they were going to get, but I rapidly spotted a clear pattern in how quickly I could migrate each team.
Teams looking for a like-for-like migration were painful to migrate. We immediately required extensive analysis of their existing setup, careful re-writing of data pipelines, and of course, they wanted to run both systems in parallel before finally committing to the new technology. I had 200 teams to migrate, and this was going to take forever!
Teams that started new projects were a dream in comparison. They needed some simple initial setup, and then they could be on thier way! I was left with a strange situation whereby some teams were rating the new systems as a dratitic success, and some were complaining vociferously about delays and difficulty! It certainly made the monthly status updates a challenge to write!
Ultimately, I found that encouraging teams to start afresh, even if they had critical existing reports, was much more effective and actually met with relatively little resistance. It gave teams a chance to actively assess what they really used, strip out unnecessary items, and focus on adopting best practice for the new technologies.
Peoples initial instinct was to replace like-for-like, but encouragement to trial a new approach often led to a change of heart and a pivot to a better way of working.
Summary
Acquiring some tech debt is inevitable, but small amounts can actually be desirable (or at least benign) if we are heavily focussed on projects with high time-to-market benefits.
Avoiding too much tech debt is predictably bad, and follows a similar process of regular debt. It spirals and grows until it becomes unmanageable.
If Tech bankruptcy has occurred - consider more radical, benefit giving alternatives instead of like-for-like migrations. It requires ambition and energy, but can often allow us to focus on delivering genuine values as opposed to handing over cash for past mistakes. This doesn’t happen in the real world examples, but IT is a magical industry, and rewards creativity in unique and fulfilling ways.