If you’re working with legacy code, chances are you’ve inherited some technical debt. Infact, if you’re working with code, chances you’re already surrounded by technical debt of varying sizes, at least by some measures.
Some believe that technical debt is something to be avoided, and that technical debt that exists is a dirty secret that should be hidden. The reality is that technical debt is a fact of life when code iteratively changes to deliver product solutions.
Striving for programming perfection is great in principle, but ultimately code is meant to deliver features, and there is always a good, better and best approach, with many other variations in-between.
Over the last year at Wikimedia Deutschland we have worked on refining how we record, triage, prioritize and tackle technical debt within the Wikidata and Wikibase product family.
There are many thoughts out there about how to track, tackle, and prioritize technical debt. This post is meant to represent the current status of the Wikidata / Wikibase team. Hopefully you find this useful.
Wikidata / Wikibase products and processes
Within the Wikidata and Wikibase family of products at the time of writing this we currently have ~3 product managers, ~3 engineering managers, ~14 engineers and 1 tech lead (that’s me), as well as some other departments that we as a team work with, such as UX (User Experience) and ComCom (Community Communications). The wider team for Wikidata and Wikibase is now stands at around 40 people.
The team is responsible for making sure that wikidata.org and the Wikibase software both continue to develop and progress toward their goals. To do this the team works on and maintains roughly 50 code repositories.
The core part of this work revolves around a MediaWiki extension called Wikibase which was created back in 2012 by a team of under 13. Of the original team, only 1 remains working on the Wikidata / Wikibase team.
We all work within a “Journey model”, which is modeled after the Spotify model. You can find a slightly outdated introduction to this model on Github. (I will update this if I can link to a new source of truth). A short summary would be that we act as one big team, focused around a campsite team. Journeys are proposed and teams form from the campsite around these proposals, separating into an agile process to complete whatever work was proposed. The campsite team remains all year round working on general smaller product and tech backlogs.
Introduction to our technical debt
Wikidata and Wikibase has it’s own fair share of technical debt. Some example items from the backlog of various sizes, all of which can be found on our ticket / task system can be found below.
Small: This could be something as simple as a continuous integration failure message being less than helpful and not pointing you directly to the problem you should be looking at (T282086).
Medium: Some leftover code and or configuration from a previous refactoring that is yet to be entirely cleaned up (T241975).
Large: Changing how a core system, integral to major functionality for production, is run based on a change in requirement (T265198).
Most of this tech debt is managed on a single phabricator board (wdwb-tech), and a single spreadsheet (a topic for another blog post).
Noticing and recording the debt
“Technical debt should be tracked” is one of the things folks generally agree on. There is no point in hiding it!
Anyone can create a new ticket, add it to the wdwb-tech board, and have it processed into the backlog, and work to be done by the team. This includes people on the Wikidata Wikibase team, but also external teams such as the Wikimedia Foundation, or other users and developers on Wikidata and Wikibase.
New tickets enter the boards “inbox” and gradually move their way from left to right through columns such as “research” and “discussion” eventually making their way to the “backlog” where they are deemed appropriate and ready to be prioritized.
A few rules are also setup for other tickets. Such as any ticket tagged with either wikidata or wikibase and one of any number of tech related tags will automatically be added to the wdwb-tech board inbox. These extra tags include tech-debt, performance, logspam, PHP compatibility and many more.
As the tech lead, my role currently covered “Maintenance and Tech Debt Product Ownership”, so this board is my domain, and my responsibility to make sure things progress through the process.
🕰️ In previous years, though the wdwb-tech tag and work board existed, I mainly maintained the idea of what technical debt existed in my head and on existing workboards, such as the behemoth of the Wikidata board which stands at 3000+ open tickets.
Prioritizing the debt
Now the prioritization of tickets with this tech focus are rather objectively scored based on a list of criteria. This scoring is managed in a spreadsheet, with a little bit of magic to make things easier (which will be covered in a future blog post).
The scoring mainly focuses around yes/no questions that are meant to gauge the impact of a ticket, along with an effort / time cost estimation all resulting in a single number that can be ordered. The questions that we use are by no means perfect, but it serves as a good basis for iteration and discussion around what is important and why.
Some of these questions very roughly include:
- Does a technical stakeholder of Wikidata / Wikibase care about this in some way? Stakeholders include our own teams and departments, as well as WMF teams, volunteer devs etc, all with a slightly different weighting.
- Does this improve the situation in an important area, or fix a known issue in that area? Such as security, performance, production log spam, continuous integration stability, confusing code, outdated pattern, etc.
- Does this enable the team or code to do something that we believe in principle should happen? Such as removing outdated or deprecated things, use very modern things, or possibly prevent incidents.
- Has this ticket been involved in past topics such as outages, or manual human interventions? Or is work planned on the area of code the ticket covers in the future?
This spreadsheet will be the topic of a future blog post, so watch this space for a deeper dive including some of the “magic” that actually makes it usable.
🕰️ In previous years I primarily and probably rather arbitrarily weighed tickets up against each other, forming some backlog that would be discussed with PMs and fed into a teams backlog.
Getting the debt picked up
We currently have 2 paths for tickets from the tech debt backlog to get picked up.
Small and medium tasks generally get taken to a campsite storytime session that happens every week. In this session the PM and I (Tech lead) will alternate bringing tasks to the team from our respective prioritized backlogs. This is an agreement that started at some point to avoid the extra discussion overheads around what is and isn’t important, and trying to prioritize between the product and tech focused backlogs.
The tasks will briefly be discussed, voted on to measure complexity, and if deemed to not be too complex they will be picked by the team for the week. If a task is decided to be too complex it will bounce back to the backlog and need to find another path to getting picked up.
If a task that bounces back due to complexity can be split up in some sensible uncontroversial way, then it may find it’s way back to the campsite storytime for another chance at being picked up as part of the regular weekly process.
If a task truly is “too big” and would result in a very long running focus absorbing lots of engineering time then a dedicated team would generally end up forming around the topic at some later stage determined by engineering managers along with product managers slotting work in around other annually planned roadmap items.
🕰️ In previous years I would generally discuss with product managers (PMs) & engineering managers (EMs) the top tasks that I would deem important, and why we should tackle them. PMs and EMs would then organize getting the work scheduled and picked up one way or the other.
A few choice quotes that might entice you to read more…
Technical Debt is a hidden cost that can impact value delivery, empiricism and also the team’s morale. Sooner you tackle this is better.The hidden cost of technical debt (serious-scrum)
Remember that if you are an engineer, it’s your job to raise technical debt issues as early as possible, and to make sure that you are able to explain their impact in succinct and meaningful ways. Managers: it’s your job to listen and to create the space for the issues to get worked on.How to argue the space to tackle technical debt (theengineeringmanager.com)