<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[scalable.africa - Medium]]></title>
        <description><![CDATA[A bunch of Nigerian “Senior” software developers curious about building and solving large scale problems; we try to make sense of our journey and document the process as we grow, to help others that choose to follow this path. - Medium]]></description>
        <link>https://medium.com/scalable-africa?source=rss----f4f849ab7da7---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>scalable.africa - Medium</title>
            <link>https://medium.com/scalable-africa?source=rss----f4f849ab7da7---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 08:56:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/scalable-africa" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Conventions and Unconvention]]></title>
            <link>https://medium.com/scalable-africa/conventions-and-unconvention-cf52da28daf9?source=rss----f4f849ab7da7---4</link>
            <guid isPermaLink="false">https://medium.com/p/cf52da28daf9</guid>
            <category><![CDATA[software-explanations]]></category>
            <category><![CDATA[software]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[software-convention]]></category>
            <dc:creator><![CDATA[Antisocial Extrovert]]></dc:creator>
            <pubDate>Sat, 15 May 2021 21:42:47 GMT</pubDate>
            <atom:updated>2021-05-15T21:42:47.537Z</atom:updated>
            <content:encoded><![CDATA[<p>Conventions and Unconvention</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/563/1*3fNDCJwBcgJRumO7tbVTBw@2x.jpeg" /></figure><p>There have been a series of arguments, especially in the African tech scene, around “enterprise” software engineering languages.</p><p>Granted, we all agree that it usually lacks merit. But for argument sake, lets look beyond the initial “craze”.</p><p>First we start with how large scale products are engineered, we contrast this with cooking.</p><p>Lets say you want to cook a pot of egusi (your software product), you have a team of 6 chefs (your dev team). Now each chef has their own way of cooking egusi, some use bitterleaf, some use another leaf, some prefer it with chicken, some beef, some prefer palm oil, some groundnut oil etc.</p><p>As a head chef (Engineering manager) in charge of building this pot of soup. How do you ensure you get the best out of all these chefs with their different expertise and process, but center that towards one goal?</p><p>You start by creating a single way for them to cook, a set of rules on how this pot of soup should be cooked. These rules are called <strong>Conventions.</strong></p><p>They can be: what type of leaf to use? what type of protein to use? how long should these ingredients boil for? What oil to use, maggi, thyme? etc</p><p>In the software engineering world, we call these rules “<strong>software engineering conventions</strong>”. They revolve around variable naming, folder structure, software design principles etc.</p><p>Why do we need them? In the case of our egusi soup, there is a saying “too many chef spoil the broth”. This doesn’t take into account the use of cooking conventions</p><p>A quick <a href="https://www.google.com/amp/s/amp.firstwefeast.com/eat/2013/11/10-biggest-guinness-book-world-record-breaking-foods/">google search</a> would show a host of meals which have been prepared by a large number of people. It is the same with software products</p><p>In the world of Open Source Software contributions, this is Infact key. You have millions of people contributing to a particular project, without key conventions around that project, everything falls into disarray</p><blockquote>Please forgive my cooking skills, or lack of it. No cooking ingredient was harmed (or house burned) in the composing of this article</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cf52da28daf9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalable-africa/conventions-and-unconvention-cf52da28daf9">Conventions and Unconvention</a> was originally published in <a href="https://medium.com/scalable-africa">scalable.africa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Engineering Management and The Law of Diminishing Returns]]></title>
            <link>https://medium.com/scalable-africa/engineering-management-and-the-law-of-diminishing-returns-4b89bc92f896?source=rss----f4f849ab7da7---4</link>
            <guid isPermaLink="false">https://medium.com/p/4b89bc92f896</guid>
            <category><![CDATA[engineering-mangement]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Bayo Opesanya]]></dc:creator>
            <pubDate>Mon, 03 May 2021 08:58:03 GMT</pubDate>
            <atom:updated>2021-05-03T13:07:20.426Z</atom:updated>
            <content:encoded><![CDATA[<iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2Fl2JHVUriDGEtWOx0c%2Ftwitter%2Fiframe&amp;display_name=Giphy&amp;url=https%3A%2F%2Fgiphy.com%2Fgifs%2Fdisneyzootopia-l2JHVUriDGEtWOx0c&amp;image=https%3A%2F%2Fmedia2.giphy.com%2Fmedia%2Fl2JHVUriDGEtWOx0c%2Fgiphy.gif%3Fcid%3D790b7611c13735a952666029eb1cb2a4433f02ebe2a56071%26rid%3Dgiphy.gif%26ct%3Dg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="184" frameborder="0" scrolling="no"><a href="https://medium.com/media/f85146e74353656a9c4c08fc870dbfc9/href">https://medium.com/media/f85146e74353656a9c4c08fc870dbfc9/href</a></iframe><blockquote><em>Diminishing returns, also called law of diminishing returns or principle of diminishing marginal productivity, economic law stating that if one input in the production of a commodity is increased while all other inputs are held fixed, a point will eventually be reached at which additions of the input yield progressively smaller, or diminishing, increases in output.</em></blockquote><p>— <a href="https://www.britannica.com/topic/diminishing-returns">Britannica.com</a></p><p>The job of anyone who leads Engineering at a company, is to collaborate with the business to use technology to build solutions and make profit in the most sustainable way possible. That said, one of the major struggles Engineering Managers face is summarised in one question: “How can we move faster?”</p><p>As an Engineering Manager, you may be pressured to “scale up” your team by hiring more people to match the workload, so that you can get more done quickly in less time than it currently takes. This articles seeks to help you ask another question before you determine that growing your team 3x in 5 months is the way to go, because you really shouldn’t be scaling up human teams like they are servers. Also, by hiring, you’ll be significantly raising the company’s expense on salaries. You have to be absolutely sure that it’s an expense that needs to be made and justified by the output.</p><h3>How many business-critical features are currently in development?</h3><p>Based on my experience, you definitely should limit work in progress as much as possible if you’re more interested in getting things done quicker than being/looking busy. <a href="https://lethain.com/limiting-wip/">This article</a> by Will Larson, author of the Elegant Puzzle (a book which I recommend) does a good job explaining why it is necessary, but I’ll like to add two more things:</p><ul><li>Engineering doesn’t stop after active development: Imagine that to implement a new feature, you had to create 3 new database tables, 2 new consumers and one cron job (this example is non-fiction lmao). The feature is tested, works well and is deployed. Everyone is excited. But before you jump on the next thing, you have to think about maintaining that system. The new feature comes with its maintenance burden which isn’t trivial. Now, multiply this by 4 and you can easily see how you have exponentially increased the team’s workload if those 4 systems were deployed at once. What about making changes to those systems as you get user feedback or changing business requirements? Add this to the burden of implementing another 4 of this system and you can see how quickly life can spin out of control for the team.</li><li>Documentation isn’t a silver bullet: While documentation is great and necessary, it becomes useless if not consistently updated. If you have so many things going on at once, it becomes increasingly difficult to keep documentation up to date as multiple changes occur at once and different scopes of work intersect with one another.</li><li>Hiring is expensive: This isn’t only financially, but even in the context of speed. Onboarding people takes time, and has to be done by someone on the team who already has the burden of features to build or maintain. Every new addition to the team increases <a href="https://www.solutionsiq.com/learning/blog-post/lines-of-communication/">communication lines</a>, and this will need to be accounted for.</li></ul><h3>Maybe what you should manage is speed</h3><p>The layman understanding of the Law of Diminishing Returns is “too much of anything is bad for you”. It’s good to move fast, but not too fast. It’s good to grow, but not too quickly. In this “fast-paced” world, what many people are trying to achieve with Engineering is impregnating 9 people in the hope that they’ll be able to produce a baby in 1month. It is why whole systems are oversimplified and presented as “simple integrations” or “plugins”.</p><p>If you hire too quickly, you’ll be slower.</p><p>If work grows too quickly, you’ll be slower.</p><p>If you push people to work long hours till they burnout, you’ll be slower.</p><p>You’re better off with 4 people working on 2 things, than 6 people working on 6 things.</p><p>Your best shot at greater speed is doing work in sizeable chunks and growing sustainably.</p><p>Selah.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4b89bc92f896" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalable-africa/engineering-management-and-the-law-of-diminishing-returns-4b89bc92f896">Engineering Management and The Law of Diminishing Returns</a> was originally published in <a href="https://medium.com/scalable-africa">scalable.africa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Fractional Technical management in (Africa?)]]></title>
            <link>https://medium.com/scalable-africa/fractional-technical-management-in-africa-68621b8203c1?source=rss----f4f849ab7da7---4</link>
            <guid isPermaLink="false">https://medium.com/p/68621b8203c1</guid>
            <dc:creator><![CDATA[Antisocial Extrovert]]></dc:creator>
            <pubDate>Sat, 01 May 2021 12:45:54 GMT</pubDate>
            <atom:updated>2021-05-01T12:51:48.526Z</atom:updated>
            <content:encoded><![CDATA[<p>Fractional Technical management in (Africa?)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/383/1*xre5mVgUs_q_Bg8rxBbEEQ@2x.jpeg" /></figure><p>Between January and March this year, the number of founders (both technical and non technical) who reached out to me or my friends for engineering managers was astronomical</p><p>It seems we have reached a maxima. The ratio of senior technical talent available vs the number of growing businesses requiring this talent is now highly skewed.</p><p>As we look towards remote companies and the gospel of Japa, a lot of “senior” talent opt for the more profitable path of remote work, fewer continue on a path of management. This trend unfortunately creates a barrier/ceiling in the growth of local engineering talent, as there isn’t much of a roadmap to follow from “older” and more experienced managers; we end up just “winging it”.</p><p>Early 2020, I and a couple of people who would later be best of friends, met up to discuss this exact problem. How could we break this ceiling in our respective careers as senior engineers into managing senior engineering talent.</p><p>This was the birth of what is now <a href="http://scalable.africa/">Scalable</a>.</p><p>Our mission is clear, we aim to improve on ourselves collectively, pass knowledge to the coming “generation” and in doing this, grow the local engineering ecosystem.</p><p>One of the projects we currently have is called <strong>Pipeline. </strong>It is a peer learning process of senior and junior talent. By working with a “senior” engineer on a few project, a junior learns a lot on their technical process, gain projects under their belt, is battle tested and ready to integrate into a company</p><p>While this is currently in the works, we figured the need for another more advanced project: <strong>Fractional Technical Management. </strong>A lot of companies require but do not have the resources to hire a manager on a salary of $3000 — $4000.</p><p>In this case, what if we were able to split a manager’s time between companies? A lot of “growing” companies do not need a “Good” manager’s full time attention, only for the important things around hiring, architecture, follow on review an creating of engineering processes</p><p>This leaves a lot more free time on their hands to perform these tasks for more than one company at a time.</p><p>How would this look like? Company A is looking to build out a technical team and requires the services of a technical manager, here comes Raymond.</p><p>Raymond fits the profile, and has the needed experience to perform these tasks, but his fee is a standard $3000 monthly. Unfortunately company A is still being bootstrapped and cannot afford Raymond’s fulltime services</p><p>We offer an alternative; how about $800 a month, for 10–12 hours of Raymond’s time weekly. The average full time working hours is 40–50 a week. Company A can afford that, they still get their Quota from raymond</p><p>Raymond is also able to repeat this process for Companies B, C and D. Altogether, raymond earns his complete fee, each company earns their quota from an engineering manager and are able to build out their product.</p><p>At <a href="http://scalable.africa/">Scalable</a>, we currently have a vetted list of engineering managers, with a couple of companies this is being done for. Do you need this for your startup? Reach out to me @<a href="https://mobile.twitter.com/bigbrutha_?lang=en">bigbrutha_</a> and let’s have a chat</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68621b8203c1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalable-africa/fractional-technical-management-in-africa-68621b8203c1">Fractional Technical management in (Africa?)</a> was originally published in <a href="https://medium.com/scalable-africa">scalable.africa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[99.9999% availability in the mud ]]></title>
            <link>https://medium.com/scalable-africa/99-9999-availability-in-the-mud-1990fffa4e3d?source=rss----f4f849ab7da7---4</link>
            <guid isPermaLink="false">https://medium.com/p/1990fffa4e3d</guid>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[replicaset]]></category>
            <category><![CDATA[mongodb-atlas]]></category>
            <category><![CDATA[database]]></category>
            <dc:creator><![CDATA[RT]]></dc:creator>
            <pubDate>Sun, 14 Mar 2021 19:29:26 GMT</pubDate>
            <atom:updated>2021-03-14T19:29:26.003Z</atom:updated>
            <content:encoded><![CDATA[<p>On Monday, 8th March 2021, I noticed that our servers at <a href="https://medium.com/u/be655df6bfdb">gomoney</a> started crashing all of a sudden. We run our platform in a small k8s cluster with 5 nodes (high-performance VMs). One of the pods (an individual server that runs code belonging to a domain) had crashed about 51 times in the last 12 hours since it was last restarted. This was bad, really bad. The customer success team had not reached out to complain that customers could not log into their accounts or perform certain operations, so that was a good sign (it wasn’t affecting the platform uptime). As a stopgap, we scaled each of the affected services by 2x, not like that would solve anything, but at least requests would be routed to available pods if one of them was crashing. Anyway, to the sweet stuff 😂.</p><h3>The Errors</h3><p>The error messages from the servers were logged with different error messages, which didn’t help narrow it down at all.</p><pre>MongoError: not master<br>MongoError: operation was interrupted<br>MongoError: not master and slaveOk=false<br>MongoError: Plan executor error during findAndModify :: caused by :: operation was interrupted</pre><p>The full crash logs look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VOpfBn0xkT1J_85l6caByg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LL8MhkoAmPO__Li04bSTrA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tOqUMGMB7H8tIlwZeoILiw.png" /></figure><p>At this point, we suspected it was our mongoose (a MongoDB ORM) version, so we figured we’d just upgrade to the latest mongoose version in one of the services and monitor it to see if the crash rate reduced (we setup monitoring a while back, but the config has been messed up, we’ll need to fix that soon 😭). Well, this didn’t solve anything, the pods were still crashing and at an alarming rate.</p><h3>Debugging The Issue</h3><p>We started Googling and checking everywhere (StackOverflow, Github, MongoDB docs, Mongoose docs) to see if this was an issue other people had experienced before, they had, but not in an exact way. The next thing we did was to change the way the mongo client running on the pods was connecting to the server. We played around with the connection options.</p><p>Our connection string options which were initially set to</p><pre>retryWrites=true&amp;w=majority</pre><p>was changed to</p><pre>retryWrites=true&amp;readPreference=primary&amp;replicaSet=our-replica-set-name</pre><pre>retryWrites=true&amp;readPreference=nearest&amp;maxStalenessSeconds=100&amp;replicaSet=our-replica-set-name</pre><p>Thesee also didn’t solve anything. At this point, I was personally convinced it was not from our config, because we had not changed it since initial deployment. So why was it crashing all of a sudden? 😫</p><h3>Finding The Cause (with help though)</h3><p>We set our sights on the database cluster itself, I was checking the server metrics and noticed one of the replicas was restarting more times than it should, the reason? I didn’t know. In fact, it restarted as we were on the zoom call debugging. Fun times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DdVcmGVwVm8l0JPszydU6g.png" /><figcaption>Server Restart Chart for 3rd Replica (This week)</figcaption></figure><p>The red lines in the chart above represent a “server restart” event, to compare it to what it should look like, see all three servers side by side in the same duration. You can see that the 3rd replica was obviously restarting way too many times.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*endAXQ_mcXdq_i5OVdAD4g.png" /><figcaption>Server Restart Chart for 3 Replicas (2 months)</figcaption></figure><p>It was time to call in the big guns aka MongoDB Support 😂, as we were obviously way in over our heads. We created a support ticket and presented all this information.</p><h3>The Revelation</h3><p>Apparently, that faulty replica was affected by a BI Connector (Business Intelligence MongoDB driver) process that was using up a lot of memory of that replica. In all cases, they found out that the not master and slaveOk=false errors were occurring on node 00-02 , the faulty replica. There was a mongosqld (BI Connector Process) running on that node and the process was using all of the available memory and leading to instability in the cluster.</p><p>They were seeing high heartbeat response times to and from this replica and also high replication lag (it could not get the latest data copied to it). The frequent delays were resulting in that replica <a href="https://docs.mongodb.com/manual/core/replica-set-elections">calling elections</a> because it is not able to process heartbeat requests from the other members promptly, due to the memory pressure. After an election, it could render the whole cluster useless because it could not respond to clients on time when it was elected as the primary/master. I saw a case where there was no primary for about 5 seconds. 5 seconds is a really long time for a computer. Omo.</p><p>Errors also occurred when an election had made another replica the primary making our faulty replica a secondary. In one instance node 00-01 was elected to become the primary but node 00-02 was still in the process of closing all of its current connections, this would also leave the clients in an unstable state and, boom, another crash.</p><h3>The Solution</h3><p>We disabled the BI Connector and would be provisioning an <a href="https://docs.atlas.mongodb.com/reference/faq/deployment/#what-are-analytics-nodes-">analytics node</a>—another read-only replica that won’t take part in the election process— then configure the BI Connector to be hosted by the analytics node instead of on node 00-02. This event made me realise how wonderful database technology is. The election process that happens in replica-sets is such an important event, such that it can make or break your application’s uptime. Thankfully we had set our clients to write to the primary node and read from the secondary nodes. So all the crashes happened when apps were trying to read from that faulty replica.</p><h3>Takeaways</h3><h4>What happened to our staging environment?</h4><p>It is worth mentioning that this didn’t affect our staging (test) environment at all which raised a few eyebrows, the reason being, the database cluster for our production environment runs in a replica set, while the staging environment runs as a standalone node, you know, those 512MB “SANDBOX” clusters that MongoDB Atlas provides. That said, I think we should have (and going forward will do so) run our staging environment also in a replica set, not like it would have solved anything though, but you’d want your staging environment to closely mirror the production environment on all layers (database, app, transport, etc).</p><p>Don’t be afraid to ask for help, do your own debugging first, then package all that research to whomever you’d be asking for help. That can help them build on what you already did. This is particularly important if you want to ask the support staff for help so you can narrow down the issue.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1990fffa4e3d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalable-africa/99-9999-availability-in-the-mud-1990fffa4e3d">99.9999% availability in the mud 💀</a> was originally published in <a href="https://medium.com/scalable-africa">scalable.africa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Technical debt]]></title>
            <link>https://medium.com/scalable-africa/technical-debt-51c94658761f?source=rss----f4f849ab7da7---4</link>
            <guid isPermaLink="false">https://medium.com/p/51c94658761f</guid>
            <category><![CDATA[team-management]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[debt]]></category>
            <category><![CDATA[technical-debt]]></category>
            <dc:creator><![CDATA[RT]]></dc:creator>
            <pubDate>Fri, 15 Jan 2021 20:58:39 GMT</pubDate>
            <atom:updated>2021-01-15T23:02:24.211Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T-zIJlLij6FOdal-H6P_rw.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@dancristianp?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Dan-Cristian Pădureț</a> on <a href="https://unsplash.com/s/photos/art?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Technical debt is defined as “<em>…the implied cost of additional rework caused by choosing an ill-suited solution —could be easy, limited or over-engineered— now as opposed to using an appropriate solution.</em>” [1]</p><p>It is the effort that builds up when a team makes a conscious decision to implement code that’s regarded as a <em>quick fix,</em> instead of building out the best solution. The <em>quick fix</em> is the “debt” and like any kind of debt, it accrues interest over time, so the additional effort built up which could be time, money or resources is the tech debt that builds up and makes it harder to easily implement a better solution. Tech debt is not bad and it is unavoidable; time to market pressure amongst others is what makes tech debt unavoidable, that is if the team doesn’t plan on being in development hell forever 🌝.</p><h3>Where Does Technical Debt Come From?</h3><p>All these incur technical debt because they harm productivity. Some can be directly influenced by the engineering team, others can be influenced by the management.</p><ol><li>Bad architecture</li><li>Inconsistent project structure</li><li>Code duplication</li><li>Low test coverage</li><li>Poor code documentation (or a complete lack of documentation)</li><li>Potential bugs not fixed on time</li><li>Unnecessarily complex code (over-engineered code)</li><li>Code smells</li><li>Scope creep</li><li>Lack of a QA process</li><li>Inconsistent deadlines</li><li>CTO who thinks they’re a god 👀</li></ol><h3>Are there other types of debt?</h3><p>Not all software projects issues are as a result of Technical Debt, so we may ask ourselves, are there other types of debt? These are a few types of debt which are faced by teams around the world.</p><ul><li><strong><em>Quality Debt</em></strong><em>: is </em>caused by identified defects in the software built.</li><li><strong><em>Process Debt</em></strong><em>: is </em>caused by a lack of team processes or poorly implemented processes.</li><li><strong><em>Feature Debt</em></strong>: <em>is </em>caused by a team releasing useless or delayed features. It can also be caused by releasing incomplete features with hope to go back and finish and never going back.</li><li><strong><em>User Experience Debt</em></strong>: <em>is </em>caused by inconsistent or poor user experience which has to be redone later.</li><li><strong><em>Skill Debt</em></strong>: <em>is </em>caused by a lack of skills which are required by the team.</li></ul><h3>Categorization</h3><p>How is debt accrued you ask? Well, tech debt can be introduced into software projects in several ways. These three types of debt:</p><ol><li><strong>Deliberate technical debt</strong> can be described as intentional choices (i.e., shortcuts) made to deliver on time.</li><li><strong>Unintentional technical debt</strong> can be described as an accumulation of debt over time as a result of things like ignored coding standards and practices, outdated technical design, or debt incurred from constant rework and ad-hoc features added to legacy code.</li><li><strong>Unavoidable technical debt </strong>can be are caused by factors out of the control of the development team. A third-party system upgrade can force the team to rework some part of the system. Or a language’s/library’s version becomes outdated making the dependent code vulnerable to attacks. Systems or tools that the software depends will evolve and can negatively impact your software.</li></ol><h3>Scenarios</h3><ol><li>Say you have an API with an endpoint that takes 1000ms to return a response. Clients consuming that endpoint may wait over a 1500ms —due to reduced network speeds— to receive a response from the server. If you refactor the database queries or introduce a cache layer, we can bring the round trip time (RTT) to 500ms. If you choose to leave it, the performance will degrade over time and the RTTs for that resource will only get worse as the database grows or modifications are made to that resource. This is an example where technical debt is incurred because of the platform architecture.</li><li>Say you worked on a large part of a large codebase and didn’t write any tests because you had to continue meeting deadlines —I know you wanted to write tests but you just couldn’t. This is not your fault, as the business doesn’t see writing tests as important because it doesn’t directly impact the end-user; their words, not mine 🤷🏿‍♀️. After a few years, you get another gig and move to a new company. Now Raymond, oh poor Raymond; “the new guy”, has inherited the codebase. Before her can make changes to the codebase, he has to go through a lot your code to understand your thought process. To get new features out, he has to understand that neatly nested ternary that made you feel smart or that map-reduce that felt satisfying to pull off. The additional time it will take Raymond to understand and refactor the codebase would have been significantly reduced if you had written the tests originally like you intended to. Similarly, if you have been allowed to refactor that part of the project earlier, it would have made it easier to modify now. Some Raymond’s would duly inform management that a rewrite would be a better alternative than managing the current mess. This is a very common case that almost all the devs I know have experienced and it always ends with tech debt being accumulated.</li></ol><p>There are a ton of other examples of how tech debt can be accrued, if I were to list them out I don’t think I’d ever stop typing 😭.</p><h3>My experience</h3><p>In 2019 when my team built the first version of <a href="https://medium.com/u/be655df6bfdb">gomoney</a>, we handled certain account properties as a single number; from the account balance to the account’s transfer limits. It was an easy, albeit naive implementation which later came back to haunt us because we eventually built a lot of systems around those two main account properties. Over time the business needed to have specific limits for different kinds of transactions and this posed a problem as the current implementation only supported one limit per account. The ensuing refactor affected about 49 files. That’s a considerable surface area of a project of its size, other parts of the platform were not affected as we’d gone ahead to ensure proper separation of concerns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r5-V3L1JsEh4WZwUgenesQ.png" /></figure><p>In my six years as a developer, I’ve had to refactor legacy codebases, rebuild parts of or entire systems, either because the original author of the code didn’t use descriptive variable names, didn’t architect a robust solution or over-engineered the solution. As I’ve heard and read over the years “Your code will be read more times than it’ll be written” [4], so in the bid to get that new feature out, be intentional about how much debt you want to take on and put things in place ahead of time so when you or someone else wants to pay it back, it won’t be a hassle.</p><h3>Technical Debt Is Like Tetris</h3><p>Eric Higgins describes it perfectly, “PMs work with software developers to prioritize features will be built and released to customers next. Finishing a Tetris row is like shipping a feature. Shipping complex features require more rows. The software requires time more time to build correctly and tradeoffs are always made due to business requirements. The tradeoffs are represented by the “spaces” between bricks in Tetris. All code has technical debt. That’s normal and we can keep playing Tetris with a few gaps.” [2]</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/320/1*PL6ykKNsbDJgJ5YTP1WHmg.gif" /><figcaption>Me playing Tetris badly 😭</figcaption></figure><p>In our case, we paid our debt when we saw it fit to pay it back, and not before. Doing that would have been premature optimization; premature optimization is not fun at all, because you’ll be optimizing for scenarios that haven’t happened yet, based on assumptions, not experience or fact. I have a personal rule, “first make it work, then make it robust”. I urge you to not try to be a perfectionist, be like a multi-pass compiler, don’t optimise on the first pass, that’s what other passes are for 🙃😌.</p><h3>Prioritization</h3><p>This is a source of great tension between product management and software development teams.</p><ul><li>Product managers often have to prioritize features over addressing technical debt because new features for customers have the potential to generate revenue.</li><li>In contrast, software developers recognize that they could more quickly deliver high-quality features and functionality if they could remove/reduce the technical debt, which impacts the stability and the performance of the software as it accumulates.</li></ul><p>I mentioned tech debt categories earlier, however, the categorization does not help with prioritization. Categorizing technical debt using customer impact as the clincher gives the product and development teams a basis for prioritization. The team’s focus should be placed on paying back the technical debt that impacts performance and security which is usually on par with building out new features and functionality.</p><p>In this way, we make sure that there’s no reduction in user experience due to a slow, unresponsive or buggy application or platform. It also ensures that we respect data privacy and protection of the customer’s sensitive data by addressing potential security issues.</p><h3>Solutions</h3><p>The team at the Agile Alliance detailed a few ways to tackle technical debt [3].</p><ol><li>Include technical debt consideration in iteration-level activities — build tech debt into the team’s per-sprint roadmap</li><li>Factor technical debt into release-level activities —build tech debt into the product’s roadmap.</li><li>Factor technical debt when implementing user stories — whenever possible developers on the team should pay back technical debt.</li></ol><h3>Conclusion</h3><p>This isn’t a fairytale, and there are no happy endings. Paying down technical debt does not always end in more users and more profit for the business but it’s essential. You can have well-architected code with 100% test coverage, but you may have no reasonable revenue. On the flip side, your company could be one large script running some messy code that makes you a ton of money. Everyone on the team — from management to developers — should share an understanding of what technical debt is. They should know when to acquire tech debt and when to pay it back.</p><h3>Additional Reading</h3><ul><li><a href="https://www.agilealliance.org/introduction-to-the-technical-debt-concept/">Introduction to the Technical Debt Concept | Agile Alliance</a></li><li><a href="https://www.nngroup.com/articles/ux-debt/">UX Debt: How to Identify, Prioritize, and Resolve</a></li><li><a href="https://devops.com/pay-skills-debt-technical-debt/">Pay Down the Skills Debt Before the Technical Debt - DevOps.com</a></li><li><a href="https://medium.com/@dlondon4/introducing-feature-debt-the-unconventional-sibling-of-tech-debt-4e046609f53a">Introducing Feature Debt — the unconventional sibling of Tech Debt</a></li><li><a href="https://hackernoon.com/there-are-3-main-types-of-technical-debt-heres-how-to-manage-them-4a3328a4c50c">3 Main Types of Technical Debt and How to Manage Them | Hacker Noon</a></li></ul><h3>Citations &amp; References</h3><p>[1] <a href="https://en.wikipedia.org/wiki/Technical_debt">Wikipedia definition for Technical Debt</a></p><p>[2] <a href="https://medium.com/@erichiggins/technical-debt-is-like-tetris-168f64d8b700">Technical Debt Is Like Tetris</a></p><p>[3] <a href="https://www.agilealliance.org/project-management-and-technical-debt/">Project Management and Technical Debt</a></p><p>[4] <a href="https://www.goodreads.com/quotes/835238-indeed-the-ratio-of-time-spent-reading-versus-writing-is">Robert C. Martin Quote</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=51c94658761f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/scalable-africa/technical-debt-51c94658761f">Technical debt</a> was originally published in <a href="https://medium.com/scalable-africa">scalable.africa</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>