<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Akis Tsiotsios on Medium]]></title>
        <description><![CDATA[Stories by Akis Tsiotsios on Medium]]></description>
        <link>https://medium.com/@akis.tsiotsios?source=rss-c74451cac557------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*03kiWWosgtl3u9Lx.jpg</url>
            <title>Stories by Akis Tsiotsios on Medium</title>
            <link>https://medium.com/@akis.tsiotsios?source=rss-c74451cac557------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:02:07 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@akis.tsiotsios/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Handling the Data Science duality]]></title>
            <link>https://medium.com/@akis.tsiotsios/managing-the-data-science-debt-8699f9e1ce33?source=rss-c74451cac557------2</link>
            <guid isPermaLink="false">https://medium.com/p/8699f9e1ce33</guid>
            <category><![CDATA[data-science-projects]]></category>
            <category><![CDATA[scrum]]></category>
            <category><![CDATA[technical-debt]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Akis Tsiotsios]]></dc:creator>
            <pubDate>Wed, 10 Nov 2021 12:17:23 GMT</pubDate>
            <atom:updated>2024-10-04T18:01:28.585Z</atom:updated>
            <content:encoded><![CDATA[<p>The days when Data Science (DS) was perceived as a series of experimentation exercises and broad data-driven insights seem to belong to the past. Most companies have acknowledged that realising benefits with DS expands forcefully beyond the exploration &amp; insights generation steps; it ends up in designing and engineering <em>products</em> that partly look and behave like any other software.</p><p>The explosion of VC-backed MLOps startups trying to seamlessly ingest SW engineering principles and automation steps into the DS development process are a good sign that the industry is hungry for products. So is the wide adoption of scrum project management frameworks for operating DS teams (which naturally gives rise to heated debates between the data scientists and the business as discussed later).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NT9JUlO4GYWsLVfxiSqoIA.png" /><figcaption>Search interest for “MLOps” based on google trends</figcaption></figure><p>Does this transition from research-oriented to software-oriented mindset mean that DS has converged to the well-desired plateau of productivity? Perhaps the current impeding factor is that <strong>the right balance between the modelling &amp; the software dimensions of Data Science is yet to be found</strong>. Acknowledging equally the particulars and the value of each independent dimension, but also their profound indivisible interaction seems like the first step towards an organic DS framework.</p><h4>The technical debt</h4><p>Focusing first on the engineering dimension of DS product development. As with any SW, the decision making during the development process aims at avoid hitting the extremes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LtOD_OjzPNezSTRl4hSdFQ.png" /></figure><p>On the far left side lies the pitfall of over-engineering in a premature way in an effort to anticipate and compensate for anything related to product scalability, maintainability, extensibility, robustness etc. Naturally, the risk comes down to deploying nothing at all as the premature design complexity increases.</p><p>On the other side lies the technical debt that gets accumulated under a <em>deploy-now-test-later</em> mindset (that sometimes stays just at the deploy-now step). That might, sooner or later, compromise the value of existing products and pause the development of new ones, demanding an intervention to manage the technical debt.</p><p>Scrum masters, product owners and SW architects &amp; engineers aim at taking the right, iterative and collective decisions to avoid the two extremes.</p><h4>The modelling debt</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LSbYlxayLslqXdmcD-KyPA.png" /></figure><p>Now, <strong>the modelling aspect of DS is a beast of its own</strong>. Here, data scientists are assigned with the task of investigating the predictive/analytical value of various hypotheses and models that could <em>eventually </em>become products; eventually, since there is no proof in advance that a new model (regardless of how well engineered) would have any better-than-random or better-than-naïve analytical value. Hence, an independent modelling process with its own methodology, metrics and extremes needs to be followed.</p><p>Premature modelling optimisation must sound more than familiar to any DS team out there; over-doing it with testing more and more features, data manipulations, hypotheses &amp; models, problem formulations, datasets and evaluation metrics while a DS project is still in a Proof of Concept (PoC) stage. Answering with certainty whether the model performance is good enough (what’s <em>good </em>anyway?) and evaluating the cost &amp; benefits of alternative modelling decisions is not an easy task.</p><p>On the other end of the spectrum, the modelling debt usually arises in the lack of proper problem formulation, data sanity check, modelling and evaluation during the PoC stage. Who needs a proper ranking model or setting up a complex A/B testing mechanism to evaluate the model performance? Then, the deployed model might disguise itself as a successful product until the sudden discovery that the promised intelligence does not meet the market criteria. <strong>Data Science products fail silently as they say</strong>. Interestingly, post-managing the modelling debt is even trickier that managing technical debt. It usually comes down to a whole new round of data explorations, experimentations and evaluations that looks more like a completely new exercise rather than an incremental improvement.</p><h4>The Data Science duality</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/703/1*dKbcTNV_OMiYJ7vVrxSTXA.png" /><figcaption>Data Science product: the (un)lucky heir</figcaption></figure><p>Now, <strong>Data Science is all about the <em>combination </em>of the modelling and the engineering components</strong>. And as the curse of dimensionality dictates, complexity (and hence decisions) grows exponentially with respect to the number of variables of a problem. One could say that DS is the (un)lucky heir of modelling and SW; lucky as it inherits the intrinsic thrill from both, but also unlucky because it has to face the challenges &amp; responsibilities of their combined effect.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1008/1*yi4NbxpcSdFo0FdoJcC9EQ.png" /><figcaption>The Data Science realities space</figcaption></figure><p>Many DS teams must feel familiar with the example scenarios in the Data Science realities space shown above.</p><ul><li>The “modelling luxury” scenario shows an example of a team with a stronger modelling mindset and little productionisation experience. The initial focus might be mainly on the analytical performance of the algorithm that justifies additional complexities and perhaps some future technical debt (with an unknown owner). During the deployment phase, the hidden requirements behind the modelling decisions might come as a big surprise and severely compromise productionisation. Hidden technical requirements that reflect to anything related to data flows, infra &amp; systems, skillset, resources and roles.</li><li>The “fraud intelligence” scenario reflects a more SW-oriented, quick-delivery team that perceives DS mainly through the engineering filter. Adopting an “analysis is paralysis” moto, the derived intelligence of a DS product might not get thoroughly iterated and evaluated before and monitored after production. Regardless of how well engineered the DS product may be, its analytical value can be later found to fail the (unexamined) market criteria, to be outperformed by a competitor’s solution, or to be no better than a naïve or random alternative.</li></ul><p>Given this dual aspect of DS, it becomes clear that trying to impose one-size-fits-all frameworks to handle DS projects is likely to fail.</p><p>Take scrum project management for example. It is an excellent framework for navigating the “premature optimisation vs technical debt” crossroads towards productionisation. It surely takes more than some lonely Data Scientists to deploy things to production, and working under a scrum framework establishes the notions of heterogeneous roles, planning &amp; alignment, iterative delivery and MVPs.</p><p>That being said, overfitting to a scrum framework can cause serious friction to the modelling dimension of DS. Planning well-scoped, time-constrained tasks and sprints does not always play nicely together with exploratory exercises such as literature review, data sanity checks &amp; manipulations, hypothesis testing, problem formulation and model development &amp; evaluation.</p><p>How to strike then the right balance when working on inherently multi-dimensional DS projects? Maybe a good start is to answer the following:</p><ol><li>Is this two-dimensional nature of Data Science widely acknowledged across your data-intensive business? Is the contribution of each separate dimension equally valued and is their combined effect well understood?</li><li>How integrated is the product team in the day to day DS development process?</li></ol><p>(1) is really important for ensuring alignment across all levels. Alignment in apprehension and expectations that will then lead to alignment in intention and direction. This goes up to senior management which will introduce functions, processes and objectives that are uniquely adapted to the DS duality. But also down to Data Scientists, who will be called to own and be accountable for every aspect of this two-dimensional spectrum and grow their skillset respectively.</p><p>(2) is about having a Product Manager maestro behind every DS task, analysis, project and product. The product maestro ensures that the DS process leads to products with real business value by helping the team ask questions, make conclusions and take decisions; the DS duality leads to longer development journeys and an enormous amount of decisions that can overwhelm lonely Data Scientists in no time. The need for a team behind DS projects is required not only for engineering pipelines but also for navigating within a complex web of decisions.</p><h4>The two-stroke DS framework</h4><p>At <a href="https://www.marinetraffic.com/en/ais/home/centerx:30.2/centery:59.9/zoom:11">MarineTraffic</a> we have to deal with exciting and demanding DS challenges that revolve around the shipping industry. Our intelligent products affect the decision making within a trillion-dollar economy and their output is constantly assessed within a competitive market. Understanding profoundly the Data Science specifics and continuously improving our processes and frameworks is at the core of what we do.</p><p>Following a series of ongoing iterations, successes and failures, we currently work towards establishing a two-stroke DS framework that acknowledges and addresses both dimensions of the DS duality with equal weight and with a strong product mindset, but handles each one differently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H_9FPPjAi8sZZvnzT2RgFg.png" /><figcaption>The two-stroke Data Science development framework</figcaption></figure><p>The Product Manager &amp; the Data Scientist are the basic owners of the <strong>PoC step</strong>, working hand in hand towards translating a business challenge into a set of meaningful data, hypotheses, conclusions and eventually models &amp; products.</p><p>Data Engineers are official co-owners of a PoC but they get involved in an ad-hoc, dynamic manner in order to a) assist the Data Scientist with producing the right historical dataset either through one-off extracts or by automating new data flows, and b) to serve as <strong>delivery consultants</strong><em> </em>and challenge the hidden technical debt behind the considered modelling options.</p><p>We seek answers to the following fundamental questions when prioritising successive PoC iterations</p><ul><li>What is the main data/modelling action that needs to be tested &amp; how?</li><li>What real world problem is it addressing and what is the business value behind it?</li><li>What is the potential analytical value behind it? How does it score compared to other actions based on our domain knowledge and intuition?</li><li>What is the effort required in order to produce a meaningful historical dataset that would allow us to analyse, model &amp; evaluate?</li><li>What would be the future effort required in order to put to production the modelling action at hand given that we prove its value?</li></ul><p>Due to the exploratory nature of this step and the limited requirements for interfaces &amp; synced resources, we adopt<strong> </strong>no strict time-based frameworks here, but a clean &amp; well-defined workflow (Kanban) &amp; a close and frequent interaction between the Data Scientists and the Product Managers. This leaves room for creativity, brainstorming and most importantly deep dives on meaningful data. A high-level objective here would be to yield <strong>iterative conclusions </strong>on the business value behind the solution; mainly on its analytical value, while questioning its applicability.</p><p>To give an example, while working on a series of PoC iterations aiming at finding improvements for the MarineTraffic ship ETA (estimated time of arrival) product, we used the above criteria to gradually assess/prioritise over 40 different data/modelling actions that ranged from doing a simple data manipulation operation to changing completely our problem formulation. A great bunch of the suggested actions scored a low priority and was discarded or parked merely due to the potential high technical debt. Other actions, that seemed like a natural incremental step forward at the moment, were found to be not that imperative from a wider analytical perspective. Watching the list grow with more &amp; more actions, only a few of which got a green light, was quite relieving among other things; it seemed like a step towards accepting the complexity of the DS projects and familiarising ourselves with the mindset of deciding <em>what not to do </em>when trying to yield conclusions.</p><p>Given the positive outcome &amp; sign-off during the PoC stage of a DS project, the <strong>productionisation step </strong>is set in motion, with the ownership being now equally spread across multiple teams and functions, including Data Scientists and ML/Data/SW/DevOps Engineers. The product maestro is still the centre of gravity and the faster, SW-oriented and collective nature of this stage is the perfect fit for scrum principles.</p><p>Here, the Data Scientist wears the additional hat of the <strong>modelling consultant </strong>who aims at preventing the hidden appearance of the modelling debt:</p><ul><li>How might an engineering decision affect the modelling assumptions and requirements?</li><li>Has there been a compromise in the signed-off model performance while working on the productionised version of the algorithm?</li></ul><p>And then working strategically towards the true essence behind the MLOps buzzword:</p><ul><li>How can we automate the modelling debt evaluation by continuously challenging the intelligence and the value that our product has promised to deliver?</li><li>What improvements should we make to our processes, systems and workflows in order to accelerate future interventions that manage the dual DS debt or to minimise it in the first place?</li></ul><p>Evidently, both stages of the two-stroke framework are iterative and heavily product-oriented. However, each one follows its own principles trying to adapt the weight of each DS dimension without disregarding the other.</p><p>The strategy is single; to accept the inescapable duality of Data Science and use it to generate niche, intelligent products. But to achieve that, the tactics have to be diverse, multiple and ever-growing. This is after all what keeps us, the Data Scientists, perpetually busy and excited. Isn’t it?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8699f9e1ce33" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>