<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Don Diego on Medium]]></title>
        <description><![CDATA[Stories by Don Diego on Medium]]></description>
        <link>https://medium.com/@dimpac?source=rss-3db32b5765a2------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*gMy7tG3dODY3oF8i</url>
            <title>Stories by Don Diego on Medium</title>
            <link>https://medium.com/@dimpac?source=rss-3db32b5765a2------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 19:06:25 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@dimpac/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Why Data Management matters (Part I)]]></title>
            <link>https://medium.com/@dimpac/why-data-management-matters-part-i-704e457c9b4d?source=rss-3db32b5765a2------2</link>
            <guid isPermaLink="false">https://medium.com/p/704e457c9b4d</guid>
            <category><![CDATA[metadata-management]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Don Diego]]></dc:creator>
            <pubDate>Mon, 23 May 2022 08:11:21 GMT</pubDate>
            <atom:updated>2022-05-24T08:10:12.705Z</atom:updated>
            <content:encoded><![CDATA[<p>In 2021, during a conversation with a senior leader in the data space (who also happens to work in the consumer finance space), I asked him about his organisation’s Data Management strategy. To my surprise, he replied that data management was not something he considered relevant for the type of work they were doing.</p><p>Answers like this are common, and although Engineers and some leaders in small and medium organisations may not give Data Management the saliency it deserves, it will only be a matter of time until reckoning will come during their data journey.</p><p>Let me provide a straightforward example:</p><p>It is effortless to create foundational technical debt in data. Let’s assume we have two critical reports Report 1 and Report 2.</p><p>Report 1 relies on Data Elements 1, 2 and 3. These Data Elements are coming from Golden Sources. So far, so good.</p><p>However, another team has been tasked to create Report 2, that relies on Data Elements 2, 4, and 5. They found that Data Source 1 could provide the data elements they required to deliver the report swiftly, however they were unaware the data contained in that data source was the result of transformations that altered the output of data elements 2, 4, and 5, although they have the same semantics and belong to the same domain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dvW4Ayvm7ueId39aI1MSZA.png" /><figcaption>Data Element 2, 4 and 5 and Data Element 2&#39;, 4&#39; and 5&#39; have different implementations and potentially different outputs, although they have the same semantics and belong to the same domain.</figcaption></figure><h3>Foundational Technical Debt</h3><p>Why am I calling the development of Report 2 “Foundational Technical Debt”? The reason is that it is coming from inception, when decisions about where to source the data from are made. This type of technical debt is usually costly to remediate due to the amount effort required in reconciliation and testing. If you don’t have a mature automated data testing practice, this effort will be multiplied by 2 or 3.</p><h3>What Data Management can’t do</h3><p>Before I delve into why data management is important, I would like to clarify what Data Management can’t do. Data Management can’t give you solution at the core data engineering level. Continuous Integration and Development of best practices are not in the scope of Data Management. Therefore, it is worth mentioning that I am focussed on technical debt generated by the lack of a Data Management strategy or foundational technical debt.</p><h3>Data Management 101</h3><p>Before we go into providing practical solutions (Which may come in a future entry), first let’s agree on a standard definition of Data Management. This definition has been used within the BCS Data Management Specialist Group.</p><p>Data management is a (&lt;strikethrough&gt;corporate&lt;/strikethrough&gt;) service that helps to provide information services by controlling or coordinating the definitions and usage of reliable and relevant data.</p><p>The definition of Data Management looks ok, but we will need to break it down a bit further.</p><p><strong>Data Management is a Service</strong></p><p>This aspect is essential to understand. Data Management is not just a Data Catalogue or a Metadata Repository. Data Management is a service that should be available to everyone directly working on the definition of data elements in the organisation. It should be available, ubiquituous and well documented. And yes, when I say service, I mean self-service.</p><p><strong>Data Management controls and coordinates the definition of data</strong></p><p>This aspect is perhaps where the devil is since there is no clarity about what “control and coordinates” mean in this context.</p><p>“Controls and coordinates” mean the definition of a process to guarantee data elements are clearly defined within their domains. Golden sources or golden datasets have been identified as their source of truth, and also aspects like data quality and data lineage have been appropriately documented throughout its lifecycle.</p><p>Let’s break down this definition a bit:</p><p><em>Data Elements are defined in their domain.</em></p><p>The quality of the data, the data pipeline, and data readability are concerns of the domain. The people who are expected to know the data best are responsible for managing it. The data ownership model will be distributed rather than centralised</p><p><em>Data elements are linked to a golden source or golden data source</em></p><p>I will use the definitions of Golden Source and Golden Data Source defined by Piethein Strengholt in his book <a href="https://www.amazon.com.au/Data-Management-Scale-Enterprise-Architecture/dp/149205478X">Data Management at Scale</a></p><p><em>Golden source</em></p><p>A golden source is the authoritative application where endorsed, curated and reliable data is managed within a particular context. It is usually the result of transforming one or more golden datasets.</p><p>Golden dataset</p><p>A golden dataset is the authoritative original data created. This dataset is genuine and unique and must be accurate, complete, and known. A golden dataset is constructed of raw data elements</p><p><em>Data Element</em></p><p>A Data Element is an atomic unit of human readable information that has precise semantics. They have an identifiable name and act as the glue to other data elements in the data lifecycle.</p><p><em>Data Quality requirements for the data elements are properly documented</em></p><p>Anyone who has worked before in regulatory initiatives will understand how complex Data Quality can become. It would require another article or a group of articles to surface the details of what a good DQ programme may entail. To keep this post reasonable, I will summarise what a good DQ programme should have in scope:</p><ol><li>A clear definition of Data Quality controls through the Data Lifecycle</li><li>Tools and Technologies to measure, monitor, report, and action (prioritise issues) on Data Quality levels</li></ol><h3>To be continued</h3><p>There are a few aspects that I have left out and are also an essential part of a good Data Management Strategy. Topics like Reference and Data Management, Data Governance, and Data Security. I will write about them in a future post.</p><p>I will also propose a practical solution, I promise.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=704e457c9b4d" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>