<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Antonios Chalkiopoulos on Medium]]></title>
        <description><![CDATA[Stories by Antonios Chalkiopoulos on Medium]]></description>
        <link>https://medium.com/@antonios-lensesio?source=rss-8791b20b99b5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*pVeMaPxQ3FYeIKad</url>
            <title>Stories by Antonios Chalkiopoulos on Medium</title>
            <link>https://medium.com/@antonios-lensesio?source=rss-8791b20b99b5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 14:30:26 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@antonios-lensesio/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A DataOps perspective on App and Data Democratization]]></title>
            <link>https://medium.com/lensesio/a-dataops-perspective-on-app-and-data-democratization-99377aaeb2ad?source=rss-8791b20b99b5------2</link>
            <guid isPermaLink="false">https://medium.com/p/99377aaeb2ad</guid>
            <category><![CDATA[dataops]]></category>
            <category><![CDATA[microservices]]></category>
            <category><![CDATA[kubernetes]]></category>
            <category><![CDATA[kafka]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Antonios Chalkiopoulos]]></dc:creator>
            <pubDate>Wed, 06 May 2020 09:29:16 GMT</pubDate>
            <atom:updated>2020-05-06T17:08:03.023Z</atom:updated>
            <content:encoded><![CDATA[<p>How DataOps facilitates access to data and apps, and helps to scale a data-driven company</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*TfUbmluhcyUNCX-C9jFi4A.jpeg" /><figcaption>Photo from <a href="https://unsplash.com/@johnnyhammer">João Marcelo Martins</a></figcaption></figure><p>THE ROLE OF<br>DATA ENGINEERING<br>IN SOCIALIZING DATA</p><p>The #DataOps approach for socializing data and applications makes them accessible to everyone in an organization: removes all barriers to enable ethical innovation, productivity, observability and deliver business outcomes.</p><p><em>On how the future of data engineering is looking..</em></p><h3><strong>Data engineering in 2020</strong></h3><p>Data Engineers continuously fly the flag for open-source technology. When projects mature, they try to share them with their organization — but invariably fall into several challenges along the way.</p><p>When solution architects review them, they typically suggest a centralized data platform initiative. This requires:</p><ul><li>Ensuring enterprise <strong>security</strong> and <strong>compliance</strong></li><li>Identifying repeatable and scalable <strong>best practices</strong></li><li>Enabling cataloging, sharing and <strong>collaboration</strong></li></ul><p>The Data Platform Team, who tends to the infrastructure and service, has separate priorities that are usually:</p><ul><li>Self-service capabilities to avoid being the <strong>bottleneck</strong></li><li><strong>Fine-grained </strong>permissions over a set of Open Source data technologies</li><li><strong>Interoperation</strong> with existing vendor and in-house solutions</li></ul><p>Platform Users like developers, data scientists, analysts care about:</p><ul><li><strong>Observability</strong> over their apps and data</li><li>Operational capabilities that increase <strong>productivity</strong></li><li>A solid <strong>path to production</strong> while meeting business requirements</li></ul><h3><strong>Why is data democratization valuable?</strong></h3><p>The main key elements of democratization are: productivity, quality and improved data ethics via increased transparency.</p><p>When data and apps democratization isn’t a priority:</p><ul><li><strong>Data consumers </strong>have <strong>poor visibility</strong> and understanding of data and processes, thus low confidence</li><li><strong>Business</strong> domain experts<strong> </strong>and IT<strong> </strong>don’t <strong>collaborate</strong> to drive business outcomes</li><li><strong>Engineers</strong> struggle to understand the bigger picture and role of a <a href="https://martinfowler.com/articles/data-monolith-to-mesh.html">data-mesh</a></li></ul><p>Access to data and apps is often <strong>unequal</strong> and depends on the <strong>technical ability</strong> of people and <strong>teams</strong>.</p><p>To assess how democratized your business is, consider answering the following questions :</p><p><strong>As a</strong> <em>[Data Engineer | Data Scientist | Data Analyst | Software Engineer | Product Manager ]</em> <strong>in the</strong> <em>[Mortgages Team | Mobile Team | Data Team | Marketing Team etc..],</em> <strong>how easily can I…</strong></p><ul><li><strong>Discover</strong> available datasets, apps and insights</li><li><strong>Search metadata</strong> about a dataset or an app</li><li><strong>Explore</strong> raw or fairly unprocessed data</li><li><strong>Generate new datasets</strong> from existing data</li><li><strong>Create and share</strong> refined data and insights</li><li><strong>Ingest</strong> new data sources</li><li><strong>Deploy </strong>and monitor application logic</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tKFRnVoBAln8cIFw" /></figure><h3>How about socializing data and apps?</h3><p>A senior software engineer typically uses up to 20 different tools on a daily basis to access apps and data.</p><p>In addition to improving engineering productivity, the <strong>biggest opportunity</strong> for IT is to realize the benefits of a data platform — beyond the core engineers. When people with business domain expertise are able to access and build with data, they drive business outcomes.</p><blockquote>How to move from democratizing to socializing data and apps?</blockquote><p>Socializing data and apps is about giving <strong>complete transparency</strong> over all data and apps to the entire organization, irrespective of positions or skills. A data platform provides an opportunity to bring data and engineering AND business people together to create applications. For example:</p><p><strong>A</strong> <em>[ Business Manager | Analyst | Executive ]</em> <strong>may not need to deeply understand</strong><em> </em>[<em> Apache Kafka | Kubernetes | Microservices | ETL etc..],</em> <strong>a high level of clarity will </strong>help them drive better business outcomes.</p><h3><strong>`Extreme Data Nudity`at Work</strong></h3><p>A look at a data initiative for a large retail bank in Europe shows some interesting results in socializing data technology.</p><p>The business objective was defined as:</p><blockquote>Provide <strong>complete transparency</strong> over all <strong>data</strong> and <strong>apps</strong> in a real-time data platform for more than 1M customers applying <strong>DataOps</strong> principles across people with high- and low-technology skill levels.</blockquote><p>Technology used included <strong>Apache Kafka</strong>, <strong>OpenShift</strong> and <strong>spring-boot</strong> micro-services.</p><p>Adhering to data ethics regulations and going beyond GDPR, transformed their business in just a few weeks and resulted in:</p><ul><li>A set of simple and easy to remember data policies emerged, that continuously monitored and masked <strong>sensitive data</strong></li><li>The ten different product/engineering teams started <strong>collaborating </strong>with increased efficiencies, identifying re-usable datasets and application logic</li><li>A <strong>data-mesh</strong> topology revealed how upstream and downstream consumer networks operated, resulting in fewer outages</li><li>A <strong>common language</strong> and terminology was established across multiple domains as business users started engaging the technology</li><li>Improved <strong>Data quality</strong> emerged, as issues were identified and resolved</li></ul><blockquote>Similar to the “three amigos” agile software method, DataOps brings the platform, business and development teams together to collaborate and deliver fantastic outcomes:</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/595/0*NNzQBV0Qb-s0r6xY" /><figcaption>#DataOps three-amigos</figcaption></figure><h3>Self-service data operations</h3><p>The second step of a DataOps approach is: operations.</p><p>Imagine self-service operations where anyone has the tools to operate data, understand application logic and perform ETL. A world where moving data is <strong>as seamless as copy &amp; paste</strong>. When this happens it’s one of the <strong>biggest enablers</strong> of data democratization.</p><p><a href="https://multithreaded.stitchfix.com/blog/2016/03/16/engineers-shouldnt-write-etl/">Nobody should focus on writing and maintaining data pipelines or ETL.</a></p><p>A Data platform can <strong>simplify</strong> ETL and <strong>train</strong> data users for simple data operations.</p><p>When applications can be defined with a simple and technology agnostic language such as SQL, a data platform team empowers anyone to power data transformation and integration.</p><h3><strong>DataOps Is The Way To Go</strong></h3><p>If you’re interested in data platform initiatives using technologies like <strong>Apache Kafka</strong>, or <strong>Kubernetes,</strong> DataOps is the approach for successful outcomes. It improves ethics for data and apps, delivers value-based pricing that won’t break your bank and gives a great collaborative experience to your entire company.</p><blockquote><em>Kudos to all inspirational though leaders who think and talk on how the future of data engineering will look like! (like </em><a href="https://medium.com/@dmateusp">Daniel Mateus Pires</a> and many more)</blockquote><p>We are <a href="https://lenses.io/">Lenses.io</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99377aaeb2ad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/lensesio/a-dataops-perspective-on-app-and-data-democratization-99377aaeb2ad">A DataOps perspective on App and Data Democratization</a> was originally published in <a href="https://medium.com/lensesio">lenses.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>