<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Bert Baeck on Medium]]></title>
        <description><![CDATA[Stories by Bert Baeck on Medium]]></description>
        <link>https://medium.com/@bertbaeck?source=rss-7e81eadd67fa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*FGuC3jquIRcr1Fc5fjmPPw.jpeg</url>
            <title>Stories by Bert Baeck on Medium</title>
            <link>https://medium.com/@bertbaeck?source=rss-7e81eadd67fa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 01:55:49 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@bertbaeck/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[There Are No Alps]]></title>
            <link>https://bertbaeck.medium.com/there-are-no-alps-52b8508a15de?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/52b8508a15de</guid>
            <category><![CDATA[entrepreneurship]]></category>
            <category><![CDATA[alps]]></category>
            <category><![CDATA[stoicism]]></category>
            <category><![CDATA[obstacles]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Mon, 11 May 2026 11:39:08 GMT</pubDate>
            <atom:updated>2026-05-11T11:39:08.748Z</atom:updated>
            <content:encoded><![CDATA[<p>The mountain doesn’t decide whether you cross it.</p><p>You do.</p><p>Hannibal had elephants and snow. Napoleon had cannon and ice. Jobs had an engineer who said it couldn’t be done. None of them was denying reality because the Alps were real, the cold was real, and the mouse really hadn’t been built yet. They just refused to let the obstacle have the final word before they’d even started.</p><p>Most founders don’t fail at the summit. They fail at the basecamp, in the quiet moment where they look up, believe the mountain more than the goal, and decide the climb isn’t worth starting.</p><p>Today, find your Alp. The pitch you’ve been rehearsing instead of sending. The hard conversation. The product decision you keep “thinking about.”</p><p>Look at it honestly, <em>and then refuse to grant it authority it hasn’t earned.</em></p><p>There shall be no Alps!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/568/0*Wlm6AbgO1oBU3rJT.jpeg" /></figure><p>The ⛰️ mountain is you. <br>The mountain has always been you… <br>My advice to you: Get on the bike or run.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=52b8508a15de" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Loneliness of the Great Unlock]]></title>
            <link>https://bertbaeck.medium.com/the-loneliness-of-the-great-unlock-01f0255bdca8?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/01f0255bdca8</guid>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Wed, 25 Jun 2025 00:57:40 GMT</pubDate>
            <atom:updated>2025-06-25T00:57:40.620Z</atom:updated>
            <content:encoded><![CDATA[<h3>Invention doesn’t start with applause. <br>It starts in silence. Often, in loneliness.<br>And I know this — because I’ve lived it.<br>With <a href="http://www.trendminer.com/"><strong>TrendMiner</strong></a>, and still today with <a href="http://www.timeseer.ai/"><strong>Timeseer.AI</strong></a>.</h3><p>It was a sunny afternoon in early spring, April 2023. I was driving my Taycan, my first electric car, to <em>Hans Lemmens Golf</em> for a club fitting. I had recently picked up golf with my two sons because I made a quiet bet on the future: As they grow older, quality time with them will likely only shrink. Interests shift. Lives get busy. And I wanted something we could keep doing together—something that ages well. Golf felt like the perfect long game. A skill we could return to across decades. A steady way to stay connected, even as life changes. The Taycan was beautiful but impractical. The trunk barely fit a golf bag, and range anxiety was real. Eventually, I traded it for a Jeep again.</p><p>But that day, what stayed with me wasn’t the car — it was a podcast I had chosen intentionally. A deep dive into the thinking of Jeff Bezos. Midway through the episode, he said:</p><blockquote><strong><em>“Invention requires a long-term willingness to be misunderstood.”</em></strong></blockquote><p>That line landed hard. I pulled over and wrote it in my journal. Not just because it sounded clever, but because it gave <em>language</em> to a pattern I’d been living for years. A pattern I hadn’t been able to name. Now, someone like Jeff Bezos was saying it. And it clicked. Because since 2020, I’ve been building <strong>Timeseer.AI</strong> — a company we founded with a simple but future-facing belief: <strong>Data reliability will become operational infrastructure. </strong>It will shift left into the IT middleware stack. Funny enough, in 2019, I already thought we were <em>late</em> to the party. Now it’s 2025… and in many ways, we still are.</p><p>But I saw it coming. What I call <strong>the great unlock</strong>. It was the moment when three conditions finally converged:</p><ul><li>The ability to <strong>store time-series data at scale</strong>, both in the cloud and on-premises, is no longer the overhyped promise of “big data.”</li><li>The ability to <strong>compute at scale</strong>, with affordable, fast processing across hybrid environments.</li><li>Most importantly, companies <strong>began building data-driven applications</strong>—not just analyzing data but <em>depending</em> on it to make decisions, run operations, and automate processes.</li></ul><p>That was the unlock: <strong>Not when data became available, but when people began to rely on it. </strong>And exactly at that moment, <strong>data quality shifted from a technical detail to an operational risk</strong>. Trust in data became non-negotiable. And yet, pitching that truth — trying to sell a <em>data reliability platform</em> before the pain was obvious — was incredibly difficult. Most people didn’t get it. Not because it wasn’t true. But it was too early. That’s what I’ve come to understand as <strong>the loneliness of the great unlock</strong>.</p><h3>1. Being early feels like being wrong</h3><p>When you’re first to a new idea, you’re not met with applause. You’re met with confusion. You pitch a problem people don’t feel — yet. You get polite nods, blank stares, and deferrals. It’s not rejection — it’s a soft no wrapped in uncertainty.</p><p>And in that silence, self-doubt creeps in. Am I wrong?<br>Am I solving a problem that doesn’t exist?<br>That day in the car, Bezos gave me the answer.</p><blockquote><strong><em>“Invention requires a long-term willingness to be misunderstood.”</em></strong></blockquote><p>He wasn’t talking about branding. He was talking about <em>endurance</em>. The emotional fortitude it takes to keep going when the market hasn’t caught up to your vision.</p><h3>2. The idea maze must be walked alone</h3><p>Balaji Srinivasan describes innovation as navigating an <em>idea maze</em> — a web of dead ends, contradictions, and uncharted turns. You don’t just have a vision. You <em>earn</em> it by walking paths no one else has cleared yet. That’s how I built <strong>TrendMiner</strong>, long before “self-service industrial analytics” had a name. And it’s what I’m doing again with <strong>Timeseer.AI</strong>. Even with brilliant people beside you, the original belief — the founder’s intuition — is something you carry alone.</p><blockquote><strong><em>“The idea maze must be walked alone.” — Balaji Srinivasan</em></strong></blockquote><p>There’s no blueprint for creating a category. Only the maze. And your willingness to stay inside it.</p><h3>3. The category always comes after the conviction</h3><p>When we started talking about <strong>data reliability</strong>, there was no category. No buyer persona. No budget line. And most critically, many companies didn’t even realize they had a problem. Everyone said <em>“data quality is important,”</em> but no one was treating it as a <strong>must-solve-now</strong> issue. There was no product-market fit. No built-in urgency. People asked us for proof. <em>“Where is this already going wrong?” </em>But of course, you need early clients to surface the horror stories, and we were starting. It was the classic early-mover paradox: the cure feels optional until the pain becomes visible. I had to explain, again and again, that this wasn’t about dashboards or databases.</p><p>It was about <strong>trust</strong>. If sensor data becomes the heartbeat of industrial decision-making, then someone needs to continuously ensure that data is accurate, timely, and complete, not once but always. Today, the market is catching up. We can convince early adopters. We have repeatable use cases. But back then, we were early. And that meant walking a lonely road.</p><p>That’s the essence of <strong>the loneliness of the great unlock</strong> — you’re not building something <em>on trend</em>. You’re building the thing <em>before the trend exists</em>.</p><h3>4. Loneliness is not a signal to stop — it’s proof you’re leading</h3><p>We glamorize innovation. But in practice, it looks like this: Pitching something 50 times before someone says “yes.” Building through silence. Being the only one who sees it — and still choosing to build.</p><blockquote><strong><em>“The lonely path of vision feels wrong… until it feels inevitable.”</em></strong></blockquote><p>When <strong>TrendMiner</strong> finally took off, people said, <em>“Of course. Makes total sense.” </em>But they didn’t see the five years before that. The loneliness. The stubbornness. The days when belief was all I had. I’m feeling that again now with <strong>Timeseer.AI</strong>. And that’s how I know I’m on the right path (I hope)</p><h3>Ever since then…</h3><p>That day in the Taycan has stayed with me —obviously not because of the car, but because of the clarity it brought. The insight I hadn’t been able to articulate until then.</p><blockquote><strong><em>“Invention requires a long-term willingness to be misunderstood.”</em></strong></blockquote><blockquote><strong><em>“The idea maze must be walked alone.”</em></strong></blockquote><blockquote><strong><em>“The lonely path of vision feels wrong until it feels inevitable.”</em></strong></blockquote><p>So if you’re building something new that doesn’t yet fit into the world’s categories, let me tell you this: <strong>You’re not broken. You’re likely just early. </strong>But don’t stop there. Being early doesn’t mean ignoring product-market fit. You still need to solve a problem they <strong>want</strong> to solve <em>now</em>, where <strong>budget exists</strong>, where the <strong>pain is visible</strong>, with a product you can build in a market you can still <strong>lead</strong> before incumbents catch up. Even if you’re solving the problem of the future, you still need early adopters today. You still need to prove value — early, clearly, and repeatedly because belief alone won’t build the category. <strong>Proof will!</strong></p><p>This is <strong>the loneliness of the great unlock</strong>.<br><strong>Embrace it. You’re not alone in it.<br></strong>You’re just <strong>first</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=01f0255bdca8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Combating the Brain Drain in Process Manufacturing]]></title>
            <link>https://bertbaeck.medium.com/combating-the-brain-drain-in-process-manufacturing-a318bbd2f106?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/a318bbd2f106</guid>
            <category><![CDATA[iiot]]></category>
            <category><![CDATA[industrial-ai]]></category>
            <category><![CDATA[brain-drain]]></category>
            <category><![CDATA[manufacturing]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Wed, 26 Feb 2025 08:54:40 GMT</pubDate>
            <atom:updated>2025-02-26T08:54:40.620Z</atom:updated>
            <content:encoded><![CDATA[<p>Process manufacturing faces a looming <strong>brain drain</strong> as seasoned experts retire and younger employees churn through jobs. Manufacturers risk losing decades of <strong>“tribal knowledge”</strong> — the hard-earned, experience-based know-how that keeps plants running smoothly. Below, I explore the scale of this knowledge loss, its risks, and how Industrial IoT (IIoT), AI, and modern training tools are helping capture and transfer critical expertise.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YiqSBKSpEw_6YH2an1CZ7A.png" /></figure><h3>The Scope of Knowledge Loss from Retirements and Turnover</h3><p><strong>Aging workforce exodus:</strong> Manufacturing has a rapidly graying workforce. As of 2017, nearly one-quarter of U.S. manufacturing workers were <a href="https://themanufacturinginstitute.org/research/the-aging-of-the-manufacturing-workforce/#:~:text=expansion%20and%20record,productivity%20of%20their%20older%20workers">55</a> or older, and retirement rates are accelerating. Over the next decade, more than <em>2.7 million</em> baby boomers are projected to retire from U.S. manufacturing jobs, taking their deeply embedded knowledge with them. In the broader industrial sector, roughly 25% of the population will be over 50 by 2025 — including about <em>12 million</em> manufacturing employees with years of expertise. Manufacturers are keenly aware of this trend: <strong>97% of firms express concern about brain drain</strong> due to retirements, with nearly half saying they are <em>“very concerned”</em> about losing institutional know-how.</p><p><strong>High turnover among younger workers:</strong> Compounding the issue, younger employees in manufacturing tend to have higher turnover and shorter tenures than previous generations. Few millennials or Gen Z workers plan to stay in one company for decades as their predecessors did. One industry analysis found a <em>staggering 51% turnover rate </em>for Gen Z and millennial manufacturing hires at the six-month mark. While that may not be universal, even U.S. government data shows the average tenure of a new manufacturing employee is only about <strong>5.3 years</strong>. These “job-hopping” early-career workers often leave before they can fully absorb the knowledge of veteran colleagues — or they take whatever knowledge they’ve gained to a new employer. The result is a shrinking window for knowledge transfer between generations.</p><p><strong>Tribal knowledge at risk:</strong> The combined effect of mass retirements and youthful turnover is an unprecedented threat to manufacturers’ institutional memory. Critical process wisdom that lives in experienced operators’ heads — optimal machine settings, subtle cues that prevent failures, workaround solutions honed over the years — may evaporate as people walk out the door. “When they leave or retire, all the tribal knowledge accumulated over the years disappears with them,” one analysis noted, creating skill gaps that <em>“flummox the newer workforce”</em> and erode a plant’s performance. Nearly <strong>61% of engineers</strong> surveyed say <a href="https://www.imts.com/read/article-details/Knowledge-is-Power-and-Profit/1249/type/Read/1/category/Business/65#:~:text=advantage%E2%80%94Ford,%20GE,%20and%20Amazon%20come,Start%20with%20some%20basic">knowledge loss is critical when employees depart</a>. In short, manufacturing is facing a knowledge retention crisis of enormous scale, often dubbed a “brain drain.”</p><h3>Economic and Operational Risks of Lost Knowledge</h3><p>Failing to address this brain drain carries serious economic and operational consequences. When veteran expertise is lost, <strong>productivity and quality can suffer immediately.</strong> One study found that, on average, <a href="https://www.imts.com/read/article-details/Knowledge-is-Power-and-Profit/1249/type/Read/1/category/Business/65#:~:text=WealthLosing%20knowledge%20doesn%E2%80%99t%20just%20make,so%20that%20they%20can%20be"><em>42% of the skills</em> needed to perform a given job are known only by the person in that role </a>— meaning if that individual leaves, nearly half of the job’s critical knowledge is lost, and a replacement must learn it from scratch. This gap can translate directly into lost output and higher costs. The loss of seasoned employees has been directly tied to “losing money” for firms, as the remaining staff cannot fill about half of the departing expert’s capabilities.</p><p>Key operational risks include:</p><ul><li><strong>Longer downtime and safety incidents:</strong> Less experienced teams may struggle to troubleshoot equipment issues or prevent mistakes, leading to more frequent or prolonged downtime. <a href="https://www.rewo.io/knowledge-loss-unscheduled-downtime/#:~:text=It%20would%20be%20wrong%20to,cost%20can%20be%20doubled%20to">Unplanned downtime already costs industrial manufacturers an estimated <strong>$260,000 per hour</strong></a> on average. If critical know-how for quick fixes is missing, an outage that a veteran might have resolved in minutes could last hours — a costly prospect in a plant where stoppages can burn over $1 million in a single incident. Even minor errors can escalate into safety hazards or environmental incidents in process industries when expert oversight is absent.</li><li><strong>Reduced efficiency and quality:</strong> Tribal knowledge often underpins fine-tuning process parameters and quality control. Losing that intuition can mean higher scrap rates, more rework, and suboptimal process settings. For example, manufacturing professionals report that when senior engineers leave, there is often a <a href="https://www.ease.io/blog/manufacturing-brain-drain/"><strong>loss of mentorship</strong> for younger staff</a> and a dip in day-to-day efficiency. Products may take longer to market, and error rates can rise, hurting customer satisfaction. In one survey of engineers, 40% observed that the retirement of senior employees had increased loss of knowledge in their organization, yet <em>49% of companies had no formal practices</em> to capture or transfer that expertise — a recipe for operational drift.</li><li><strong>Innovation slowdown:</strong> Brain drain doesn’t just impact routine operations; it also undercuts innovation. Companies mired in filling skill gaps and “reinventing the wheel” may have less bandwidth for R&amp;D or continuous improvement. A thinning bench of technical experts can leave firms less able to tackle complex projects or adopt new technologies, putting them at a competitive disadvantage. Strategic growth initiatives can stall if manufacturers are constantly scrambling to replace know-how.</li><li><strong>Higher costs and hiring challenges:</strong> Replacing experienced employees is expensive. The Society for Human Resource Management estimates that recruiting and training a new hire costs an average of <a href="https://lrshrm.shrm.org/blog/2017/10/essential-elements-employee-retention"><strong>6 to 9 months of that position’s salary</strong></a>. Beyond direct costs, inefficiencies from knowledge gaps (like repeated mistakes or slower ramp-up) act as a hidden tax on operations. Moreover, many manufacturing roles are already complex to fill due to skills shortages. A joint study by Deloitte and the Manufacturing Institute warns the U.S. manufacturing skills gap could leave <a href="https://www2.deloitte.com/us/en/insights/industry/manufacturing/manufacturing-industry-diversity.html"><strong>2.1 million jobs unfilled by 2030</strong>,</a> potentially costing $1 trillion in lost revenue in that year alone. In short, manufacturers that ignore knowledge loss risk operational disruption and significant economic loss.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/901/1*zfynuVB1amc-ImyRWvgo-A.jpeg" /></figure><ul><li><strong>Morale and turnover spirals:</strong> There’s also a human factor. When seasoned mentors leave, and chaos ensues, remaining employees can become frustrated or disengaged. Losing “institutional memory” often forces teams to relearn lessons the hard way, which can sink morale. High turnover can then feed on itself — departures lead to overwork and stress for those left behind, prompting even more people to quit. This creates a vicious cycle of attrition. A “brain drain” thus not only removes skills but can also undermine the culture and stability of the workforce.</li></ul><p><strong>Bottom line:</strong> The cost of doing nothing is steep. <strong>Manufacturing leaders rank knowledge loss among their top concerns</strong>, alongside labor shortages and rising costs. Yet traditionally, knowledge management has been an “important-but-not-urgent” issue for many firms. With the coming wave of retirements and continued turnover, that attitude is changing — manufacturers can no longer afford to let critical expertise walk out the door unchecked.</p><h3>The Perfect Storm: Why NOW Is the Defining Moment to Solve the Brain Drain Crisis in Manufacturing</h3><p>What makes this moment different from just 18 months ago is the <strong>unprecedented acceleration of AI, Industrial IoT (IIoT), and digital transformation</strong> — technologies that are no longer just hype but real, tangible solutions that can <strong>augment human expertise, drive efficiency, and future-proof industrial knowledge</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gL_aBSdPji484vt3n4LaWg.png" /></figure><p>Key <strong>market inflection points</strong> are reshaping how manufacturers could combat knowledge loss.</p><ol><li><strong>AI-powered workforce solutions and AI co-pilots</strong> now allow companies to embed <strong>expert-level decision-making into intelligent systems</strong>, guiding operators in real time as if a seasoned mentor were standing next to them.</li><li><strong>IIoT platforms and lakehouse architectures</strong> provide the <strong>data backbone for capturing and structuring operational insights</strong>, ensuring that knowledge is no longer locked inside individual minds but accessible across the enterprise.</li><li>Meanwhile, <strong>digital twins for simulation and autonomous operations</strong> enable organizations to create <strong>real-world training environments</strong>, allowing less-experienced employees to practice complex decision-making in a risk-free, virtual setting.</li><li>The rapid adoption of <strong>5G connectivity and cost-effective IoT hardware</strong> ensures that knowledge is <strong>transferred instantly across plants and locations</strong>, eliminating silos and empowering a more agile, connected workforce.</li><li>At the same time, industry-wide shifts like <strong>the rise of data-sharing economies, as-a-service business models, and AI-powered cybersecurity</strong> provide manufacturers with <strong>unprecedented access to external expertise, best practices, and real-time insights</strong> without the burden of maintaining everything in-house.</li><li><strong>AI integration and augmentation (AI + IoT =&gt; Internet of Behaviors)</strong> transforms industrial decision-making, turning years of undocumented human intuition into <strong>scalable, AI-enhanced knowledge systems</strong>.</li><li><strong>Gen-AI and Large Language Models (LLMs)</strong> have now reached a point where they can <strong>analyze massive volumes of historical process data, recognize patterns, and provide contextual recommendations</strong> — effectively <strong>preserving and amplifying</strong> the wisdom of experienced workers.</li></ol><p>These shifts are happening <strong>faster than anyone anticipated</strong>. In the last 12 months, AI has advanced to a point where it can now <strong>do what was previously thought impossible — capture complex, experience-based manufacturing knowledge and make it instantly accessible to new employees</strong>.</p><p>The factories that <strong>embrace AI-driven knowledge management today</strong> will not only <strong>prevent operational disruptions caused by brain drain</strong> but also <strong>gain a competitive advantage by scaling expertise, reducing downtime, and ensuring continuous process optimization</strong>.</p><p>The future of manufacturing is no longer just about automation — it is about <strong>augmenting human expertise with AI</strong> to create a <strong>smarter, more resilient workforce</strong>. Those who seize this moment will <strong>define the next era of industrial excellence</strong> — those who don’t risk fading into obsolescence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ne8NC5pegNl8foJs8Tv19A.png" /></figure><p>I am validating and quantifying this as a stand-alone software opportunity. If you like the topic and it concerns you, please share your opinion and insights.</p><p><a href="https://bertbaeck732.lpages.co/chatiiotai-problem-hypothesis/">ChatIIoT.AI - Problem Hypothesis</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a318bbd2f106" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lessons from Boeing: Why Industrial Data Quality is a Matter of Safety and Trust]]></title>
            <link>https://bertbaeck.medium.com/lessons-from-boeing-why-industrial-data-quality-is-a-matter-of-safety-and-trust-9a55d93e0fe8?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/9a55d93e0fe8</guid>
            <category><![CDATA[iiot]]></category>
            <category><![CDATA[industrial-data]]></category>
            <category><![CDATA[data-quality]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Wed, 29 Jan 2025 10:50:34 GMT</pubDate>
            <atom:updated>2025-01-29T10:50:34.048Z</atom:updated>
            <content:encoded><![CDATA[<p>The aerospace industry is built on precision, safety, and reliability. Yet, Boeing’s recent challenges — including the 737 MAX crisis and ongoing quality control concerns — have shown how <strong>poor data quality can have catastrophic consequences</strong>. These failures underscore a critical truth: in industrial operations, <strong>bad data leads to bad decisions</strong> — and in high-stakes industries, the cost can be lives lost, regulatory penalties, and shattered trust.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0wPMdSo6U6TI-qjB.png" /></figure><h3>Key Takeaways from Boeing’s Data Quality Failures</h3><h4>1. Faulty Sensor Data Can Lead to Systemic Failures</h4><p>The tragic crashes of the <strong>737 MAX in 2018 and 2019</strong> were traced back to a single faulty <strong>Angle of Attack (AoA) sensor</strong> that fed incorrect data into an automated control system, resulting in fatal nosedives. The lack of <strong>redundancy and real-time validation</strong> in Boeing’s data pipeline proved disastrous.</p><p>✅ <strong>Lesson for industrial operations</strong>: Critical sensor data must be <strong>validated, redundant, and continuously monitored</strong> to prevent automation failures.</p><h4>2. Poor Data Governance Creates Manufacturing Defects</h4><p>Recent Boeing reports highlight <strong>loose bolts, fuselage misalignment, and incomplete inspections</strong> — suggesting <strong>gaps in quality control data</strong>. When critical inspections rely on inaccurate or missing data, defects go unnoticed, leading to costly rework, delays, and potential safety risks.</p><p>✅ <strong>Lesson for manufacturers</strong>: <strong>Data observability and traceability</strong> are essential to ensuring product quality and preventing costly recalls.</p><h4>3. Regulatory Non-Compliance Stems from Data Issues</h4><p>Boeing faced billions in fines and increased regulatory scrutiny from the <strong>FAA and NTSB</strong> due to <strong>inaccurate risk reporting and missing compliance data</strong>. When critical data is not transparent, industries lose regulatory approval and customer confidence.</p><p>✅ <strong>Lesson for compliance teams</strong>: <strong>Automated compliance monitoring and real-time anomaly detection</strong> ensure that industrial organizations stay ahead of regulations and audits.</p><h4>4. AI and Automation Depend on High-Quality Data</h4><p>Boeing relies on AI-powered maintenance and automated flight controls, but <strong>flawed input data</strong> makes these systems unreliable. <strong>AI is only as good as the data it learns from</strong>.</p><p>✅ <strong>Lesson for AI-driven industries</strong>: AI-driven insights and automation <strong>must be powered by clean, accurate, and continuously monitored data</strong> to avoid false predictions and operational risks.</p><h3>How Timeseer.AI Ensures Data Integrity for Industrial Operations</h3><p>The challenges Boeing faced are not unique to aerospace. <strong>Every industrial company relying on IoT and OT sensor data</strong> must ensure <strong>data quality, observability, and real-time monitoring</strong> to prevent failures, optimize performance, and maintain regulatory compliance.</p><p>At <strong>Timeseer.AI</strong>, we help industrial leaders:</p><ul><li><strong>Detect and prevent sensor data anomalies</strong> before they cause failures.</li><li><strong>Ensure data accuracy, completeness, and traceability</strong> in manufacturing and operations.</li><li><strong>Monitor compliance and quality metrics in real time</strong> to avoid penalties and risks.</li><li><strong>Power AI and automation with trusted data</strong> for better decision-making.</li></ul><p><strong>Data quality is not just about efficiency — it’s about trust, safety, and long-term resilience.</strong> Boeing’s struggles highlight the urgent need for industrial companies to <strong>prioritize data integrity</strong> before errors escalate into costly disasters.</p><p>🔍 <strong>Want to learn how Timeseer.AI can help protect your industrial operations?</strong> Contact us today: bert@timeseer.AI</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9a55d93e0fe8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TOP 10 IoT/OT Data Quality and Data Engineering Predictions for 2025]]></title>
            <link>https://medium.com/timeseer/top-10-iot-ot-data-quality-and-data-engineering-predictions-for-2025-0ea0627f3c7f?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/0ea0627f3c7f</guid>
            <category><![CDATA[iiot]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[data-observability]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Wed, 11 Dec 2024 11:07:29 GMT</pubDate>
            <atom:updated>2024-12-11T11:15:00.798Z</atom:updated>
            <content:encoded><![CDATA[<p><em>By Bert, CEO and Co-founder of Timeseer.AI</em></p><p>As we approach 2025, IoT and OT data management stands at a pivotal moment. The exponential growth in connected devices, projected to surpass <a href="https://iot-analytics.com/number-connected-iot-devices/"><strong>30 billion globally</strong></a>, and the increasing demand for real-time insights have elevated data quality and engineering to the forefront of digital transformation. The IoT data management market is growing at a <a href="https://www.marketsandmarkets.com/Market-Reports/internet-of-things-market-573.html?gad_source=1&amp;gclid=CjwKCAiA9bq6BhAKEiwAH6bqoNpTMy7giXge7QV3zjNaWr0WZRNzYwZO5KAgbkheZiWYvqNfFaEIIRoCB3UQAvD_BwE"><strong>CAGR of 14%-19%</strong></a>, reflecting the urgency and opportunity within this space. This growth rate is notably higher than that of the broader enterprise data management market, with a CAGR ranging from <a href="https://www.marketsandmarkets.com/Market-Reports/enterprise-data-management-market-69219339.html?utm_source=chatgpt.com">9.5% to 12.4%</a>. This disparity underscores the heightened urgency and opportunity within the IoT/OT data management sector compared to traditional IT-centric data management sector.</p><p>At Timeseer.AI, we bring a unique perspective to the challenges of IoT and OT professionals. Our platform is designed to meet their needs, enabling connected industries to transform their data into trusted, actionable insights. Unlike IT-centric solutions, our expertise in time-series and sensor-driven data sets us apart, positioning us to lead the way in advancing IoT data quality and engineering for a smarter, more connected future.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ez5PfVPIeNtLtcwyIrPIwg.png" /></figure><p>Drawing from our expertise, extensive industry research, and insights into the daily challenges faced by our customers, we present our top 10 2025 predictions for IoT/OT data quality and data engineering. These predictions are grounded in years of experience and a deep understanding of the evolving needs within the field</p><h3>1. Data Accountability Will Shift to Data Producers</h3><p>What I expect to happen is that the responsibility for IoT/OT data quality will <strong>increasingly shift</strong> to data producers. Organizations will begin formalizing this approach, recognizing that data producers — those closest to the data — are best positioned to validate its correctness, interpret anomalies, and ensure it is fit for purpose. This evolution will embed data quality accountability into producer workflows, which the right tools and processes can support to enable success.</p><p>This prediction is grounded in findings from our recent survey titled <em>“A Short Survey on Data Quality and Data Operations</em>.” We <strong>surveyed over 100 IoT/OT data professionals</strong> and asked, <em>“Who in your organization is ultimately accountable for the quality of IoT/OT data?”</em> The responses were fragmented and inconsistent, with no clear consensus. Some pointed to the data infrastructure team, others to data engineering, and a few to the data producers themselves. This ambiguity reflects a widespread lack of ownership over IoT/OT data quality in many organizations today.</p><p>As IoT/OT environments grow increasingly data-driven, this lack of accountability is no longer sustainable. Companies are starting to recognize that data producers are uniquely equipped to take ownership of data quality. In 2025, this shift will gain momentum, paving the way for clearer accountability, better governance, and more trusted IoT/OT data pipelines.</p><h3>2. Data Quality Will Further Shift Left</h3><p>In IoT/OT environments, data quality is undergoing a significant transformation — from being a feature to becoming the platform’s foundation. Historically, data quality was often an afterthought, addressed inconsistently by individual analytics solutions or data-driven apps. Each application operated with its own definition and approach to data quality, leading to fragmentation and misalignment. This siloed approach is no longer sustainable as data leaders demand one version of the truth — a unified, standardized approach to data quality that ensures consistency across systems.</p><p>This shift is driving data quality further “left” in the pipeline, meaning it is embedded earlier in the data lifecycle, closer to the data source. By addressing data quality at the ingestion and processing stages, organizations can ensure both <strong>QA (quality assurance)</strong> and <strong>QC (quality control)</strong> are consistently maintained. This early intervention not only minimizes downstream errors but also enables IoT/OT data to serve as a reliable foundation for analytics, AI, and decision-making. As data platforms evolve, expect data quality to solidify its role as the central pillar of IoT/OT ecosystems rather than just an add-on feature.</p><h3>3. Data Quality Management will go beyond Detection with Data Cleansing as the Cornerstone</h3><p>In IoT/OT data environments, ensuring data quality is no longer just about detecting anomalies; it’s about resolving them effectively to maintain trust and enable actionable insights. While data observability is vital for identifying issues, unresolved errors in critical data streams erode trust in downstream analytics and decision-making processes. Data cleansing is the essential next step, ensuring IoT/OT data remains reliable, accurate, and fit for purpose.</p><p>For critical data used in downstream analytics, functionalities like <strong>interactive data validation</strong> will increasingly emerge and be handled by data analysts or data stewards. For instance, metering data used for billing in water utilities must be accurate. Also, a predictive model optimizing plant performance must account for and detect spurious sensor behavior. When anomalies arise, these issues are escalated to a human for resolution, as autonomous cleaning is not permitted in such high-stakes scenarios.</p><h3>4. AI-Driven Data Quality Takes Center Stage</h3><p>AI and ML are no longer buzzwords — they’re essential for ensuring data reliability. They play a critical role in data quality pipeline configuration and delivering accurate, precise alerts in a manageable way. Innovations like <strong>co-pilots for pipeline configuration, the proposal of recommended data quality checks, and the interpretation of results are becoming increasingly important</strong>. In 2025, expect a surge in adopting AI/ML models for (near) real-time anomaly detection, adaptive quality checks, and automating repetitive data cleaning tasks. With <strong>only </strong><a href="https://www.precisely.com/resource-center/analystreports/lebow-report-2024"><strong>12% of organizations AI-ready</strong></a>, this highlights a critical gap and wake-up call for businesses relying on IoT data pipelines. AI readiness — having the infrastructure, skills, and quality data to leverage IoT insights — is essential. Without it, businesses risk falling behind in a data-driven world.</p><h3>5. Data as a Product Will Expand</h3><p>Data is evolving into a strategic asset managed like a product, a concept often called ‘data as a product.’ While this is not a new or novel idea — financial services and e-commerce industries have been leveraging this concept for years — adopting this approach is now accelerating across IoT/OT-focused industries. This approach involves treating data with the same care and processes as physical products — defining ownership, quality standards, and lifecycle management. For IoT/OT data, this means ensuring it is always available, accurate, and enriched for various use cases — ranging from operational optimization to predictive maintenance and advanced AI applications.</p><p><strong>Managing data like a product also includes setting up clear data governance frameworks, establishing metrics to monitor quality, and implementing feedback loops for continuous improvement</strong>. Dedicated data product teams will be created and become commonplace in connected industries, ensuring that sensor and telemetry data are well-curated, reliable, and ready for decision-making. These teams will act as the bridge between captured data(uncurated raw data) and actionable insights, ensuring reliable, purpose-fit data that drive significant business value from IoT ecosystems. This growing adoption highlights the increasing recognition of the strategic importance of well-managed data across industries reliant on connected systems.</p><h3>6. Real-Time Data Quality Monitoring Becomes the Norm</h3><p>Next year, data quality monitoring and data observability will become a non-negotiable requirement for IoT/OT ecosystems in the manufacturing, oil &amp; energy, and utilities industries. With IoT systems expected to generate <strong>80 zettabytes of data annually</strong> (<a href="https://blogs.idc.com/2021/01/06/future-of-industry-ecosystems-shared-data-and-insights/?utm_source=chatgpt.com">IDC</a>), traditional monitoring methods can not keep up with the complexity and speed required. Real-time monitoring tailored for data quality will allow organizations to instantly detect and resolve issues, preventing downtime and reducing costs.</p><p>In <strong>manufacturing</strong>, where unplanned downtime costs can reach <strong>$260,000 per hour</strong> (<a href="https://www.iiot-world.com/predictive-analytics/predictive-maintenance/the-actual-cost-of-downtime-in-the-manufacturing-industry/?utm_source=chatgpt.com">IIoT World</a>), real-time data quality systems will flag inconsistencies in machine telemetry or production metrics before they cause costly disruptions. For <strong>energy and utilities</strong>, real-time data quality monitoring will stabilize grids, prevent leakages or outages, and enable the seamless integration of renewable energy sources.</p><p>With the IoT analytics market expected to reach <strong>$45 billion by 2025</strong>, real-time data quality monitoring is no longer a “nice-to-have” feature but a business-critical capability. As industries increasingly rely on connected systems, ensuring that data is accurate, consistent, and actionable will be the foundation for operational resilience, cost savings, and competitive advantage in IoT/OT environments.</p><h3>7. Do It Yourself (DIY) IoT/OT Data Quality Approaches Will Break under the Pressure of the Data Demands</h3><p>Relying on DIY approaches to IoT/OT data quality will no longer stay viable as IoT ecosystems’ scale, complexity, and demands outgrow manual efforts. While builders enjoy crafting their own data tests and rules, centralized data teams realize that data quality goes far beyond writing checks the moment they go beyond PoC (proof of concept) and PoV (proof of value).</p><p>Effective data quality management requires systems prioritizing relevance over raw statistical significance, ensuring issues are ranked by their actual business impact rather than just numerical thresholds. It also demands efficient processing at scale to handle massive IoT/OT data volumes, fleet-wide asset management capabilities for running and summarizing checks, and AI-driven tools for data cleaning, augmentation, and result interpretation. So, the true challenge isn’t creating data quality checks but building the infrastructure to run them continuously, managing configurations at scale, and enabling actionable insights without being overwhelmed by false alarms due to alert fatigue.</p><p>As IoT devices generate ever-increasing volumes of time-series data, patchwork <strong>DIY solutions will struggle </strong>to keep up with operational needs, leading to inefficiencies and missed opportunities. Teams will further embrace purpose-built systems that offer seamless automation, intelligent assistance, and scalability to ensure data remains reliable, actionable, and fit for purpose. DIY approaches, while tempting for quick fixes, will ultimately break under the pressure of IoT/OT data demands, making robust, integrated data quality frameworks a critical requirement for long-term success.</p><h3>8. AI and Data Governance Will Take Priority</h3><p>IoT/OT industries will adopt unified data governance frameworks to overcome the growing challenge of fragmented, siloed data. With <strong>57% of organizations citing data silos as a significant barrier</strong> (Precisely-Lebow, 2024), seamless integration across systems will no longer be optional. IoT/OT ecosystems generate diverse streams of sensor and telemetry data, and with unified governance, these streams remain connected, leading to inefficiencies and missed opportunities for actionable insights.</p><p>AI-driven governance will play a pivotal role by enabling real-time integration and contextual understanding of data across silos. For example, in manufacturing, unified governance will allow machine data from different facilities to be seamlessly aggregated, providing a single source of truth for predictive maintenance and production optimization. Similarly, in utilities, governance systems will ensure accurate, consistent data across thousands of IoT devices, enabling reliable billing, compliance, and operational decisions.</p><h3>9. IoT/OT Data Quality Will Prove to be the Enabler for AI</h3><p>AI models are only as good as the data they’re trained on. By addressing the fact that 67% of organizations distrust their data and ensuring high-quality IoT/OT data, businesses can unlock AI-driven use cases such as predictive maintenance and energy optimization.</p><h3>10. Low-Code Platforms Will Dominate</h3><p>In 2025, low-code interfaces for data quality management will gain significant traction in IoT/OT environments, surpassing traditional code-based approaches. Unlike IT or APM (Application Performance Management) domains, IoT/OT industries face a critical shortage of personnel with coding expertise. The complexity and scale of IoT/OT data pipelines demand intuitive, user-friendly tools that empower engineers, operators, and analysts to manage data quality without deep technical expertise. <strong>This</strong> <strong>shift toward low-code platforms will enable broader adoption, faster implementation, and greater scalability</strong>, paving the way for efficient and accessible data quality management across connected industries.</p><h3>The Strategic Imperative: Why Data Quality Matters</h3><p>By 2026, data quality will no longer be a back-office concern but a strategic enabler. Companies failing to address these trends risk falling behind as leaders scale their IoT capabilities, integrate AI, and make data-driven decisions.</p><p>At <strong>Timeseer.AI</strong>, we are at the forefront of this transformation. Our purpose-built platform empowers industries to turn IoT data into a trusted, strategic asset. As pipelines expand and the need for reliable data grows, we ensure your data quality is scalable, secure, and actionable.</p><p>Let’s shape the future of IoT data together.</p><p><strong>Sources:</strong></p><ol><li><a href="https://www.lebow.drexel.edu/sites/default/files/2024-09/drexel-lebow-precisel-data-integrity-trends-insights-2025-outlook.pdf">Precisely-Lebow Report 2024</a></li><li><a href="https://iot-analytics.com/">IoT Analytics</a> Research (2024)</li><li><a href="https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-data-driven-enterprise-of-2025">McKinsey — Data-Driven Enterprise of 2025</a></li><li>Gartner — IoT and Edge Computing Trends (2024)</li><li>Forrester — DataOps in Practice (2024)</li><li><a href="https://www.idc.com/">IDC</a> FutureScape: IoT Predictions (2024)</li><li>MarketsandMarkets — Data Quality as a Service Report (2024)</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0ea0627f3c7f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/timeseer/top-10-iot-ot-data-quality-and-data-engineering-predictions-for-2025-0ea0627f3c7f">TOP 10 IoT/OT Data Quality and Data Engineering Predictions for 2025</a> was originally published in <a href="https://medium.com/timeseer">Timeseer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top 10 things I learned at IMPACT 2024 Summit#dataObservability]]></title>
            <link>https://bertbaeck.medium.com/top-10-things-i-learned-at-impact-2024-summit-dataobservability-1f90783345b3?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f90783345b3</guid>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[industrial-automation]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Thu, 14 Nov 2024 21:01:40 GMT</pubDate>
            <atom:updated>2024-11-14T21:01:40.086Z</atom:updated>
            <content:encoded><![CDATA[<p>I attended the MCdata Impact Conference today and am excited to share insights that will shape our platform design at Timeseer.AI, especially with a focus on IoT/OT data.</p><p>I guess we’re the first to post insights… just <em>one minute</em> after the conference wrapped up! 🏆📅 Talk about real-time reporting! 😉</p><p>At <a href="http://www.timeseer.ai/">Timeseer.AI</a>, our mission is to help the connected industrial economy thrive by providing reliable IIoT data. We aim to increase the power and value of data, ensuring it can be treated as a true asset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vUN6pL-O6VMCzOfDd9Qx2Q.jpeg" /></figure><p>Here are the top 10 things I’ve learned about the future of data quality management:</p><ol><li><strong>Top 3 Data Challenges Across Industries</strong><br>Across the board, the top three most significant pain points that enterprises are facing are AI readiness, ensuring <strong>data quality</strong>, and implementing effective data governance.</li><li><strong>Data Cleansing is more prominent in IoT/OT data quality management.</strong> <br>In horizontal IT data quality tools, the most attention is paid to data quality assurance (QA) rather than quality control (QC). The emphasis is on flagging problems, data quality monitoring, incident management, and root cause analysis. In IoT, data quality management requires a balance of both QA and QC — QC involves cleaning or solving actual data issues after they arise, either automatically or semi-automatically (through interactive data validation). Gen-AI augmentation is critical here, enabling more proactive monitoring and prevention.</li><li><strong>Alert Fatigue is Real</strong><br>Controlling alert volume is essential to avoid overwhelming teams. Ideas include sending alerts only for new events and changes within the last 24 hours and setting custom thresholds based on the frequency of occurrences.</li><li><strong>Fine-Tune Alert Sensitivity with Metric Monitoring</strong><br>Custom data quality metrics are an excellent way to manage alert fatigue effectively. They allow teams to adapt alert sensitivity to stay within manageable limits. One client limited daily alerts to a maximum of 5 and adjusted sensitivity accordingly.</li><li><strong>IoT/OT Data is still a Different Animal.</strong><br>There’s a noticeable gap in connectivity between time-series and no-SQL databases and support for IoT/OT time series data. This will be an area to watch for future partnerships with companies like timeseer.AI</li><li><strong>APM Maturity vs. Data Observability: A Huge Gap</strong><br>Data deserves the same respect and attention as Application Performance Monitoring (APM), but the market isn’t quite there yet. There is a significant gap between APM maturity and data observability. Standard practices in APM are not widely adopted in data operations. Conclusion: <strong>Data needs more respect.</strong></li><li><strong>Go Wide vs. Go Deep<br></strong>The difference between data engineers and data analysts/stewards is their approach: data engineers typically go wide, monitoring a large number of sources and tables for quality, while data analysts go deep, focusing on understanding the root cause of data issues related to source data.</li><li><strong>Low-Code Solutions Are Gaining Traction</strong><br>Point-and-click (low-code) tools are increasingly popular with business users, surpassing code-based solutions (like YAML) due to their ease of use and accessibility.</li><li><strong>AI Agents as Data Stewards</strong><br>There is a lot of buzz about future AI agents that could understand what data means and how it’s used. These agents will act as virtual data stewards by profiling data and transforming data management. These co-pilots and AI agents will become a critical component of data quality management.</li></ol><p>Thanks for reading my top <em>10</em> insights from the MCdata Impact Conference! (If you counted only 9… well done! I&#39;m just testing if you’re paying attention 😉.)</p><p>I am excited to keep these insights in mind as we develop smarter data solutions for IoT/OT data producing clients! #DataManagement #IoT #AlertFatigue #DataGovernance #DigitalTransformation #AI</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f90783345b3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[OT/IoT Data Good Enough for Operations Isn’t Good Enough for AI and Analytics ]]></title>
            <link>https://bertbaeck.medium.com/ot-iot-data-good-enough-for-operations-isnt-good-enough-for-ai-and-analytics-96edc339e7c1?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/96edc339e7c1</guid>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[sto]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[databricks]]></category>
            <category><![CDATA[data-observability]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Tue, 19 Mar 2024 21:33:33 GMT</pubDate>
            <atom:updated>2024-03-19T21:33:33.442Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*AAR1H3iBNaYkTeKgmuuB8g.png" /></figure><h3><strong>Navigating the Data Quality 🚢❄️ Iceberg in Industrial Manufacturing</strong></h3><p>In the vast ocean of industrial operations, where control systems, SCADA, alarms, and safety mechanisms navigate the ship, <strong>data integrity often sails under the radar</strong>. Yet, alongside these technological captains, the human in the loop, the domain expert, stands as the co-captain, ensuring a vigilant watch over the seas. As we venture into the deep waters of data historians and warehouses, we encounter an iceberg of challenges that can sink the unsinkable: data defects. 🛠️📈</p><h3><strong>The Peril of Overlooking Data Quality</strong> 🌊</h3><p>The journey begins with a seemingly harmless assumption: if data is good enough for day-to-day operations (because of the presence of all the layers of protection), surely it’s fit for digital operations. However, this belief is the tip of the iceberg. Beneath the surface lies a complex reality where data defects become permanent fixtures once archived in a historian or warehouse. These defects magnify their impact, affecting everything from business dashboards, data products, and data shared with suppliers and clients to AI models and predictive maintenance strategies. 🚨🔍</p><h3><strong>The Ripple Effect Across the Enterprise</strong> 🌐</h3><p>Compromised data quality casts a wide net of consequences, affecting every facet of an organization. Here’s a breakdown of its far-reaching impact:</p><ul><li><strong>Data Teams and Scientists: </strong>Lose valuable time due to constant data issues, a daily pain point. Data</li><li><strong>Operations:</strong> Missed opportunities to reduce costs and boost EBITDA with data defects causing issues like increased energy expenses, unplanned downtime, and higher maintenance costs.</li><li><strong>Business Decision-Makers:</strong> Eroded trust in decision-making processes when basing decisions on flawed OT/IoT data.</li><li><strong>Client Relations:</strong> The practice of sharing data for compliance or transparency can backfire, risking client loss and revenue impact if the data is of low quality.</li><li><strong>Reputation:</strong> The most severe impact is on the company’s reputation, where unreliable data exposed to the public can cause irreparable damage.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tUZhXt3BMFEvScckR14jFg.png" /></figure><h3>Beyond Operations: The Broader Impact 🌐</h3><p>Data quality extends its tentacles beyond the operational level, influencing reports, data products, AI models, and even how data is shared with clients and suppliers. Ensuring data reliability is becoming a corporate responsibility, underpinning the notion that <strong>data is an asset, not a liability.</strong> 🤝💼</p><h3><strong>A Proactive Approach to Data Quality</strong> 🔄</h3><p>However, the industry needs a paradigm shift, moving from a reactive to a proactive stance on data quality. Addressing data issues at their source would not only save time and resources but also prevent the cascading effects of poor data quality downstream. It’s not just a journey from “garbage in, garbage out” to a culture of data integrity and reliability; it’s also a must to succeed in digital operations and digital transformation.</p><p>There is one huge problem: Resource scarcity in manufacturing operations is the bottleneck. Doing more with fewer people has been the business case for industrial automation for years. But this has a breaking point. Data stewardship and ownership need to come from the people who understand the data, who are already drowning and in firefighting mode. Executives who spend budgets on “digital” in a broader sense need to incorporate this.</p><h3><strong>AI: The Beacon of Hope</strong> 🤖❤️</h3><p>Artificial Intelligence (AI) emerges as a powerful ally in this quest. It’s now widely accepted that AI needs data quality, but data quality also needs AI, primarily if you work with high volumes of OT/IoT data. In the case of OT/IoT, both Discriminative AI and Generative AI are necessary to tame the time series data.</p><p>Use cases break up into three buckets:<strong> auto-completion, co-pilots, and auto-summary</strong>, applied to the configuration of data quality checks, setting up data pipelines, enhancing result interpretation and complexity reduction, implementing adaptive data quality checks using foundational models, smart data imputation, metadata learning, translating insights into human-readable text, assisting in data cleansing, and ensuring that data is not just clean but also contextually relevant and actionable and many more.</p><h3><strong>Embarking on the Data Quality Voyage</strong> 👣</h3><p>For organizations embarking on the journey to enhance data quality, leveraging technology platforms that assess, monitor, and improve data quality is crucial. Here’s how different roles can benefit from and contribute to this initiative:</p><ul><li><strong>Data Scientist:</strong> Enhance your workflow and reduce time spent on cleansing by utilizing platforms like timeseer.AI. Improve AI model quality by ensuring your analytics are based on reliable data.</li><li><strong>Data Engineer:</strong> Set up and maintain data quality gates and establish data pipelines where reliable data flows seamlessly, supporting the foundation for accurate analytics and decision-making.</li><li><strong>Data Steward (Domain Expert):</strong> Partner with business leaders to detect fleet-wide anomalies using sensor and data acquisition systems, ensuring data integrity and relevance across the organization. And you’ll be able to validate data in a manageable way (yes, you can!)</li><li><strong>Business Leader:</strong> Build a case for data quality initiatives, leveraging insights from data stewards and the capabilities of platforms like timeseer.AI to drive organizational change and improve decision-making processes.</li></ul><p>Watch out for <a href="http://www.timeseer.ai/">timeseer.AI</a> here. 🌟🛠️</p><h3><strong>In Conclusion</strong> ✨</h3><p>The dialogue on data quality in industrial analytics is more than a technical discussion; it’s a strategic imperative. As we navigate through the complexities of IoT and big data, adopting a proactive, AI-enhanced approach to data quality is not just beneficial; it’s essential for navigating the industry&#39;s future with confidence and clarity. 🚀🌍</p><p>To the readers, remember: in the digital age, the value of your data is only as good as the insights it can generate. <strong>Prioritizing data quality is the first step toward unlocking its full potential. 🗝️💡</strong></p><p>More about this topic was uncovered in the following video together with Aveva Benelux:</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FQ8V9xNuyX_Y%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DQ8V9xNuyX_Y&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FQ8V9xNuyX_Y%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/9566ceafb972e0eaff8203db4939d84b/href">https://medium.com/media/9566ceafb972e0eaff8203db4939d84b/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=96edc339e7c1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[ Exploring OT/IoT Time Series Data Products ]]></title>
            <link>https://bertbaeck.medium.com/exploring-ot-iot-time-series-data-products-f44332e4507c?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/f44332e4507c</guid>
            <category><![CDATA[product-data]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[timeseries]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Thu, 07 Mar 2024 20:29:59 GMT</pubDate>
            <atom:updated>2024-03-07T20:33:59.471Z</atom:updated>
            <content:encoded><![CDATA[<p>Diving into the world of IoT, let’s talk about time series data products 🌐.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*Xstal8JnaAq6GAw2lqp4pQ.png" /></figure><p>Ever wondered about data products? 🤔 A data product is essentially a data “container” 📦 that solves a business problem or is monetized 💰. They’re crafted for users at various maturity levels, whether internal or external. Here are some examples:</p><p>Designed for users at various levels, they cater to internal and external stakeholders.</p><h3>Data products explained 🔍💡</h3><p>Exploring the diverse landscape of data products reveals a spectrum of tools designed to harness the power of data in unique ways. Here’s a brief overview:</p><ul><li><strong>Static Data Products </strong>(datasets, reports): These are the foundational blocks, providing snapshots of data at a specific point in time 📊. Ideal for historical analysis and baseline reporting, they serve as the bedrock for informed decision-making.</li><li><strong>Rendered Data Products </strong>(analytics views): Offering a more interactive experience, these products transform data into visual analytics and dashboards 📈. They allow users to explore trends, patterns, and insights through a graphical interface, making data more accessible and understandable.</li><li><strong>Dynamic Data Products </strong>(request-driven/subscribed): These are highly responsive and tailored to user requests or subscriptions 🔄. Whether it’s real-time data feeds, personalized reports, or alerts, dynamic products adapt to provide the most relevant and current information.</li><li><strong>Functional Data Products</strong> (algorithms): The powerhouses of data-driven decision-making, these products leverage algorithms to offer predictions, optimizations, and automated insights 🤖. From machine learning models to complex analytical tools, they process vast datasets to uncover actionable intelligence.</li></ul><p>Each type of data product plays a crucial role in the data ecosystem, catering to different needs and objectives. Together, they unlock the full potential of data for organizations and individuals alike.</p><p>Examples include:</p><ul><li>A dynamic dashboard that tracks real-time environmental conditions (e.g., emissions) across multiple locations 🏭.</li><li>An application analyzing equipment performance over time to predict maintenance needs 🛠️.</li><li>Operational data shared 🤝 with suppliers or end clients.</li><li>A model forecasting energy consumption patterns based on historical data to optimize usage 🔋.</li></ul><h3>But, What Sets Data Products Apart? 🤨</h3><p>It’s not just about the tech! Data products focus on the people and process side, covering the entire lifecycle of data 🔄. It’s about bringing “product thinking” to data, prioritizing business use over technology.</p><p>Unlike static data sets, IoT time series data products focus on capturing the nuances of time-bound data, emphasizing the importance of timely, accurate insights for decision-making 🕒.</p><h3>What are key characteristics of IoT/OT data products? ✨</h3><ol><li><strong>Data Quality</strong>: This is the cornerstone. With IoT, the integrity of time series data is paramount. Ensuring accuracy, completeness, and timeliness in data collection and processing builds trust and reliability in the data product 🌟.</li><li><strong>Discoverability</strong>: Given the vast amount of data generated, these products must be easily searchable and reusable across the organization. A comprehensive metadata registry aids in this endeavor 🕵️‍♂️.</li><li><strong>Security</strong>: Protecting sensitive IoT data, especially with personal or proprietary information, requires robust security measures and compliance with standards 🔐.</li><li><strong>Observability</strong>: The dynamic nature of IoT data means changes happen in real time. Tools that provide immediate anomaly detection and root cause analysis are crucial for maintaining data integrity and product reliability 🚀.</li><li><strong>Context:</strong> For IoT data, context is key to unlocking its full potential. This includes understanding where the data comes from, the environment in which it’s collected, and the timing and conditions under which it’s gathered. Context helps in interpreting the data correctly, ensuring that insights and actions derived from IoT applications are relevant and accurate.</li></ol><p>While the promise of data products, particularly those driven by IoT, holds immense potential for innovation and efficiency, it often remains unfulfilled due to challenges in maintaining high data quality and properly contextualizing the data. Addressing these critical aspects is essential for unlocking the true value of data products and realizing their full impact.</p><p>Let’s harness the power of IoT time series data to drive forward-thinking solutions! 🚀 #IoT #TimeSeriesData #DataQuality #AIoT</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f44332e4507c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Timeseer.AI Makes Its Mark at Hannover Messe 2023]]></title>
            <link>https://medium.com/timeseer/timeseer-ai-makes-its-mark-at-hannover-messe-2023-6db4a9965f40?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/6db4a9965f40</guid>
            <category><![CDATA[predictive-maintenance]]></category>
            <category><![CDATA[data-observability]]></category>
            <category><![CDATA[data-quality]]></category>
            <category><![CDATA[iot]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Mon, 24 Apr 2023 09:52:15 GMT</pubDate>
            <atom:updated>2023-04-25T11:23:50.451Z</atom:updated>
            <content:encoded><![CDATA[<p>This week 📅 <a href="https://www.linkedin.com/company/timeseer/">Timeseer.AI</a> attended Hannover Messe 2023 🌎in Germany. Our team, together with over 6500 exhibitors from 📊 75 countries and approximately 215,000 visitors, traveled to the Hannover Fairground to learn about and experience the solutions that Industry 4.0, 5G, and autonomous systems offer to businesses around the world.</p><p>Below 👇 are my four key takeaways:</p><h3>#1. We’ve made BI dashboards sexy; now it’s time to make them reliable.</h3><p>I walked around the fair and noticed that many industrial vendors still focus on descriptive analytics (BI dashboards), including popular industrial examples such as overall equipment effectiveness (OEE), energy monitoring, and asset condition-based monitoring (CBM). I talked to many software suppliers, and they agreed that their data consumers do not always trust the underlying data in their dashboards. This is extremely undesirable since it’s a no-brainer that the data used in dashboards <em>must</em> be accurate, reliable, and consistent.</p><p>In reality, raw data in dashboards is frequently subjected to aggregation, segmentation, summarization, grouping, and averaging. Consequently, it is much more susceptible to data defects or issues with data reliability. In an industrial context, we can identify various types of data quality issues such as gaps or duplicated time stamps; sensor data integrity issues like drift, wrong placement, degradation, and broken sensors; and IoT-related data issues such as network or bandwidth problems and battery issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*__I87TDqmMYIYYqxhqC_9Q.png" /><figcaption>. Wrong data is visualized in the CBM dashboard due to averaged outliers to show a trendline.</figcaption></figure><p>Consumers lose trust in their dashboards not because of the dashboard itself but because of the lack of reliability of the underlying data. If companies want to create trust in their dashboards, they need to ensure trust in the data used. These two aspects go hand in hand as accurate and reliable data is essential to ensuring that the insights and decisions derived from the dashboards are meaningful and actionable. By implementing appropriate data quality controls and data governance processes, companies can improve the reliability of the data used in their dashboards and therefore build trust with their consumers.</p><h3>#2. It’s time for predictive maintenance (PdM) to mature.</h3><p>Despite the potential value and ROI, a significant number of IoT software companies continue to prioritize predictive maintenance as their primary use case. However, the reality is that scaling this use case beyond the initial asset to similar assets is not as straightforward as it may seem. The data is often insufficiently reliable, and the necessary context is often lacking to guarantee success. While adding a vibration sensor to your asset is a crucial first step, it is not adequate in and of itself. If every similar asset needs manual data modeling, it would destroy the business case.</p><h3>#3. Data guardians are on the rise.</h3><p>Currently, Industrial DataOps is a highly pertinent strategic topic, especially with the industry rapidly transitioning from data collection and storage to analytics and AI. However, it is now widely acknowledged that critical middleware issues must be addressed, and data infrastructures must be the focus to achieve mature AI. To realize this goal, DataOps can be broken down into components such as data integration, contextualization, data fusion, data quality, and data observability. At Hannover Messe 2023, companies such as <a href="http://www.highbyte.com/">HighByte</a>, <a href="https://www.elementanalytics.com/">Element</a>, <a href="http://www.cognite.com/">Cognite</a>, and Timeseer.AI advocated for these crucial data guardianship principles. It was heartening to see that we are all complementary entities working towards a common goal instead of being direct competitors. And I foresee a future where we can elevate each other.</p><h3>#4. As a Belgian, I still prefer our beer of course, 🍺 but German bratwurst is better 🌭 .</h3><p>In every German trade show, it’s a common sight to see people enjoying half-liter beers in thick glasses, which can give the event a festive Oktoberfest vibe. However, when it comes to the quality of beer, I still prefer Belgian beer 🍻 without being chauvinistic 😇. In my opinion, Duvel still reigns supreme as the best beer in the world. For that, I would like to express my gratitude to Flanders Investment and Trade for inviting us to their booth for a delightful after-hours aperitif.</p><h3>Timeseer.AI to the Rescue 🚀</h3><p><a href="http://www.timeseer.ai/">Timeseer.AI</a> can help with the issues mentioned above and can be integrated into 3rd party analytics solutions such as dashboard software and self-service analytics tools. The value here is that data is verified and conditioned (cleaned) before being consumed in dashboards. Each dashboard in production that is based on reliable data can have our quality stamp ‘proofed and verified by Timeseer.AI’. Actually, one of our IoT clients (<a href="http://www.versasense.com/">Versasense</a>) uses Timeseer.AI to proof and verify the data in their CBM dashboards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ovq8phJ2jzlP3h1eNk0h1g.png" /><figcaption>Timeseer.AI’s data quality core integrated into an IoT customer’s backend and UI/UX.</figcaption></figure><p>In the case of <a href="http://www.verasense.com/">Versasense</a>, <a href="http://www.timeseer.ai/">Timeseer.AI</a> has also had its impact on the AI/analytics side. And to understand, this is our value proposition on the ML and AI-ops side:</p><p>Data models are now trained on clean, reliable data periods resulting in a model quality improvement. In the Versasense case, the model improved by over 50%. Our partner also confirmed that they experienced a 60% reduction in time spent on manually cleaning or correcting the data.</p><p>Likewise, incorporating data validation has value at interference (when the model is running live). Unfortunately, data quality or sensor quality integrity issues are still the main cause of too many false alerts in predictive models. However, validating data at inference will significantly reduce the number of false alerts. In this case, Versasense noticed 70% fewer false alerts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8dOWcrlrQ3MAia5Ar01JBQ.png" /><figcaption>Data quality issues are detected and flagged at inference to reduce the amount of false alerts</figcaption></figure><h3>Concluding Thoughts</h3><p>Data quality is often a small feature of analytics platforms, but what companies need is one single version of the truth. This is where Timeseer.AI comes into play because providing that data truth is our core — our entire platform is built for this purpose. And to achieve success, data quality belongs in the middleware. By integrating <a href="http://www.timeseer.ai/">Timeseer.AI</a> into your own application, you can realize significant value such as:</p><p>&gt;&gt; Increased trust in dashboards by verifying and validating the underlying data.</p><p>&gt;&gt; Improved model quality by basing trained models on Timeseer-proofed and validated data.</p><p>&gt;&gt; Fewer false alerts of predictive models due to data integrity checks at inference.</p><p>&gt;&gt; 50–60% reduction in time spent on cleaning data quality.</p><p>If you want to understand how Timeseer.AI fits into your architecture, please reach out to me at <a href="/service/mailto:bert@timseer.AI">bert@timeseer.AI</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6db4a9965f40" width="1" height="1" alt=""><hr><p><a href="https://medium.com/timeseer/timeseer-ai-makes-its-mark-at-hannover-messe-2023-6db4a9965f40">Timeseer.AI Makes Its Mark at Hannover Messe 2023</a> was originally published in <a href="https://medium.com/timeseer">Timeseer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI SOBERING 2023: DATA CENTRICITY NEEDED FOR THE NEXT STEP IN AI MATURITY]]></title>
            <link>https://medium.com/timeseer/ai-sobering-2023-data-centricity-needed-for-the-next-step-in-ai-maturity-17d9f9b79972?source=rss-7e81eadd67fa------2</link>
            <guid isPermaLink="false">https://medium.com/p/17d9f9b79972</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[iot]]></category>
            <category><![CDATA[data-observability]]></category>
            <category><![CDATA[industrial-ai]]></category>
            <category><![CDATA[data-quality]]></category>
            <dc:creator><![CDATA[Bert Baeck]]></dc:creator>
            <pubDate>Tue, 14 Mar 2023 09:21:04 GMT</pubDate>
            <atom:updated>2023-03-14T09:21:04.081Z</atom:updated>
            <content:encoded><![CDATA[<p>To date, 63% of companies are still in the <a href="https://www.accenture.com/us-en/insights/artificial-intelligence/ai-maturity-and-transformation?c=acn_glb_aimaturityfrompmediarelations_13124019&amp;n=mrl_0622">experimentation phase</a> with AI, and 80% of all <a href="https://www.roboticsbusinessreview.com/ai/almost-80-of-ai-and-ml-projects-have-stalled-survey-says/">AI projects are still stuck</a> at the Proof of Concept level. These are sobering statistics. Wouldn’t you agree?</p><p>In this article, I will update the insights I presented in my <a href="https://bertbaeck.medium.com/ai-sobering-8f11bff5f2e7">2020 AI Sobering</a> blog and explain why data quality is now getting so much attention and importance. I will also sketch the steps organizations can take towards ensuring organization-wide reliable data and provide some key recommendations on how to get started from where you are on your journey to AI maturity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*KPBKY-ZXSzTJXf8A.jpg" /></figure><h3>#DATA AND AI ARE EATING SOFTWARE</h3><p>In 2011, <a href="https://en.wikipedia.org/wiki/Marc_Andreessen">Marc Andreessen</a> famously claimed that “Software would be eating the world.” He also predicted that software companies would disrupt traditional industries. Since then, we have seen many industries transform. Following the lead of Amazon, Netflix, Airbnb, etc, numerous companies have started using their data to their advantage. However, from what we have observed today, I believe that it is instead “AI and data that are eating software.”</p><p>And this is because most companies have become not only software companies but also data companies that use their data as strategic assets to compete in their respective markets. This shift to data-driven decision-making, advanced automation, and AI models in production (at scale) is necessary to remain competitive. Of course, not all industries are developing at the same pace or are under the same pressure; however, it is for sure happening.</p><h3>#AI SOBERING DEFINED</h3><p>During my period working in venture capital, I had over 200 conversations with AI companies. From these talks, I started to see a trend that I later called <a href="https://bertbaeck.medium.com/ai-sobering-8f11bff5f2e7">AI Sobering</a>. I called this trend “AI Sobering” because I noticed a huge discrepancy between the expected outcomes and the actual outcomes of the AI initiatives of these companies. That the state of the AI market is sobering is not entirely a surprise. A lot can be linked to where AI is on the famous <a href="https://www.gartner.com/en/articles/what-s-new-in-artificial-intelligence-from-the-2022-gartner-hype-cycle#:~:text=The%202022%20Gartner%20Hype%20Cycle%E2%84%A2%20for%20Artificial%20Intelligence%20(AI,applications%2C%20devices%20and%20productivity%20tools.">hype cycle</a> of innovation (produced annually by Gartner). Over the years, AI as an umbrella topic has broken up into several smaller segments, each with its own maturity level.</p><p>For example, today Big Data is perceived as mature and productive by many, but back in 2014, it was still going through the trough of disillusionment. As you can see in the figure below, every innovation that survives once started as a technology trigger. Then these become hyped up to a peak of exaggerated expectations to eventually evolve to a plateau of productivity after moving through the trough of disillusionment. Sobering up is part of becoming mature…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*f3oN2t4a7wHwGkFJHOQE4A.png" /></figure><p>Now, two years after my initial AI Sobering article, my <strong>key conclusions have remained almost unchanged</strong>: Most companies are still experimenting with AI, and most of the earlier flagged findings are still valid. I made these conclusions by studying a recent <a href="https://www.accenture.com/us-en/insights/artificial-intelligence/ai-maturity-and-transformation?c=acn_glb_aimaturityfrompmediarelations_13124019&amp;n=mrl_0622">Accenture survey</a> which was conducted with over 1,600 C-Suite executives and data-science leaders from the world’s largest organizations. Below are some of the observations from this report that led me to my conclusions:</p><ul><li>80% of all AI projects are stuck[1] at the Proof of Concept.</li><li>50% of all AI projects fail due to poor data quality (for one out of four companies, according to International Data Corporation (IDC)).</li><li>90% of VC-backed AI companies end up as services-oriented organizations because they never evolve to 80% productization[2].</li><li>Much of what is labeled as ‘artificial intelligence’ today is not AI at all.</li></ul><p>So it begs to question: What are the reasons that so many (VC-backed) AI start-up companies fail or never achieve full productization? Here is what I think:</p><ul><li><strong>They have no focus:</strong> They serve too many non-related use cases and subsequently try to cater to the needs of too many user personas (A one-size-fits-all approach is rarely successful.)</li><li><strong>They cannibalize use cases:</strong> They do not break down the problem into smaller bite-size problems that can be solved separately in a scalable and ecosystem-friendly method. Solving a use case end-to-end is often not feasible.</li><li><strong>They use non-scalable AI models</strong>: They cannot replicate models to a similar asset, so moving to the next ‘asset or device’ would require a lot of tuning or additional services.</li><li><strong>They have an unhealthy, </strong><a href="https://venturebeat.com/business/why-ai-companies-dont-always-scale-like-traditional-software-startups/"><strong>unsustainable gross margin</strong></a><strong>:</strong> They use a cloud infrastructure that is substantial and that sometimes has hidden costs. They also rely too much on the “humans in the loop” to function at a high level of accuracy.</li></ul><p>Even given these reasons, it’s good to note that overall AI maturity is increasing but still strongly varies between industries. The expected forecast for 2024 is promising, though. After reviewing the data from the 2022 Accenture Survey mentioned above, here are a few more observations I’ve made:</p><ul><li>Only 12% of companies are already AI achievers.</li><li>The automotive industry is the pack’s leader if we exclude ‘big tech’.</li><li>The chemical industry is lagging against the industrial average.</li></ul><p>And very important to mention: According to Accenture, AI transformation is expected to take less time than digital transformation, approximately 13% or 16 months faster.</p><h3>#BARRIERS TO AI ADOPTION</h3><p>AI has been a technology buzzword for over a decade now; however, if only 12% of companies out there are real AI achievers, what’s preventing organizations from making the next step in AI maturity?</p><p>To formulate an opinion about this, I would like to reference an annual study <a href="https://www.oreilly.com/radar/ai-adoption-in-the-enterprise-2021/">O’Reilly</a> conducts on barriers to AI adoption.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ROmsXxbnS3bX0l98rh_kfQ.png" /></figure><p>Referring to the figure above, and compared with previous years, the same challenges exist today, but there are some important trends and changes. <strong>The three aspects contributing to nearly 55% of current barriers are the “lack of skilled people or difficulty hiring the required roles”, “difficulty identifying appropriate business use cases”, and “data quality issues or lack of data quality”</strong>. I have several observations about these challenges and about moving toward a more mature AI practice.</p><p><strong>#1.</strong> At 19%, the “<strong>lack of skilled people or difficulty hiring the required roles”</strong> is the highest scoring barrier. To overcome this challenge, organizations need more business users or citizen data scientists — a term we use to describe subject matter experts who have increased their data science knowledge and experience. Furthermore:</p><ul><li>There is simply a global shortage of data scientist profiles in the market. Therefore, for organizations to succeed in their overall digital transformation, they need more internal hands working with the data, i.e., business users or citizen data scientists (see figure below).</li><li>According to a <a href="https://pages.dataiku.com/2022-trends">2022 study from Dataiku</a>, we are at a crossroads because these business users and so-called citizen data scientists have started delivering more value with AI than data scientists. A key recommendation here is to hire people that are passionate about the problem, not just about the ML techniques that can be applied.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7P8LRy0SBwFYHaTu70aP7Q.png" /></figure><p><strong>#2.</strong> The “<strong>difficulty identifying appropriate business use cases”</strong> at 17% is a second barrier to AI adoption. However, as I mentioned above, since there is an increased number of citizen data scientists and business users, there will also be a resulting decrease in the difficulty of finding and identifying appropriate use cases.</p><p><strong>#3</strong>. A third barrier is the<strong> “data quality issues or lack of data quality”</strong>. It is striking to see that when moving into a more mature AI practice, the importance of data quality starts to play a more dominant role. This number has increased from 12% to 18% in less than a year, and I expect this number to grow in the coming years to become the most dominant contributor to AI maturity. Therefore, it is time to embrace the fact that to make the next step in AI maturity, organizations need <a href="https://analyticsindiamag.com/andrew-ng-to-kickstart-a-new-generation-of-ai/">better data, not better models</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/0*8s-M9nnegZJZ5k8y.jpg" /></figure><p>What this means is that companies need to guarantee trust in the data that resides in all the data stores out there and to make reliable, trustworthy data available to an ever-growing number of data consumers. Additionally, “Data is a strategic asset” needs to be more than a CXO buzz statement. In fact, if data is used at scale while the trust in that data cannot be guaranteed, then data becomes a liability. Thus, if organizations truly want to become data-driven, then data reliability is an absolute prerequisite.</p><h3>#THE GREAT UNLOCK &amp; THE IMPORTANCE OF DATA QUALITY</h3><p>Why is data quality suddenly so important? Wasn’t this an old Gartner quadrant? Yes, that’s the right observation. More than two decades ago, the first-generation data quality platforms entered the market shortly after the first-generation business intelligence and data warehouses came to the market. The first movers, such as Precisely, IBM, Informatica, and Talend, are still important today in this market and are highly profitable.</p><p>However, due to technological evolutions of recent years, organizations can now store and compute data at scale (mainly in the cloud), freeing up people and giving them time to focus on higher-order technology like ML/AI and better tooling. With these new capabilities, organizations have started building data-driven applications that are mission-critical. This is called <strong>‘the great unlock.’</strong> In fact, the leadership of many organizations promotes this evolution by encouraging their employees to work more with data in their transition to becoming more data-driven. Hence, with a growing amount of data consumers, trust in data has become crucial for success.</p><p>The big difference between the established, first-generation data quality/observability vendors and the second-generation data quality/observability vendors like Monte Carlo Data, Soda, Timseer.AI, and Databand is that these “new kids on the block” are natively integrated with all the new building blocks of the modern data stack: data integration tools, ETL/ELT, data transformation, metric stores, and many more. And these vendors offer end-to-end data observability as a core differentiator.</p><p>I predict that data quality will soon become the fastest-growing category within enterprise data. Since companies like Databricks, Snowflake, and Google BigQuery are all posting revenues above $1B, this is a significant indication that the data quality/observability market will become business-critical and huge. Let me explain: The tremendous revenue of these companies indicates that there will be further adoption of modern cloud data stores since the faster the revenues grow, the more the adoption of modern cloud data stores will grow. More importantly, all this data needs to be protected and guaranteed that it is reliable. Companies have been investing heavily in their data infrastructure over the last years, so it is a logical consequence that data leadership needs to protect the data that resides in their data stores and that is heavily used. Right?</p><h3>#RELIABLE DATA: IT’S EVERYONE’S PROBLEM!</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/368/0*ifO6_hSlDJ_muSpK" /></figure><p>The evolution towards mandatory reliable data needs to be supported by the CDO offices or the data governance teams. No one wants to and should serve as a company’s designated ‘data janitor’ — the person responsible for cleaning the data. As such, let’s have a look at each entity’s current pain regarding this issue:</p><ul><li><strong>Chief Data Officer: </strong>Providing trust in data is shifting towards corporate responsibility, especially given that today a growing amount of data consumers rely on the available data, and only a small percentage of these consumers is aware of the quality and reliability of this data.</li><li><strong>Data Teams (Data Science + Data Engineering): </strong>These teams are acutely aware of the importance of data quality and spend as much as 60% of their valuable time on cleaning and transforming data: a time-intensive, manual job no data scientist enjoys and by default is usually tackled at the end of the data pipeline (which is too late).</li><li>Data Quality should be everyone’s shared responsibility, yet in many organizations, data engineers are expected to clean the data and maintain or fix pipelines they didn’t break in the first place.</li><li><strong>Business Users: </strong>When unverified data is used for day-to-day decision-making, in key operational Excel sheets, and for troubleshooting, it’s just a matter of time before wrong decisions are made and wrong insights are derived. Obviously, when the quality of data is not known, the accuracy of decisions is also not known. The impact of this can be enormous.</li><li><strong>The Business Itself: </strong>For many business processes, as long as there is a human in the loop, there is always the possibility of error. However, if organizations truly shift towards ‘autonomous operation,’ data-driven decision-making, and advanced automation, data quality shifts into a “must-have.” There is no possible autonomous operation without reliable data.</li></ul><h3>#SIZING THE PROBLEM OF DATA QUALITY</h3><p>The size of this problem will only keep growing, and I believe this for several obvious reasons:</p><ul><li>The amount of ‘new’ data will continue to grow. In fact, 90% of the world’s data has been generated over the last two years. In an industrial context, this data growth has accelerated due to the advent of 5G and the rise of cheap sensors.</li><li>The enterprise data stack is becoming increasingly complex, so as a consequence, with each step the data takes upstream or downstream in the data pipeline, the impact of data downtime balloons.</li><li>Since data is increasingly leveraged and used, data downtime will impact more critical business processes, services, and products.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hb6eX4PJeFLYuvuWNDxK4g.png" /></figure><h3>#MAD* MATURITY MATRIX ™ AND DATA THAT BECOMES A LIABILITY</h3><p>Another piece of this puzzle is understanding the MAD Maturity Matrix and how unverified data can become a liability. So first, let’s start with the MAD maturity matrix. As seen in the two-dimensional matrix below, the maturity of data quality is plotted against the maturity of AI/data. To indicate the maturity of the AI practice, we distinguish five levels:</p><ol><li><strong>Basic:</strong> Investments are made in data storage infrastructure, and the first dashboards are up and running.</li><li><strong>Foundation:</strong> Discovery analytics (search/troubleshooting tools) are present, and the organization has dedicated people who can be labeled as citizen data scientists.</li><li><strong>Operational</strong>: The company has matured into predictive analytics and has some first AI/ML proof of concepts running, and next to the CDS teams, there are data scientists/engineers amongst the workforce resulting in full operational data pipelines.</li><li><strong>Systemic</strong>: The company is scaling up its AI practice and has the first predictive models in production, and is investing heavily in further cloud transformation.</li><li><strong>Transformational</strong>: The organization has evolved into storing its data in the cloud and is working on digital twins, and has predictive production analytics models at scale.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HRbmk5wOdLmws2X6_iR-VA.png" /></figure><p>Referring to the figure above: If organizations advance their AI/Data Maturity (X-axis) without paying the proper attention to data quality, data becomes a liability. Digital transformation will only be successful if the generated and stored data is reliable. Whenever an organization is confronted with one of the following symptoms, there is likely a situation where data has become a liability:</p><ul><li>Limited or no repeatability and scalability of AI models</li><li>Lack of trust in the data-by-data consumers or lack of trust in the predictions made by the model</li><li>Clear evidence of data downtime that has resulted in operational downtime</li><li>Suboptimal data-driven decisions</li><li>Extremely low efficiency in data projects</li><li>Difficulty in moving beyond the POC level</li></ul><p>As you can see, for many reasons, data reliability is a business-critical issue, and the size of the problem is growing. I have already mentioned several reasons for this, but let me weigh in on a few more:</p><ul><li>Most data stored today is <strong>unmonitored, unstructured, and unverified for reliability</strong>.</li><li>90% of the workforce is ignorant or unaware of the quality of data.</li><li>Nearly every data consumer will agree with the fact that they have data issues, but nobody knows the size of this impact.</li></ul><p><strong>Hence, the impact and size of data quality are unquantified today, and consequently, companies are vulnerable at scale.</strong></p><h3>#HOW DATA CAN BECOME A STRATEGIC ASSET</h3><p>I hear a lot of CDOs and data decision-makers say that data is a strategic asset. If data is truly this, and thus more than just a buzzword, organizations must treat it as such. If they mean it, they must handle it accordingly. However, this will never happen if they push through with a sole focus on AI maturity without concentrating on ensuring the quality and reliability of the data. Proper data quality management and observability are imperative to move towards full AI maturity and to achieve this impact at scale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/0*xYX95fiZ68G2NaDo.jpg" /></figure><p>Of course, this only works in steps that I’ve outlined below and that are illustrated in the figure that follows:</p><p><strong>Step 1:</strong> Provide tooling to be able to assess or score the quality of the data.</p><p><strong>Step 2:</strong> Enable the capability to monitor the quality of data over time and flag anomalies, and provide the first triage capabilities.</p><p><strong>Step 3:</strong> Provide the ability to set up data quality SLAs (and monitor against them) and provide a data cleaning pipeline/data service to provide optimal quality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7g1o9nTGJew17eA21SavAQ.png" /></figure><p>Data can become an asset if we move linearly towards the upper right quadrant, where organizations will be both optimized for data quality and transformational with AI. As an organization, if you are on the right track, here are a couple of common outcomes to detect:</p><ul><li>Organisational trusted data and decisions</li><li>Repeatable, maintainable, and scalable analytics</li><li>High useability by all data user personas</li><li>Reduced time to value</li></ul><p>To conclude, I want to share some key conclusions and some key recommendations on how organizations can more quickly arrive at the AI productivity plateau.</p><h3>#KEY CONCLUSIONS</h3><ul><li>Organizations need to accept that the bottleneck is the data, not the state of the models used today (which corresponds with what thought leaders like Andrew NG are saying).</li><li>Only 12% of the companies out there can already be labeled as AI achievers. However, it’s expected that AI transformation will take less time than digital transformation.</li><li>Data cleansing and contextualization are done and tackled often at the end of the data pipeline. As a result, data teams lose valuable time discovering the reason(s) for data anomalies.</li><li>Data quality is not yet omnipresent in most organizations that leverage their data. I think after reading this article post, it’s clear that this is not just the problem of IT or the data teams.</li></ul><h3>#KEY RECOMMENDATIONS</h3><ol><li>Organizations should make data governance a corporate strategic topic for all their companies, making not only the IT data (relational data) a scope or focus but also the operational data.</li><li>Organizations need to equip data science teams with the right data engineering/operations tools to boost and optimize their productivity, so they can save time on data cleaning and transformation.</li><li>Organizations need to invest further in creating more business users and citizen data scientists (domain experts). This step will result in many more hands that can contribute to the success of the organization’s digital transformation.</li><li>Organizations must start quantifying the quality of data and monitoring the quality of the data over time to get a better understanding of the size of the current problem and to take the first step towards data centricity.</li><li>Organizations should start by appointing data stewards (internal promotion). These are subject matter experts that understand both the business/operations and the content of the data and are responsible for ensuring the quality and fitness of the organization’s data assets.</li><li>Organizations should clean and contextualize the data closer to the source since these people understand the operational context and the content of the data. Too much time is lost finding out post-mortem what is wrong with the data.</li><li>Organizations should establish the good practice of working toward one single version of the truth regarding data. However, it should be noted that it is good practice to store several tiers of aggregated data with different data quality levels (Bronze/Silver/Gold, as seen in the figure below).</li><li>Organizations should create and expose Brown Data (dirty synthetic data by design) with available data products to expose vulnerabilities which will lead them to understand what data defects they need to protect themselves from. This is good practice. Even if issues haven’t happened yet, it doesn’t mean organizations are not vulnerable.</li><li>Organizations should invest heavily in using data while assuring a reliable foundation, and as much as possible, should create mission-critical data-driven applications. After all, the above recommendations only matter if data is used more in dashboards and data products and if it triggers workflows and tasks to generate actionable insights.</li><li>To end, all of the above recommendations are relevant only if organizations first start using data more, so to reiterate: “Do more of recommendation #9”. 😊</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9PIKO4L-lTLXYs5h0rTtCA.png" /></figure><p>This list is limited, and there are many more recommendations to give. So feel free to share your opinion/take on this. I can elaborate on these in my next blog post if needed.</p><h3>#TIMESEER.AI’S ROLE IN ENSURING THE SUCCESS OF AI MATURITY</h3><p>I represent Timeseer.AI as its co-founder and CEO. We have developed a Data Quality Management and Observability Platform for time-series data which includes sensor/device data, timestamped system metrics, financial stock data, and weather data.</p><p>The time-series space is booming! It has been the fastest-growing database segment for the past 2 years and will continue to grow. It’s important to point out here that most 2nd generation data quality tools available today are built and optimized for relational data. These tools do not work well for time-series data since time-series data is a completely different animal. (I’ll elaborate more about this in a later blog.)</p><p>By focusing on time-series data, we are making a contrarian bet on the absolute need now and in the future for a data quality management and observability platform for this data type. And we estimate, although possibly a bold prediction, that by 2030, ALL sensor data stored in time-series databases will be monitored for reliability to ensure that the data is fit-for-purpose and can be treated as a true strategic asset.</p><p>As Timeseer.AI, we have outlined <strong>our mission</strong> on two levels:</p><ul><li><strong>On the enterprise level</strong>, we aim to reduce the number of data quality and sensor data integrity incidents that hit operations by 10X.</li><li><strong>On the individual level</strong>, we aim to help bring in 50% extra productivity in the daily sensor data ops for data science teams.</li></ul><h3>#ABOUT ME</h3><p>After Software AG acquired my previous software scale-up company, I worked for three years as a partner at Smartfin Capital, a European technology Venture capital fund. During this time, I had the opportunity to investigate and study the entire market with a sole focus on IoT and <a href="https://mattturck.com/data2021/">MAD</a> (in other words: “machine learning, AI, and data” as a venture capitalist and owner of the MAD landscape <a href="https://mattturck.com/">Matt Turck</a> started calling it).</p><p><em>[1] </em><a href="https://www.accenture.com/us-en/insights/artificial-intelligence/ai-maturity-and-transformation?c=acn_glb_aimaturityfrompmediarelations_13124019&amp;n=mrl_0622"><em>https://www.accenture.com/us-en/insights/artificial-intelligence/ai-maturity-and-transformation?c=acn_glb_aimaturityfrompmediarelations_13124019&amp;n=mrl_0622</em></a><em><br>[2] Based on personal research during my VC period.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17d9f9b79972" width="1" height="1" alt=""><hr><p><a href="https://medium.com/timeseer/ai-sobering-2023-data-centricity-needed-for-the-next-step-in-ai-maturity-17d9f9b79972">AI SOBERING 2023: DATA CENTRICITY NEEDED FOR THE NEXT STEP IN AI MATURITY</a> was originally published in <a href="https://medium.com/timeseer">Timeseer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>