<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[The Semantic Layer - Medium]]></title>
        <description><![CDATA[The semantic layer is where raw data becomes trusted knowledge, and where AI finally has something solid to stand on. This publication covers the practice of building it: data governance, ontology, knowledge graphs, and the patterns that make data and AI work together. - Medium]]></description>
        <link>https://medium.com/the-semantic-layer?source=rss----476333b6b1c2---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>The Semantic Layer - Medium</title>
            <link>https://medium.com/the-semantic-layer?source=rss----476333b6b1c2---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 19:33:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/the-semantic-layer" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Self-service data product access for Microsoft Fabric - OneLake catalog]]></title>
            <link>https://medium.com/the-semantic-layer/self-service-data-product-access-for-microsoft-fabric-onelake-catalog-48e98b81246b?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/48e98b81246b</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[onelake-catalog]]></category>
            <category><![CDATA[product-data]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <category><![CDATA[microsoft-purview]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Sun, 27 Sep 2026 14:34:29 GMT</pubDate>
            <atom:updated>2026-09-27T14:34:29.744Z</atom:updated>
            <content:encoded><![CDATA[<p>In many organizations, data access still runs on emails, tickets and someone manually adding people to groups. That’s slower than it needs to be, and afterwards it’s hard to show who approved what.</p><p>This article walks through a self-service access process for data products built on Microsoft Fabric, Microsoft 365 and Microsoft Entra ID. It uses five pieces:</p><p>Thanks for reading! Subscribe for free to receive new posts and support my work.</p><ul><li>Microsoft Copilot Studio for the request conversation</li><li>Microsoft SharePoint as the data product register and the request log</li><li>Microsoft Power Automate for the approval flow</li><li>Microsoft Teams Approvals for the owner’s decision</li><li>Entra ID security groups to grant the access</li></ul><p>The same pattern works with Azure Databricks.</p><h3>The agent collects, the owner decides</h3><p>The agent never makes an access decision. It asks the user two things: which data product they want, and why.</p><p>Everything else comes from governed data. The requester’s identity comes from their Microsoft 365 session. The owner and the access group come from the data product register, looked up by product ID. The agent has no say in who approves a request or which group the user ends up in.</p><p>That leaves four clear jobs. The agent takes the request. The register says what the product is and who owns it. The owner approves or rejects. Entra ID grants the access.</p><h3>Three layers</h3><p><strong>Front door: Copilot Studio.</strong> The Data Access Assistant runs in Teams and Microsoft 365 Copilot Chat. It has two workflows exposed as tools. One lists the available data products, and the other creates an access request.</p><p><strong>Data: SharePoint.</strong> Two lists hold everything. The Data Product Repository stores each product with its owner and access group. The Access Requests list stores each request and its approval history.</p><p><strong>Process: Power Automate.</strong> A flow starts when a new access request arrives. It looks up the product, sends the approval to the owner and writes the decision back to the request.</p><p>The Access Requests list sits between the agent and the approval process. The agent writes a row, and Power Automate reacts to it. Because the two sides only meet at that list, you could swap the chat interface for a Power App later and leave the approval logic alone.</p><h3>How a request moves through the system</h3><ol><li>The user opens the Data Access Assistant in Teams or Microsoft 365 Copilot Chat.</li><li>The agent pulls the current list of data products from the repository.</li><li>The user picks a product and explains why they need it.</li><li>The agent creates an Access Requests record with the requester, product ID, justification and a Pending status.</li><li>Power Automate looks up the product owner and access group.</li><li>The owner gets an approval in Teams Approvals.</li><li>If the owner approves, the requester is added to the Entra ID security group.</li><li>The flow updates the request record with the decision.</li><li>The requester gets a notification.</li></ol><p>Rejected requests are recorded too, with the approver’s comment. Each user talks to the agent in their own 1:1 conversation, so many requests can run at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*cOWfVc402LW1iNVq.png" /></figure><h3>The SharePoint lists</h3><h4>Data Product Repository</h4><p>The repository has one row per data product. The core fields are product name, owner and access group. Store the owner as a Person column and the access group as a Person or Group column pointing at the Entra ID security group. The SharePoint item ID works fine as the product ID.</p><p>One SharePoint detail will save you trouble. If Power Automate expressions will reference a column, create it without spaces. SharePoint sets the internal column name when you create the column, and renaming the display name later doesn’t change it.</p><h4>Access Requests</h4><p>Each request holds the requester email, product ID, justification, status, decision date, decision maker, approver comment and granted group. The list is both the request queue and the audit record.</p><h3>The Copilot Studio agent</h3><p>The agent is kept simple on purpose. It has instructions that describe the access request conversation, plus the two workflows as tools.</p><h4>Get data products</h4><p>This workflow reads the Data Product Repository and returns a short list of product names and IDs. A Select action builds the structure:</p><pre>title = item()?[&#39;Title&#39;]<br>value = string(item()?[&#39;ID&#39;])</pre><h4>Create access request</h4><p>This workflow writes the request to the Access Requests list.</p><p>The field that needs protecting here is the requester email, and the AI should never be the one filling it in. Set the tool input to:</p><pre>System.User.Email</pre><p>That ties the request to the signed-in user. Nothing said in the conversation can change it.</p><h3>The approval flow</h3><p>Power Automate starts when a new Access Requests item is created. The flow resolves the requester, fetches the product by its ID and reads the owner and access group from the repository. It then creates a Teams approval assigned to the owner. If the owner approves, the flow adds the requester to the Entra ID group. Either way, it writes the final decision back to the request.</p><p>Turn off reassignment on the approval. Otherwise someone can hand the request to a different approver.</p><p>You can pull the Entra ID group’s object ID out of the SharePoint Person or Group claims value:</p><pre>last(split(outputs(&#39;Get_item&#39;)?[&#39;body/ADGroup/Claims&#39;], &#39;|&#39;))</pre><p>Both the approve path and the reject path update the request record, so every request ends in a final state.</p><h3>Microsoft Fabric and Databricks both plug in through the group</h3><p>The workflow never talks to the data platform directly. The Entra ID security group is the only integration point.</p><h4>Microsoft Fabric</h4><p>A product in the register can be a lakehouse, a warehouse or a semantic model in the OneLake catalog. You give the matching Entra ID group access to that item with the Microsoft Fabric permission that fits: item permissions, workspace roles or OneLake security roles. Once the group has its permissions, approving a request only means adding the user to the group.</p><p>You can also store a link in the repository so the agent can send users straight to the data product.</p><h4>Azure Databricks</h4><p>The same approach works with Unity Catalog. You grant privileges such as USE CATALOG, USE SCHEMA and SELECT to a group backed by the Entra ID group. The group reaches Databricks through automatic identity management or SCIM provisioning.</p><p>The approval process stays the same. Only the group tied to the product changes. Depending on how you set up synchronization, a new membership can take a while to show up in Databricks.</p><p>One repository can hold both Microsoft Fabric and Databricks products. A Platform column is enough to tell them apart.</p><h3>SharePoint is optional</h3><p>SharePoint was convenient for this build, but the pattern doesn’t depend on it. You need a reliable source that knows each data product, its owner and the group that grants access. That could be a database, an API, a catalog or a data governance platform.</p><p>If your organization already runs a data product catalog, don’t copy it into SharePoint. The agent only needs to read product information from somewhere and create the request.</p><h3>Governance controls</h3><p>Self-service access needs tight control over identity, ownership and configuration. Each risk below has a matching control.</p><p><strong>Requester impersonation.</strong> The requester email comes from the signed-in Teams identity.</p><p><strong>Wrong approver.</strong> The flow looks up the owner in the repository by product ID.</p><p><strong>Wrong access group.</strong> The group also comes from the repository.</p><p><strong>Delegated approvals.</strong> Approval reassignment is turned off.</p><p><strong>Automated misuse.</strong> Agent-to-agent connections are turned off.</p><p><strong>Repository tampering.</strong> Only the governance team can edit the repository.</p><p><strong>Missing owner.</strong> Every product must have an owner.</p><p><strong>Lingering access.</strong> You can add access expiry and automatic removal.</p><p>The Data Product Repository needs the most protection. If users could edit it, they could change a product’s owner or access group and get around the approval. Treat it as governed configuration.</p><p>The Access Requests list is your audit trail. It shows who asked, for which product, why, who approved or rejected it, when, and which group was used. That record supports access reviews and audits.</p><h3>Lessons learned</h3><p>The architecture itself is fairly simple. Most of the friction came from the Microsoft platform.</p><h4>Use an agent, not a shared Adaptive Card</h4><p>An earlier version posted an Adaptive Card in a Teams channel. It handled one response fine. Letting many people respond to the same card needed a response trigger that didn’t work reliably in the tenant. A 1:1 conversation with an agent avoids the problem.</p><h4>Plan for Dataverse and Copilot Credits</h4><p>Copilot Studio agents need Dataverse, and testing in the new designer uses Copilot Credits. A dedicated environment linked to a pay-as-you-go billing policy covered both.</p><h4>The first approval is slow</h4><p>Teams Approvals runs on Dataverse as well. The first approval in an environment starts a one-time setup, so that first request can take a while to show up.</p><h4>Watch your Power Automate expressions</h4><p>A trigger condition that used empty() on a number column stopped the flow from triggering at all.</p><p>In Select actions, write explicit expressions like item()?[...] instead of picking dynamic content. The designer tends to wrap dynamic content in an Apply to each loop you didn’t ask for.</p><h4>Check where actions sit around a Condition</h4><p>An action placed below a Condition, instead of inside one of its branches, runs for both outcomes. It’s easy to miss when you edit an existing flow.</p><h4>Your browser and network can get in the way</h4><p>Browser DNS settings and VPN configuration can break Power Platform work. In this build, some Power Platform domains were blocked and threw NetworkErrors. Switching to Edge fixed it.</p><h4>Generative agents may ask before acting</h4><p>A generative agent can ask the user to confirm before it runs a tool. If you want the request submitted right away, handle that in the agent instructions, the skill instructions and the tool description.</p><h3>Next: let users find the product before they ask for it</h3><p>Right now the agent asks one question: which data product do you want? The next step is to let users talk about the products first.</p><p>A user could ask:</p><blockquote><em>Which data products contain customer revenue information?</em></blockquote><p>The agent would search the product metadata and explain what it finds. The user might follow up:</p><blockquote><em>Which one would be suitable for analyzing revenue by region?</em></blockquote><p>The agent could describe the matching products, their owners and whatever other metadata exists. Once the user has picked one, the same conversation carries on:</p><blockquote><em>I want access to this one.</em></blockquote><p>The agent collects the justification and submits the request. The flow becomes discover, understand, select, request, approve, access. Requesting access turns into part of finding data instead of a separate ticket.</p><p>A production version should also have:</p><ul><li>A production environment and a service identity instead of personal connections</li><li>Backup owners</li><li>Approval timeouts and escalation</li><li>A way for users to check the status of their own requests</li><li>Access expiry and automatic removal</li><li>Regular access reviews</li></ul><p>You can start with SharePoint and move to another source later without touching the approval pattern. What matters is that each part keeps its own job. The data source describes the products, the agent helps people work with that information, the owner decides, and Entra ID enforces the result.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=48e98b81246b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/self-service-data-product-access-for-microsoft-fabric-onelake-catalog-48e98b81246b">Self-service data product access for Microsoft Fabric - OneLake catalog</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your Data Catalog Has 1 million Assets and Nobody Can Find Anything. Here Are the KPIs That Fix It.]]></title>
            <link>https://medium.com/the-semantic-layer/your-data-catalog-has-1-million-assets-and-nobody-can-find-anything-here-are-the-kpis-that-fix-it-45d98b6e6ee2?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/45d98b6e6ee2</guid>
            <category><![CDATA[data-catalog]]></category>
            <category><![CDATA[metadata-management]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[microsoft-purview]]></category>
            <category><![CDATA[databricks]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Wed, 15 Jul 2026 06:02:18 GMT</pubDate>
            <atom:updated>2026-07-15T06:02:17.947Z</atom:updated>
            <content:encoded><![CDATA[<p>A data catalog with 1 million data assets and a 1 percent search success rate is not a catalog. It is a graveyard with a search bar. The asset count on your quarterly slide tells you nothing about whether anyone has ever found what they needed in there.</p><p>The teams running healthy catalogs measure something different. They measure whether the catalog actually works as a product, and when a metric goes red, they investigate the root cause instead of polishing the dashboard.</p><h3>TL;DR</h3><ul><li>Asset counts, glossary term counts, and steward counts measure how busy your team is, not whether the catalog is useful.</li><li>Replace inventory metrics with four outcome categories: Findability, Trust, Engagement, and Curation Health (also known as Metadata Quality).</li><li>A failing KPI is the start of an investigation, not the end. Most catalog problems have organizational root causes hiding behind technical symptoms.</li><li>Every KPI in this article maps to telemetry available in Microsoft Purview Unified Catalog, Collibra, Alation, Atlan, Databricks Unity Catalog, Snowflake Horizon, or any modern catalog with audit logs.</li><li>Treat your catalog like an e-commerce site, not a filing cabinet. Track searches, clicks, and conversions.</li><li>A phased rollout is included so you do not try to wire up twelve metrics in week one.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qT6S-nKdJP8gyUkc0QR-1g.png" /></figure><h3>Why “Assets Cataloged” Is The Worst KPI in Data Governance</h3><p>“We cataloged 1 million assets this year.”</p><p>Then someone searches “quarterly revenue” and gets zero results, because the certified data product is called “Finance Reporting”</p><p>Cataloging an asset is the easy part. It is the part you can automate, schedule, and forget. The hard part is making that asset findable, understandable, trustworthy, and used. Those four things have nothing to do with how many assets are in the catalog.</p><p>The pattern repeats across every catalog metric that gets reported up. Glossary terms defined. Stewards assigned. Domains created. Lineage scans completed. Each one is true. None of them tell you whether the catalog is doing its job.</p><p>A catalog has exactly one job: connect a person who needs data to the right data, fast, with enough context to use it correctly. Every KPI worth tracking is a measurement of how well that job is being done. And every red KPI is a question, not an answer.</p><h3>The Four Outcomes a Useful Catalog Delivers</h3><p>A catalog worth its budget delivers four outcomes:</p><ol><li><strong>Findability.</strong> People can find the right data when they look.</li><li><strong>Trust.</strong> People believe what they find.</li><li><strong>Engagement.</strong> People come back and act on what they find.</li><li><strong>Curation Health (Metadata Quality).</strong> The catalog stays useful over time.</li></ol><p>Each one has metrics behind it. None of them are asset counts. And each one will eventually go red, which is when the real work starts.</p><h3>Findability: Does the Search Bar Actually Work?</h3><p>Findability is the easiest pillar to measure because most catalogs log the data for you. In Microsoft Purview Unified Catalog, the Classic Catalog Adoption Report under Health Management is your starting point. Collibra exposes the same telemetry through its Insights module. Alation, Atlan, and data.world all surface search analytics natively. Databricks Unity Catalog and Snowflake Horizon log query and search activity into system tables you can query directly. Whatever tool you run, the data is sitting there.</p><p><strong>Zero-Result Search Rate.</strong> Of all searches performed in the last 30 days, what percentage returned no results? This number is the single best leading indicator of catalog health. A rate above 20 percent means your users are walking away empty-handed one in five times. They will stop coming back. Track the trend by user persona: a high zero-result rate among business users is a vocabulary problem, not a metadata coverage problem.</p><p><strong>Top Zero-Result Terms (the working list).</strong> Pull the actual search strings that returned nothing. This is your roadmap. Each term is a user telling you what they expected to find. The fix is rarely to add more assets. It is to add a glossary synonym, rewrite a description, or rename a data product. Track the count of unique zero-result terms over time. A shrinking list is the truest sign your catalog is learning.</p><p><strong>Time to First Useful Asset.</strong> Pick five common questions a new analyst would ask in their first week (“where is the active customer table?”, “which dataset has churn predictions?”). Have a non-expert search and time how long until they land on a usable asset with enough metadata to act on it. Track the median monthly. Under two minutes is healthy. Over five minutes means your catalog is theoretical.</p><p><strong>Search-to-Click Conversion Rate.</strong> Of all searches that returned results, what percentage led to the user opening at least one asset detail page? A search that returns 200 results and zero clicks is a failed search dressed up as a successful one. Aim above 60 percent. Below 30 percent means your search relevance is broken.</p><p>Findability gets people to the asset page. Trust decides whether they actually use it. And Trust is the pillar where root cause analysis matters most.</p><h3>Trust: Do People Believe What They Find?</h3><p>A catalog full of assets nobody trusts is just a Wikipedia for data nobody reads. Trust signals are what convert a curious browser into an actual user.</p><p><strong>Certified Asset Coverage on Critical Data Elements.</strong> Of your CDEs, what percentage carry an active “Certified” or equivalent endorsement from a domain owner? In Microsoft Fabric this is the certification badge on workspace items. In Microsoft Purview Unified Catalog this is the curated, governed status set by a steward. Databricks Unity Catalog uses certified tags on tables and views. Collibra, Alation, and Atlan all have their own endorsement workflows. Below 80 percent on CDEs and your users will keep building shadow extracts. They do not trust the uncertified, and they are correct not to.</p><p><strong>Metadata Completeness on Critical Data Elements.</strong> Skip the overall metadata fill rate. It will look great and mean nothing because the trivial assets bring up the average. Score only the CDEs against required fields: owner, description, classification, certified status, lineage upstream and downstream, sensitivity label. Anything below 95 percent on CDEs is a finding, not a tile.</p><p><strong>Description Quality Score (sampled).</strong> Sample twenty asset descriptions per quarter and score them against a rubric: does it explain what the data is, who it is for, when it refreshes, and what it should not be used for? Report the average. This is the most subjective KPI in the article and the most predictive of user adoption. A catalog full of one-line descriptions copied from column names is a catalog nobody trusts.</p><p><strong>Lineage Completeness on Reported Data.</strong> For every asset that feeds a regulatory report, an executive dashboard, or a customer-facing AI agent, can lineage be traced back to source with no gaps? The KPI is binary at the asset level, percentage in aggregate. This single number is the difference between a one-week audit response and a three-month one.</p><p><strong>Data Quality Pass Rate on Trusted Assets.</strong> A certified asset that fails its own quality rules is worse than an uncertified one. Pair certification with active quality monitoring through Microsoft Purview Data Quality, Soda, Monte Carlo, Great Expectations, or whatever your stack runs. Track the pass rate on certified CDE assets specifically. A drop here is an emergency, not a tile.</p><p>Now the metric goes red. What do you do next?</p><h3>When Trust Breaks, Stop Reporting and Start Investigating</h3><p>This is the part most governance teams skip. The data quality dashboard turns red, the team adds a comment, and the meeting moves on. Six months later the same asset is still failing and nobody trusts the catalog anymore.</p><p>A red KPI is not the answer. It is the question.</p><p>When the data quality pass rate on a certified asset drops, you do not just open a ticket. You investigate. The five-whys technique borrowed from Toyota’s production system works as well in data governance as it does on a factory floor. Each “why” peels back a layer.</p><p>Walk through a real example.</p><p>The pass rate on the certified Customer Master table dropped from 98 percent to 71 percent overnight. <strong>Why?</strong> A new column showed 30 percent nulls. <strong>Why?</strong> The upstream CRM started writing nulls where it used to write a default value. <strong>Why?</strong> A product team deployed a CRM configuration change last Tuesday. <strong>Why?</strong> Nobody told the data team because there was no contract between the CRM and the data platform. <strong>Why?</strong> No producer-consumer agreement exists for any source system feeding the lakehouse.</p><p>The KPI flagged a null spike. The actual problem is organizational. You do not fix this with a data quality rule. You fix it with a data contract framework, a change notification process, and a producer accountability model. The dashboard told you something was wrong. The investigation told you what to actually build.</p><p>The same logic applies across the Trust pillar:</p><p>When <strong>certified coverage</strong> drops, ask why stewards stopped certifying. Is the certification process too heavy? Did a key steward leave? Did leadership stop sponsoring the program? The metric is a symptom. The cause is usually capacity, ownership, or sponsorship.</p><p>When <strong>description quality</strong> is low, ask why descriptions are not being written. Are stewards rewarded for it? Is there a template? Is the catalog UI making it painful? Most of the time the answer is that nobody owns the work and nobody measures it.</p><p>When <strong>lineage completeness</strong> is poor, ask why lineage is broken. Is it manual ingestion that bypasses scanners? Is it transformation logic in stored procedures the catalog cannot parse? Is it cross-platform handoffs (Snowflake to Databricks to Power BI) where lineage stitching fails? Each cause has a different fix, and “we need better lineage” is not a fix.</p><p>The discipline is simple. Every red KPI generates a written root cause analysis before any remediation work starts. The output is not a fix, it is a hypothesis about why the metric went red, and a plan to test it. Skip this step and you will spend the next year fixing symptoms while the underlying disease keeps spreading.</p><p>This is also the section where you earn credibility with engineering. Engineers respect root cause discipline. They do not respect dashboards.</p><h3>Engagement: Are People Coming Back?</h3><p>Most catalogs measure user counts. A user who logged in once last quarter is not a user. A user who searches, opens an asset, and then takes action (requests access, queries the data, links it to a project) is a user. Engagement metrics separate the two.</p><p><strong>Repeat User Rate.</strong> Of users who logged in last month, what percentage came back this month? A flat or declining number means your catalog is a one-visit destination. A rising number means it is becoming part of how people work.</p><p><strong>Asset Page Click Depth.</strong> When a user lands on an asset page, do they engage with the metadata (lineage, schema, quality scores, related assets) or leave immediately? Track average tabs viewed per asset visit, plus bounce rate. A high bounce rate on critical assets is a description quality problem, not a search problem.</p><p><strong>Top-Visited Assets and Long-Tail Coverage.</strong> Plot the distribution of asset visits. A healthy catalog has a fat head (your top 50 assets get heavy traffic) and a meaningful long tail (assets 200 to 2,000 still get occasional traffic). A catalog where 90 percent of traffic hits 1 percent of assets means the rest is dead inventory. Cull or improve.</p><p><strong>Self-Service Access Request Rate Through the Catalog.</strong> What percentage of access to governed data is initiated through the catalog versus through Slack, email, or tickets? In Microsoft Purview this is access policy traffic. In Unity Catalog it is grant requests. In Immuta or Privacera it is policy-driven access. Whatever the tool, the question is the same. Aim above 60 percent. Below 30 percent means people still see the catalog as documentation, not as a workflow.</p><p><strong>Data Product Engagement (not just count).</strong> For each published data product, track views, click-through to underlying assets, access requests, and downstream usage. A data product with 500 views and 2 access requests has a conversion problem. The description oversells, the access process is too painful, or the product itself is not what users actually need. Counting published data products tells you none of this.</p><p>When an engagement metric goes red, run the same investigation discipline. Repeat user rate dropping is rarely about the catalog UI. It is usually about a single bad first experience that taught users the catalog could not be trusted. Find that experience.</p><h3>Curation Health (Metadata Quality): Will the Catalog Still Work in Two Years?</h3><p>Different organizations call this pillar different things. Some governance teams use <strong>Curation Health</strong>, framing the work as ongoing care of the catalog. Others use <strong>Metadata Quality</strong>, framing the metadata itself as a data product with its own quality bar. Both terms point at the same thing: the catalog’s content has to stay accurate, complete, and current, or the rest of your KPIs collapse.</p><p>Pick the term that resonates with your organization. The metrics are the same.</p><p>Catalogs decay. Owners leave. Descriptions go stale. Lineage drifts. Domains accumulate orphan assets. These KPIs catch decay before it kills the catalog.</p><p><strong>Active Steward Coverage.</strong> How many of your domains have a steward who curated, certified, or updated at least one asset in the last 30 days? A steward who has not touched anything in 90 days is not a steward. They are a name in a field. Report active stewards, not assigned ones.</p><p><strong>Asset Staleness Rate.</strong> What percentage of assets in the catalog have not been refreshed, accessed, or curated in the last 12 months? On its own this is just an inventory metric. Pair it with traffic data: a stale asset that nobody visits is a candidate for archive. A stale asset that gets weekly traffic is a candidate for urgent re-curation.</p><p><strong>Orphan Asset Count.</strong> Count assets with no owner, no domain, or no description. Track the trend. A growing orphan count means automated scans are outpacing curation. The fix is either more curation capacity or better automation rules at ingestion.</p><p><strong>Glossary-to-Asset Linkage Rate.</strong> Of your defined business glossary terms, what percentage are linked to at least one cataloged asset? Of your CDEs, what percentage are linked to at least one glossary term? Glossary terms with no linkage are vocabulary nobody uses. Assets with no glossary linkage are data nobody can describe in business language. Both gaps are findability problems waiting to happen.</p><p><strong>Metadata Freshness on Critical Assets.</strong> When was the last time a CDE’s description, owner, or classification was reviewed by a human? Set a freshness SLA per asset tier (CDEs reviewed every six months, regulated assets every quarter, everything else annually). Track the percentage of assets within SLA. This is the metadata-quality equivalent of a data freshness rule, applied to the catalog itself.</p><p><strong>User Feedback Loop Closure Rate.</strong> When a user reports a wrong description, a missing owner, a broken lineage, or a stale certification, how long until it is resolved? Track median time to closure and the ratio of open to closed feedback items. A growing backlog of unresolved feedback is a catalog losing its user base in slow motion.</p><p>When metadata quality degrades, the root cause is almost always organizational: stewards have day jobs, the program has no executive sponsor, or the curation process is too heavy for the value it delivers. Investigate accordingly.</p><h3>Where the Catalog Telemetry Already Lives</h3><p>Most of these KPIs do not require new tooling. The data exists today, regardless of what catalog you run.</p><p>In Microsoft Purview Unified Catalog, findability data sits in Health Management’s Classic Catalog Adoption Report. Trust and metadata quality telemetry come from asset metadata APIs and the audit log. Engagement comes from search and asset interaction events.</p><p>In Collibra, the equivalent telemetry sits in Insights and the Activity Stream. Alation exposes it through its built-in usage analytics. Atlan surfaces engagement and search data natively. Databricks Unity Catalog and Snowflake Horizon log query, search, and grant events into system tables you can query like any other dataset. Open-source options like OpenMetadata and DataHub expose the same signals through their APIs.</p><p>Pipe the raw telemetry into a lakehouse or warehouse you already run (Microsoft Fabric, Databricks, Snowflake, BigQuery, take your pick). Model it semantically. Surface it through Power BI, Tableau, Looker, or whatever your business uses. Sensitivity-label and certify the report itself, because nothing kills credibility faster than a governance scorecard that breaks its own rules.</p><p>Treat your catalog scorecard like a data product. Eat your own cooking.</p><h3>A Phased Rollout So You Do Not Drown in Metrics</h3><p>Twelve KPIs across four pillars is not a week-one project. Phase it.</p><p><strong>Quarter 1.</strong> Pick four, one per pillar. Zero-Result Search Rate (Findability). Certified Asset Coverage on CDEs (Trust). Self-Service Access Request Rate Through the Catalog (Engagement). Active Steward Coverage (Curation Health / Metadata Quality). These four force you to identify your CDEs, your active steward population, your access workflow, and your search baseline.</p><p><strong>Quarter 2.</strong> Add Top Zero-Result Terms (the working list, with a quarterly closure target) and Metadata Completeness on CDEs. Both turn the quarter-one metrics into actionable backlogs. Establish the root cause analysis discipline now, while the metric set is small.</p><p><strong>Quarter 3.</strong> Layer in Time to First Useful Asset, Repeat User Rate, and Lineage Completeness on Reported Data. By now you have the curation muscle to act on what these reveal.</p><p><strong>Quarter 4.</strong> Add the rest. Tune thresholds per domain. A regulatory reporting domain deserves a higher CDE certification bar than a marketing analytics sandbox.</p><h3>Practical Takeaways</h3><ul><li>Stop reporting assets cataloged, glossary terms defined, and stewards assigned as success metrics. They measure inventory, not value.</li><li>Organize your catalog scorecard around four outcomes: Findability, Trust, Engagement, Curation Health (or Metadata Quality, whichever term lands better in your organization).</li><li>A red KPI is the start of an investigation, not the end. Run a five-whys root cause analysis on every metric that breaks, especially in Trust. Most catalog problems are organizational symptoms wearing a technical disguise.</li><li>Anchor trust and metadata quality metrics to your Critical Data Elements. CDE coverage matters more than full-catalog coverage.</li><li>The telemetry exists in whatever catalog you run, whether Microsoft Purview, Collibra, Alation, Atlan, Unity Catalog, Snowflake Horizon, or open-source options. Stop asking for new tools. Start querying the audit logs.</li><li>Treat the catalog like an e-commerce site. Track searches, clicks, conversions, and bounce rates. Amazon does not guess what shoppers want. Neither should you.</li><li>Phase the rollout. Four KPIs the business reads beats twelve KPIs in a spreadsheet nobody opens.</li></ul><h3>What a Healthy Catalog Sounds Like a Year From Now</h3><p>A year from now, when leadership asks whether the catalog is working, you can answer with volume (“we cataloged 50,000 assets”) or you can answer with outcomes.</p><p>“Zero-result searches dropped from 34 percent to 11 percent. Time to first useful asset for new analysts is under 90 seconds. Sixty-four percent of access to governed data now flows through self-service in the catalog, up from 12 percent. Repeat user rate doubled. And when our certified Customer Master table failed last quarter, we traced it to a missing data contract with the CRM team and shipped a producer notification framework two weeks later.”</p><p>The first answer is a status update. The second is a product working as intended, with a team that knows how to fix it when it breaks.</p><p>A catalog is not a filing cabinet. It is a search engine for your business. Measure it like one. And when it stops working, do not just log the symptom. Find the cause.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=45d98b6e6ee2" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/your-data-catalog-has-1-million-assets-and-nobody-can-find-anything-here-are-the-kpis-that-fix-it-45d98b6e6ee2">Your Data Catalog Has 1 million Assets and Nobody Can Find Anything. Here Are the KPIs That Fix It.</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Semantic Models Don’t Bring Meaning. People Do.]]></title>
            <link>https://medium.com/the-semantic-layer/semantic-models-dont-bring-meaning-people-do-4750c29a8fe9?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/4750c29a8fe9</guid>
            <category><![CDATA[microsoft-purview]]></category>
            <category><![CDATA[power-bi]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[semantic-model]]></category>
            <category><![CDATA[microsoft-fabric]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Mon, 22 Jun 2026 10:36:13 GMT</pubDate>
            <atom:updated>2026-06-22T10:36:14.831Z</atom:updated>
            <content:encoded><![CDATA[<p>The word “semantic” is doing too much work, and it hides where the real work lives.</p><p>If you have ever gone looking for a clear explanation of how a semantic model brings semantics to your data, you have probably noticed the documentation is thin. There is a reason for that. “Semantic model” does two or three different jobs under one word, and most of the writing about it never says which one it means.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zvjUVPwrDeKS7d5ZmUgzog.png" /></figure><h3>What a semantic model actually adds</h3><p>Look at what a Power BI / Fabric semantic model puts on top of raw tables. It maps a vocabulary, so FCT_SLS.AMT_NET becomes Net Sales. It encodes relationships, so the engine knows an order belongs to a customer and can traverse that without you writing the join. It encodes hierarchies, so Date to Year to Quarter to Month carries roll-up meaning. And it defines measures.</p><p>The measures are where the value sits. Revenue gets defined once, in DAX, with its inclusions and exclusions pinned down, so everyone who asks for revenue gets the same number. That is the real job: fix the meaning of a metric so it is not re-derived inconsistently across fifty reports. A single source of truth for definitions is worth a lot.</p><h3>But it is a thin kind of meaning</h3><p>Notice what kind of meaning that is. It is business meaning for consumption and aggregation. It is not formal semantics in the knowledge-representation sense. The model has no inference, no reasoning, no portable definition of what a customer actually is. It is a dimensional model wearing business clothing.</p><p>The word itself is inherited. It goes back to the Business Objects universe in the nineties and the old SQL Server BI Semantic Model, where “semantic” meant business-friendly abstraction over a physical schema. It never meant logically specified.</p><p>In practice three different things wear the badge:</p><ul><li>The metrics layer (Power BI semantic model, dbt, Cube): meaning as consistent calculations and relationships for querying.</li><li>The catalog layer (Microsoft Purview Unified Catalog, and its peers): meaning as defined terms, classifications, lineage, and stewardship. It describes and classifies. It does not compute.</li><li>The ontology and knowledge-graph layer: meaning as formal concept definitions a machine can reason over.</li></ul><p>The documentation is vague partly because vendors conflate these. You open a page about semantics and cannot tell which of the three it is claiming.</p><h3>All of this is technical. Meaning is not.</h3><p>Every artifact above is built by data people in a modeling tool. But the meaning does not live in the data or the tool. It lives with the business. Active customer means something because people agreed it does. The model only encodes the agreement.</p><p>So the moment you want business users involved, the instinct is to hand them the artifact. Sit them in front of the glossary, teach them a simplified modeling tool, run a workshop, define four hundred terms. That produces the glossary graveyard. Definitions nobody bound to data, nobody maintains, nobody queries. Meaning that is not connected to something and not consumed decays within a quarter.</p><p>The reframe that works is to split the two things the word was hiding. Meaning is business-owned. Encoding is technical. Business users do not write the DAX or the mappings. They own the definition and they validate the encoding. The artifact’s only job is to stay traceable back to a definition a named human agreed to.</p><h3>What that looks like in practice</h3><p>Start from the numbers people argue about, not from the data model. Do not ask what customer means in the abstract. Ask which reported figures two departments compute differently. Revenue, active customer, churn, qualified lead. Wherever the numbers disagree, there is an undefined term, and that is exactly where business semantics earns its keep. It also gives you a business reason to do the work, which you need to get anyone’s time.</p><p>Capture each term in business language first. Definition, owner, synonyms, and the edge cases that are the whole point. Does a trial count as a customer. Does a refunded order count as revenue. In a Purview governance domain that is a glossary term with a named steward, and the steward being a real person with a real job matters more than the tool.</p><p>Then bind the term to the physical assets and to the measure that implements it. Then round-trip it. Show the owner what their term currently computes, in their language, and ask if the number matches what they meant. That last step is the one everyone skips, and it is where you catch the drift between intent and encoding.</p><p>Federate the whole thing by domain instead of centralizing meaning in one modeling team. That maps cleanly onto governance domains and data products, and it is the only version that scales past a handful of metrics.</p><h3>The AI shift cuts both ways</h3><p>There is a new reason to care. When a business user asks an agent how many active customers there were in the Nordics last quarter, the agent has to resolve active customer to the right encoding or it returns confident garbage. The thin semantics that was fine for a human analyst is not fine for a model.</p><p>The same models also collapse the old translation gap. A business user describes a definition in plain language. The model drafts the measure or the term-to-asset mapping. The human confirms. Natural language is becoming a real authoring surface for semantics, which is the first thing that has ever lowered the barrier for business users instead of just renaming the tool.</p><h3>One caution</h3><p>Resist over-formalizing. Most business semantics needs three things: agreed definitions, clear ownership, and binding to data. Reserve the heavier formal modeling for the few places where concepts genuinely have to be reasoned over, not for the eighty percent that just needs one measure everyone trusts.</p><p>The model gives you structure. Meaning is a human agreement. Treat it as one, and the rest of the stack has something solid to point at.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4750c29a8fe9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/semantic-models-dont-bring-meaning-people-do-4750c29a8fe9">Semantic Models Don’t Bring Meaning. People Do.</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Product vs AI Product: The Hard Truth About Why They Cannot Be Built the Same Way]]></title>
            <link>https://medium.com/the-semantic-layer/data-product-vs-ai-product-the-hard-truth-about-why-they-cannot-be-built-the-same-way-70840893b371?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/70840893b371</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[ai-products]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[data-mesh]]></category>
            <category><![CDATA[product-data]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Tue, 02 Jun 2026 07:39:20 GMT</pubDate>
            <atom:updated>2026-06-02T07:39:20.298Z</atom:updated>
            <content:encoded><![CDATA[<p>In 2025, enterprises poured <strong>$684 billion</strong> into AI, and <strong>$547 billion of that, more than 80%, failed to deliver business value</strong> (<a href="https://www.pertamapartners.com/insights/ai-project-failure-statistics-2026">RAND, via Pertama Partners, 2026</a>). MIT’s NANDA team studied 300 of those projects and found that <strong>95% of corporate generative AI pilots produced zero measurable P&amp;L impact</strong> (<a href="https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf">MIT NANDA, State of AI in Business 2025</a>). MIT’s diagnosis is uncomfortable: it is not the models, not the talent, and not the regulation. It is a “learning gap,” brittle workflows, and shallow integration between the AI tools and the organizations buying them. Read between those lines and you will see something most data teams resist: a data product and an AI product are not the same thing, and the playbook that works for one will quietly sink the other.</p><h3>TL;DR</h3><p>A data product is a contract. An AI product is a probability. The first guarantees a shape, a freshness, and a meaning. The second guarantees nothing except a distribution of outputs. The MIT and RAND numbers above show what happens when enterprises ignore that distinction: brittle integration, no feedback loops, no clear ownership, and pilots that never scale. This article walks through the foundational differences across three dimensions, architecture, governance, and business value, and shows what changes when you build, monitor, and sell AI products on their own terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ljjTJppKD2XnTWBBxk-OA.png" /></figure><h3>Why This Distinction Matters Right Now</h3><p>Five years ago, a data engineer’s job was simple to describe. Move data from A to B. Make sure the numbers add up. Land it in a warehouse. Build it into a dashboard. The unit of work was a <strong>table</strong>, and the table either matched the spec or it did not.</p><p>Then came data products. The data mesh paradigm, introduced by Zhamak Dehghani in 2018, asked teams to treat data as a product with owners, contracts, SLAs, and discoverability (<a href="https://www.thoughtworks.com/about-us/events/webinars/core-principles-of-data-mesh/data-as-a-product">Thoughtworks, 2025</a>). That shift was hard but conceptually clean. A data product is still deterministic. Same input, same output. Pass or fail.</p><p>AI products broke that model. The unit of work is no longer a table. It is a <strong>decision</strong> made by a probabilistic system that can drift, hallucinate, and surprise you in production. MIT’s NANDA initiative studied 300 public AI deployments, 52 enterprise interviews, and 153 senior leader surveys for its “GenAI Divide” report and found that <strong>95% of corporate generative AI pilots produce zero measurable P&amp;L impact</strong>, despite $30 to $40 billion of enterprise spend (<a href="https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf">MIT NANDA, State of AI in Business 2025</a>, as covered by <a href="https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/">Fortune, 2025</a>). Gartner reached a similar conclusion in April 2026: <strong>only 28% of AI infrastructure projects deliver their promised return</strong>, and <strong>57% of I&amp;O leaders have already seen at least one AI project fail in their own ranks</strong> (<a href="https://www.gartner.com/en/newsroom/press-releases/2026-04-07-gartner-says-artificial-intelligence-projects-in-infrastructure-and-operations-stall-ahead-of-meaningful-roi-returns">Gartner, 2026</a>).</p><p>If you cannot tell the two species apart, you will keep using the wrong tools on the wrong problem. Let’s start with the definition that fixes the rest.</p><h3>A Data Product Is a Contract. An AI Product Is a Probability.</h3><p>A data product is a reusable, governed data asset with a clear owner, a defined consumer, documented quality rules, and a contract. The contract spells out the schema, the freshness, the semantics, and the SLA (<a href="https://www.getdbt.com/blog/the-four-principles-of-data-mesh">dbt Labs, 2024</a>). If you query the customer_revenue_daily data product on Tuesday, you get the same answer you would have gotten on Monday for Monday&#39;s data. Determinism is the whole point.</p><p>An AI product is the opposite. It is a probabilistic function trained on a frozen snapshot of the world. The same prompt can produce different outputs. The same model can degrade silently as the world moves on. Industry estimates suggest <strong>91% of ML models fail in production due to model drift</strong> (<a href="https://enhancedmlops.com/advanced-ml-model-monitoring-drift-detection-explainability-and-automated-retraining/">EnhancedMLOps, 2025</a>). That number is not an indictment of ML engineers. It is the cost of treating a probability distribution like a deterministic table.</p><p>Think of it like this. A data product is a vending machine. You press B4, you get a Snickers, every time. An AI product is a bartender. You ask for “something refreshing,” and the answer depends on the bartender’s mood, what is in stock, and whether the last customer drank the last lime. Both are useful. Neither is interchangeable with the other.</p><p>That single distinction drives everything else. Hold it in your head as you read the next three sections.</p><h3>The Architecture Split: Pipelines vs Feedback Loops</h3><p>The clearest difference between a data product and an AI product shows up the moment you draw the architecture.</p><p>A data product is built around a <strong>pipeline</strong>. Source systems feed ingestion. Ingestion lands raw data in Bronze. Transformations clean it into Silver. Aggregations expose it as Gold. A governance layer sits across all three with quality rules, lineage, and access policies. The data flows in one direction. Every step is deterministic and reproducible.</p><p>An AI product is built around a <strong>feedback loop</strong>. You train a model on historical data. You serve predictions. You capture outcomes. You measure drift. You retrain. The loop never closes. The architecture needs components a data product never had:</p><ul><li>A <strong>feature store</strong> to keep training and serving features consistent.</li><li>A <strong>model registry</strong> to version models the way you version code.</li><li>A <strong>vector store</strong> if you are doing retrieval-augmented generation.</li><li>An <strong>evaluation harness</strong> to score outputs against gold sets and human feedback.</li><li>A <strong>drift monitor</strong> that compares production input distributions to training distributions.</li><li>A <strong>shadow deployment</strong> capability to test new models without breaking users.</li></ul><p>A data engineer who has spent a decade building pipelines often underestimates this. They look at an AI product and see “another pipeline with a fancy step in the middle.” That is the trap. Without the feedback loop, you do not have an AI product. You have a model that worked once and will silently rot. A 2024 survey found that <strong>67% of enterprises reported critical model issues going unnoticed for over a month</strong> because they lacked proper monitoring (<a href="https://enhancedmlops.com/advanced-ml-model-monitoring-drift-detection-explainability-and-automated-retraining/">EnhancedMLOps, 2025</a>).</p><p>The architectural takeaway is simple. Build a data product like a bridge. Build an AI product like a living organism.</p><h3>The Governance Split: Pass-Fail vs Continuous Risk</h3><p>Now to governance, where the wheels really come off for most teams.</p><p>A data product is governed with <strong>rules that are either satisfied or violated</strong>. Did the freshness SLA hold? Did every row pass the not-null check? Did lineage capture every upstream dependency? Modern catalogs now offer observability views and SLO tracking for exactly this purpose, where a Silver metric drifting raises an incident routed to the owning domain. Binary. Auditable. Clean.</p><p>An AI product cannot be governed with pass-fail rules because <strong>the failure modes are continuous, contextual, and sometimes invisible</strong>. A hallucination is not a schema violation. A biased recommendation is not a missing row. A confidently wrong answer is not a null value. <strong>47% of enterprise AI users have made major decisions based on hallucinated outputs</strong>, and knowledge workers now spend an average of <strong>4.3 hours per week verifying AI outputs</strong> before trusting them (<a href="https://www.getmaxim.ai/articles/the-state-of-ai-hallucinations-in-2025-challenges-solutions-and-the-maxim-ai-advantage/">Maxim AI, 2025</a>).</p><p>That changes governance from a compliance checklist into an operational discipline. For AI products, you need:</p><ol><li><strong>Continuous evaluation</strong> against curated test sets that grow with every incident.</li><li><strong>Human-in-the-loop review</strong> for high-stakes outputs, with the review captured as training data.</li><li><strong>Output observability</strong> that captures the prompt, the context, the retrieved chunks, and the response.</li><li><strong>Risk tiers</strong> so a chatbot that recommends a coffee blend is governed differently than one that approves a loan.</li><li><strong>Model cards and audit trails</strong> that document training data, evaluation results, and known limitations.</li></ol><p>Governance platforms are racing to bolt on AI-ready features that classify, label, and protect the data feeding AI workloads, but the cultural shift matters more than the tool. Your governance committee needs to stop asking “did it pass the test?” and start asking “how confident are we, and what is our exposure if it is wrong?”</p><p>If your governance model still produces a green check, you are governing a data product. If it produces a risk score and a residual confidence interval, you are governing an AI product.</p><h3>The Ownership Split: One Domain vs Many Domains</h3><p>A data product has a clear owner. The Finance domain owns revenue_actuals. The Marketing domain owns campaign_attribution. Federated computational governance, the fourth principle of data mesh, exists precisely to keep ownership local while enforcing global standards (<a href="https://www.thoughtworks.com/about-us/events/webinars/core-principles-of-data-mesh/data-as-a-product">Thoughtworks, 2025</a>).</p><p>An AI product crosses domains by default. A churn prediction model touches data from product, billing, and support. A copilot that drafts customer emails depends on CRM data, product documentation, brand voice guidelines, and policy rules. <strong>Who owns the model?</strong> The data engineering team that built the pipeline? The ML team that trained the model? The product team that ships the user experience? The legal team that owns the risk?</p><p>This is the silent killer of AI projects. RAND found that <strong>33.8% of enterprise AI projects are abandoned before they ever reach production</strong> (<a href="https://www.pertamapartners.com/insights/ai-project-failure-statistics-2026">Pertama Partners, 2026</a>). A large slice of those die not from technical failure but from ownership gridlock. No one wants to sign their name next to the loan approval bot.</p><p>The fix is uncomfortable but necessary. Treat the AI product as its own domain, with an owner who has accountability for the model’s behavior end-to-end, including data sourcing, prompt design, evaluation, monitoring, and incident response. That owner needs the same authority a data product owner has, plus a risk budget. Without that, you are governing by committee, and committees do not ship.</p><h3>The Business Value Split: Reuse vs Recovery</h3><p>A data product creates value through <strong>reuse</strong>. You build customer_360 once, and 14 teams query it for the next three years. The marginal cost of each new consumer is low. The ROI compounds.</p><p>An AI product creates value through <strong>decisions that are right more often than the baseline</strong>. The ROI depends on three numbers most teams never compute upfront: the cost of a correct decision, the cost of an incorrect one, and the volume of decisions per unit time. A fraud model that catches 92% of fraud is worthless if the 8% it misses is the highest-dollar fraud, and the 5% false positive rate alienates your best customers.</p><p>This is why successful AI deployments achieve massive returns while the average flounders. The 20% that succeed often hit a <strong>383% return on investment</strong>, but only because they ruthlessly tied model performance to a financial outcome before they started building (<a href="https://www.softwareseni.com/why-80-percent-of-artificial-intelligence-projects-fail-while-successful-implementations-achieve-383-percent-return-on-investment/">SoftwareSeni, 2025</a>).</p><p>If you cannot draw a line from a model decision to a dollar saved, earned, or risked, do not build the AI product. Build the data product first, instrument the manual decision, and prove the value lever exists. Then automate. MIT NANDA’s research is blunt on the root cause: the 95% of pilots that go nowhere fail not because the models are bad, but because organizations skip the integration and learning work that lets a probabilistic system improve over time (<a href="https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf">MIT NANDA, 2025</a>).</p><h3>Side by Side: The Differences That Matter</h3><p>Run the two species against each other on nine dimensions and the gap becomes impossible to miss.</p><p><strong>Unit of value.</strong> A data product delivers a table or a dataset. An AI product delivers a decision or a generated output.</p><p><strong>Behavior.</strong> A data product is deterministic. An AI product is probabilistic.</p><p><strong>Failure mode.</strong> A data product fails through schema breaks and freshness misses. An AI product fails through drift, hallucination, and bias.</p><p><strong>Quality check.</strong> A data product passes or fails a fixed set of rules. An AI product needs continuous evaluation and risk scoring.</p><p><strong>Architecture.</strong> A data product is a one-way pipeline. An AI product is a closed feedback loop.</p><p><strong>Ownership.</strong> A data product lives inside a single domain. An AI product crosses domains and needs an accountable model owner.</p><p><strong>Monitoring.</strong> A data product has SLAs on freshness and completeness. An AI product has SLOs on accuracy, plus drift detection and output observability.</p><p><strong>Governance tooling.</strong> A data product relies on catalog quality rules and lineage. An AI product relies on AI governance, model cards, and an evaluation harness.</p><p><strong>ROI pattern.</strong> A data product compounds through reuse. An AI product compounds through decision accuracy at scale.</p><h3>What You Just Covered</h3><p>Five ideas anchor everything above.</p><p>A data product is a <strong>contract</strong>, an AI product is a <strong>probability</strong>. The first guarantees a shape and a freshness. The second guarantees only a distribution of outputs.</p><p>The architecture is different in kind, not degree. A data product flows one way through a <strong>pipeline</strong>. An AI product lives inside a <strong>feedback loop</strong> of training, serving, observing, and retraining. Without that loop, you have a model that worked once and will silently rot, which is how <strong>91% of ML models fail in production</strong> (<a href="https://enhancedmlops.com/advanced-ml-model-monitoring-drift-detection-explainability-and-automated-retraining/">EnhancedMLOps, 2025</a>).</p><p>Governance changes from <strong>pass-fail rules to continuous risk</strong>. A schema check cannot catch a hallucination. A null check cannot catch a biased recommendation. You need evaluation harnesses, human-in-the-loop review, output observability, and risk tiers, not just SLAs.</p><p>Ownership has to cross domains. A data product can live inside Finance or Marketing. An AI product almost always touches three or four domains at once, which is why <strong>33.8% of AI projects are abandoned before production</strong> (<a href="https://www.pertamapartners.com/insights/ai-project-failure-statistics-2026">RAND/Pertama, 2026</a>). Without a single accountable model owner with risk budget, ownership gridlock kills the project.</p><p>Value is measured differently. Data products compound through <strong>reuse</strong>. AI products compound through <strong>decision accuracy at scale</strong>. The 20% that succeed often hit <strong>383% ROI</strong> because they tied model performance to a financial outcome from day one (<a href="https://www.softwareseni.com/why-80-percent-of-artificial-intelligence-projects-fail-while-successful-implementations-achieve-383-percent-return-on-investment/">SoftwareSeni, 2025</a>). The 80% that fail did not.</p><h3>Closing</h3><p>The $547 billion of AI investment that failed to deliver value in 2025, and the 95% of pilots that produced no P&amp;L impact, did not fail for a single reason. MIT names the learning gap. RAND names abandoned scope and runaway cost. Gartner names data readiness. The thread tying those diagnoses together is this: the operating model that ships a data product does not ship an AI product. A data product is a bridge. An AI product is an organism. Build them differently. Govern them differently. Sell them differently.</p><p>The teams that win in 2026 will be the ones who can hold both shapes in their head at once and choose the right one for each problem. That is the foundational difference. Everything else is implementation.</p><h3>Sources</h3><ul><li><a href="https://mlq.ai/media/quarterly_decks/v0.1_State_of_AI_in_Business_2025_Report.pdf">MIT NANDA: The GenAI Divide, State of AI in Business 2025 (PDF)</a></li><li><a href="https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/">Fortune: MIT Report, 95% of Generative AI Pilots at Companies Are Failing, August 2025</a></li><li><a href="https://www.pertamapartners.com/insights/ai-project-failure-statistics-2026">RAND/Pertama Partners: AI Project Failure Statistics 2026</a></li><li><a href="https://www.gartner.com/en/newsroom/press-releases/2026-04-07-gartner-says-artificial-intelligence-projects-in-infrastructure-and-operations-stall-ahead-of-meaningful-roi-returns">Gartner: AI Projects in I&amp;O Stall Ahead of Meaningful ROI, April 2026</a></li><li><a href="https://www.gartner.com/en/newsroom/press-releases/2025-02-26-lack-of-ai-ready-data-puts-ai-projects-at-risk">Gartner: Lack of AI-Ready Data Puts AI Projects at Risk, February 2025</a></li><li><a href="https://www.softwareseni.com/why-80-percent-of-artificial-intelligence-projects-fail-while-successful-implementations-achieve-383-percent-return-on-investment/">SoftwareSeni: Why 80 Percent of AI Projects Fail While Successful Implementations Achieve 383 Percent ROI</a></li><li><a href="https://www.getmaxim.ai/articles/the-state-of-ai-hallucinations-in-2025-challenges-solutions-and-the-maxim-ai-advantage/">Maxim AI: The State of AI Hallucinations in 2025</a></li><li><a href="https://enhancedmlops.com/advanced-ml-model-monitoring-drift-detection-explainability-and-automated-retraining/">EnhancedMLOps: Advanced ML Model Monitoring, Drift Detection, and Automated Retraining</a></li><li><a href="https://www.thoughtworks.com/about-us/events/webinars/core-principles-of-data-mesh/data-as-a-product">Thoughtworks: Core Principles of Data Mesh and Data as a Product</a></li><li><a href="https://www.getdbt.com/blog/the-four-principles-of-data-mesh">dbt Labs: The 4 Principles of Data Mesh</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=70840893b371" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/data-product-vs-ai-product-the-hard-truth-about-why-they-cannot-be-built-the-same-way-70840893b371">Data Product vs AI Product: The Hard Truth About Why They Cannot Be Built the Same Way</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Context as a Product]]></title>
            <link>https://medium.com/the-semantic-layer/context-as-a-product-a2c13aaeea9b?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/a2c13aaeea9b</guid>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[context-engineering]]></category>
            <category><![CDATA[metadata-management]]></category>
            <category><![CDATA[semantic-layer]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Fri, 22 May 2026 13:22:34 GMT</pubDate>
            <atom:updated>2026-05-22T13:22:34.207Z</atom:updated>
            <content:encoded><![CDATA[<p>For a long time, metadata has been an afterthought. You build a data warehouse, and somewhere in the catalog there are column descriptions, lineage, and an owner who left the company two years ago. The metadata is there to help with the data, the way footnotes help with a book. Nothing more.</p><p>That is changing.</p><p>Data is no longer being used only by people. An AI model now reads a column description in a way no analyst would. People skim. People ignore. People ask a colleague when something is unclear. A model does none of that. It takes what it finds and treats it as the truth. If the description is wrong, the answer is wrong. If the classification is missing, the wrong data leaks out. If two tables are related and the catalog does not say so, the model makes something up.</p><p>That is why metadata stops being an afterthought and starts being a product. Something with consumers. Something with versions. Something you ship.</p><p>Before getting to the shipping part, it is worth being clear about what we are actually talking about.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sFgyyzG119FHcdp3KUMZAg.png" /></figure><h3>Context, by itself</h3><p>Context is what you need to know in order to use data correctly. Not the data itself, but everything around it. What the values mean. Where they came from. What they can be used for. What is missing that someone might assume is there. How the data actually behaves, instead of how the documentation says it behaves.</p><p>People carry context without thinking about it. An analyst opens a new table and pauses before writing a query. That pause is context being gathered. An engineer tells a new joiner not to trust a column from before March 2024. That is context being passed on. A steward writes a careful note on a sensitive field. That is a small piece of context being saved.</p><p>Most of this work is invisible. It lives in conversations, in chat threads, in the comments of old notebooks, and in the heads of people who have been around long enough to know.</p><p>The data sits in a database. The context usually does not. That gap is the problem.</p><p>For a long time, the gap did not matter much. The people using the data filled it in by asking, reading, or learning the hard way. The data could afford to be poor on context because the people using it had the context in their heads.</p><p>That is changing. The new users of data are not just people. They are models, agents, and automated pipelines. None of them carry context. Whatever they need has to be written down somewhere they can read it. The gap between what the data says and what the data means used to be closed by a human. Now it has to be closed by the system.</p><h3>Context as a product</h3><p>A context product is context that has been written down and packaged so it can be shipped. Instead of leaving it in heads, chat threads, and old notebooks, you turn it into something with a clear address, a version, and documentation. Any consumer (a person, a BI tool, a model, an agent) can read it and use the data correctly.</p><p>A context product usually includes:</p><ul><li>What the entities are and how they connect</li><li>What each field means in plain business terms</li><li>What is sensitive, and which laws apply to it</li><li>Where the data came from and how fresh it is</li><li>What the data can and cannot be used for</li><li>The shape of the data and how to get to it</li></ul><p>A data product without a context product is an API or a table with a vibe. A context product without a data product is a brochure. The two belong together.</p><h3>Context is not just metadata</h3><p>It is worth being careful with the word. Metadata, the way most catalogs use it, is the structural and descriptive information about data: types, lengths, descriptions, owners, refresh times, lineage. That is part of context, but it is the easy part. Adding better descriptions to a catalog and calling the job done is the most common mistake in this space.</p><p>Context is everything around the data. It includes the metadata, and it also includes four things that catalogs rarely cover.</p><p>What the data means. A column called AE_SEV_GRD is not &quot;a single digit between 1 and 5.&quot; It means how serious an adverse event was, scored on the standard scale clinical trials use, where a 3 is &quot;severe but not immediately life-threatening&quot; and the gap between a 3 and a 4 is a call a clinician had to make under pressure. The real meaning lives in a medical vocabulary, not in a database catalog.</p><p>Why the data exists. Why it was collected and under what consent. The same column, in the same shape, can be fine to use in one trial and off-limits in another. The only thing that decides it is the consent form the patient signed.</p><p>How the data behaves in practice. Which joins make sense and which produce nonsense. Which numbers are meaningful and which are misleading. The fact that nobody trusts values before March 2024 because the upstream system was misconfigured. The fact that the documentation says one thing and the people who use the data every day know to do something else.</p><p>What the data does not contain. Which patients are missing because the protocol excluded them. Which adverse events were never coded because the site coordinator gave up. Which numbers look reasonable but rest on assumptions that no longer hold.</p><p>Metadata is what a database can tell you about itself. Context is what an experienced colleague would tell you before you started writing the query. That second part has always lived in people’s heads, in chat threads, and in the comments of old notebooks. Turning it into something a machine can read is not really a cataloging job. It is a knowledge capture job that happens to look like engineering.</p><p>A context product is closer to a domain model than to a data dictionary. The catalog is where it lives. The work is everything else.</p><h3>The clinical trial example</h3><p>Pharma is a good place to see this clearly for two reasons. The stakes are clear, because patient safety and regulators force them to be. And the work is predictable. Every trial follows a written protocol that says what data will be collected, how, when, and from whom. The industry shares vocabularies. The data shapes are standard. A Phase III trial run out of Stockholm looks a lot like one run out of Copenhagen. That shared structure is what makes context products easier to build here than almost anywhere else.</p><p>A clinical trial produces patient data, adverse event records, lab values, protocol deviations, consent records, and many derived tables. The sponsor uses some of it inside the company, shares some with a CRO, shares some with regulators, and might share anonymized parts with a public registry.</p><p>Today, most of that sharing happens through documents, scripts, and emails. The receiving side reads the protocol PDF, looks at the data dictionary, asks the sponsor what AE_SEV_GRD actually means, finds out it is the standard severity scale used in clinical trials, writes a transformation, and gets to work. Multiply that by every consumer and every trial, and you have a tax on the whole industry: everyone paying to figure out the same things, on their own.</p><p>A context product changes this. The sponsor publishes the trial’s context as a package: the entity model, the vocabularies, the consent and use rules, the lineage back to source systems, the sensitivity of each field, the rules that come from the protocol. The CRO points its analytics platform at the context, and the platform now knows, without asking, that AE_SEV_GRD follows the standard severity scale, that this group of patients cannot be used for new research, and that the lab values are in SI units in this study and conventional units in the last one.</p><p>The same context can feed an AI agent that a regulator might use to ask, “show me all serious adverse events in patients over 65 across the program.” The agent does not need a human to translate the question into SQL across three different schemas. It reads the context, reasons over it, and runs the query against the right data under the right rules.</p><p>Clinical trials are a clean example because the rules are not negotiable, but the same pattern fits in many other places. Insurance claims. Genomic data. Financial market data. ESG reporting. Anywhere a group of consumers all pay the same cost to figure out the same data, on their own.</p><h3>Why this is happening now</h3><p>Two things are forcing it.</p><p>First, AI agents have no human judgment to fall back on. A person who reads a confusing description goes and asks someone. An agent guesses. The cost of bad metadata used to be a slow analyst. The cost now is a confident, wrong answer delivered to a regulator. That changes what “good enough” looks like.</p><p>Second, the agents are increasingly not yours. The same model running inside your company is also running inside your partners, your vendors, and your customers. If you want the data you share to be used correctly across all of them, you cannot expect the other side to piece together context from a PDF. You have to send the context with the data.</p><p>This is the place the data mesh argument was always heading. Data-as-a-product was always context-as-a-product underneath. The original writing just did not have a reason to put context in the title. AI gives it the reason.</p><h3>What is hard about it</h3><p>A few things are not solved.</p><p>There is no shared standard for what a context product looks like. The closest comparisons are OpenAPI for services and Schema.org for the web, and neither of them quite fits. ODPS, the Open Data Product Specification, is trying to cover part of it. Metric layer projects like dbt’s semantic layer, Cube, and AtScale cover another part. Knowledge graph formats (RDF, OWL, property graphs) cover yet another. None of them cover the full thing, and the camps barely talk to each other.</p><p>Versioning is unsolved. When the meaning of a field changes, and it will, every consumer downstream has to know. Ideally the old meaning stays queryable for historical analysis. Context products need something like semver, plus a way to mark deprecation, plus a way to say “this field meant X until Q2 2025 and Y after.”</p><p>Governance has to travel with the context. A context product that says what data exists without saying who can use it, and for what, is worse than useless. Consent, jurisdiction, purpose, and retention all need to be built in from the start, not added on at the end.</p><p>And most catalogs today were not built for this. They were built for humans browsing a web UI, not for agents pulling structured context through an API. The major catalog vendors are racing to add programmatic access, but the original assumptions go deep, and the cleanup is real work.</p><h3>What the ecosystem looks like if this works</h3><p>If context products become real, the thing you get is composability. An AI agent working with data from three organizations can reason about how the entities connect, because each context product spells it out. A new analytics tool can read a context product and immediately offer correct queries, because the meaning is explicit. A regulator can audit not just the data, but the meaning the data was used under.</p><p>The interesting part is that this stops being a single-vendor story. The value of a context product comes from being readable by anyone: your own tools, partner tools, agents you have never heard of, agents that do not yet exist. That is uncomfortable for vendors who would rather keep the context inside their platform. It is also exactly why the discipline matters. The semantic layer becomes a product surface, not a back office.</p><p>Most organizations are nowhere near this. Most catalogs are still aimed inward. Most metadata is still an afterthought. But the direction is set. The question is whether your context is going to be a product you maintain on purpose, or a side effect that someone else’s agent has to guess at.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2c13aaeea9b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/context-as-a-product-a2c13aaeea9b">Context as a Product</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Questions That Actually Matter When You Build a Data Product]]></title>
            <link>https://medium.com/the-semantic-layer/the-questions-that-actually-matter-when-you-build-a-data-product-7c4148098cff?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/7c4148098cff</guid>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[data-mesh]]></category>
            <category><![CDATA[product-data]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[data-platforms]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Fri, 15 May 2026 07:16:49 GMT</pubDate>
            <atom:updated>2026-05-15T07:16:49.057Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ohrjJRa6u_YaoVMKd70nAA.png" /></figure><p>Most data products fail quietly. Not because the pipelines broke or the model drifted, but because nobody asked the right questions before writing the first line of SQL. The API get built, the metrics get defined, the models get shipped, and then six months later someone notices that no decisions have changed and no one is using the thing. The team blames adoption. The truth is usually that the product was answering a question nobody actually had.</p><p>There is a useful instinct, borrowed from product design, that says any new thing should clear three bars at once: someone has to want it, the organization has to be willing to fund it, and the team has to be able to build it without setting itself on fire. Most data teams obsess over the third bar because architecture diagrams are fun and stakeholder conversations are not. They skip the first two and discover, painfully, that “we built the thing we said we would” is not the same as “the thing we built mattered.”</p><p>There is also a quieter movement in the data product world to treat a data product the way software teams treat APIs. Write down, up front, who owns it, who is allowed to use it, what guarantees it offers, what it costs to run, how it will evolve, and how it can be retired. This is good discipline. It forces teams to articulate the things they usually leave implicit and then argue about later. But a specification is a contract, not a product. A perfectly described data product can still be useless. Specifying it is necessary; it is not sufficient.</p><p>Underneath all of this, the technical scaffolding still matters. Shared definitions matter: deciding what a “customer” even is before three teams ship three different versions of the concept and spend the next year reconciling them in spreadsheets. This is less about elegance and more about saving your future self from a thousand small arguments over whether a churned trial user counts. Data contracts do something similar at the boundary between producers and consumers. They turn implicit handshakes into something you can actually enforce when an engineer renames a field on a Friday afternoon without telling anyone. These are guardrails. They keep the building from falling down. They are not where products die.</p><p>What actually decides whether a data product lives or dies is product management, and it is the part data teams are most likely to under-invest in. Good product management, in this context, is not a role. It is a discipline. It is the practice of holding a clear, evidence-based answer to four questions at all times: who is this for, what problem are we solving for them, why is solving it worth more than the alternatives we could be working on, and how will we know if we are right. A product manager (or anyone playing that role, including the analytics lead or the data engineer who happens to be closest to the user) is the person responsible for not letting the team drift away from those answers under the gravitational pull of interesting technical work.</p><p>In practice, executing this well looks unglamorous. It starts with discovery: real conversations with the people whose decisions you intend to change, before any modeling begins. Not a survey, not a JIRA ticket from a stakeholder, but a conversation where you ask what they did last week, what they got wrong, and what they wish they had known. Then it becomes synthesis: writing down, in plain language, the problem you believe you are solving and the bet you are making about why this data product will solve it. If you cannot fit that on one page, you do not understand it yet. From there it becomes prioritization, which is mostly the art of saying no to good ideas in service of one important one. Then delivery in small, observable increments, so you can tell quickly whether your bet was right. And finally, the hardest part: revisiting the original problem statement after launch and being honest about whether it actually got solved. Most teams skip this step because it is psychologically expensive. The ones that do it are the ones whose data products compound in value rather than slowly decay.</p><p>Good product management is also the thing that protects the human questions below from being optional. It is what makes “who is the user” a precondition for funding rather than a footnote in the readme. It is what turns “what decision changes” from a slogan into a checkpoint nobody is allowed to skip. Without that discipline, the questions that follow are just nice-sounding ideas. With it, they become the operating system of the team.</p><p>Products die on the human side. The questions I keep coming back to, the ones that actually predict whether a data product will land, tend to make people uncomfortable in planning meetings, which is usually a sign they are the right ones to ask.</p><p>The first is <strong>who the actual user is, and whether anyone has watched them work.</strong> Not the persona deck. Not the executive sponsor who requested it. The person who will, on a Monday morning, open a tab and look at this thing. If no one on the team can describe that person’s day, the product is being built for an abstraction, and abstractions do not have feedback loops.</p><p>The second is <strong>what decision changes because of this.</strong> This is the question that exposes most dashboards as decorative. If a stakeholder cannot point to a specific decision (pricing, staffing, prioritization, intervention) that will be made differently because of your work, you are not building a product. You are building a comfort object. Comfort objects have their place, but they should not be funded like products.</p><p>The third question is <strong>who loses if this succeeds.</strong> Every data product reshuffles power. A new model makes one team look better and another worse. A unified customer definition strips a team of the flexibility they used to bend numbers in their favor. A forecasting tool replaces a director’s gut feel with an algorithm. Pretending this is not happening is naive, and the people who lose will quietly sabotage adoption in ways that look, from the outside, like “the data is wrong” or “the tool is confusing.” It is rarely the tool.</p><p>The fourth is <strong>who owns it on a Tuesday at 8pm when it breaks.</strong> Data products are not artifacts, they are commitments. Specifications can demand that you name an owner, but the document does not do the owning; a person does. If you cannot name the human who will respond when the numbers look weird, when the upstream schema changes, when a VP asks “is this right?” at the worst possible moment, then you do not have a product, you have a prototype.</p><p>The fifth, and the one teams skip most often, is <strong>how we will know we were wrong.</strong> Success metrics are everywhere. Kill criteria are rare. Without them, every data product becomes a sunk cost: someone built it, someone defended the roadmap slot, and now nobody wants to be the one who admits it should be retired. Defining, up front, the conditions under which you will deprecate the product is the single most underrated act of organizational hygiene in this entire discipline.</p><p>There is one more question I have come to take seriously, which is <strong>what the product teaches its users.</strong> Good data products do not just deliver answers, they shape how the organization thinks. A well-designed metric becomes a shared vocabulary. A badly-designed one becomes a shared delusion. Once a number lives on a dashboard, people optimize for it, regardless of whether it represents what you meant. So before you ship, ask: if this metric becomes the thing people game, am I okay with how they will use it?</p><p>None of this means the technical and contractual layers do not matter. Without shared definitions you get conceptual chaos. Without contracts you get silent breakage. Without a clear specification of ownership, access, and lifecycle, you get a graveyard of orphaned dashboards. But these are the conditions for a data product to be possible, not the conditions for it to be valuable. Value lives in whether a real person, with a real decision, in a real organization with real politics, actually uses the thing and is better off for it.</p><p>The teams I have watched build durable data products do something deceptively simple. They spend an unreasonable amount of time, before any code is written, sitting next to the people who will use the product. They ask them what they currently do, what they currently believe, what they are afraid of being wrong about, and what they would do differently if they had a better answer. Then they build the smallest possible thing that changes one of those answers, and they watch what happens.</p><p>Specifications, contracts, and shared definitions are just tooling. Talk to the human first. Then write the SQL.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c4148098cff" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/the-questions-that-actually-matter-when-you-build-a-data-product-7c4148098cff">The Questions That Actually Matter When You Build a Data Product</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Governance Runs on Incentives, Not Mandates.]]></title>
            <link>https://medium.com/the-semantic-layer/data-governance-runs-on-incentives-not-mandates-b4ae8c8a91e9?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/b4ae8c8a91e9</guid>
            <category><![CDATA[knowledge-management]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[microsoft]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-management]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Fri, 08 May 2026 05:25:12 GMT</pubDate>
            <atom:updated>2026-05-08T05:25:12.316Z</atom:updated>
            <content:encoded><![CDATA[<p>Your governance program is not failing because your platform is bad. It is failing because the person you asked to document fifty datasets has a backlog of 400 tickets, no time, and no reason to care.</p><p>That is the whole story. Data governance runs on incentives. You have been running it on mandates.</p><h3>TL;DR</h3><p>Data governance has two completely different incentive structures running in parallel, and they do not match. Executives see reduced risk, faster AI, and cleaner audits. The people on the floor, including data engineers, analysts, and business users, see more meetings, more forms, and more accountability with zero extra pay. They tell you the quiet part out loud: “it is not in my job description, so I am not going to do it.” They are right. Until you change the incentive structure on both sides, no platform, no mandate, and no town hall will close the gap.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cI7YR2ZSISZ8DaV8GDqVkA.png" /></figure><h3>The Executive Pitch That Keeps Failing</h3><p>Walk into any boardroom right now and the data governance pitch sounds the same. Reduce regulatory risk. Unlock AI use cases. Improve data quality. Consolidate platforms. Pass audits. Be ready for the next acquisition.</p><p>Every one of those points is true. Every one of those points is also completely irrelevant to the analyst who just got asked to write business descriptions for 200 columns by Friday.</p><p>This is the gap nobody wants to talk about. Executives buy governance because they see the upside. The people who actually have to do governance work see only the downside. Until you fix that asymmetry, you can buy the best data governance platform on the market and your coverage dashboard will still be flat six months later.</p><p>The platform vendors will not tell you this. Their demos show clean lineage, glowing quality scores, and AI-ready datasets. What the demos do not show is the human being who had to sit down and tag every column. That person is the entire program.</p><p>So before you spend another dollar on a governance platform, ask the harder question: who benefits, who pays the cost, and how are you closing that gap?</p><h3>Two Worlds, Two Scoreboards</h3><p>Picture two people in your organization on the same Monday morning.</p><p>The first is the Chief Data Officer. Their bonus is tied to AI initiatives shipping on time, audit findings going down, and a vague bucket called “data maturity.” They walked out of last week’s leadership meeting with a clear story: governance is the foundation for everything we want to do this year. They are bought in. They are excited. They are about to send an email that starts with “Team, as part of our enterprise governance initiative…”</p><p>The second person is a senior data engineer on the marketing analytics team. They have a release shipping Wednesday. The pipeline they own breaks every other Friday because of upstream schema changes nobody warned them about. Their last performance review measured velocity, on-time delivery, and SLA hits. Governance was not on the list.</p><p>Now the email lands. The engineer is being asked to:</p><p>Fill in business glossary terms for the datasets they own. Approve access requests as a data owner. Attend a weekly governance working group. Document data lineage in a tool they have never used. Tag sensitive columns according to a classification scheme that did not exist last quarter.</p><p>What is the rational response? Ignore the email and ship the release. That is not laziness. That is the system working exactly as designed. You measure them on velocity, you get velocity. You ask them to slow down for governance, and they do the math.</p><p>Two scoreboards. Two worlds. Same company.</p><h3>The Question Nobody Answers Honestly</h3><p>Sit in enough data steward training sessions and you will hear the same question, dressed up in different clothes, every single time.</p><p>“What’s in it for me?”</p><p>It comes out as “How will this be measured in my goals?” or “Who is paying for the time I spend on this?” or the most direct version, “Am I getting compensated for taking on this role?”</p><p>Most governance programs answer this question with a slide that lists benefits to the company. Faster decisions. Less rework. Better trust in data. Fewer fire drills.</p><p>Read those benefits again. Every single one of them accrues to the organization, not the person. None of them show up in a performance review. None of them lead to a raise. None of them get someone promoted.</p><p>If you cannot answer “what’s in it for me” with something that lives in their world, your program is running on goodwill. Goodwill burns out fast.</p><p>The honest answers usually look something like this. You will spend less time fielding the same question fifty times because the answer will be in the governance platform. Your name on a well-documented dataset gets seen by leadership and helps your case for promotion. The classification work makes you the only person who actually knows where the regulated data lives, which is leverage. We are reducing your other workload to make room for this, on paper, with your manager.</p><p>If you cannot make any of those statements true, you do not have a governance program. You have a wishlist.</p><h3>“It Is Not in My Job Description”</h3><p>Talk to enough business users about data governance and you will hear this exact sentence, word for word, on repeat.</p><p>“It is not in my job description, so I am not going to do it.”</p><p>The first time you hear it you might bristle. After the tenth time, you realize they are right. Strictly speaking, governance work is not in their job description. Nobody updated the JD when the program launched. Nobody added it to the goal-setting template. Nobody told their manager to weight it during performance reviews. The expectation just appeared one day in an email from the CDO.</p><p>So when a finance analyst gets asked to validate the business glossary definition of “active customer,” what they actually hear is “do me a favor, on top of everything else, with no recognition.” Of course they push back. The system gave them no reason not to.</p><p>This is where most programs make their biggest mistake. They treat the pushback as a culture problem. They send another email about how governance is “everyone’s job.” They run a town hall. They show another slide about the importance of data trust. None of it works, because none of it addresses the actual issue. The work is genuinely not in the job description. Pretending otherwise is a small lie that every adult in the room recognizes.</p><p>This is also where the gap between management and the floor gets exposed most clearly. The executive sponsor truly believes governance is part of everyone’s job because in the executive view of the world, “improving the data” is a self-evident good and a shared responsibility. The business user lives in a different world. Their world is a JIRA backlog, a quarterly target, and a manager who measures them on output. Asking them to take on accountability without changing any of those signals is a one-way trade. They notice.</p><p>The fix is the boring administrative work most programs avoid. Update the job description. Get HR involved. Add governance responsibilities to the role definition for owners and stewards. Add them to the goal-setting template for the teams you depend on. Make sure the line manager knows that part of their team’s evaluation now formally includes this work.</p><p>Once governance is in the job description, the pushback flips. The analyst who said “it is not my job” stops saying it, because now it is. The manager who quietly de-prioritized it can no longer do that without consequence. The CDO stops having to beg for time. The work becomes part of how the team operates, not an extra ask.</p><p>It sounds bureaucratic. It is bureaucratic. That is the point. Governance lives or dies in the formal systems that allocate human time and reward. If you are not willing to change those systems, you are not running a governance program. You are running a request, and requests are easy to ignore.</p><h3>Why Top-Down Mandates Quietly Die</h3><p>Top-down works for things people already want to do. It does not work for things that add friction to their day.</p><p>When governance is announced as an executive initiative and pushed down through the org chart, here is the choreography that follows. The CDO sends the email. The VPs forward it with a thumbs up. Directors schedule a kickoff. Managers tell their teams to attend. The teams attend, nod, and go back to their actual work. Three months later, dashboards show 12 percent of datasets documented and falling. Six months later, someone calls a “reset.”</p><p>This is not a failure of communication. This is what happens when accountability flows downward but value does not.</p><p>The pattern has a name in change management literature. It is called compliance theater. People do enough to avoid getting in trouble and not one ounce more. You can spot it by looking at your governance platform. Are descriptions copy-pasted? Are owners assigned to the same five people across hundreds of datasets? Are tags applied in big batches at the end of the quarter? Theater.</p><p>The reason theater shows up is simple. The cost of doing the work is real. The cost of not doing the work is fuzzy. People optimize for what is real.</p><p>If you want governance to stick, you have to flip that equation. The cost of not doing it has to be specific and visible to the person whose behavior you want to change. The cost of doing it has to be reduced, not added on top of their existing job.</p><p>That is change management. It is harder than buying a tool.</p><h3>The Five Faces of Governance Theater</h3><p>If you want to know whether your program is real or theatrical, you do not need a maturity assessment from a consultancy. You need to look for five patterns. They show up in every stalled program I have ever seen.</p><p>The copy-paste descriptions. Open ten random datasets in your governance platform and read the descriptions out loud. If three or more of them sound generic, like “This table contains customer information related to transactions,” you have copy-paste. People are filling in the field to make the validation pass. They are not describing anything. The platform is a checkbox, not a knowledge asset.</p><p>The five-name owner list. Export your list of data owners and sort by frequency. If the same five names show up across hundreds of datasets, those people are not actually owners. They are placeholders, usually senior managers who got volunteered to make the dashboard turn green. Real ownership is distributed because real ownership requires actual context about the data, and no human being has context on 200 datasets.</p><p>The end-of-quarter blitz. Look at when documentation activity happens in your platform. If 70 percent of it lands in the last two weeks of every quarter, you have a blitz. People are doing just enough work to hit a metric, then disappearing until the next deadline. The work is performative. Nothing has actually changed about how data is used day to day.</p><p>The unread glossary. Pull the analytics on your business glossary. Most glossaries get heavy traffic the week they launch and almost nothing after that. If your glossary is not getting steady weekly views from people outside the governance team, the people writing definitions are writing for an audience of zero. That is demoralizing, unsustainable, and a strong signal that the rest of the org does not see the value yet.</p><p>The orphaned working group. Almost every governance program kicks off with a council or working group. Most councils are full of senior leaders for the first three meetings, then get quietly delegated to junior representatives, then start being canceled because of “scheduling conflicts.” If your council can disappear for two months without anyone noticing, the program has lost its center of gravity.</p><p>Spot two or more of these and you are running theater. The instinct in most organizations is to respond with stricter enforcement: tighter validation rules, escalation emails, executive nudges. That makes things worse. It treats the symptom and ignores the disease. The disease is the incentive gap. Until you fix that, every enforcement action just teaches people to do better theater.</p><h3>Rewrite the Benefits in the Reader’s Language</h3><p>Here is a small experiment. Take your last governance pitch deck. Strip out every benefit that is phrased in terms of the company. Compliance, risk, AI readiness, audit, M&amp;A, board reporting, all of it. What is left?</p><p>In most decks, almost nothing.</p><p>Now rebuild it. For each role you are asking to change, write three benefits in their language.</p><p>For a data engineer: fewer 2 a.m. pages because schema changes get caught upstream. Fewer “what does this column mean” Slack pings. A trail of ownership that protects you when something breaks at 11 p.m. on a Sunday.</p><p>For an analyst: trustworthy data that does not need to be re-validated every time. A common glossary so you stop arguing with finance about what “active customer” means. Less time hunting for the right table.</p><p>For a product manager: faster experiment setup because the data is already documented. Confidence that the metrics in your dashboard match the metrics in the executive report. Fewer last-minute fire drills because someone deprecated a table without telling you.</p><p>For a data scientist: a feature store with provenance, so your model does not get retrained on data that quietly changed schema. AI governance that protects you from being blamed when a model drifts because of upstream issues you could not see.</p><p>Notice none of those benefits mention the company. They are all selfish, in the good way. They speak to pain the person already feels. That is the only kind of benefit that gets someone to change behavior on a Monday morning.</p><p>The rule of thumb: if the benefit could be cut from the slide without changing the listener’s day, it is not really a benefit. It is a wish.</p><h3>The Middle Layer Is Where Programs Live or Die</h3><p>Top-down does not work. Bottom-up does not scale. The actual leverage point is your middle managers.</p><p>This is the layer that translates strategy into work. They write the goals. They run the standups. They sign off on time spent. If a director tells their team that governance work counts toward their performance review, it suddenly counts. If they do not, it does not, no matter what the CDO said in the all-hands.</p><p>So if you are running a governance program and you have not won over middle management, you are pushing on a string. The CDO can scream into the void all day. The line manager has more influence over whether someone documents a dataset than the entire executive team combined.</p><p>What does winning over middle management actually look like? Three concrete moves.</p><p>First, you give them air cover. Their team’s velocity will dip during the rollout. You need to acknowledge that publicly and protect them from the inevitable “why are you behind on your roadmap” conversation. If you do not, they will quietly de-prioritize governance the moment delivery pressure shows up, which is always.</p><p>Second, you give them tools that make them look good. A dashboard that shows their team’s documentation coverage, ownership, and quality scores. Something they can put in their own quarterly review to show progress. Make them the hero of the story.</p><p>Third, you tie a slice of their bonus or rating to it. Not a huge slice. Five percent will do. But it has to be real, and it has to show up in the system that drives their pay. If governance is not in the comp plan, it is not real.</p><p>This last one is uncomfortable. Most organizations recoil from changing comp structures. Do it anyway. The companies that get this right are the ones that stop pretending governance is a separate, free, parallel workstream.</p><h3>The Compensation Question Is Not Optional</h3><p>Let me say the quiet part out loud. If you are asking someone to take on data steward duties, own a domain, sign off on access requests, attend governance councils, and field stewardship questions, you are giving them a second job.</p><p>You can dance around this for a while. You can call it “part of the role” and stretch the definition of their existing job description. You can promise visibility and growth. You can lean on goodwill.</p><p>But the people who do this work know exactly what is happening. They are doing extra work without extra pay. The good ones either burn out or leave for somewhere that pays them for the work they actually do. The ones who stay become quietly cynical, which is worse.</p><p>The serious organizations have stopped dancing. They have created formal stewardship roles with formal job ladders. They pay a stewardship premium, sometimes a flat dollar amount, sometimes baked into the band. They write the responsibilities into the job description so the work is visible during reviews.</p><p>You do not need to do all of this on day one. But you do need a credible answer to “am I getting paid for this.” If your answer is “it counts toward your career growth,” that is not an answer. That is a delay tactic.</p><h3>AI Governance Makes the Incentive Problem Twice as Hard</h3><p>If you think data governance is a tough sell, wait until you start asking for AI governance.</p><p>Data governance asks people to document datasets. AI governance asks them to document model intent, training data lineage, evaluation results, fairness assessments, drift monitoring, prompt templates, retrieval sources, and the business reasoning behind every decision the model makes. It is data governance with an order of magnitude more documentation, more accountability, and more emotional weight, because if the model gets it wrong, somebody is going to be in a meeting explaining why.</p><p>The incentive gap gets worse here for three reasons.</p><p>First, the people building AI systems are usually the most in-demand and least patient with paperwork. Data scientists and ML engineers are operating under intense pressure to ship something the business is excited about. Asking them to slow down to document model risk feels like asking them to argue against their own success. Their incentive is speed and impact. Governance feels like the opposite of both.</p><p>Second, the surface area is bigger and changes faster. A dataset is relatively static. A model is a moving target with new versions, new prompts, new retrieval sources, and new failure modes every sprint. The documentation cost is not one-time. It is continuous. That breaks the goodwill model immediately, because goodwill works for one-time asks and falls apart for ongoing ones.</p><p>Third, the consequences of failure are now visible to executives in a way they never were for plain data quality issues. A bad dashboard is awkward. A biased loan model is a press release. The scrutiny is higher, which means the documentation expectations are higher, which means the workload is heavier, which means the incentive gap is wider.</p><p>The same rules apply, but the stakes are higher. You have to translate AI governance benefits into the builder’s language. A model card protects you when leadership asks why the model behaved a certain way. Drift monitoring saves you from being paged six months from now when the model quietly stopped working. Evaluation logs become your evidence in the post-mortem that is definitely going to happen. Prompt versioning means you can roll back when a “minor tweak” turns out to be catastrophic.</p><p>You also have to stop pretending AI governance can be a side quest. Either it is built into the model development lifecycle from day one, with sprint capacity allocated to it, or it is a checklist somebody fills in after the fact. The after-the-fact version is actively dangerous because it creates the illusion of safety while doing none of the work.</p><p>The teams that get this right treat AI governance as a release gate, not a documentation chore. The model does not ship until the governance artifacts exist, and the time to produce them is on the project plan from the beginning. That single change rewires the incentive. The work goes from “extra” to “required to ship,” and engineers respond to required-to-ship the way they always have. They do it.</p><p>If you remember nothing else about AI governance, remember this. With data, you might still get away with goodwill for a while. With AI, you cannot. The pace is too fast and the surface area is too big. Build governance into the workflow from day one or spend the next three years chasing it.</p><h3>A Practical Sequence That Actually Works</h3><p>If you are running a program right now and reading this with a slow pit in your stomach, here is the unglamorous sequence that tends to work.</p><p>Pick one domain. Not the whole company. One. Marketing analytics, finance reporting, customer data, whichever one has the most pain and the most willing leadership. Get it right there before you go anywhere else.</p><p>Sit with the people in that domain and listen. What do they spend time on that a governance platform could eliminate? What questions do they get asked over and over? What broke last quarter? Build the value story for governance out of their actual pain, not the pitch deck.</p><p>Negotiate the trade. For every hour of governance work you are asking for, name an hour of existing work that gets removed, automated, or deprioritized. If you cannot name it, you are stacking. Stacking does not work.</p><p>Get the manager bought in before the team. Make sure governance work shows up on the team’s official roadmap, not in a side document. If it is not on the roadmap, it does not exist.</p><p>Show the win. Six weeks in, point at something specific that got better because of the work. A question that did not need to be asked. A pipeline change that did not break a downstream report. Make the value visible in the language of the team.</p><p>Then, and only then, expand. Use the first domain as the proof point and the playbook for the next one.</p><p>This is slow. It is not the org-wide rollout your executive sponsor is asking for. It is the only thing that works.</p><h3>What You Just Read</h3><p>The argument was simple, even if the work is not.</p><p>Executives buy governance for benefits that show up at the company level. Practitioners pay the cost in their own time and energy. Until those two scoreboards are aligned, no amount of technology, training, or top-down mandate fixes the gap.</p><p>The fix is not a better platform. The fix is rewriting the benefit story in the language of the person being asked to do the work, winning the middle management layer, and being honest about compensation. Without those three things, your program is running on goodwill, and goodwill burns out.</p><h3>Closing Thought</h3><p>The governance programs that work are not the ones with the loudest executive sponsorship or the prettiest platform. They are the ones where the data engineer two layers down quietly thinks, “yes, this is making my job better, and someone noticed I did the work.”</p><p>Fix the incentives and the documentation fills in on its own. Keep mandating and you can keep buying platforms forever.</p><p>Your move.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4ae8c8a91e9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/data-governance-runs-on-incentives-not-mandates-b4ae8c8a91e9">Data Governance Runs on Incentives, Not Mandates.</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Welcome to The Semantic Layer]]></title>
            <link>https://medium.com/the-semantic-layer/welcome-to-the-semantic-layer-8d6f330f86da?source=rss----476333b6b1c2---4</link>
            <guid isPermaLink="false">https://medium.com/p/8d6f330f86da</guid>
            <category><![CDATA[semantic-layer]]></category>
            <category><![CDATA[knowledge-graph]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[ontology]]></category>
            <dc:creator><![CDATA[Marco Österlin]]></dc:creator>
            <pubDate>Wed, 06 May 2026 09:09:36 GMT</pubDate>
            <atom:updated>2026-05-06T09:09:36.407Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*p8ZKwy8_dMUz5grhrcgD0A.png" /></figure><h3>Why this publication exists</h3><p>There is a gap between governed data and useful AI, and the field is still figuring out how to close it.</p><p>For the last fifteen years, data governance has mostly been about catalogs, classifications, lineage, and compliance reporting. The work mattered, but the audience was small: data stewards, governance leads, and the occasional auditor. Most engineers and analysts treated governance as something that happened around their work, not in it.</p><p>That is changing. AI changes it.</p><p>When a large language model retrieves from your data estate, what it finds becomes part of an answer that someone trusts. The quality of that answer depends on whether the data was classified correctly, whether the relationships between entities are explicit, whether sensitive information was excluded, whether the lineage is intact, whether the business meaning is captured anywhere a machine can read it. Governance stopped being back-office work the moment data started feeding generative systems directly.</p><p>This is what I mean by the semantic layer.</p><h3>What the semantic layer means here</h3><p>In the BI world, “semantic layer” usually refers to the metric definitions sitting between a warehouse and a dashboard. That meaning is real and useful, but it is too narrow for what the field is becoming.</p><p>In this publication, the semantic layer is the connective tissue between raw data and trustworthy AI. It includes:</p><ul><li>Data governance, the foundation that makes anything else trustworthy: catalogs, lineage, ownership, stewardship, and quality</li><li>Information protection, the rules of access: classification, sensitivity labeling, DLP, and the controls that decide what data is allowed to flow where</li><li>AI governance and observability, the controls on the model side: monitoring behavior, evaluating outputs, and validating that models actually do what they were intended to do</li><li>Ontologies and knowledge graphs, the explicit structure that turns rows into meaning and gives AI something it can reason over</li><li>Business semantics and metric layers, the shared vocabulary that humans, BI tools, and AI systems all depend on</li><li>The platforms, tools, and patterns that bring all of this together</li></ul><p>If you work on any of those things and you suspect the next few years are going to demand they all work together, this publication is for you.</p><h3>Vendor and product agnostic</h3><p>This publication is not tied to any single vendor, platform, or product.</p><p>Databricks, Microsoft Fabric, Microsoft Purview, Collibra, Alation, Atlan, Databricks Unity Catalog, Snowflake Horizon, Informatica, dbt, Cube, AtScale, Neo4j, TigerGraph, Stardog, Palantir Foundry, and whatever ships next quarter all have something to teach about how this layer gets built. So do open source projects, internal platforms, and the patterns that emerge before any tool catches up to them.</p><p>If a piece is honest, useful, and grounded in real work, the platform it talks about does not matter. Pieces that compare approaches across vendors are particularly welcome, because most practitioners have to make those comparisons themselves and the public material is usually written by the vendors competing in them.</p><h3>Open to collaborators</h3><p>This is explicitly a multi-author publication. I am one writer. The semantic layer is bigger than any one practitioner, any one vendor stack, and any one geography.</p><p>If you work in this space and you want to write here, please do. That includes:</p><ul><li>Data governance leads with real implementation experience to share</li><li>Engineers building knowledge graphs, ontologies, or semantic models in production</li><li>AI and ML practitioners working on grounding, retrieval, and trust</li><li>Architects designing the platforms that hold this all together</li><li>Anyone with a sharp opinion and the work to back it up</li></ul><p>You do not need a large following. You do not need to write often. You do not need to agree with anything I have published. You need a piece worth reading and the willingness to put your name on it.</p><p>If you are interested, reach out and you can start publishing!<br>The editorial bar is honesty, usefulness, and respect for the reader’s time. The rest is up to you.</p><h3>What you will find here</h3><p>Three kinds of writing.</p><p>Implementation work. Patterns from real engagements, with the actual decisions and trade-offs left in. If a label taxonomy did not survive contact with a customer, that goes in. If an ontology design saved a project, that goes in too.</p><p>Opinionated takes. Where the field is going, what the industry is getting wrong, where the gap between marketing and reality is widest. These will not be neutral.</p><p>Field notes. Shorter pieces from delivery work, conferences, community discussions, and whatever the writers here are building.</p><p>What you will not find: vendor marketing dressed up as articles, AI-generated filler, or pieces that summarize what someone else already wrote.</p><h3>Who started this</h3><p>Marco Österlin, Microsoft MVP for Microsoft Purview, specializing in Data Catalogs and Metadata Management, and Principal Consultant at Twoday in the Nordics. I implement data governance and AI platforms for enterprise customers and write about what I learn doing it. But this publication is not about me. It is about the people building this layer, wherever they are and whatever they use to build it.</p><h3>What to do now</h3><p>Subscribe if any of this sounds like work you do or work you want to understand. Reach out if you want to write here. Push back if you disagree with anything published. The field is too young and the stakes are too high for any of this to be a one-way conversation.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8d6f330f86da" width="1" height="1" alt=""><hr><p><a href="https://medium.com/the-semantic-layer/welcome-to-the-semantic-layer-8d6f330f86da">Welcome to The Semantic Layer</a> was originally published in <a href="https://medium.com/the-semantic-layer">The Semantic Layer</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>