<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Select Star on Medium]]></title>
        <description><![CDATA[Stories by Select Star on Medium]]></description>
        <link>https://medium.com/@selectstar?source=rss-509053c0c088------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*LfJxi2mLMJKVcEgPmuJ5zQ.png</url>
            <title>Stories by Select Star on Medium</title>
            <link>https://medium.com/@selectstar?source=rss-509053c0c088------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 02:48:36 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@selectstar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Select Star Joins Snowflake: A New Chapter for AI + Data Discovery]]></title>
            <link>https://medium.com/selectstar/select-star-joins-snowflake-a-new-chapter-for-ai-data-discovery-0db3e670df08?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/0db3e670df08</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[metadata]]></category>
            <category><![CDATA[data-discovery]]></category>
            <category><![CDATA[snowflake]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Mon, 24 Nov 2025 17:05:47 GMT</pubDate>
            <atom:updated>2025-11-24T17:05:47.690Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MIsbYQAPjYuUvM331-SkMQ.png" /></figure><p>I’m excited to share that <a href="https://www.snowflake.com/en/blog/snowflake-acquire-select-star.html">Snowflake has announced</a> its plans to acquire the Select Star platform. This marks a new chapter in our mission to bring intelligent metadata context to every organization and expand Snowflake Horizon Catalog’s view of enterprise data for agentic AI experiences.</p><p>From the beginning, our vision at Select Star has been to make data easier to find, understand, and use. That means not only capturing what data exists, but also where the data comes from, who uses it, and how it’s being used. Whether it was simplifying data governance, accelerating self-service analytics, or cutting down the cost of unused data assets, we’ve seen how powerful metadata context can be. It’s evolved from a nice-to-have to the core of how modern data teams operate.</p><h3>AI + Data Thrives With Metadata Context</h3><p>As AI becomes more integrated into organization workflows, context becomes even more critical. A model or agent can only perform as well as the context it has. Without knowing where data originates, how it has been shaped, or which teams depend on it, AI cannot produce reliable results. The foundation of trustworthy AI is trustworthy metadata.</p><p>We saw this firsthand as customers have leveraged Select Star’s lineage, popularity, entity relationships, and other connected metadata to supercharge their AI for data. With our universal metadata model that connects more than 25 integrations, our customers have gained a new level of clarity of their data models and their data ecosystem. For us, this reinforced a belief we held from the start: metadata is the connective tissue that turns raw data into something usable and dependable.</p><p>This context has become even more important with the rise of AI. Through our work on the MCP (Model Context Protocol) Server, Ask AI, and automated semantic model generation, we watched how powerful AI becomes when it can rely on accurate and up-to-date metadata. These experiences showed us that the next generation of data intelligence will require an embedded metadata layer that sits across the entire data ecosystem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xprs9glfufOyocN1lOIYHg.png" /></figure><p>Joining Snowflake will allow us to take this vision further. One of our priorities will be to bring Select Star’s industry-leading data lineage and data discovery capabilities into Snowflake’s Horizon Catalog, which will help agentic AI experiences like Snowflake Intelligence and Cortex Code deeply understand an enterprise’s data and how to extract insight from it.</p><h3>Looking Ahead</h3><p>Expanding our metadata context layer beyond Snowflake and bringing our existing integrations into Snowflake’s Horizon Catalog will give teams a consistent view of lineage, ownership, and usage across every connected system. With that foundation in place, customers can spend more time answering business questions and less time stitching together the story behind their data. It becomes easier to identify the right datasets, avoid redundant work, and confidently support business initiatives across the organization.</p><p>I am excited to scale our impact and continue building the kind of data intelligence platform we always envisioned. I’m deeply grateful to our customers, our team, and our investors for being a part of our journey, and looking forward to what we will create together with Snowflake.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lmsJnVBB9A6L2K_0cDBguw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0db3e670df08" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/select-star-joins-snowflake-a-new-chapter-for-ai-data-discovery-0db3e670df08">Select Star Joins Snowflake: A New Chapter for AI + Data Discovery</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building Semantic Data Models: From BI to AI]]></title>
            <link>https://medium.com/selectstar/building-semantic-data-models-from-bi-to-ai-55a5949b4364?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/55a5949b4364</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Tue, 28 Oct 2025 19:31:13 GMT</pubDate>
            <atom:updated>2025-11-12T00:22:00.261Z</atom:updated>
            <content:encoded><![CDATA[<p>​AI is showing up in more BI tools and analytics workflows, particularly as natural language analytics. But results often fall short because LLMs don’t know your business: the metric definitions, valid join paths, and constraints buried in SQL, dashboards, wikis, and tribal knowledge. Without this context, even the best LLMs are just guessing.</p><p>A semantic data model fixes this. It puts definitions, joins, and rules in one place so both people, BI tools and AI use the same logic. When the data model is clear, a SQL query, a Looker Explore, and a natural language question return the same number.</p><p>This post distills a talk at dbt Coalesce by Select Star’s Head of Data Strategy &amp; Customer Success, Alec Bialosky, into a practical guide. We cover why the semantic data model matters, what to include and exclude, and a step-by-step process to build and validate one you can ship quickly.</p><h4>​<strong>What is a semantic data model and why is it important for AI</strong></h4><p>A semantic data model is a blueprint that encodes your business definitions (metrics, dimensions, entities) and the allowed relationships, in a form both BI tools and LLMs can interpret. With that context in place, AI becomes useful for analytics in four concrete ways:</p><ol><li><strong>Shared meaning for machines.</strong> Business definitions for metrics, dimensions, and relationships are encoded in the semantic data model so AI can interpret them the same way your team does.</li><li><strong>Query correctness.</strong> The semantic data model provides the schema, valid joins, and relationships so LLMs generate accurate SQL rather than hallucinations.</li><li><strong>Context for reasoning.</strong> With metrics, hierarchies, and relationships encoded, AI can explain why results look the way they do, not just return numbers.</li><li><strong>Foundation for automation.</strong> The model lets AI build, test, and refine analytics workflows without reinventing metric logic each time.</li></ol><h4>​<strong>What to include in your semantic data models</strong></h4><p>A useful semantic data model is intentionally streamlined. Start with the parts that help people and AI ask and answer business questions: clear definitions, safe joins, and the calculations teams rely on. Keep pipeline and staging details in code, not in the model. As adoption grows, you can add breadth, but begin with the minimum that unlocks trusted answers.</p><p><strong>Include</strong></p><ul><li><strong>Entities and dimensions.</strong> The core things you analyze, the grain for each, and the attributes you slice by. Examples: customer, order, product, date, region.</li><li><strong>Metric calculations.</strong> Canonical formulas for KPIs and derived measures. Capture filters, windowing, and null handling.</li><li><strong>Relationships.</strong> Allowed join paths with the expected cardinality. Make the safe path the default and call out disallowed shortcuts.</li><li><strong>Business rules and language.</strong> Synonyms, common phrases, and examples that map to model elements. This helps both humans and AI route questions correctly.</li></ul><p><strong>Exclude</strong></p><ul><li><strong>Raw or staging tables.</strong> Keep the model at the reporting layer. If a field is not used for analysis, it does not belong here.</li><li><strong>ETL and pipeline details.</strong> Transform steps and implementation details live in your code, not in the model. <strong>One‑off or experimental metrics.</strong> If it is not standardized and broadly useful, keep it out until it is.</li></ul><h4>​<strong>How to build a semantic data model</strong></h4><figure><img alt="How to build your semantic data model with Select Star" src="https://cdn-images-1.medium.com/max/1024/0*2mkb19m8MOHzraPy.png" /></figure><p>​To build a semantic data model your BI and AI can trust, follow these six steps. Start with the questions your business asks most often, align on canonical metrics, reuse trusted assets, make ownership explicit, document in plain language, then iterate with feedback.</p><p>​<strong>1) Start with the Business Questions</strong></p><p>Identify the decisions teams make and the questions they ask most. Group by theme. For each theme, list the entities involved, key slices (time, region, segment), and the primary KPI. This gives you the first pass at entities, dimensions, and metrics.</p><p>Examples to seed your list:</p><ul><li>What was revenue last quarter by region and segment?</li><li>How many active customers churned within 30 days of their first order?</li><li>Which products drive repeat purchases in the first 90 days?</li></ul><h4><strong>2) Align on Core Metrics</strong></h4><p>Bring Finance, Product, and Operations together. Agree on canonical formulas and grains. If two versions are required, define both and document when to use each.</p><p>Document each KPI in plain language so anyone can reproduce the number:</p><ul><li>Business definition and reporting grain</li><li>Formula (include key filters)</li><li>Owner</li><li>Last updated</li></ul><h4><strong>3) Leverage What Already Exists</strong></h4><p>Do not reinvent definitions if trusted dashboards already exist. Audit the reports leaders rely on and lift formulas, filters, and join paths directly. This shortens time to value and provides an answer key for validation.</p><h4><strong>4) Make Ownership Explicit</strong></h4><p>Assign domain owners and metric owners. Capture a lightweight change log so downstream users can see what changed and why. Define a simple process for proposing new metrics and retiring unused ones.</p><p>Ownership checklist:</p><ul><li>Named owners per entity and metric</li><li>Changelog with date, change summary, and approver</li><li>Review cadence to reconcile BI vs model differences</li></ul><h4><strong>5) Document and Share in Plain Language</strong></h4><p>Write descriptions that nontechnical stakeholders can understand. Add synonyms for key terms, sample values, and example questions. Spell out safe join paths and expected cardinality. Publish the docs where people already work and make them searchable.</p><p>What to publish and share:</p><ul><li>Plain‑English definition and formula</li><li>Synonyms and common phrases (for AI routing)</li><li>Example questions mapped to metrics</li><li>Diagram of allowed joins between core entities</li></ul><h4><strong>6) Iterate with Feedback Loops</strong></h4><p>Treat the model as a living asset. Validate early and often, then improve based on real usage rather than long design meetings.</p><p>How to validate and gather feedback:</p><ul><li>Parity with BI: compare LLM answers to canonical dashboards for 10 high‑value questions</li><li>Consistency checks: same question across tools should match, even with varied phrasing</li><li>Usage‑based tests: prioritize questions people ask most</li><li>Drift monitoring: alert when model outputs diverge from BI or when upstream assets change</li></ul><h4><strong>Automating semantic data model generation with Select Star</strong></h4><p>If you already have dashboards that leaders trust, you can bootstrap the model by extracting the formulas and relationships that power those dashboards. Many teams find that this flips the effort profile from mostly building to mostly testing, which speeds up rollout and improves reliability.</p><p>Select Star can help automate <a href="https://www.selectstar.com/solutions/semantic-model-generation">semantic model generation</a>. We can reverse engineer metric logic, joins, and entities from BI tools and warehouse lineage, then output a clean YAML model with entities, dimensions, relationships, metrics, synonyms, and sample values. We keep the model lean by importing only fields that are actually used. You can publish to the dbt Semantic Layer or your warehouse, and make the model available to AI tools. We also track dependencies and changes, and surface drift when source assets move.</p><p>Want to add guide rails to your AI tools? <a href="https://www.selectstar.com/demo">Connect with our team</a> to see semantic model generation in action.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=55a5949b4364" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/building-semantic-data-models-from-bi-to-ai-55a5949b4364">Building Semantic Data Models: From BI to AI</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[​Improve Data Governance with Column-level Lineage in Amazon Redshift using Select Star]]></title>
            <link>https://medium.com/selectstar/improve-data-governance-with-column-level-lineage-in-amazon-redshift-using-select-star-63256849e489?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/63256849e489</guid>
            <category><![CDATA[data-lineage]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[redshift]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Tue, 28 Oct 2025 15:06:01 GMT</pubDate>
            <atom:updated>2025-10-28T15:06:01.398Z</atom:updated>
            <content:encoded><![CDATA[<p>Organizations today manage increasingly complex data ecosystems where data flows through multiple systems and undergoes numerous transformations. With Amazon Redshift, a fully managed data warehouse service, you can analyze petabytes of data across your data warehouse, operational database, and data lake. However, as your data pipelines grow more sophisticated, tracking how data moves and transforms becomes challenging. You need visibility into your data’s journey — from source to destination — to maintain data quality, ensure compliance, and make informed business decisions.</p><p><a href="https://www.selectstar.com/">Select Star</a> is an automated metadata context platform that automatically analyzes and documents your data to provide you with a single source of truth for data. It provides features such as data catalog, lineage, usage analysis, and AI assistants to help data teams govern and manage their data effectively. Select Star serves as a co-pilot for data teams, assisting in data governance, data migration, self-service analytics, data democratization, and cost optimization initiatives. The platform allows users to easily organize and govern their data, making it simple for everyone to find the right data for their needs.</p><p><a href="https://aws.amazon.com/redshift/">Amazon Redshift</a> is a fully managed data warehouse service that lets you analyze petabytes of data across your data warehouse, operational database, and data lake. As organizations build increasingly complex data architectures with Amazon Redshift at their core, they face the challenge of tracking how data flows through multiple systems and transformations. Without clear visibility into these data movements, it becomes difficult to maintain data quality, ensure compliance, and make informed business decisions.</p><p>This post shows you how to implement column-level <a href="https://www.selectstar.com/product/data-lineage">data lineage</a> for Amazon Redshift using Select Star. You’ll learn how to gain detailed visibility into your data transformations, understand dependencies, and track data movement across your warehouse environment. By following these steps, you can better govern your data, quickly troubleshoot issues, and confidently manage changes to your data models.</p><h3>Column-level data lineage: A crucial tool to understand data</h3><p>When you manage data in Amazon Redshift, you need to track how your data moves and changes throughout your systems. Data lineage helps you do this by creating a detailed map of your data’s journey: from where it originates, through its transformations, to where it’s used. With column-level lineage, you can track individual data elements as they flow through your system, helping you maintain data integrity, ensure compliance, and derive accurate insights.</p><p>Think of data lineage as your data’s story in Amazon Redshift. It shows you where your data comes from, how it transforms as it moves through your systems, and where your teams ultimately use this data in their workflows. This visibility becomes particularly valuable when you’re troubleshooting data quality issues or planning changes to your data models. For example, if you find an inconsistency in a report, you can trace the data back to its source to identify where the issue originated. Similarly, before modifying a source table, you can see exactly which downstream reports and analytics might be affected by your changes.</p><p>This comprehensive view of your data’s journey helps you work more efficiently with Amazon Redshift. You can troubleshoot issues faster by quickly identifying the root cause of data discrepancies. You can make changes to your data models with confidence, knowing exactly what downstream impacts to expect. For compliance requirements, you can easily track sensitive data throughout your system. Most importantly, you can understand the complex dependencies between different data assets, helping you maintain a reliable and efficient data ecosystem.</p><h3>The Benefits of Data Lineage</h3><p>When you use data lineage with Amazon Redshift, you gain transparency into your data models that helps you work more efficiently. Instead of manually investigating each step in complex pipelines when issues arise, you can quickly trace problems to their source by following the lineage map. For example, if you notice incorrect values in a dashboard, you can follow the data’s path backward through your transformations to find exactly where the discrepancy began.</p><p>Data lineage also helps you understand how changes to your data models affect your entire system. Before you modify a table, schema or update a transformation pipeline, you can see exactly which downstream reports and analytics will be impacted. This visibility helps you prevent unexpected disruptions and plan your updates strategically. For instance, if you need to modify a frequently used source table, you can identify all dependent dashboards and notify the relevant teams before making the change. This proactive approach helps you maintain data reliability while confidently evolving your Amazon Redshift environment.</p><h3>Data Lineage with Select Star</h3><p>With Select Star, you can visualize and understand how data flows through your Amazon Redshift environment. The lineage feature automatically maps relationships between your data assets by analyzing SQL statements from your connected data sources. This gives you a clear view of where your data comes from and where it flows, helping you understand dependencies across tables, columns, and dashboards. When you plan changes to your data assets, you can quickly see how these changes might affect your entire data environment.</p><p>Select Star’s lineage view provides four ways to explore your data relationships:</p><ol><li><strong>Upstream</strong>: Shows the immediate upstream dependencies in a tree hierarchy, helping you trace where your data originates</li><li><strong>Downstream</strong>: Shows the immediate downstream dependencies in a tree hierarchy, letting you see what depends on your data</li><li><strong>Downstream Dashboards</strong>: Shows all dashboard dependencies downstream with extended information like Top User and dashboard Popularity, helping you understand data consumption</li><li><strong>Explore</strong>: Shows an advanced lineage graph that allows navigation of data flow at the column level, giving you the most detailed view of your data relationships</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RFdoSJ_zBefLOh6T.png" /></figure><h3>Getting Started with Select Star and Redshift</h3><p>By setting up column-level lineage with Select Star, you can gain detailed insights into your Amazon Redshift data environment. This feature helps you understand exactly how your data flows and transforms across your warehouse. In the following sections, you’ll learn how to configure column-level lineage for your Amazon Redshift cluster. After completing these steps, you’ll be able to track data relationships at a granular level, helping you better manage your data assets and understand their dependencies.</p><h3>Step 1: Create a new data source</h3><ul><li>Go to Select Star Settings and click on “Data” in the sidebar.</li><li>Click “+ Add” to create a new Data Source.</li><li>Choose “Redshift” as the Source Type and provide the required information, including<br>Display Name: This value is Redshift by default, but you can override it if desired.<br>Cluster Name: The name of your AWS Redshift cluster in the AWS management console. Also known as “Cluster identifier” by AWS.<br>Database: The name of the database in AWS Redshift you’ve given us access to.<br>AWS Region: ID of the AWS region where the cluster was created. For example us-east-2,us-west-1, eu-central-1</li><li>Click Connect</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1tk-Qo6IoZcKxeJN.jpeg" /></figure><h3>Step 2: Set up CloudFormation stack</h3><p>Select Star recommends using AWS CloudFormation for a safe and auditable integration setup.</p><ul><li>A form will be displayed. Click the “Open CloudFormation” button. This will open a new window to the AWS Management Console and proceed to create the CloudFormation stack. Ensure you’re logged into the correct AWS account where your Redshift cluster resides.‍‍</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2Gvsv8fEoACyk-MI.jpeg" /></figure><ul><li>On the Create Stack form, enter the required parameters:<br>Amazon Redshift cluster name<br>Comma-separated list of database names<br>Your database username (this will only be used to create a dedicated “select_star” user)</li><li>​Also, on the same Create Stack form, set “Configure S3 logging” and “Restart Cluster” fields to “true” for automatic configuration.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WLJOMaN3ATGzFSro.jpeg" /></figure><ul><li>Review the information carefully. At the bottom of the form, check the box that says “I acknowledge that AWS CloudFormation might create IAM resources.”</li><li>Click the “Create stack” button to initiate the process.</li><li>​Wait for the stack status to change from “CREATE_IN_PROGRESS” to “CREATE_COMPLETE” in the Stack info tab. This typically takes about 5 minutes and you will need to refresh the tab to see the progress.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G718u8rtSMHW5mWT.jpeg" /></figure><h3>Step 3: Confirm authorization</h3><ul><li>Once the CloudFormation stack is created, copy the Role ARN from the Outputs tab.</li><li>Return to Select Star and provide the Role ARN in the authorization form.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Pw0P69x1LBUxQN3q.jpeg" /></figure><h3>Step 4: Select databases and schemas</h3><ul><li>Choose the databases and schemas you want to load into Select Star.</li><li>Select Star will generate lineage for the selected databases and schemas.</li></ul><p>After completing these steps, Select Star will begin syncing your Redshift metadata. Within 24–48 hours, you’ll be able to explore column-level lineage for your Redshift environment directly within the Select Star platform.</p><h3>Future-Proof Your Governance Strategy with Data Lineage</h3><p>Column-level data lineage for Amazon Redshift with Select Star makes it easier to see and understand how data flows through your warehouse environment. In addition to Amazon Redshift, Select Star <a href="https://www.selectstar.com/resources/redshift-data-governance">integrates</a> with AWS services like AWS Glue and Amazon Quicksight, as well as widely used databases and business intelligence tools. Together, these integrations provide a complete view of your data ecosystem.</p><p>With column-level lineage in place, your team can govern data assets more effectively and simplify compliance across the organization. This visibility makes it easier to plan data changes, understand dependencies, and quickly resolve issues by tracing them back to their source. As your data environment scales, having this clear picture of data movement becomes essential for maintaining quality, reliability, and trust in your analytics.</p><p>To get started with Select Star for Amazon Redshift, you can visit <a href="https://www.selectstar.com/resources/redshift-data-governance">AWS Marketplace</a> to deploy the solution today. If you’d like to see the features in action, <a href="https://www.selectstar.com/demo">schedule a demo</a> with our team.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=63256849e489" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/improve-data-governance-with-column-level-lineage-in-amazon-redshift-using-select-star-63256849e489">​Improve Data Governance with Column-level Lineage in Amazon Redshift using Select Star</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[dbt Coalesce 2025 Highlights: dbt + Fivetran Merger, Open Data Infrastructure, dbt Fusion and MCP…]]></title>
            <link>https://medium.com/selectstar/dbt-coalesce-2025-highlights-dbt-fivetran-merger-open-data-infrastructure-dbt-fusion-and-mcp-894025c81ec1?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/894025c81ec1</guid>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Wed, 15 Oct 2025 15:33:44 GMT</pubDate>
            <atom:updated>2025-11-04T16:16:53.432Z</atom:updated>
            <content:encoded><![CDATA[<h3>dbt Coalesce 2025 Highlights: dbt + Fivetran Merger, Open Data Infrastructure, dbt Fusion and MCP Server</h3><p>​Coalesce 2025 opened in Las Vegas with more than 2,000 people in the room and over 14,000 online. The opening keynote set two clear headlines: dbt’s announced merger with <a href="https://www.fivetran.com/">Fivetran</a> and a major spotlight on the <a href="https://www.getdbt.com/product/fusion">dbt Fusion engine</a>, now in dbt Platform preview. Together, they signal a shift toward open data infrastructure and AI‑ready analytics. The message was to integrate where it matters without locking yourself to a single compute engine or BI stack. Here’s a practitioner’s recap of the opening keynote and what your data team should know.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FWqJsAvIwwXo%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DWqJsAvIwwXo&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FWqJsAvIwwXo%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube-nocookie" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/7697962787e7621704fc0ca6e52c5a64/href">https://medium.com/media/7697962787e7621704fc0ca6e52c5a64/href</a></iframe><h4>The big news: dbt + Fivetran</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Cr9L0_eimepFpN_8.jpg" /></figure><p>The <a href="https://www.getdbt.com/blog/dbt-labs-and-fivetran-merge-announcement">dbt + Fivetran merger</a> brings ingestion, transformation, and metadata closer together without dictating your compute layer. On stage, dbt Labs CEO Tristan Handy reiterated commitments to openness and compatibility across engines, catalogs, BI tools, and LLMs. Fivetran CEO George Fraser highlighted its deepening data lake investments, including turnkey Iceberg destinations where Fivetran acts as the storage steward for compaction, deletes, and other lakehouse hygiene. Fraser also addressed community concerns, saying there would be no changes to dbt licensing or naming.</p><p><strong>Why it matters:</strong> tighter loops between ingestion, transformation, and metadata mean fewer seams and fewer surprises. The roadmap deliberately avoids a walled garden, emphasizing open standards and multi-engine choice over lock-in, with compatibility across catalogs, BI tools, and LLMs. And for teams leaning into lakehouse patterns, Iceberg-backed destinations make lakes feel more turnkey without constant storage babysitting.</p><h4>Fusion: dbt rewritten as a compiler (and why data teams should care)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*R6euh6sKg1SfqK44.jpg" /></figure><p><a href="https://www.getdbt.com/product/fusion">dbt Fusion</a> is a new engine written in Rust that works like a SQL compiler. It understands each warehouse’s dialect, tracks project state, and compiles your code across engines. You feel this in the editor with the dbt VS Code extension: errors show before a query runs, you can preview the output of a single CTE, and Compare Changes gives you a data diff so you can see what will change. When you rename a column, Fusion updates downstream references and flags anything that might break. You can also validate the same project on Snowflake, Databricks, BigQuery, ClickHouse, and more.</p><p>Beyond the editor, Fusion helps reduce waste in production. <a href="https://www.getdbt.com/blog/announcing-state-aware-orchestration">State Aware Orchestration</a> skips models whose inputs have not changed, so you avoid empty runs. Freshness settings like build_after and updates_on: all align rebuilds with your SLAs. Column-aware and aggregated testing removes duplicate scans while keeping quality high. The gains are already being realized: <a href="https://eqtgroup.com/">EQT Group</a>, one of the world’s largest private equity investors, reported about 60% faster runtimes and 45% lower warehouse cost after adopting Fusion. A dbt internal project saw 9% savings from reuse and 55% from tuning, for a total of 64% cost savings.</p><h4>AI-ready analytics in dbt with MCP, MetricFlow, and agents</h4><p>The keynote’s AI story was not about having a model write SQL. It focused on giving models the right context and tools, and on building in error correction. The <a href="https://docs.getdbt.com/blog/introducing-dbt-mcp-server">dbt MCP server</a>, now generally available as a remote service with OAuth, is a governed bridge for IDEs and model providers. It lets them read models, lineage, and metrics and, when permitted, run dbt commands with proper authentication.</p><p><a href="https://www.getdbt.com/blog/open-source-metricflow-governed-metrics">MetricFlow</a> will return to open source under Apache 2.0, which moves the industry toward deterministic, governed metrics that teams can trust across tools. dbt Agents are also on the way, with developer and observability agents to propose and apply changes and troubleshoot failures, and discovery and analyst agents to help people find the right data and answer questions using approved definitions.</p><p><a href="https://www.nbim.no/">Norges Bank Investment Management</a> (NBIM) rolled out conversational analytics through the MCP server across the organization. They saw roughly 10x the adoption of their previous catalog, fewer tickets, and faster time to insight, and now run hundreds of production agents that monitor systems and take action because answers are grounded in governed context.</p><p>For context beyond dbt, <a href="https://www.selectstar.com/">Select Star</a> fills the gap by giving agents and other tools end-to-end data context: where a field originated in a source system, which Fivetran connector brought it in, which dbt model produced it, which semantic definition or MetricFlow metric describes it, and which dashboards and notebooks depend on it. <a href="https://www.selectstar.com/solutions/ai-ready-data">Agents</a> can also see owners, certifications, usage, and PII tags so they choose the right asset and explain why. Those same signals make it easy to audit answers and retire stale assets when usage drops.</p><h4>Your path to open, AI-ready analytics with dbt + Fivetran and Fusion</h4><p>The Coalesce 2025 opening keynote put the focus on metadata. Fusion and agent-driven workflows can increase throughput and confidence if your metadata is reliable and visible. Here’s where to start.</p><ul><li>Pilot Fusion on a bounded project and benchmark runtime and cost. Turn on CI diffs and impact analysis so reviewers can see the effect of a change.</li><li>Stand up the remote dbt MCP server in a sandbox, scope permissions carefully, and log and review agent actions.</li><li>Require certified assets for any production answer.</li><li>Align metrics by pairing MetricFlow with your existing layer and certifying a small set of cross-team metrics first.<br>Plan for a multi-engine reality by enforcing naming and ownership standards and watching for duplicates and forked models.</li></ul><p>Want deeper context for AI and analytics in dbt and across the rest of your stack? <a href="https://www.selectstar.com/demo">Connect with our team</a> to see how Select Star adds business definitions, end-to-end lineage, ownership, and certifications, and usage signals that give AI and agents a trusted starting point.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=894025c81ec1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/dbt-coalesce-2025-highlights-dbt-fivetran-merger-open-data-infrastructure-dbt-fusion-and-mcp-894025c81ec1">dbt Coalesce 2025 Highlights: dbt + Fivetran Merger, Open Data Infrastructure, dbt Fusion and MCP…</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Scaling dbt Docs with an Automated Data Catalog]]></title>
            <link>https://medium.com/selectstar/scaling-dbt-docs-with-an-automated-data-catalog-d0fbe3f9a226?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0fbe3f9a226</guid>
            <category><![CDATA[data-catalog]]></category>
            <category><![CDATA[data-discovery]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[dbt]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Thu, 02 Oct 2025 02:51:37 GMT</pubDate>
            <atom:updated>2025-10-21T15:01:57.231Z</atom:updated>
            <content:encoded><![CDATA[<p>​Using <a href="https://docs.getdbt.com/docs/build/view-documentation#dbt-docs">dbt Docs</a> as your data dictionary? dbt Docs is a great way to start documenting your data models, sources, and columns, but it can become brittle as projects grow. Yaml files can drift, sharing is clunky, and business users think of it as too technical or developer-only documentation. If you want everyone to use the same definitions across tools, how do you keep data documentation current and visible across your stack?</p><p>In this guide, you’ll learn a practical way to scale dbt Docs alongside an <a href="https://www.selectstar.com/product/data-catalog">automated data catalog</a> so definitions stay accurate and easy to find. Keep schema.yml as the source of truth, manage changes through pull requests in GitHub or Bitbucket, and surface definitions in BI and other team tools so everyone is working from the same meaning.</p><h4>What are dbt Docs?</h4><p>dbt Docs are the documentation site and metadata generated from your dbt project (models, sources, columns, tests, and lineage) using descriptions defined in schema.yml (properties) and artifacts produced by dbt runs.</p><p><strong>How dbt Docs are built</strong></p><p>You write table and/column descriptions in YAML. dbt produces artifacts during runs that are saved as JSON files (semantic_manifest.json, manifest.json, catalog.json, run_results.json, and sources.json). The `dbt docs generate` command builds a static documentation site. Teams can view documentation locally with the dbt docs serve command or in dbt Cloud’s Catalog.</p><p><strong>Who uses dbt Docs</strong></p><p>Primarily analytics and data engineers. Broader adoption requires surfacing definitions where business users work like BI tools and data catalog search.</p><p><strong>Why dbt Docs matter</strong></p><p>Keeping schema.yml accurate is foundational. It gives you versioned, reviewable definitions and reduces drift across tools.</p><p>Example schema.yml</p><pre>​models:<br>- name: orders<br>  description: &quot;Finalized customer orders used for revenue reporting&quot;<br>  columns:<br>    - name: order_id<br>      description: &quot;Unique identifier for an order&quot;<br>    - name: order_total<br>      description: &quot;Sum of line item totals after discounts and tax&quot;</pre><h4>Common Challenges with Using and Scaling dbt Docs</h4><p>First, let’s look at where dbt Docs usually break. These are the dbt documentation challenges we see most often when teams try to scale docs across projects and stakeholders.</p><h4><strong>Data documentation beyond dbt</strong></h4><p>dbt Docs only covers assets managed within your dbt project. If you use BI tools like Tableau or Power BI, or need to document tables outside of dbt in Snowflake, those assets aren’t included automatically. While dbt exposures can help track these dependencies, they require manual maintenance and don’t pull in the actual documentation from those systems.</p><h4><strong>Shareability and usability</strong></h4><p>Many teammates never open dbt Docs. Non‑technical users want <a href="https://www.selectstar.com/product/entity-relationship-diagrams">ERDs</a>, <a href="https://www.selectstar.com/product/data-lineage">lineage</a>, and in‑app search. When definitions live only in YAML, they are hard to find. Definitions need to be where work happens: BI tooltips, search, and lineage.</p><h4><strong>Keeping YAML up to date at scale</strong></h4><p>Ownership is unclear, reviews are ad hoc, and coverage slips. The result is drift between schema.yml, dashboards, and what people actually trust. Changes to schema.yml need to be treated like code: propose edits, review in a pull request, and merge on a regular cadence.</p><h4><strong>Onboarding and search</strong></h4><p>Property references tell you what a column is, not how to use it. New hires need examples, links to source models, and consistent naming, or they bounce. Descriptions need to be paired with links to the source model and a short example query.</p><h4><strong>Fragmented documentation</strong></h4><p>Multiple projects and documentation sites create duplicate truths. Screenshots and side docs are a sign the official docs are not easy to use. We recommend to pick a single source of truth, route all edits through it, and remove duplicates over time.</p><h4>dbt Docs vs. Data Catalog: Choosing Your Source of Truth</h4><p>There are a few workable ways to run <a href="https://www.selectstar.com/resources/the-complete-guide-to-data-documentation">documentation at scale</a>. Start by choosing where your single source of truth lives: in dbt (schema.yml) or in your data catalog. Many teams choose a hybrid: draft in the catalog and publish back to dbt by pull request. Whatever you pick, write it down, set owners, and follow it consistently.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4WOf1zdprlgxe_bHs7ILZw.png" /></figure><p><strong>Approach 1: dbt as the source of truth.</strong> Keep authoritative table and column definitions in schema.yml. This keeps definitions versioned and close to the code, reviewed in pull requests, and tied to build changes. Many teams pair this with a catalog for drafting and discovery.</p><p><strong>Approach 2: catalog as the source of truth.</strong> Use a catalog UI for fast drafting and collaboration, and treat dbt Docs as a published view. This is easy to adopt but can drift unless you have a clear publish step back to dbt.</p><p><strong>Our recommendation: hybrid, governance first.</strong> Author in a catalog if you like, but make dbt the canonical record. Route changes through pull requests, publish accepted definitions back to schema.yml on a regular cadence (daily works well), and surface definitions in BI and lineage so people see them in context.</p><p>Govern change like code. <a href="https://www.selectstar.com/resources/data-ownership">Assign owners</a> for key models, propose edits through pull requests, and aim for small, frequent merges. Visibility matters too; most people will not open the docs site, so bring definitions to where they work.</p><p>Finally, measure the basics: documentation coverage, median PR merge time, and the volume of “what does this mean?” questions. These show whether the process is working and where to adjust.</p><h4>How Select Star’s dbt Docs Sync Keeps schema.yml Up-To-Date</h4><p>At Select Star, we recommend the hybrid approach, drafting in a catalog and keeping dbt as the record. Select Star’s data catalog supports this natively through <a href="https://docs.selectstar.com/integrations/dbt/dbt-docs-sync">dbt Docs Sync</a> functionality. A daily pull request updates schema.yml, approvals happen in Git or Bitbucket repository, and the same definitions appear in your catalog, BI, and lineage.</p><h4>1) Connect your repo and dbt to Select Star</h4><ul><li>Host your dbt project in GitHub or Bitbucket.</li><li>In Select Star, connect to dbt and point to the repo/project path.</li></ul><h4>2) Author &amp; validate in Select Star data catalog</h4><ul><li>Draft and edit table and column descriptions in Select Star. Take advantage of AI-powered documentation that can suggest descriptions based on metadata, SQL queries, existing documentation, and more.</li><li>Review and mark changes as validated.</li></ul><h4>3) Publish and review in a daily PR</h4><ul><li>Validated changes are batched into one scheduled daily pull request that updates schema.yml. If a previous PR is still open, dbt Docs Sync closes it and opens a fresh one to avoid conflicts.</li><li>Owners can review and merge. The change history is captured in the repo.</li></ul><p><strong>Example PR diff</strong></p><pre>​- description: &quot;Sum of line item totals&quot;<br>+ description: &quot;Sum of line item totals after discounts and tax&quot;</pre><p>Beyond documentation, a data catalog like Select Star supports everyday analytics and AI workflows. ERDs and column‑level lineage show how models and sources connect and help teams trace issues to the right upstream change, speeding up root‑cause analysis. Cross‑platform search spans dbt and your BI tools so people can find models, columns, dashboards, and definitions in one place, which reduces “what does this mean?” pings and shortens onboarding. Ownership and usage insights point to what to document, consolidate, or deprecate next, improving trust and lowering cost. Read more about what modern data catalog tools can offer in addition to scaling dbt Docs and how to evaluate them in our guide on <a href="https://www.selectstar.com/resources/data-catalog-tools">data catalog tools</a>.</p><h4>Keep dbt Documentation Up-to-Date and Discoverable</h4><p>With up-to-date documentation, the rest of the work gets easier. Definitions meet people where they work (BI tooltips, search, lineage), onboarding speeds up, and analysis is more consistent.</p><p>The data team at <a href="https://www.selectstar.com/resources/how-nib-enables-data-democratization">nib</a>, an Australian health and travel insurer serving more than 1.5 million people in Australia and New Zealand, had datasets created by multiple teams, which led to multiple dbt repositories and scattered dbt Docs. They wanted one searchable place for documentation. Select Star brought the docs from each repo into a single catalog and made datasets easy to find. Business users update definitions in Select Star, and approved changes sync back to dbt. <a href="https://www.selectstar.com/resources/how-nib-enables-data-democratization">Learn more.</a></p><p><a href="https://www.selectstar.com/demo">Book a demo</a> to see the daily PR workflow and Select Star’s dbt Doc Sync in action.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*z5IVn7mPJY8uRezXyRRBDQ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0fbe3f9a226" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/scaling-dbt-docs-with-an-automated-data-catalog-d0fbe3f9a226">Scaling dbt Docs with an Automated Data Catalog</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[​Automated Metadata Management on AWS with Select Star]]></title>
            <link>https://medium.com/selectstar/automated-metadata-management-on-aws-with-select-star-05b7fc6fd708?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/05b7fc6fd708</guid>
            <category><![CDATA[data-discovery]]></category>
            <category><![CDATA[redshift]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[metadata-management]]></category>
            <category><![CDATA[aws]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Tue, 30 Sep 2025 13:43:29 GMT</pubDate>
            <atom:updated>2025-10-14T15:52:24.487Z</atom:updated>
            <content:encoded><![CDATA[<p>In today’s digital age, data is more than just a resource — it’s the foundation upon which modern enterprises build their strategies, make decisions, and drive innovation. However, the sheer volume and complexity of data can often be overwhelming, leading to inefficiencies and missed opportunities. That’s where Select Star comes in. Available on the AWS Marketplace and Redshift Service Ready with a Data &amp; Analytics competency, Select Star is a cutting-edge <a href="https://www.selectstar.com/">metadata management platform</a> designed to help organizations manage, govern, and optimize their data with unprecedented ease and accuracy.</p><h4><strong>A Modern Solution for Modern Data Challenges</strong></h4><p>As businesses grow, so does their data. From customer interactions and transactional records to operational metrics and beyond, the amount of data generated daily is staggering. Managing this data effectively is crucial for businesses looking to gain insights, maintain compliance, and make data-driven decisions. Traditional methods of data cataloging — often manual and fragmented — are no longer sufficient. They are time-consuming, prone to errors, and unable to keep pace with the dynamic nature of modern data environments.</p><p>Select Star was created to address these challenges head-on. Founded over five years ago, Select Star has quickly established itself as a leader in metadata management, offering a platform that automates the <a href="https://www.selectstar.com/product/data-catalog">discovery</a>, <a href="https://www.selectstar.com/product/data-access-control">classification</a>, and <a href="https://www.selectstar.com/product/ask-ai">documentation</a> of data across an organization. By automating these processes, Select Star not only reduces the manual effort required but also enhances the accuracy, consistency, and accessibility of data.</p><h4><strong>Core Features of Select Star</strong></h4><h4><strong>Automated Metadata Processing</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L54mgyjWhJtp1KXVjrGctw.png" /></figure><p>At the heart of Select Star’s offering is its ability to <a href="https://www.selectstar.com/resources/metadata-ai">automatically process and manage metadata</a>. Metadata, the data about data, is essential for understanding and using data effectively. It includes information about the origin, structure, and usage of data, helping organizations to categorize and organize their data assets. Select Star automates the collection and processing of metadata, ensuring that it is always up-to-date and accurate. This automation eliminates the need for manual entry, reduces the risk of errors, and saves valuable time for data teams.</p><h4><strong>Data Lineage and Governance</strong></h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdemo.arcade.software%2F9Hmo1L7BhWjCxJ4jDty4&amp;display_name=Arcade&amp;url=https%3A%2F%2Fapp.arcade.software%2Fshare%2F9Hmo1L7BhWjCxJ4jDty4&amp;image=https%3A%2F%2Fapp.arcade.software%2Fog%2F9Hmo1L7BhWjCxJ4jDty4&amp;type=text%2Fhtml&amp;schema=arcade" width="1024" height="640" frameborder="0" scrolling="no"><a href="https://medium.com/media/ccf8753b4fb0d927670610e492a62741/href">https://medium.com/media/ccf8753b4fb0d927670610e492a62741/href</a></iframe><p><a href="https://www.selectstar.com/product/data-lineage">Data lineage</a>, the ability to trace the flow of data from its source to its final destination, is a critical component of data governance. It allows organizations to understand where their data comes from, how it is transformed, and how it is used across the enterprise. This visibility is essential for maintaining data quality, ensuring compliance with regulations, and making informed decisions. Select Star provides robust data lineage mapping, giving organizations a clear view of their data’s journey. This feature helps businesses maintain data integrity, mitigate risks, and comply with industry standards.</p><h4><strong>Utilization Scoring and Insights</strong></h4><figure><img alt="Select Star surfaces popularity and downstream counts so you can prioritize critical tables and deprecate unused ones." src="https://cdn-images-1.medium.com/max/846/0*MRdD63xJTK_-lNeQ.jpg" /><figcaption>Select Star surfaces popularity and downstream counts so you can prioritize critical tables and deprecate unused ones.</figcaption></figure><p>Understanding how data is being used within an organization is key to maximizing its value. Select Star offers utilization scoring, a feature that assigns scores to data based on its usage. This allows organizations to identify which data assets are most valuable and how they are being utilized. By gaining insights into data usage, businesses can prioritize the most critical data, optimize their data management strategies, and make better use of their resources. This feature is particularly valuable for large enterprises where data can often be siloed across departments, leading to inefficiencies and missed opportunities.</p><h4><strong>Flexible Deployment and Scalability</strong></h4><p>One of Select Star’s key strengths is its flexibility in deployment. Whether a business is a small startup or a large enterprise, Select Star can be deployed in a variety of environments — on-premises, in the cloud, or in a hybrid setup. This flexibility allows organizations to choose the deployment model that best suits their needs and infrastructure. Moreover, Select Star is designed to scale as the business grows, accommodating increasing data volumes and additional users without compromising performance or reliability.</p><h4><strong>Integration with AWS: A Seamless Experience</strong></h4><p>Select Star’s deep integration with AWS services enhances its functionality and makes it an ideal choice for organizations already using AWS. This integration allows businesses to leverage Select Star’s powerful features within their existing AWS environment, creating a seamless data management experience.</p><h4><strong>Amazon S3 for Data Storage</strong></h4><p>Select Star integrates with <a href="https://aws.amazon.com/pm/serv-s3/">Amazon S3</a>, one of the most popular and reliable cloud storage solutions. By storing data in S3, organizations can ensure that their data is secure, easily accessible, and compliant with industry standards. Select Star’s automated metadata processing works seamlessly with S3, making it easy to catalog and manage data stored in the cloud.</p><h4><strong>AWS Glue for Data Cataloging</strong></h4><p><a href="https://www.selectstar.com/integrations/aws-glue">AWS Glue</a> is another key integration that enhances Select Star’s capabilities. AWS Glue is a managed ETL (extract, transform, load) service that automates the process of discovering and cataloging data. By integrating with AWS Glue, Select Star can automatically discover, catalog, and manage data across the organization, reducing the complexity of data management and ensuring that data is always readily available for analysis.</p><h4><strong>Amazon Redshift for Data Warehousing</strong></h4><p>For organizations using <a href="https://www.selectstar.com/integrations/amazon-redshift">Amazon Redshift</a> for data warehousing, Select Star provides a powerful tool for managing and analyzing data. By cataloging data in Redshift, Select Star enables users to gain deeper insights into their data, make more informed decisions, and optimize their data warehousing strategy. This integration also simplifies the process of managing large data sets, ensuring that data is always accurate and up-to-date.</p><h4><strong>Real-World Metadata Applications and Success Stories</strong></h4><p>Select Star is not just a tool. It’s a solution that delivers tangible results for businesses across various industries. For example, a global financial services firm recently implemented Select Star to streamline their data management processes. The firm was struggling with manual data cataloging, which was time-consuming and prone to errors. By adopting Select Star, they were able to automate their metadata processing, improve data accuracy, and reduce the time spent on data management by 50%. This not only freed up resources for more strategic initiatives but also improved the reliability of their data, leading to better business decisions.</p><p>In another case, a large healthcare provider used Select Star to enhance their data governance practices. With the growing complexity of healthcare regulations, maintaining compliance was a significant challenge. Select Star’s data lineage mapping provided the visibility they needed to track data flows and ensure compliance with regulatory requirements. As a result, the provider was able to reduce compliance-related issues by 30%, while also improving their overall data management strategy.</p><h4><strong>Empowering Your Data Journey with Automated Metadata Management</strong></h4><p>In a world where data is king, having the right tools to manage and govern that data is crucial. Select Star offers a comprehensive, automated solution that simplifies data cataloging, enhances data governance, and provides valuable insights into data utilization. With its deep integration with AWS, flexible deployment options, and powerful features, Select Star is the ideal platform for organizations looking to unlock the full potential of their data.</p><p>To learn more about how Select Star can transform your data management practices, visit <a href="https://www.selectstar.com/demo">connect with our team</a> for a personalized demo. Discover how Select Star can help you turn your data into a strategic asset, driving innovation, compliance, and growth in your organization.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=05b7fc6fd708" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/automated-metadata-management-on-aws-with-select-star-05b7fc6fd708">​Automated Metadata Management on AWS with Select Star</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Select Star Joins Snowflake and Other Industry Leaders to Launch Open Semantic Interchange: Turn BI…]]></title>
            <link>https://medium.com/selectstar/select-star-joins-snowflake-and-other-industry-leaders-to-launch-open-semantic-interchange-turn-bi-8bed02c4fd33?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/8bed02c4fd33</guid>
            <category><![CDATA[semantic-model]]></category>
            <category><![CDATA[ai-ready-data]]></category>
            <category><![CDATA[semantic-layer]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Tue, 23 Sep 2025 16:41:04 GMT</pubDate>
            <atom:updated>2025-09-30T15:06:01.037Z</atom:updated>
            <content:encoded><![CDATA[<h3>Select Star Joins Snowflake and Other Industry Leaders to Launch Open Semantic Interchange: Turn BI Dashboards into AI-Ready Semantic Models</h3><p>​A semantic model centralizes business logic so every surface, BI and AI, uses the same meaning. Snowflake’s Open Semantic Interchange (OSI) is an ecosystem initiative to make those semantics portable across tools. Select Star is excited to be a launch partner supporting this mission to improve interoperability and accelerate outcomes for joint customers.</p><h4><strong>Why Most Teams Need a Semantic Layer</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SnJmJ4Af1ndqj2v_.jpg" /><figcaption><em>How a semantic layer sits between data sources in the physical data layer and BI and other analytics tools in the presentation layer (Source: </em><a href="https://moderndata101.substack.com/p/the-semantic-movement-the-story-of"><em>Modern Data 101</em></a><em>).</em></figcaption></figure><p>A semantic layer turns scattered logic (copied SQL in dashboards, conflicting metric definitions across BI tools, and transformation code spread across dbt/ETL pipelines) into shared understanding. In practice, that looks like a few concrete wins. Your teams get consistent answers across tools because each metric or dimension has one governed definition. You do less rework and fewer handoffs because business logic lives centrally instead of being copied across dashboards. You gain explainability and trust with end‑to‑end lineage and clear ownership, which speeds audits and incident triage. And you get portability for the future as definitions move between tools without lock‑in. Most importantly, the data context (owners, glossary terms, policies, and usage) travels with the definition so semantics are actually operational.</p><p>Shinji Kim, founder and CEO of Select Star, explains why semantic layers matter: “AI doesn’t just need data; it needs understanding. A semantic model provides that by defining shared metrics, terms, and relationships across your data. Without it, you risk inconsistent answers and a lack of clarity around how results are produced. It’s the context that grounds AI in how your business actually works.”</p><h4>Snowflake and Other Industry Leaders Partner on an Open, Interoperable Future: What OSI Means for Data Leaders</h4><p>Open Semantic Interchange (OSI) is an open source initiative led by Snowflake and ecosystem partners to create a universal, vendor‑neutral semantic model specification. Its goal is to standardize fragmented data definitions and make semantic metadata portable across tools and platforms. By enabling the exchange of semantics in a standard, open format, OSI improves interoperability and helps enterprises deliver consistent metrics and definitions across dashboards, notebooks, and machine learning models.</p><p>“The Open Semantic Interchange is a critical step toward unlocking a new era of interoperability across the data and AI ecosystem,” said Snowflake spokesperson. “By working with partners like Select Star, this initiative ensures a common, standardized understanding of semantic data across tools. This will enhance interoperability and vendor neutrality, providing organizations with greater flexibility and efficiency in building their data infrastructure, and simplifying data operations.”</p><p>In today’s mixed‑tool reality (multiple BI platforms, notebooks, and emerging AI apps), even “settled” KPIs drift as teams copy logic between systems or migrate tools. OSI provides a common way to package and exchange semantic metadata so they remain consistent wherever they are consumed. As a launch partner, Select Star is contributing real‑world feedback and use cases to help the initiative focus on day‑one outcomes such as KPI consistency, safer migrations, and AI readiness, without forcing a new modeling tool or vendor lock‑in.</p><h4><strong>Why data leaders should care</strong></h4><ul><li><strong>Consistent answers across BI and AI.</strong> Shared semantic metadata reduces drift so dashboards and assistants agree.</li><li><strong>Lower migration and modernization cost.</strong> Move definitions forward when consolidating or trialing new tools without re‑modeling from scratch.</li><li><strong>Faster time to adoption.</strong> Standardized interchange shortens the path from pilot to production for new analytics and AI surfaces.</li><li><strong>Explainability on day one.</strong> When semantics travels with lineage, owners, and definitions, audits and incident triage move faster.</li><li><strong>Ecosystem momentum.</strong> Vendors and partners align on a common way to interchange semantic metadata, reducing one‑off integrations.</li></ul><p>Learn more about the Open Semantic Interchange on <a href="https://www.snowflake.com/en/blog/open-semantic-interchange-ai-standard/">Snowflake’s blog</a>.</p><p>​<strong>How Select Star Delivers an Interoperable Semantic Layer for Snowflake</strong></p><p>Select Star is a metadata context platform that helps data teams turn the BI they already trust into an interoperable semantic layer for Snowflake. We reverse engineer dashboards and SQL from Looker, Tableau, Power BI, and other BI tools to generate governed metric and dimension definitions in days, not months. Each definition carries ownership, glossary terms, policies, and end-to-end lineage so answers are explainable and auditable. We help data teams reconcile duplicate KPIs, establish stewardship, and align semantics with Snowflake’s Open Semantic Interchange so they can travel across tools as OSI matures.</p><h4>How Select Star Autogenerates Semantic Models</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*93sf2Az5sJEgYaNdPR0ETQ.png" /><figcaption>Select Star turns your existing BI logic into a governed semantic model for Snowflake Intelligence, Cortex Analyst, and other AI tools.</figcaption></figure><ol><li><strong>Connect.</strong> You connect Select Star to your BI tool (Looker, Tableau, Power BI).</li><li><strong>Scan. </strong>Select Star automatically scans dashboards, worksheets, and metrics.</li><li><strong>Reverse-engineer. </strong>Select Star generates a semantic model with metric definitions and source lineage.</li><li><strong>Govern and extend. </strong>You can review, govern, and extend the yaml file.</li><li><strong>Publish. </strong>Make the semantic models available to Snowflake Cortex Analyst, ChatGPT, Claude, Cursor, or any AI tools right away.</li></ol><p>Explore more on semantic models with our post on <a href="https://www.selectstar.com/resources/snowflake-cortex-analyst">self-service data analytics and Snowflake Cortex Analyst</a>.</p><h4>Get Snowflake‑Ready Semantic Models from the BI You Already Have</h4><p>Teams like Faire, Datasembly, and Opendoor show what is possible when you leverage rich data context — data lineage, usage, ownership, and definitions — to ground analytics and AI. <a href="https://www.selectstar.com/case-studies/faire-slashes-data-pipeline-costs-with-snowflake-and-select-star">Faire</a> achieved ~70% pipeline cost reduction and ~80% fewer debugging hours on Snowflake by using lineage and usage insights to standardize work. <a href="https://www.selectstar.com/case-studies/datasembly">Datasembly</a> completes projects 90%+ faster and reports $30k+ annual Snowflake cost savings through context‑driven consolidation. <a href="https://www.selectstar.com/resources/amazon-quicksight-self-service-analytics">Opendoor</a> reports faster triage and self‑serve gains, along with major analytics cost reductions, by making governed metrics discoverable and trusted.</p><p>Ready to see this on your data without rebuilding from scratch? <a href="http://www.selectstar.com/demo">Connect with our team</a> and see your dashboards reverse‑engineered into a governed semantic model, complete with lineage and OSI‑aligned interoperability, so your teams get consistent, explainable answers in days, not months.</p><h4><strong>Join the Open Semantic Interchange (OSI) Initiative</strong></h4><p>Fragmented definitions are one of the biggest barriers to trustworthy AI. OSI brings industry leaders together around an open, vendor-neutral specification so definitions can travel across tools without drift. If you want to help shape a more open and connected data future, start with Snowflake’s official OSI announcement <a href="https://www.snowflake.com/en/blog/open-semantic-interchange-ai-standard/">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8bed02c4fd33" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/select-star-joins-snowflake-and-other-industry-leaders-to-launch-open-semantic-interchange-turn-bi-8bed02c4fd33">Select Star Joins Snowflake and Other Industry Leaders to Launch Open Semantic Interchange: Turn BI…</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Metadata is the New Oil: Fueling the AI-Ready Data Stack]]></title>
            <link>https://medium.com/selectstar/metadata-is-the-new-oil-fueling-the-ai-ready-data-stack-933f8678dd95?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/933f8678dd95</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-management]]></category>
            <category><![CDATA[metadata]]></category>
            <category><![CDATA[data]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Tue, 23 Sep 2025 15:27:48 GMT</pubDate>
            <atom:updated>2025-09-23T15:27:48.819Z</atom:updated>
            <content:encoded><![CDATA[<p>​We’ve all heard the saying: “Data is the new oil.” But like oil, raw data isn’t valuable on its own. It needs context to be useful, something to help us understand what it is, where it came from, and how to apply it. That’s where metadata comes in.</p><p>Metadata provides the context for AI tools and systems to understand your data’s meaning and structure, including things like lineage, usage, and ownership. This kind of context is essential for building AI-ready data environments, enabling AI to interpret, navigate, apply, and govern data more effectively across complex environments.</p><p>As data teams scale, dashboards multiply, and AI tools begin generating SQL and insights, metadata has become the most important (and most underinvested) layer in the modern data stack. It tells you what data is being used, how it’s connected, where it came from, and whether you can trust it. Without it, you’re flying blind.</p><p>This perspective was central to a recent episode of the <a href="https://softwareengineeringdaily.com/2025/09/04/context-aware-sql-and-metadata-with-shinji-kim/">Software Engineering Daily podcast</a>, where Select Star founder and CEO <a href="https://www.linkedin.com/in/shinjikim">Shinji Kim</a> discussed the growing importance of metadata in enabling trust, discovery, and AI accuracy.</p><p>In this post, we’ll explore how metadata follows a similar lifecycle to oil: it must be extracted, refined, applied, and governed. And just like oil powered the industrial age, metadata is powering the AI-driven data era.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fopen.spotify.com%2Fembed%2Fepisode%2F6rtUFXvZblxXEVEtJX7gfm%3Futm_source%3Doembed&amp;display_name=Spotify&amp;url=https%3A%2F%2Fopen.spotify.com%2Fepisode%2F6rtUFXvZblxXEVEtJX7gfm&amp;image=https%3A%2F%2Fimage-cdn-ak.spotifycdn.com%2Fimage%2Fab67656300005f1f2cce417b7df660be6599a9ab&amp;type=text%2Fhtml&amp;schema=spotify" width="456" height="152" frameborder="0" scrolling="no"><a href="https://medium.com/media/056fdc5af229abaf1c1df43de5113e74/href">https://medium.com/media/056fdc5af229abaf1c1df43de5113e74/href</a></iframe><h3>​Extracting Metadata: Capturing Context from the Modern Data Stack</h3><p>Metadata used to mean <a href="https://www.selectstar.com/resources/the-complete-guide-to-data-documentation">documentation</a>. Today, it’s so much more.</p><p>At its core, metadata is information about your data. It includes things like table names, column descriptions, lineage, usage patterns, popularity, and ownership. Good metadata gives teams the context they need to understand how data is structured, where it flows, and how it’s used throughout the organization.</p><p>Every query that gets run, dashboard that gets loaded, or pipeline that gets deployed creates metadata: behavioral signals, lineage, usage patterns, and ownership trails. The challenge isn’t generating metadata; it’s capturing it in a structured, scalable way.</p><p>Modern metadata platforms extract this context passively across your stack:</p><ul><li><strong>Query logs</strong> show how data is accessed and joined</li><li><strong>BI tools</strong> reveal which dashboards are actually being used (and by whom)</li><li><strong>Transformation layers</strong> trace how raw fields become metrics</li></ul><p>This extraction phase builds the foundation for your internal knowledge graph. It connects people, assets, and activity into something you can reason about. Select Star builds its metadata knowledge graph by observing query activity, analyzing usage patterns, and mapping relationships across data systems.</p><h3>​Refining Metadata: Turning Raw Signals into Actionable Context</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdemo.arcade.software%2F9Hmo1L7BhWjCxJ4jDty4&amp;display_name=Arcade&amp;url=https%3A%2F%2Fapp.arcade.software%2Fshare%2F9Hmo1L7BhWjCxJ4jDty4%3Fref%3Dshare-link&amp;image=https%3A%2F%2Fapp.arcade.software%2Fog%2F9Hmo1L7BhWjCxJ4jDty4&amp;type=text%2Fhtml&amp;schema=arcade" width="1024" height="640" frameborder="0" scrolling="no"><a href="https://medium.com/media/81ade2d486fb5a9e8ddae53e0c361b4f/href">https://medium.com/media/81ade2d486fb5a9e8ddae53e0c361b4f/href</a></iframe><p>Raw metadata is like crude oil: messy, inconsistent, and overwhelming. To be useful, it must be refined.</p><p>Refining metadata means:</p><ul><li><strong>Scoring popularity</strong>, so teams can prioritize what matters</li><li><strong>Tracing lineage</strong>, so people understand impact and dependencies</li><li><strong>Modeling semantics</strong>, so metrics and terms are consistent across tools</li><li><strong>Auto-documenting</strong>, so tribal knowledge becomes institutional memory</li></ul><p>Once refined, metadata becomes more than documentation. It becomes the connective tissue of your data workflows. It helps analysts choose the right table. It lets AI tools generate more accurate SQL. It shows stakeholders where their KPIs come from.</p><p>Shinji Kim outlines metadata in three critical levels: physical metadata, which includes structural information like table names and column types; usage patterns, which capture how data is accessed and queried; and business context, which includes semantic layers, metric definitions, and ownership. Each level plays a distinct role in enabling trust, discovery, and automation across the data stack</p><h3>​Applying Metadata: Building Trust, Enabling Automation, and Improving AI Accuracy</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/869/0*WIxSIsSBD8GoPOU5.png" /></figure><p>So what can you actually <em>do</em> with good metadata? Consider the common situation where six dashboards all show slightly different “active users” numbers. With high-quality metadata in place, you can quickly see which metric is most commonly used, trace the data powering it, understand how it’s calculated, and identify who is responsible for maintaining it.</p><p>That level of clarity is just one example. Metadata delivers value across the stack in many ways:</p><ul><li><a href="https://www.selectstar.com/case-studies/pennant-services-m-a-data-management"><strong>Build trust</strong></a>: Teams stop second-guessing dashboards when lineage and definitions are visible</li><li><a href="https://www.selectstar.com/case-studies/how-productboard-is-increasing-productivity-by-80-with-select-star"><strong>Accelerate onboarding</strong></a>: New analysts find what they need without pinging five people</li><li><a href="https://www.selectstar.com/resources/amazon-quicksight-self-service-analytics"><strong>Enable self-service</strong></a>: Business users explore with confidence when context is embedded</li><li><a href="https://www.selectstar.com/resources/snowflake-cost-optimization"><strong>Reduce cost</strong></a>: By identifying unused or duplicated assets, you can clean the house</li><li>​<a href="https://www.selectstar.com/resources/text-to-sql-llm"><strong>Improve AI accuracy</strong></a>: Metadata helps LLMs avoid hallucinations by narrowing context</li></ul><p>When metadata is structured and reliable, large language models can understand schemas, follow lineage, and generate queries that reflect how your business actually works. This kind of context is essential for analytics workflows and especially critical for AI tools that need to reason over real, production-grade data.</p><p>From onboarding to AI enablement, companies are already seeing the impact of applied metadata. <a href="https://www.selectstar.com/customers">Select Star’s case studies</a> show how it works in practice.</p><h3>​Metadata Governance: Keeping AI and Analytics Reliable at Scale</h3><p><a href="https://www.selectstar.com/resources/how-alphasense-harnessed-metadata-to-control-dashboard-sprawl">Metadata</a> also plays a critical role in preventing failure. When metadata is out of date, dashboards may still load, but they often show the wrong numbers or rely on outdated logic. AI systems may generate incorrect queries. Teams are left making decisions based on incomplete or outdated context.</p><p>This challenge is made worse by the fact that so much metadata still lives in tribal knowledge, spread across Slack threads, Notion pages, or in someone’s head. As organizations layer in AI agents and automation, the cost of stale or missing context compounds. What begins as a minor oversight can quickly ripple into system-wide issues.</p><p>As Shinji pointed out, good metadata not only defines context, but it also helps maintain context. Good metadata keeps semantic layers in sync, flags downstream impact when things change, and ensures teams aren’t left guessing when the data shifts underneath them.</p><p>Good governance goes beyond avoiding failure and keeps metadata accurate and usable without adding unnecessary overhead. Select Star supports this by automatically propagating tags, updating documentation, assigning ownership, and monitoring changes to lineage and usage patterns. These automated workflows help teams stay aligned and scale governance across the organization.</p><h3>Metadata as the New Oil</h3><p>Metadata isn’t just technical exhaust. It’s fuel. It powers modern data governance, drives self-service analytics, and enables AI to work with precision. Like oil, metadata becomes exponentially more valuable when it’s extracted, refined, and strategically applied across your stack.</p><p>For data teams operating in fast-paced, AI-driven environments, metadata offers a competitive advantage. It ensures analysts can find trusted assets, stakeholders understand metrics, and AI tools generate accurate results. Teams that invest in metadata aren’t just cleaning up documentation. They’re building AI-ready data systems that are faster, more resilient, and ready to scale with confidence.</p><p>In a world where AI is only as good as the context you give it, metadata has become the new oil.</p><p>Let’s talk metadata. <a href="https://www.selectstar.com/">Connect with the Select Star team</a> and see what we can unlock together.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=933f8678dd95" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/metadata-is-the-new-oil-fueling-the-ai-ready-data-stack-933f8678dd95">Metadata is the New Oil: Fueling the AI-Ready Data Stack</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Best Metadata Management Tools for Analytics and AI]]></title>
            <link>https://medium.com/selectstar/best-metadata-management-tools-for-analytics-and-ai-43d05dfbc52b?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/43d05dfbc52b</guid>
            <category><![CDATA[data-lineage]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[metadata-management]]></category>
            <category><![CDATA[data-catalog]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Sun, 21 Sep 2025 20:59:36 GMT</pubDate>
            <atom:updated>2025-10-07T15:31:54.258Z</atom:updated>
            <content:encoded><![CDATA[<p>​AI assistants and LLMs are only as good as the data they rely on. Without consistent metadata (representation of data lineage, definitions, ownership, and usage), AI answers are often incomplete at best and misleading at worst. Metadata management tools give data teams the foundation they need to make analytics and AI reliable, explainable, and compliant.</p><p>This guide explains how today’s best metadata management platforms provide that context layer: what features matter, how they differ from platform-native options, and where open source fits. Along the way, we’ll highlight practical benefits, compare categories, and share how organizations using Select Star have accelerated trust and self-service by making metadata central to their data strategy.</p><h4>Why Metadata Management Tools Matter Now</h4><p>Data stacks have evolved. In the past, many organizations ran analytics on a single platform. Today, most rely on a mix of cloud services, databases, and specialized tools. This variety creates flexibility but also introduces complexity: metadata is scattered across disconnected systems.</p><p>Without proper metadata management, teams face:</p><ul><li>Duplicate assets and conflicting reports</li><li>Unclear or missing definitions</li><li>Gaps in ownership and accountability</li><li>Compliance blind spots and audit delays</li></ul><p>For analytics teams, these problems compound, and they directly impact the accuracy of AI tools built on top of company data. Models are trained on inconsistent inputs, reports lose credibility, and compliance risks grow.</p><p>Metadata management tools address these challenges by making modern data stacks more transparent and trustworthy. Dashboards can pull from approved, well-defined datasets so analysts trust the numbers they see. Audits move faster because lineage and ownership are already documented. Most importantly, data becomes a reliable business asset, <a href="https://www.selectstar.com/resources/metadata-ai">fuel for accurate analytics and dependable AI</a> instead of a source of doubt.</p><h4>​Top Metadata Management Tools for Analytics and AI in 2025</h4><p>As more organizations rely on AI assistants and automated analytics, the choice of AI-ready metadata tools determines how accurate and explainable those outputs will be. The right tool ensures that analysts can discover the right datasets quickly, business leaders trust the metrics they see, and AI models are trained on consistent, well-documented inputs.</p><p>While the market for metadata tools can feel crowded, most options fall into three clear categories. Below we’ll break down each type, the problems they solve, and the trade-offs to consider, so you can decide which approach best fits your analytics and AI strategy.</p><h4>End-to-End Metadata Management Tools</h4><p>End-to-end metadata management platforms, often referred to as enterprise metadata catalog software, are built to support modern, complex data stacks and provide visibility across multiple systems. They bring together features like automated discovery, cross-platform lineage, and AI-powered search in one environment. By capturing and connecting metadata from different warehouses, BI tools, and pipelines, these platforms give analysts faster ways to find trusted datasets and help explain AI outputs with clear traceability.</p><p>They also include strong governance and access controls, making them well-suited for organizations that must meet regulatory requirements or want to enforce consistent standards across teams. Because they are vendor-agnostic, end-to-end platforms offer flexibility for companies with diverse technologies and evolving strategies.</p><p>This category is best suited for enterprises or fast-growing organizations that need interoperability across platforms, a governed foundation for analytics, and AI-ready metadata without being locked into a single ecosystem. Here are some of the leading enterprise metadata management platforms available in 2025:</p><ol><li><a href="http://selectstar.com/">Select Star</a></li><li><a href="http://alation.com/">Alation</a></li><li><a href="http://atlan.com/">Atlan</a></li><li><a href="https://coalesce.io/">Coalesce</a></li><li><a href="http://collibra.com/">Collibra</a></li><li><a href="https://dataedo.com/">Dataedo</a></li><li><a href="https://www.datagalaxy.com/en/product/data-catalog/">DataGalaxy</a></li><li><a href="https://data.world/">data.world</a></li><li><a href="https://www.informatica.com/">Informatica</a></li><li><a href="http://secoda.co/">Secoda</a></li></ol><h4>Platform-Native Metadata Tools</h4><p>Platform-native metadata features, sometimes called cloud-native metadata governance tools, are built directly into cloud warehouses and lakehouse platforms like Snowflake, BigQuery, Databricks, or AWS Glue. They provide basic cataloging, tagging, and lineage within the boundaries of a single ecosystem, offering a fast way to get started without introducing new vendors or systems.</p><p>Because these features are tightly integrated with their host platforms, setup is minimal, and the user experience feels seamless. Analysts and engineers can quickly search tables, review lineage, and apply access controls without leaving their core environment. However, the trade-off is limited reach: these tools rarely extend beyond their parent platform, making it difficult for teams with hybrid or multi-cloud stacks to gain a full picture of their data landscape.</p><p>This category is best for organizations heavily invested in one cloud ecosystem that want straightforward, low-overhead metadata management, but may outgrow the limitations as their stack diversifies.</p><ol><li><a href="https://docs.aws.amazon.com/prescriptive-guidance/latest/serverless-etl-aws-glue/aws-glue-data-catalog.html">AWS Glue Data Catalog</a></li><li><a href="https://www.databricks.com/product/unity-catalog">Databricks Unity Catalog</a></li><li><a href="https://cloud.google.com/dataplex">Google Cloud Dataplex</a></li><li><a href="https://www.microsoft.com/en-us/security/business/microsoft-purview">Microsoft Purview</a></li><li><a href="https://www.oracle.com/middleware/technologies/enterprise-metadata-management.html">Oracle Enterprise Metadata Management</a></li><li><a href="https://www.sap.com/products/data-cloud/data-intelligence/features.html">SAP Data Intelligence</a></li><li><a href="https://www.snowflake.com/en/product/features/horizon/">Snowflake’s Horizon Catalog</a></li></ol><h4>Open Source Metadata Tools</h4><p>Open source metadata management tools provide flexibility and control for teams that want to customize their approach. Many are evolving into AI-ready metadata tools with lineage tracking, glossary support, and connectors designed for modern data stacks. Maintained by active developer communities, these projects evolve quickly with new integrations, features, and connectors that can be adapted to fit unique workflows.</p><p>For analytics teams, open source tools can deliver robust lineage tracking, discovery, and glossary functionality without licensing costs. They are particularly appealing for organizations that want to avoid vendor lock-in or experiment with emerging approaches to metadata and AI readiness. However, the freedom comes with responsibility: successful deployment requires strong engineering resources to manage hosting, upgrades, and ongoing support.</p><p>This category is best for technically mature teams with the bandwidth to build and maintain their own solutions, and for organizations that value customization and community-driven innovation over turnkey simplicity.</p><ol><li><a href="https://www.amundsen.io/">Amundsen</a></li><li><a href="https://atlas.apache.org/">Apache Atlas</a></li><li><a href="https://datahub.com/">DataHub</a></li><li><a href="https://egeria-project.org/">Egeria</a></li><li><a href="https://github.com/Netflix/metacat">Metacat</a></li><li><a href="https://open-metadata.org/">OpenMetadata</a></li></ol><h4>Key Features to Look for When Selecting a Metadata Management Solution</h4><p>Choosing the right platform starts with the must‑have metadata management features that set leading tools apart. These capabilities go beyond basic cataloging to address the real pain points analytics teams face every day: time wasted hunting for the right dataset, dashboards that contradict each other, and AI models trained on incomplete or inconsistent information.</p><p>The features below are table stakes for <a href="https://www.selectstar.com/resources/metadata-management-for-ai">modern metadata management</a>. Whether delivered through AI-ready metadata tools or metadata discovery platforms, these capabilities improve analyst speed, raise trust in dashboards, and make AI outputs more accurate and explainable.</p><p><strong>Automated Metadata Discovery</strong></p><ul><li>What it means: Continuously scans connected systems to capture tables, columns, schemas, tags, and descriptions.</li><li>Why it matters: Keeps metadata accurate at scale and eliminates manual upkeep.</li></ul><p><a href="https://www.selectstar.com/product/data-lineage"><strong>Cross-Platform Data Lineage</strong></a></p><ul><li>What it means: Shows how data flows, transforms, and connects across systems, down to the column level.</li><li>Why it matters: Builds trust, simplifies audits, and supports impact analysis by showing dependencies and traceability.</li></ul><p><a href="https://www.selectstar.com/resources/business-glossary-data-governance"><strong>Business Glossary</strong></a></p><ul><li>What it means: A governed space to define metrics, dimensions, and business terms, then link them to physical assets.</li><li>Why it matters: Aligns teams on consistent definitions, reduces errors in reporting, and saves time spent debating terms.</li></ul><p><a href="https://www.selectstar.com/solutions/ai-ready-data"><strong>Search and AI Assistants</strong></a></p><ul><li>What it means: Enables natural language search and AI-powered recommendations across datasets and assets.</li><li>Why it matters: Speeds up discovery, empowers self-service, and helps AI tools return accurate, explainable answers.</li></ul><p><strong>Usage Analytics</strong></p><ul><li>What it means: Surfaces query history, dashboard views, ownership, recency, and popularity signals.</li><li>Why it matters: Guides users and AI toward trusted, high-value datasets while flagging stale or unused assets to cut costs.</li></ul><p><a href="https://www.selectstar.com/solutions/change-management"><strong>Impact Analysis</strong></a></p><ul><li>What it means: Detects downstream breakage risk from schema or model changes, then alerts owners and consumers.</li><li>Why it matters: Prevents broken dashboards and failed AI pipelines by catching issues before they cascade.</li></ul><p><a href="https://www.selectstar.com/product/data-access-control"><strong>Governance</strong></a></p><ul><li>What it means: Supports fine-grained access control, PII tagging, and audit trails.</li><li>Why it matters: Reduces regulatory risk by providing verifiable evidence for audits, and clarifies ownership and accountability.</li></ul><p><a href="https://www.selectstar.com/integrations"><strong>Integration Coverage</strong></a></p><ul><li>What it means: Offers connectors and APIs for warehouses, BI, ETL, orchestration, and ML platforms.</li><li>Why it matters: Provides a unified view of all data to reduce blind spots and ensure all assets are discoverable and trustworthy.</li></ul><h4>How Leading Teams Use Metadata Management</h4><p>Metadata management goes far beyond storing technical details. The right tools deliver measurable results for real companies. Here are a few examples:</p><ul><li><strong>Insurance</strong>: <a href="https://www.selectstar.com/case-studies/how-nib-achieved-data-discovery">nib</a>, a leading Australian health insurer, used automated discovery and lineage tracking to improve visibility, remove duplicates, and stay compliant, achieving a 90% increase in efficiency.</li><li><strong>Finance</strong>: <a href="https://www.selectstar.com/resources/snowflake-dynamic-data-masking">Rise Analytics</a>, an analytics platform for credit unions, adopted metadata management to improve visibility, track sensitive data, and simplify audits. The result: higher adoption and immediate impact for analysts and compliance teams.</li><li><strong>Manufacturing</strong>: <a href="http://selectstar.com/case-studies/how-xometry-uses-select-stars-column-level-lineage-to-eliminate-data-outages">Xometry</a>, a global manufacturing marketplace, implemented column-level lineage across pipelines. They saved 200+ engineering hours and achieved 36× faster debugging.</li><li><strong>Real Estate</strong>: <a href="https://www.selectstar.com/resources/amazon-quicksight-self-service-analytics">Opendoor</a>, a digital real estate platform that simplifies buying and selling homes, enabled self-service analytics in Amazon QuickSight, giving business users direct access to trusted data without constant support from engineers.</li></ul><p>These case studies highlight how metadata management tools like Select Star enable faster discovery, stronger compliance, and more accurate analytics across industries.</p><h4>Preparing Your Data Stack for AI with Metadata</h4><p>Metadata management is a foundation for trust, governance, and AI-readiness today. Without it, teams waste time, lose trust, and face compliance risk. Now is the right time to evaluate your current practices: identify gaps, set priorities, and explore tools that fit your ecosystem and resources.</p><p>Start small with a high‑value use case. Demonstrate ROI early, then expand gradually to other areas of the business. Modern metadata tools make this process far easier than manual cataloging — shortening audits, reducing risk, and accelerating AI readiness. Even one well‑executed project can fast‑track analytics and prepare your data for AI initiatives.</p><p>Want to see this in action? <a href="https://www.selectstar.com/demo">Request a demo</a> to discover how Select Star can help your team turn scattered metadata into a trusted, searchable data foundation in just days.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sHeK0P63rOhO4HPJ.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=43d05dfbc52b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/best-metadata-management-tools-for-analytics-and-ai-43d05dfbc52b">Best Metadata Management Tools for Analytics and AI</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Entity Relationship Diagrams (ERDs) vs. Data Lineage]]></title>
            <link>https://medium.com/selectstar/entity-relationship-diagrams-erds-vs-data-lineage-2ae088e7920b?source=rss-509053c0c088------2</link>
            <guid isPermaLink="false">https://medium.com/p/2ae088e7920b</guid>
            <category><![CDATA[data-lineage]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[entity-relationship]]></category>
            <category><![CDATA[data-catalog]]></category>
            <category><![CDATA[erd]]></category>
            <dc:creator><![CDATA[Select Star]]></dc:creator>
            <pubDate>Wed, 03 Sep 2025 15:11:42 GMT</pubDate>
            <atom:updated>2025-10-01T21:46:41.451Z</atom:updated>
            <content:encoded><![CDATA[<p>Data-driven businesses gather and store massive amounts of data on every part of their operation. As data volumes increase, data models become more complex, and finding and understanding data for timely decision-making is much more difficult.</p><p>Entity relationship diagrams (ERDs) and data lineage are essential tools for understanding and managing complex datasets. While both improve data visibility, they serve different purposes: ERDs visualize how data tables relate to each other, while data lineage shows how data moves and transforms across systems.</p><p>At Select Star, we’ve seen firsthand how automated column-level lineage and ERDs accelerate onboarding, reduce debugging time, and streamline governance workflows at companies like <a href="https://www.selectstar.com/resources/column-level-data-lineage-examples">Xometry, Faire, and others</a>. In this post, we’ll explain what ERDs and data lineage are, highlight their differences and use cases, and explain how each supports better data governance and analytics.‍</p><h3>What Are ERDs and Data Lineage? Definitions and Use Cases</h3><h3>What Are Entity Relationship Diagrams (ERDs)?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3v33WONRJq20paI8" /><figcaption>Example of an ERD (Source: <a href="https://www.lucidchart.com/pages/er-diagrams">Lucidchart</a>)</figcaption></figure><p>An <a href="https://www.selectstar.com/product/entity-relationship-diagrams">Entity Relationship Diagram (ERD)</a> visually maps how tables in a relational database are connected through relationships such as primary and foreign keys. This helps analysts and engineers understand how to join tables and build accurate queries.</p><p>Traditionally, ERDs were created by data architects to enforce relational data models. They define:</p><ul><li><strong>Primary keys</strong> — columns that uniquely identify rows in a table.</li><li><strong>Foreign keys</strong> — columns that reference primary keys in other tables to establish relationships.</li></ul><p>These constraints ensure data integrity and enable efficient querying. But with the rise of modern cloud data warehouses like Snowflake, Databricks, and BigQuery, which often don’t require enforced keys, many organizations lack up-to-date ERDs or data models altogether.</p><p>Creating ERDs manually in these environments is time-consuming. That’s why modern tools like Select Star can automatically infer ERDs by analyzing SQL query history, identifying common join patterns and underlying relationships.</p><h3>What Is Data Lineage?</h3><p><a href="https://www.selectstar.com/product/data-lineage">Data lineage</a> tracks how data moves and transforms across your data ecosystem, from raw ingestion to transformations, modeling, and reporting. It answers critical questions like:</p><ul><li>Where did this data come from?</li><li>What processes or queries changed it?</li><li>Which dashboards or models depend on it?</li></ul><p>Understanding lineage is essential for:</p><ul><li><strong>Change management</strong> — knowing what downstream assets are affected.</li><li><strong>Data governance</strong> — ensuring compliance and traceability.</li><li><strong>Data quality</strong> — identifying root causes of issues.</li></ul><p>While lineage can be manually documented, platforms like Select Star can <a href="https://www.selectstar.com/resources/the-complete-guide-to-data-lineage-benefits-techniques-and-best-practices">automatically extract lineage</a> by parsing metadata and SQL query logs, capturing both table-level and column-level dependencies.</p><p>Before we dig into comparing the tools, and explaining why they’re best when used together, here’s a quick overview of their similarities and differences:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KGxO38G2YsWGvyAGMNchfg.png" /></figure><h3>Deep Dive: Entity Relationship Diagrams (ERDs) and Data Modeling</h3><p>Entity relationship diagrams, or ERDs, are an essential part of data modeling and architecture. They show the relationships between different data assets, making it easy to understand which disparate datasets have been combined and how they are joined together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_bUlt5ald6UvFB_E.png" /><figcaption><em>ERD shows which datasets are joined together and which fields they are joined on</em></figcaption></figure><p>In a relational database, entities correspond to tables. The columns in the table — the information the organization wants to collect — are called attributes. For example, if a college has an entity called Students, within the entity would be attributes like Student Name and Student Major. Courses would be a different entity. Within that entity would be attributes like Instructor and Prerequisites.</p><p>Relationships are how entities interact. They might be one-to-one, one-to-many, or many-to-many. Each student has multiple courses, but only one major. An instructor might teach only one course but be associated with 100 students. An entity relationship diagram puts all this information in a clear, visual format. It’s a bird’s-eye view that facilitates database design and debugging, because redundancies and design flaws are easy to spot.</p><p>ERDs also enhance internal communication. The graphic nature of an ERD eliminates the confusion that might arise between stakeholders who talk about the same topics using different vocabulary.</p><h3>Deep Dive: Data Lineage and Data Model Dependencies</h3><p>Data lineage tracks data’s origin and transformations, ensuring reliability, accuracy, and quality. It’s like a personal history of each unit of data.</p><p>While table-level data lineage only tracks data’s movement between tables, column-level data lineage is a fine-grained map that provides context for each table, column, and dashboard between your data warehouse and your BI tools.</p><p>When you understand data’s origin, ownership, transformations, and dependencies, you can trace problems to their source. You can also prevent problems by identifying potential downstream impacts before implementing a change.</p><p>Data lineage creates confidence that your single source of truth is always accurate and up-to-date.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fdemo.arcade.software%2F9Hmo1L7BhWjCxJ4jDty4&amp;display_name=Arcade&amp;url=https%3A%2F%2Fapp.arcade.software%2Fshare%2F9Hmo1L7BhWjCxJ4jDty4&amp;image=https%3A%2F%2Fapp.arcade.software%2Fog%2F9Hmo1L7BhWjCxJ4jDty4&amp;type=text%2Fhtml&amp;schema=arcade" width="1024" height="640" frameborder="0" scrolling="no"><a href="https://medium.com/media/ccf8753b4fb0d927670610e492a62741/href">https://medium.com/media/ccf8753b4fb0d927670610e492a62741/href</a></iframe><p>‍A common way to present lineage is with a lineage graph, which visualizes the flow of data across your data pipelines. While this can be helpful to understand how data traverses from asset to asset, it can also be helpful to see downstream dependencies listed out in a more hierarchical view to quickly digest larger amounts of information.</p><h3>Data Lineage Use Cases</h3><p>Data lineage makes efficient data management and governance possible. The most common use cases for column-level data lineage include:</p><ul><li><strong>Root cause analysis</strong>: When a pipeline breaks, data lineage enables teams to quickly track down the transformation that caused the problem and fix it.</li><li><strong>Impact analysis</strong>: Data lineage shows the data team exactly how data is being used downstream, so they can head off pipeline issues before making a change.</li><li><strong>Automated data maintenance</strong>: Column-level data lineage allows companies to automate routine maintenance of their data catalog, ensuring data remains fresh, accurate, and compliant.</li><li><strong>Audits</strong>: Auditors can easily track and verify data origins and movement and can understand its context, supporting data integrity.</li><li><strong>Database optimization</strong>: Column-level lineage helps teams identify rarely-used columns that can be archived or deprecated. Streamlining and removing obsolete columns optimizes storage costs and leads to faster queries.</li></ul><h3>Comparing ERDs vs. Data Lineage: Key Differences and Use Cases</h3><p>ERDs and data lineage are different tools with a common goal: to enhance the understanding, management, and effective use of data within the organization data team.</p><p>Both tools offer valuable context to enable quick, strategic decision-making. Both present information in a logical manner that simplifies troubleshooting and compliance. Both provide visibility into the structure and flow of data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UDzWJ-X2kuJiuFIVBFHZZQ.png" /></figure><p>However, each tool takes its own approach to these functions, and each presents the data in its own form.‍</p><h3>Leveraging the combined power of data lineage and ERDs</h3><p>Together, ERDs and data lineage help organizations effectively manage their data. Combining them gives teams a clear understanding of data’s structure, relationships, and dependencies, improving the ability to make timely, strategic business decisions.</p><h3>ERDs and Data Lineage for Data Governance</h3><p>Column-level data lineage is the backbone of <a href="https://www.selectstar.com/solutions/data-governance">data governance</a>. It supports regulatory compliance by providing the traceability to data’s origin and how it has transformed between your data warehouse and BI tools. When errors occur, data lineage makes it possible to trace the problem back to the source.</p><p>ERDs enhance governance by making the entire data model clear, even for less-technical stakeholders. This clarity fosters productive communication about data concepts such as quality and data privacy. They also act as an early warning system — making redundancies and entanglements apparent before they have a chance to cause data errors. This helps organizations streamline data management and operations with accurate data.</p><h3>ERDs and Data Lineage for Data Documentation</h3><p>As a company grows and <a href="https://www.selectstar.com/solutions/data-democratization">data is democratized</a>, new tables are added and data models evolve. Manually establishing column-level lineage or ERDs is resource-intensive and unsustainable due to the dynamic nature of data models keep changing. Keeping up with these changes manually becomes impossible as companies expand.</p><p>With modern data governance platforms like Select Star, data teams are finally able to <a href="https://www.selectstar.com/resources/the-complete-guide-to-data-documentation">automate documentation</a> of key relationships across datasets. These insights not only help sharing knowledge and context within the data team, but also allow automating the data governance and data management workflows, such as data classification and notifications.</p><h3>Why You Need Both ERDs and Data Lineage</h3><p>Together, ERDs and data lineage provide the structural and operational context needed to manage data effectively. ERDs help you visualize the relationships between datasets, while data lineage reveals how data moves, transforms, and is used across your stack. Used in tandem, they empower faster decision-making, better collaboration, and more confident data governance.</p><p>Want to see it in action? <a href="https://www.selectstar.com/demo">Book a demo</a> to learn how Select Star helps automate ERDs and data lineage for your data team.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2ae088e7920b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/selectstar/entity-relationship-diagrams-erds-vs-data-lineage-2ae088e7920b">Entity Relationship Diagrams (ERDs) vs. Data Lineage</a> was originally published in <a href="https://medium.com/selectstar">Select Star</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>