<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Yu Ishikawa on Medium]]></title>
        <description><![CDATA[Stories by Yu Ishikawa on Medium]]></description>
        <link>https://medium.com/@yu-ishikawa?source=rss-11f93f9d3bca------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*J03GgQ9qI-G3qgZB0CzMyw.jpeg</url>
            <title>Stories by Yu Ishikawa on Medium</title>
            <link>https://medium.com/@yu-ishikawa?source=rss-11f93f9d3bca------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 20:56:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@yu-ishikawa/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Calling Typed AI from SQL: Jev, dbt, and DuckDB]]></title>
            <link>https://yu-ishikawa.medium.com/calling-typed-ai-from-sql-jev-dbt-and-duckdb-7aa7340155e3?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/7aa7340155e3</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[analytics-engineering]]></category>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[jevs]]></category>
            <category><![CDATA[duckdb]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Sat, 19 Sep 2026 06:59:42 GMT</pubDate>
            <atom:updated>2026-09-19T07:01:32.265Z</atom:updated>
            <content:encoded><![CDATA[<h3>Calling Typed AI from SQL: Jev, dbt, and DuckDB</h3><p><strong>Subtitle:</strong> A small Jaffle Shop experiment — wiring pattern, not a benchmark.</p><p>I wanted to see whether typed AI could live where analysts already work: inside SQL and dbt, not in a separate notebook or microservice.</p><p>This post walks through a minimal integration on the Jaffle Shop demo dataset: a Python UDF registered through <strong>dbt-duckdb</strong>, calling <strong>Jev</strong> (TypeSafe <a href="https://docs.typesafe.ai/">System One</a>), and materializing structured marketing fields into a normal dbt mart.</p><p>The dataset has about 100 customers. I scored the top 20 by order count. That is enough to prove the wiring works. It is not enough to prove the model is correct or that per-row inference belongs in production.</p><h3>What we are proving (and what we are not)</h3><p><strong>Proving:</strong></p><ul><li>A dbt SQL model can call a Python UDF that hits a typed AI API.</li><li>Answers come back as structured fields (choice, score, NOUL) — not free text to parse later.</li><li>The result is a table you can query, test, and join like any other mart.</li></ul><p><strong>Not proving:</strong></p><ul><li>Campaign ROI or model accuracy on real traffic.</li><li>Warehouse-scale throughput (one HTTP call per row).</li><li>That you need only one dbt installation.</li></ul><p>Keep that framing in mind for everything below.</p><h3>Architecture</h3><p>The pipeline has two dbt environments on purpose.</p><ol><li><strong>dbt Fusion</strong> builds everyday Jaffle Shop marts into local DuckDB (make build-dbt-v2).</li><li><strong>dbt-duckdb</strong> loads a plugin, registers jev_market_customer(state_json) on the connection, and builds one marketing mart (make build-jev-udf).</li></ol><p>Fusion’s DuckDB adapter cannot register Python UDFs today. A sidecar dbt-duckdb environment is the pragmatic trade-off: keep Fusion for normal transforms; reach for dbt-duckdb when you need SQL-callable Python.</p><pre>dbt Fusion (marts)          dbt-duckdb (UDF mart)<br>        |                              |<br>        v                              v<br>   DuckDB tables  ──────────&gt;  jev_market_customer()<br>                                      |<br>                                      v<br>                               TypeSafe API (Jev)<br>                                      |<br>                                      v<br>                        int_customer_marketing_actions</pre><h3>Step 1: Register the UDF via a dbt-duckdb plugin</h3><p>In profiles-jev/profiles.yml:</p><pre>module_paths:<br>  - src<br>plugins:<br>  - module: jaffle_shop_core_v2.jev_plugin</pre><p>The plugin initializes a TypeSafe client and registers a scalar function on each DuckDB connection:</p><pre>def register_jev_market_customer(conn, client):<br>    def jev_market_customer(state_json: str) -&gt; str:<br>        return market_customer_json(state_json, client)</pre><pre>    conn.create_function(<br>        &quot;jev_market_customer&quot;,<br>        jev_market_customer,<br>        side_effects=True,<br>    )</pre><p>side_effects=True is important. DuckDB must know this function leaves the database — every invocation is a network round trip.</p><p>Behind the UDF, Python asks Jev three typed questions in a single system_one call:</p><ul><li><strong>choice</strong> — next best marketing action (winback, loyalty_reward, hold, …)</li><li><strong>NOUL</strong> — likelihood to redeem a promo</li><li><strong>score</strong> — campaign priority for the week</li></ul><p>The SDK returns structured objects. We serialize them to JSON for SQL to parse.</p><h3>Step 2: Build customer state in SQL, call the UDF</h3><p>Model int_customer_marketing_actions is disabled by default (enable_jev_udf_models: false). When enabled, it:</p><ol><li>Samples the top N customers by count_lifetime_orders (default 20).</li><li>Assembles a JSON document: profile fields plus recent order line items.</li><li>Calls jev_market_customer(customer_state) once per row.</li><li>Extracts typed columns with json_extract.</li></ol><p>The heart of the model:</p><pre>select<br>    customer_id,<br>    jev_market_customer(customer_state) as marketing_json<br>from customer_state</pre><p>Downstream columns include next_best_offer, likely_to_redeem_promo, and campaign_priority. dbt tests guard the shape: not_null, unique, accepted_values on the offer enum, bounds on confidences.</p><p>Example rows (top 5 by campaign_priority):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/856/1*v-0gbOHU2P-wwQ5EqQI6ew.png" /></figure><p>On this slice Jev mostly labeled heavy buyers as loyalty_reward. That anecdote is not the point. The point is the output is a <strong>table in the warehouse</strong>, not a Slack message or a CSV export from a notebook.</p><h3>One row in, one row out</h3><p>We deliberately omit customer_name from the JSON state sent to the API.</p><p><strong>Input</strong> (trimmed):</p><pre>{<br>  &quot;profile&quot;: {<br>    &quot;customer_id&quot;: &quot;3483e39a-260f-4974-9366-50c2d833307f&quot;,<br>    &quot;count_lifetime_orders&quot;: 231,<br>    &quot;customer_type&quot;: &quot;returning&quot;,<br>    &quot;lifetime_spend&quot;: 1396.02<br>  },<br>  &quot;recent_orders&quot;: [<br>    {<br>      &quot;ordered_at&quot;: &quot;2025-08-31T00:00:00&quot;,<br>      &quot;items&quot;: [{&quot;product_name&quot;: &quot;tangaroo&quot;, &quot;product_type&quot;: &quot;beverage&quot;}]<br>    },<br>    {<br>      &quot;ordered_at&quot;: &quot;2025-08-30T00:00:00&quot;,<br>      &quot;items&quot;: [{&quot;product_name&quot;: &quot;tangaroo&quot;, &quot;product_type&quot;: &quot;beverage&quot;}]<br>    }<br>  ]<br>}</pre><p><strong>Output:</strong></p><pre>{<br>  &quot;next_best_offer&quot;: &quot;loyalty_reward&quot;,<br>  &quot;next_best_offer_confidence&quot;: 0.91,<br>  &quot;likely_to_redeem_promo&quot;: 0.71,<br>  &quot;campaign_priority&quot;: 1.64,<br>  &quot;campaign_priority_confidence&quot;: 0.47<br>}</pre><p>No regex. No “extract the JSON from the assistant message.” The types are part of the contract.</p><h3>Latency on N = 20</h3><p>I measured wall time with a small harness (make experiment-jev-article) after building the mart. Calls run sequentially (threads: 1 in the Jev profile).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/652/1*2UMEdWWbqirxFVT9Y5MnSg.png" /></figure><p>Latency is dominated by the TypeSafe API, not DuckDB. Roughly <strong>150 ms per warm call</strong> extrapolates to about three seconds for twenty customers. Fine for a demo. Not a design for scoring millions of rows inside SELECT.</p><h3>Reproduce it</h3><p>Prerequisites: Python with <a href="https://github.com/astral-sh/uv">uv</a>, a local DuckDB file from Jaffle Shop seeds, and JEV_API_KEY (or TYPESAFE_API_KEY) in .env.</p><pre>make build-dbt-v2              # Fusion: staging + marts<br>make build-jev-udf             # dbt-duckdb: marketing mart via UDF<br>make experiment-jev-article    # optional: latency JSON + response capture</pre><p>The experiment step writes article/artifacts/jev_responses.jsonl and jev_latency_summary.json, and loads raw.jev_benchmark_runs for SQL analysis.</p><p>Full instructions and tests (make test-jev) are in the repo README.</p><p><strong>Repository:</strong> <a href="https://github.com/yu-iskw/jaffle-shop-meets-dbt-v2">github.com/yu-iskw/jaffle-shop-meets-dbt-v2</a></p><h3>Limitations worth saying out loud</h3><p><strong>Small sample.</strong> Twenty customers from a toy shop. Use this to learn the pattern, not to pick campaigns.</p><p><strong>Two dbt environments.</strong> Operational overhead is real. Worth it only when SQL-callable UDFs are the goal.</p><p><strong>One API call per row.</strong> Production patterns batch customers, pre-compute scores in Python, or run inference on a schedule — then load results as tables.</p><p><strong>Duplicate API work.</strong> Building the mart and running the benchmark harness both call Jev unless you instrument the UDF path directly. I kept them separate so the article has clean latency numbers without modifying the plugin.</p><h3>Closing thought</h3><p>The interesting question is not whether customer 3483e39a… should get loyalty_reward. It is whether <strong>typed AI belongs in the analytics layer at all</strong>.</p><p>My answer after this experiment: yes, as a <strong>materialized side path</strong>. Version the questions in Python, register a UDF, land answers in a mart, test them in dbt — and keep inference off the critical path of every ad-hoc query.</p><p>If you are already on dbt and DuckDB, Jev plus a dbt-duckdb plugin is a small, honest way to try that pattern without standing up a new service.</p><p><em>Thanks for reading. Questions and PRs welcome on the repo.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7aa7340155e3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Data Platforms to Knowledge Platforms]]></title>
            <link>https://yu-ishikawa.medium.com/from-data-platforms-to-knowledge-platforms-0a6647fb998b?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/0a6647fb998b</guid>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[data-platforms]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[knowledge-management]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Tue, 01 Sep 2026 15:02:26 GMT</pubDate>
            <atom:updated>2026-09-01T15:02:26.598Z</atom:updated>
            <content:encoded><![CDATA[<p>In the AI era, producing an analysis is becoming cheap. Turning analytical outputs into governed, reusable organizational assets is becoming the harder — and more strategic — problem.</p><p>For much of the modern data era, the purpose of a data platform appeared relatively clear. It collected data from operational systems, transformed that data into usable models, and delivered it to analysts, dashboards, applications, and machine-learning systems. The platform was judged by familiar properties: reliability, scalability, freshness, discoverability, cost, security, and developer productivity.</p><p>Generative AI does not make those concerns obsolete. It changes the environment around them. Coding agents can create pipelines, modify semantic models, write queries, construct dashboards, produce interactive applications, and explain results. Analytics agents can move between business questions, governed metrics, source data, narrative interpretation, and operational action. The marginal cost of producing a plausible analysis or a new interface is falling.</p><p>That abundance creates a new scarcity: attention, confidence, shared meaning, and institutional memory. If charts, reports, notebooks, and small applications can be generated on demand, which outputs deserve to persist? If analytical methods can be packaged as agent skills, which skills deserve organizational trust? If an agent can generate a correct answer once, what makes that capability reliable enough to reuse?</p><p>These are not primarily product-selection questions. They require us to distinguish <em>artifacts</em> from <em>assets</em>, and then to revisit the meanings of BI, data analytics, the data platform, and the emerging knowledge platform.</p><blockquote><em>The strategic shift is from governing scarce production to governing abundant production: deciding which outputs become reusable capabilities, which claims become institutional knowledge, and which actions may be delegated.</em></blockquote><p>Data becomes organizational knowledge through semantics, analysis, review, and publication — not through movement alone.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0WFv7Vlh-1E97-ys7Mn5TA.png" /><figcaption>Data becomes organizational knowledge through semantics, analysis, review, and publication — not through movement alone.</figcaption></figure><h3>Artifact and asset answer different questions</h3><p>In ordinary technology language, the terms are often blurred. A dashboard may be called a BI artifact, a data asset, or both. An agent skill may be described as a file bundle, a reusable capability, or a product. The ambiguity becomes expensive when generation is cheap, because organizations can no longer afford to govern every output as if it were durable.</p><p>A useful distinction is this:</p><ul><li>An <strong>artifact</strong> is a bounded product of an activity: a chart, query result, report, notebook, dashboard definition, prompt, model card, agent trace, skill package, or generated application.</li><li>An <strong>asset</strong> is something the organization intentionally manages across a lifecycle because it is expected to create future value — or material risk — through use and reuse.</li></ul><p>Artifact describes <em>origin and form</em>. Asset describes <em>institutional status and stewardship</em>. This is compatible with the lifecycle orientation of <a href="https://www.iso.org/standard/83053.html">ISO 55000:2024</a>, which frames asset management around realizing value while balancing performance, risk, and expenditure.</p><p>artifact = produced entity tied to an activity<br>asset = governed resource or capability + expected future value + lifecycle commitment</p><p>The categories overlap without being identical. A one-off agent answer is an artifact, but usually not an asset. A reviewed metric definition is both. A reusable skill is initially an artifact of authoring; it becomes an asset when it acquires an owner, contract, evaluation suite, release process, access policy, usage evidence, maintenance budget, and retirement path. A deployed agent service is better understood as an operated capability, while its configuration, evaluations, runs, and answers are artifacts with different retention needs.</p><p>This prevents two opposite mistakes. The first is <em>artifact inflation</em>: cataloging every generated output until discovery becomes noise. The second is <em>asset invisibility</em>: failing to recognize that semantic definitions, evaluation sets, verified queries, decision histories, and agent skills may be more valuable than the dashboard or chat surface through which users encounter them.</p><p>Promotion from artifact to asset should therefore be explicit. The minimum gates are identity, purpose, ownership, provenance, quality or evaluation evidence, an access model, discoverability, versioning, maintenance expectations, and a retirement rule. Promotion is not a claim of truth. It is a commitment to stewardship.</p><p>An artifact becomes an asset through institutional commitment — not merely because it was saved.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KJyKRxVS9a1tG-ofbuIZ9Q.png" /><figcaption>An artifact becomes an asset through institutional commitment — not merely because it was saved.</figcaption></figure><h3>The function lens still helps — if context is first-class</h3><p>There is a productive way to unify transformations, metrics, dashboards, skills, and agents: treat them as functions. A transformation maps source relations to a derived relation. A metric maps governed data and dimensions to a value. A dashboard maps data, filters, parameters, and identity to a visual result. A skill maps a task and context to a structured output.</p><pre>output = F(data, parameters, question; semantics, time, identity, policy, version, model)</pre><p>The terms after the semicolon are not implementation details. Row-level security can legitimately produce different results for different identities. A metric can retain its name while its definition changes. An agent response can vary with instructions, model, tools, retrieved context, and current data. Reproducibility therefore means reconstructing the effective invocation, not merely preserving visible inputs.</p><p>The function abstraction also clarifies boundaries. A dashboard is normally a bounded analytical function: its metrics, dimensions, filters, and visual grammar are substantially predetermined. A skill is a reusable procedure with an input contract, permitted capabilities, expected outputs, and validations. An agent is a policy that selects and composes tools or skills in pursuit of a goal. The agent is not simply a larger function; it possesses discretion over which functions to invoke and when to stop.</p><p>Consequently, assurance must scale with discretion. A metric can be tested through deterministic assertions. A skill needs contract and scenario tests. An agent requires behavioral evaluations, traces, budget and stop conditions, and controls over publication and action.</p><h3>Traditional analytics was organized around delivery</h3><p>Traditional data analytics was largely a human-centered process of turning curated datasets into reports, models, experiments, and decisions. Traditional BI industrialized the recurring part of that process: shared metrics, parameterized dashboards, scheduled distribution, access control, and stable visual conventions. The traditional data platform supplied the reliable storage, transformation, compute, orchestration, cataloging, and governance beneath both.</p><p>This architecture solved a real coordination problem. Organizations needed a place where people could find accepted numbers, inspect them through bounded filters, and share a durable representation. The BI tool became a semantic layer, calculation engine, publication system, access boundary, content catalog, and collaboration surface — often all at once.</p><p>AI unbundles that bundle. A conversational interface can sit inside or outside BI. A coding agent can generate a custom application. An external agent can consume a semantic model through an API. An interactive artifact can be shared without becoming an official dashboard. Yet unbundling the interface does not eliminate the institutional functions that BI performed. It exposes them.</p><h3>Future analytics is organized around inquiry and delegation</h3><p>AI-era analytics expands the unit of work from “execute this query” to “investigate this question.” An agent may discover sources, reconcile identifiers, transform unstructured text, test hypotheses, compare explanations, generate visualizations, and draft a recommendation. Recent research shows why reducing this to text-to-SQL is inadequate. The 2026 <a href="https://arxiv.org/html/2603.20576v1">Data Agent Benchmark</a> evaluates end-to-end tasks involving multiple databases, inconsistent join keys, unstructured text, and domain knowledge; its best reported model achieved 38 percent pass@1 on 54 queries. This is one early benchmark, not a universal score, but it usefully demonstrates the distance between query generation and dependable analytical work.</p><p>Industry and research are converging on the same response: agents need governed context, not raw schema alone. A 2025 enterprise text-to-SQL system at LinkedIn combined database metadata, historical queries, wikis, and code in a knowledge graph, yet expert review still found only 53 percent of responses correct or close to correct on its internal benchmark. The important lesson is not the particular number. It is that enterprise analytics depends on semantics, organizational memory, and evaluation in addition to model capability.</p><p>Future analytics therefore has four simultaneous outputs:</p><ul><li><strong>answers</strong> for immediate use;</li><li><strong>artifacts</strong> that stabilize a particular analysis;</li><li><strong>assets</strong> that encode reusable evidence, meaning, or method; and</li><li><strong>actions</strong> whose consequences must feed back into later reasoning.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pRTJW3kqObmcOh_itnNwDg.png" /><figcaption>The future platform retains reliable data delivery while adding context, reusable methods, agents, governed publication, and feedback.</figcaption></figure><p>The future platform retains reliable data delivery while adding context, reusable methods, agents, governed publication, and feedback.</p><p>ConceptTraditional center of gravityAI-era center of gravityBIRecurring metrics, dashboards, distributionBounded, governed analytical products among many interfacesData analyticsHuman-led query, model, interpretationHuman-agent inquiry, evaluation, claims, and decisionsData platformReliable data storage, transformation, and accessGoverned evidence and capability substrate for humans and agentsKnowledge platformUsually fragmented across catalogs, documents, BI, and workflow toolsShared context, methods, artifacts, decisions, outcomes, and institutional memory</p><h3>The knowledge platform is a missing institutional layer</h3><p>A knowledge platform should not be defined as a warehouse with a chatbot, a document repository with embeddings, or a knowledge graph alone. It is the layer that makes organizational context usable across people, software, and agents. It connects data to business concepts; concepts to approved methods; methods to claims and artifacts; claims to decisions; and decisions to observed outcomes.</p><p>As of September 2026, major data-platform roadmaps visibly converge on pieces of this architecture. Google describes a Knowledge Catalog that aggregates metadata, semantic models, data products, usage context, and third-party catalogs for agent retrieval. Microsoft Fabric IQ combines semantic models, ontologies, graphs, data and operations agents, and planning. Databricks combines governed data, metrics, business rules, dashboards, applications, coding assistance, and domain agents. Snowflake exposes governed semantic views as queryable objects and agent tools. These are product-specific implementations, but the shared direction is more important than any product name: the platform boundary is moving from data management toward context management.</p><p>That convergence should be interpreted carefully. A catalog does not become a source of truth merely by calling itself a knowledge catalog. Automatically inferred relationships may be useful hypotheses rather than accepted definitions. An ontology can encode disagreements as easily as consensus. A semantic model can be consistent and still encode the wrong business policy. The knowledge platform must preserve who asserted a meaning, on what basis, for which scope, and with what review status.</p><p>The resulting architecture is not a replacement stack in which knowledge supersedes data. It is cumulative:</p><ol><li>The <strong>data platform</strong> makes observations reliable, accessible, and legitimate for use.</li><li>The <strong>analytics platform</strong> makes methods executable and inquiry productive.</li><li>The <strong>BI layer</strong> standardizes recurring metrics, bounded exploration, and publication.</li><li>The <strong>agent platform</strong> delegates tools, procedures, and decisions under policy.</li><li>The <strong>knowledge platform</strong> connects evidence, meaning, methods, artifacts, decisions, and outcomes across time.</li></ol><p>These layers may be implemented by several products. “Knowledge platform” is primarily an architectural responsibility, not necessarily a new procurement category.</p><h3>Data becomes evidence; analytics produces claims</h3><p>A database record is not automatically evidence for a business claim. It becomes usable as evidence only within a context that establishes what it represents, how it was produced, which population it covers, what transformations were applied, what limitations are known, and whether the use is legitimate.</p><pre>evidence = data + semantics + provenance + quality + scope + legitimate use</pre><p>This strengthens the meaning of a data product. A data product does not merely make a dataset discoverable and easy to query. It makes a class of observations eligible for specified forms of organizational reasoning. Its contract should include intended and inappropriate uses, not only schema, freshness, and service levels. This extends the organizational principles of <a href="https://martinfowler.com/articles/data-mesh-principles.html">data mesh</a> from data delivery toward evidential responsibility.</p><p>Data analytics then turns evidence into claims: propositions about what happened, why it happened, what may happen next, or what should be done. Those claim types require different support. “Revenue declined” is descriptive. “Churn caused most of the decline” is explanatory. “Revenue will recover” is predictive. “We should change onboarding” is normative. An agent can move smoothly between these categories even when the evidence cannot.</p><p>A trustworthy analytical claim therefore needs a proposition, scope, temporal reference, definitions, evidence, method, uncertainty, limitations, and responsibility. A chart can support a claim, but the chart is not the claim. A dashboard can define a space in which users construct claims without asserting one fixed conclusion.</p><p>Provenance supplies the connective tissue. <a href="https://www.w3.org/TR/prov-o/">W3C PROV</a> distinguishes entities, activities, and agents and records how entities were used, generated, derived, and attributed. <a href="https://openlineage.io/docs/1.48.0/spec/object-model">OpenLineage</a> operationalizes related ideas for datasets, jobs, and runs. <a href="https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html">C2PA Content Credentials</a> demonstrate how provenance can be cryptographically bound to media assets. None is a complete specification for analytical knowledge, but together they point toward claim-level lineage across data, semantics, methods, actors, and published artifacts.</p><h3>Skills are procedural assets; agents are delegated actors</h3><p>An agent skill packages procedural knowledge for reuse. Anthropic’s <a href="https://console.anthropic.com/docs/en/agents-and-tools/agent-skills/overview">Agent Skills</a> model combines instructions, metadata, scripts, and resources; its significance is conceptual even outside one platform. A skill externalizes a repeatable method so that the method can be discovered, versioned, tested, distributed, and improved independently of a single conversation.</p><p>But a folder of instructions is still an artifact. It becomes a production asset only when the organization governs its purpose, accepted evidence, permissions, input and output contracts, validations, failure modes, owner, compatibility, and retirement. Skills are therefore closer to internal APIs or software libraries than to clever prompt snippets.</p><p>An agent is different. It selects and composes tools or skills, maintains state, and exercises discretion in pursuit of a goal. Functionally, it is a delegated actor. Once analytics is considered alongside the full data platform, three forms of authority become visible:</p><ul><li><strong>Epistemic authority:</strong> selecting evidence, interpreting results, and formulating claims.</li><li><strong>Engineering authority:</strong> modifying pipelines, semantic models, tests, dashboards, skills, and applications.</li><li><strong>Operational authority:</strong> changing business systems or initiating actions based on analysis.</li></ul><p>These authorities should not be collapsed. An agent allowed to query data is not automatically authorized to publish an official forecast. An agent allowed to draft a metric change is not automatically authorized to merge it. An agent allowed to recommend an intervention is not automatically authorized to execute it.</p><p>Access, analysis, publication, modification, and action are separate delegations. Each step increases institutional consequence.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*V-3P8PQQNG2tv16lsTc1SQ.png" /><figcaption>Access, analysis, publication, modification, and action are separate delegations. Each step increases institutional consequence.</figcaption></figure><blockquote><em>Least privilege for agents must include least epistemic privilege: grant only the analytical discretion, publication status, and action authority required for the role.</em></blockquote><p>Open protocols reinforce the separation of concerns. The <a href="https://modelcontextprotocol.io/specification/2026-07-28">Model Context Protocol</a> distinguishes resources, tools, and prompts; MCP Apps can return interactive interfaces inside a conversation. These mechanisms improve portability and composition. They do not decide which resource is authoritative, which tool is safe for a claim, which prompt represents policy, or which result deserves publication. Interoperability moves the governance boundary; it does not remove it.</p><h3>Artifacts remain the publication and memory layer</h3><p>Agent conversations are productive environments for inquiry. They are weak systems of institutional record. Results may depend on transient context, mutable data, tool availability, model behavior, and unrecorded assumptions. They are difficult to discover, cite, compare, supersede, and review independently of the original interaction.</p><p>An artifact stabilizes analytical work. It gives an output an identity, a version, an owner, a status, and a relationship to evidence. The artifact may be a report, dashboard, notebook, model card, interactive application, verified answer, decision record, experiment result, or reviewed agent-generated analysis.</p><pre>published artifact = claim or analytical space + evidence trace + context + provenance + status</pre><p>Three publication forms remain especially important:</p><h3>Static snapshot</h3><p>A snapshot records an accepted conclusion in a particular context. Its purpose is not perpetual freshness; it is temporal accountability. Board materials, experiment conclusions, incident analyses, and regulatory submissions often need to remain historically stable even as newer evidence appears.</p><h3>Parameterized dashboard</h3><p>A dashboard defines a bounded space of possible analyses. It combines approved metrics, dimensions, filters, representations, and permissions. Reproducing a dashboard result requires parameter values, data time, semantic-model version, dashboard version, and effective authorization context.</p><h3>Agentic or generated artifact</h3><p>An agentic artifact records the stabilized outcome of an adaptive process. It should reference the original question, relevant skills and tools, semantic definitions, material evidence, configuration versions, limitations, and review or evaluation status. It does not need to expose unrestricted internal reasoning; it needs a sufficient evidential trace.</p><p>Different artifact forms serve different epistemic purposes. Publication converts transient analysis into inspectable organizational memory.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bXttRJ1j1x_eJMt_HWkJjQ.png" /><figcaption>Different artifact forms serve different epistemic purposes. Publication converts transient analysis into inspectable organizational memory.</figcaption></figure><p>Interactive sharing surfaces — including environments such as <a href="https://support.claude.com/en/articles/9547008-publish-and-share-artifacts">Claude Artifacts</a> and interfaces delivered through MCP Apps — matter because they lower the cost of distributing rich analytical experiences. Yet a sharing surface is an envelope, not an authority. It does not automatically provide semantic ownership, delegated data access, analytical provenance, cataloging, promotion, retention, or supersession.</p><p>Storage is likewise insufficient. A repository containing thousands of dashboards and documents may still fail to reveal the accepted interpretation, its owner, why it changed, or whether it remains valid. Institutional memory requires discoverability, attribution, review state, contestability, supersession, expiry, and withdrawal.</p><h3>A reference architecture for knowledge production</h3><p>The knowledge-platform perspective suggests seven logical planes. They need not live in one product:</p><ol><li><strong>Evidence plane:</strong> data products, transformations, quality, lineage, temporal state, and legitimate-use constraints.</li><li><strong>Meaning plane:</strong> metrics, semantic models, ontologies, entities, relationships, definitions, and exceptions.</li><li><strong>Capability plane:</strong> queries, statistical methods, models, APIs, tools, and skills.</li><li><strong>Agency plane:</strong> human workflows, agents, orchestration, delegation, approvals, and action boundaries.</li><li><strong>Assurance plane:</strong> tests, evaluations, reviews, observability, policy, red teaming, and incident feedback.</li><li><strong>Artifact plane:</strong> snapshots, dashboards, notebooks, applications, claims, decision records, versions, and history.</li><li><strong>Learning plane:</strong> decisions, actions, outcomes, adoption, counter-evidence, and changes to evidence, meaning, or method.</li></ol><p>Identity, privacy, security, cost, ownership, and lifecycle management cut across all seven. This architecture also reveals a key control: the system must distinguish machine-generated context from human-approved meaning. Automated enrichment can propose relationships and definitions; promotion policy determines when they become governed assets.</p><p>A knowledge platform closes the loop: outcomes do not merely follow analysis; they become evidence for improving future data, semantics, skills, and decisions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mKlNwUOTuNEHDUEUvNW7Iw.png" /><figcaption>A knowledge platform closes the loop: outcomes do not merely follow analysis; they become evidence for improving future data, semantics, skills, and decisions.</figcaption></figure><h3>Eight leadership design principles</h3><h4><strong>Principle 1: </strong>Separate artifact production from asset promotion</h4><p>Let people and agents explore cheaply, but require explicit gates before outputs become endorsed, reusable, or operational. Cataloging everything is not governance; it is accumulation.</p><h4><strong>Principle 2: </strong>Optimize for warranted claims, not output volume</h4><p>As generation becomes cheaper, counts of dashboards, queries, reports, and answers become poor success metrics. Measure whether important claims are supported, reviewed, discoverable, and connected to decisions and outcomes.</p><h4><strong>Principle 3: </strong>Treat semantics and context as shared infrastructure</h4><p>Every new agent or interface should not reconstruct business meaning independently. Governed metrics, entities, relationships, definitions, exclusions, and policies should be reusable across dashboards, applications, skills, and agents.</p><h4><strong>Principle 4: </strong>Govern skills as production assets</h4><p>Reusable analytical and engineering methods need owners, versions, contracts, evaluations, observability, compatibility policy, and retirement. A skills directory without these properties becomes automation sprawl.</p><h4><strong>Principle 5: </strong>Separate access, interpretation, modification, publication, and action</h4><p>These are different authorities with different consequences. Encode them separately in identity, workflow, capability design, and review policy. Read-only is not a sufficient description of an agent’s institutional power.</p><h4><strong>Principle 6: </strong>Require provenance across the complete claim chain</h4><p>Trace a published conclusion to data, semantics, method, parameters, identity context, tools, configuration, evaluations, and review status. Dataset lineage alone cannot explain an agent-generated recommendation.</p><h4><strong>Principle 7: </strong>Make epistemic status visible</h4><p>Distinguish exploration, candidate claim, reviewed analysis, endorsed asset, superseded result, and withdrawn conclusion. Promotion should be proportional to consequence; not every analysis needs bureaucracy, but every authoritative claim needs visible status.</p><h4><strong>Principle 8: </strong>Close the loop from decisions back to evidence</h4><p>Preserve which claims informed a decision, what action followed, and what outcome occurred. Without feedback, the organization remembers analyses but cannot learn whether its analytical reasoning was effective.</p><h3>Conclusion: from a system of delivery to a system of learning</h3><p>The future is unlikely to be dashboard-first or agent-first. Recurring, bounded questions will continue to benefit from dashboards. Open-ended investigation will benefit from agents. Fixed decisions and formal reviews will require snapshots. Generated interactive artifacts will create new ways to communicate and explore.</p><p>The strategic issue is not which interface wins. It is whether all interfaces operate over shared evidence and meaning; whether methods are reusable and evaluated; whether humans and agents possess appropriate authority; whether important results become durable and contestable; and whether decisions and outcomes improve the next cycle.</p><p>The data platform therefore does not disappear into a knowledge platform. Its responsibility becomes sharper: establish whether data can reliably and legitimately serve as evidence. Data analytics determines what claims that evidence supports. BI provides governed spaces for recurring measurement and publication. Agent platforms delegate methods and actions. The knowledge platform binds these elements into institutional context and memory.</p><blockquote><em>The AI-era platform should be judged not by how much analytical content it can generate, but by how well it turns selected artifacts into governed assets — and governed evidence into organizational learning.</em></blockquote><p>This does not make the platform an arbiter of truth. It makes the organization’s process of producing, challenging, publishing, revising, and learning from analytical knowledge more explicit. When the cost of generation approaches zero, that stewardship becomes the scarce capability.</p><h3>References and further reading</h3><ol><li>International Organization for Standardization, <a href="https://www.iso.org/standard/83053.html">“ISO 55000:2024 — Asset management: Vocabulary, overview and principles.”</a></li><li>W3C Provenance Working Group, <a href="https://www.w3.org/TR/prov-o/">“PROV-O: The PROV Ontology.”</a></li><li>OpenLineage, <a href="https://openlineage.io/docs/1.48.0/spec/object-model">“OpenLineage Object Model.”</a></li><li>Coalition for Content Provenance and Authenticity, <a href="https://spec.c2pa.org/specifications/specifications/2.4/specs/C2PA_Specification.html">“C2PA Technical Specification 2.4.”</a></li><li>Zhamak Dehghani, <a href="https://martinfowler.com/articles/data-mesh-principles.html">“Data Mesh Principles and Logical Architecture.”</a></li><li>Anthropic, <a href="https://console.anthropic.com/docs/en/agents-and-tools/agent-skills/overview">“Agent Skills.”</a></li><li>Model Context Protocol, <a href="https://modelcontextprotocol.io/specification/2026-07-28">“Specification, 2026–07–28.”</a></li><li>Albert Chen et al., <a href="https://arxiv.org/abs/2507.14372">“Text-to-SQL for Enterprise Data Analytics.”</a></li><li>Ruiying Ma et al., <a href="https://arxiv.org/html/2603.20576v1">“Can AI Agents Answer Your Data Questions? A Benchmark for Data Agents.”</a></li><li>Ankush Gupta et al., <a href="https://aclanthology.org/2025.acl-industry.90/">“BI-Bench: A Comprehensive Benchmark Dataset and Unsupervised Evaluation for BI Systems.”</a></li><li>Google Cloud, <a href="https://cloud.google.com/blog/products/data-analytics/introducing-the-google-cloud-knowledge-catalog">“Introducing the Google Cloud Knowledge Catalog.”</a></li><li>Microsoft, <a href="https://learn.microsoft.com/en-us/fabric/iq/overview">“What is Fabric IQ?”</a></li><li>Databricks, <a href="https://docs.databricks.com/aws/en/genie/">“Genie.”</a></li><li>Snowflake, <a href="https://docs.snowflake.com/en/user-guide/views-semantic/overview">“Overview of Semantic Views.”</a></li><li>National Institute of Standards and Technology, <a href="https://www.nist.gov/itl/ai-risk-management-framework">“AI Risk Management Framework.”</a></li><li>Anthropic, <a href="https://support.claude.com/en/articles/9547008-publish-and-share-artifacts">“Publish and share artifacts.”</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0a6647fb998b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightdash at Ubie Part 7:AgentOps as Code — Building Reliable AI Analytics Agents with Coding…]]></title>
            <link>https://yu-ishikawa.medium.com/agentops-as-code-building-reliable-ai-analytics-agents-with-coding-agents-450bba6d409c?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/450bba6d409c</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[analytics-engineering]]></category>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Fri, 28 Aug 2026 01:07:29 GMT</pubDate>
            <atom:updated>2026-08-28T06:34:58.154Z</atom:updated>
            <content:encoded><![CDATA[<h3>Lightdash at Ubie Part 7:AgentOps as Code — Building Reliable AI Analytics Agents with Coding Agents</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JdRYpuLCsqygCuZlDDDEoQ.png" /></figure><p>AgentOps as Code: coding agents iterate on versioned agent state and validate behavior through repeatable evaluations.</p><p>In Part 5 of this series, we moved beyond the BI interface and explored the official Lightdash MCP Server. That was an important shift: instead of asking people to come to a BI tool, we could bring governed analytics into AI assistants and agentic workflows.</p><p>In Part 6, we pushed the idea further by looking at secure, purpose-built analytics applications on top of Lightdash.</p><p>But there is another problem we need to solve.</p><blockquote><em>How do we make the AI analytics agents themselves reliable?</em></blockquote><p>Creating an AI agent is easy. We can give it instructions, connect it to our semantic layer, add some business knowledge, and start asking questions.</p><p>Building an agent that we can trust across dozens or hundreds of real business questions is much harder.</p><p>If an agent misunderstands the meaning of “active customer,” uses the wrong revenue definition, or presents a result in a way that violates a business convention, someone has to identify the bad behavior, understand why it happened, change the agent, test the change, make sure the fix did not break something else, and repeat until the agent behaves as expected.</p><p>This starts to look surprisingly familiar.</p><p><strong>It is software development.</strong></p><p>In this article, I want to introduce an approach I have been experimenting with in my open-source <a href="https://github.com/yu-iskw/lightdash-tools">lightdash-tools</a> project:</p><blockquote><em>AgentOps as Code: manage Lightdash AI agents as versioned engineering artifacts, define evaluations as acceptance tests, and let coding agents such as Cursor or Codex iteratively improve the agent until those tests pass.</em></blockquote><p>The interesting part is not simply that an AI can edit another AI’s prompt. The important change is that we can give a coding agent the same ingredients that make it effective at software engineering: editable artifacts, a programmable control plane, repeatable tests, observable failures, and an explicit definition of done.</p><p>Once those pieces exist, a coding agent can start <strong>developing another agent</strong>.</p><h3>From BI as Code to AgentOps as Code</h3><p>One of the reasons we originally chose Lightdash at Ubie was its strong alignment with the <strong>BI as Code</strong> philosophy.</p><p>Metrics and semantic definitions do not have to live only inside a graphical user interface. They can be represented in code, reviewed through pull requests, validated in CI, and evolved using normal software-engineering practices.</p><p>I think the same idea should apply to AI analytics agents.</p><p>An AI agent is not just a prompt. A production analytics agent has configuration, instructions, knowledge, and an evaluation contract.</p><p>An analytics agent becomes an engineered system when instructions, knowledge, configuration, evaluations, and version history are managed together.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yM1yGAHJ2pkmeMmmMcN4BQ.png" /></figure><pre>AI Analytics Agent<br>│<br>├── Agent configuration<br>│   ├── name<br>│   ├── description<br>│   ├── instructions<br>│   ├── model/capability settings<br>│   └── access scope<br>│<br>├── Knowledge<br>│   ├── terminology<br>│   ├── business definitions<br>│   ├── analytical conventions<br>│   └── internal procedures<br>│<br>└── Evaluation contract<br>    ├── representative questions<br>    ├── expected behavior<br>    └── release criteria</pre><p>Lightdash already provides many of these primitives. AI agents support configurable instructions, knowledge documents, access controls, evaluations, feedback, memory, and multiple specialized agents.</p><p>So the pieces already exist. The next question is whether we can expose those pieces to coding agents as an engineering system.</p><h3>The ai-agent-ops MCP Profile</h3><p>In Part 5, we looked at MCP mainly as a way for external AI assistants to consume Lightdash analytics. But MCP can also be a <strong>management interface</strong>.</p><p>I implemented an ai-agent-ops MCP profile in lightdash-tools that exposes a deliberately focused set of Lightdash AI-agent lifecycle APIs to an MCP-compatible coding agent.</p><p>Today the profile includes operations for listing and inspecting project agents, creating and updating agents, checking explore access, reading agent threads, creating and updating evaluation suites, running evaluations, retrieving evaluation results, and managing knowledge documents.</p><pre>Cursor / Codex / another coding agent<br>                │<br>                │ MCP<br>                ▼<br>        lightdash-tools<br>          ai-agent-ops<br>                │<br>                │ Lightdash API<br>                ▼<br>        Lightdash AI Agent<br>        ├── instructions<br>        ├── knowledge<br>        └── evaluations</pre><p>This turns the Lightdash agent into something a coding agent can <strong>inspect, modify, execute, and validate</strong>.</p><h3>Evaluations Are the Agent’s Acceptance Tests</h3><p>Lightdash evaluations let us create suites of questions and run them against an agent in batches. I like to think about them in software-engineering terms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KDvOx3kBVP0o_PrMJHtWQg.png" /></figure><p>Evaluation-driven development: diagnose failed cases, improve the implementation, and re-test until the acceptance suite is green.</p><pre>Define evaluation<br>      │<br>      ▼<br>     FAIL<br>      │<br>      ▼<br>Change instructions / knowledge<br>      │<br>      ▼<br>Run evaluation<br>      │<br>      ▼<br>     PASS</pre><p>Imagine that we have a specialized revenue-analysis agent. Our organization has several business conventions:</p><ul><li>“Net revenue” means revenue after refunds and payment-processing fees.</li><li>“Active customers” means customers with qualifying activity in the last 30 days.</li><li>Revenue-growth questions should default to month-over-month comparison unless the user specifies another period.</li><li>Internal test accounts must never be included in customer metrics.</li></ul><p>Those rules are easy for humans who work with the data every day. They are not necessarily obvious to an LLM. So we encode representative questions into an evaluation suite.</p><pre>evaluations:<br>  - key: revenue-agent-acceptance<br>    title: Revenue Agent Acceptance Tests<br>    description: &gt;<br>      Critical business semantics the revenue analyst<br>      must understand correctly.<br>    prompts:<br>      - prompt: &gt;<br>          What do we mean by net revenue?<br>        expectedResponse: &gt;<br>          Net revenue is revenue after refunds and<br>          payment-processing fees.</pre><pre>      - prompt: &gt;<br>          Which customers should be counted when<br>          I ask for active customers?<br>        expectedResponse: &gt;<br>          Customers with qualifying activity in<br>          the previous 30 days.</pre><p>Now we have something much stronger than “please make the agent good at revenue analysis.” We have an executable contract.</p><h3>Keeping the Desired State in Git</h3><p>The next step is to make the agent reproducible. lightdash-tools includes an AgentOps bundle format based on a Kubernetes-style structure.</p><pre>apiVersion: lightdash.ai/v1alpha1<br>kind: LightdashAiAgentBundle</pre><pre>metadata:<br>  name: revenue-analytics</pre><pre>spec:<br>  projectUuid: &quot;&lt;lightdash-project-uuid&gt;&quot;</pre><pre>  agents:<br>    - key: revenue-analyst<br>      name: Revenue Analyst<br>      description: &gt;<br>        Specialized analytics agent for revenue,<br>        customer, and commercial performance questions.<br>      instruction: |<br>        You are a senior revenue analyst.</pre><pre>        Follow the business definitions provided<br>        in the agent&#39;s knowledge documents.</pre><pre>        When analyzing growth without an explicitly<br>        requested comparison period, use month-over-month.<br>      enableDataAccess: true<br>      adminOnly: false<br>      evaluations:<br>        - key: acceptance<br>          title: Revenue Agent Acceptance Tests<br>          prompts:<br>            - prompt: What do we mean by net revenue?<br>              expectedResponse: &gt;<br>                Net revenue is revenue after refunds<br>                and payment-processing fees.</pre><p>This gives us a familiar GitOps workflow:</p><pre>Git<br> │ desired state<br> ▼<br>agentops plan<br> │<br> ▼<br>review diff<br> │<br> ▼<br>agentops apply<br> │<br> ▼<br>Lightdash</pre><p>For example:</p><pre>lightdash-tools agentops plan \<br>  --file agentops/revenue-agent.yaml</pre><pre>lightdash-tools agentops apply \<br>  --file agentops/revenue-agent.yaml</pre><p>There is also a drift command, so the runtime state can be compared against the desired state rather than silently diverging from what is stored in Git.</p><h3>What About Knowledge Files?</h3><p>Knowledge documents are particularly interesting because business knowledge often does not belong in one enormous system prompt.</p><pre>agentops/<br>└── revenue-agent/<br>    ├── agent.yaml<br>    ├── gate.yaml<br>    └── knowledge/<br>        ├── business-glossary.md<br>        ├── revenue-definitions.md<br>        └── analysis-conventions.md</pre><p><strong>Current implementation boundary:</strong> knowledge documents are not yet part of the LightdashAiAgentBundle schema. The GitOps bundle currently reconciles agent configuration and evaluations, while knowledge documents use their own MCP CRUD surface. Git can still be the authoring source of truth for those Markdown files.</p><h3>The Manual Red → Green Loop</h3><p>Before making anything autonomous, I think it is important to understand the loop manually.</p><pre>Revenue Agent Acceptance Tests</pre><pre>PASS  What do we mean by net revenue?</pre><pre>FAIL  Which customers count as active customers?</pre><pre>FAIL  Which comparison period should be used<br>      for unspecified revenue growth?</pre><pre>2 failed<br>1 passed</pre><p>The first failed question is a missing business definition. That belongs naturally in a knowledge document. The second is an analytical behavior preference, so it belongs in the agent instructions.</p><p>We apply the changes and run the evaluation again.</p><pre>Revenue Agent Acceptance Tests</pre><pre>PASS  What do we mean by net revenue?<br>PASS  Which customers count as active customers?<br>PASS  Which comparison period should be used?</pre><pre>3 passed<br>0 failed</pre><h3>Separate Evaluation Execution from the Release Gate</h3><p><strong>Running an evaluation is not the same thing as deciding whether an agent is releasable.</strong></p><p>The ai-agent-ops MCP profile is responsible for obtaining evaluation evidence. The release decision belongs to a separate gate.</p><pre>apiVersion: lightdash.ai/v1alpha1<br>kind: LightdashAiEvaluationGate</pre><pre>metadata:<br>  name: revenue-agent-release</pre><pre>spec:<br>  projectUuid: &quot;&lt;project-uuid&gt;&quot;<br>  agentUuid: &quot;&lt;agent-uuid&gt;&quot;<br>  evaluationUuid: &quot;&lt;evaluation-uuid&gt;&quot;<br>  policy:<br>    requireAllPassed: true</pre><pre>lightdash-tools agentops evaluate-gate \<br>  --file agentops/revenue-agent/gate.yaml \<br>  --wait \<br>  --output markdown</pre><p>This gives us a clean separation between evidence and policy:</p><pre>Evaluation<br>    │ &quot;What happened?&quot;<br>    ▼<br>Evidence<br>    │<br>    ▼<br>Release Gate<br>    │ &quot;Is this acceptable?&quot;<br>    ▼<br>PASS / FAIL</pre><h3>Now Give the Loop to a Coding Agent</h3><p>So far, nothing requires autonomous AI. A human can edit the agent, update knowledge, run evaluations, inspect failures, and repeat.</p><p>But notice what we have created: state, actions, feedback, and acceptance criteria.</p><p>That is exactly the type of environment in which a coding agent becomes useful. Instead of asking “can you improve this prompt?”, we can give it a goal:</p><blockquote><em>Make this analytics agent pass its acceptance suite.</em></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IO7kKhfyOD7gBL_ka_appw.png" /></figure><p>The autonomous AgentOps loop gives the coding agent editable state, structured tools, observable failures, and a verifiable finish condition.</p><h3>Cursor /goal and Codex Goal Mode</h3><p>This idea has become particularly practical because coding-agent harnesses are evolving from turn-by-turn assistants into systems that can pursue longer-running objectives.</p><p>A goal could look conceptually like this:</p><pre>GOAL</pre><pre>Make every evaluation in the Revenue Analyst<br>acceptance suite pass.</pre><pre>SOURCE OF TRUTH</pre><pre>The desired agent state is stored under:<br>agentops/revenue-agent/</pre><pre>YOU MAY CHANGE<br>- the agent instructions<br>- knowledge Markdown files</pre><pre>YOU MUST NOT CHANGE<br>- evaluation questions<br>- expected responses<br>- semantic-layer definitions<br>- data-access permissions<br>- release-gate criteria</pre><pre>WORKFLOW<br>1. Inspect the current agent and evaluation suite.<br>2. Run the evaluation.<br>3. Read each failed result.<br>4. Diagnose why it failed.<br>5. Update either the instructions or knowledge.<br>6. Apply/synchronize the change.<br>7. Run the evaluation again.<br>8. Repeat until the release gate passes.</pre><pre>STOP CONDITIONS<br>- all evaluations pass;<br>- the failure requires a semantic-layer or permission<br>  change outside your allowed scope;<br>- no safe intervention remains;<br>- or the iteration budget is exhausted.</pre><p>The coding agent is effectively performing <strong>test-driven development on another AI agent</strong>.</p><h3>This Is More Than Prompt Engineering</h3><pre>Traditional prompt engineering<br>Prompt → Try it → Looks better? → Maybe</pre><pre>AgentOps as Code<br>Versioned desired state<br>        ↓<br>Apply<br>        ↓<br>Evaluation suite<br>        ↓<br>Explicit failures<br>        ↓<br>Targeted change<br>        ↓<br>Regression evaluation<br>        ↓<br>Release gate</pre><p>The goal is not to find one magical system prompt. The goal is to create a <strong>controlled optimization loop</strong> around the agent.</p><h3>A Critical Guardrail: Never Let the Optimizer Rewrite Its Tests</h3><p>There is an obvious but important danger in this design.</p><p>Suppose we tell a coding agent “make every evaluation green” and give it permission to edit both the agent implementation and the evaluation expectations. The easiest solution may be to change the tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8ZL3p46Ot6-1x4PDZch37Q.png" /></figure><p>Reliable autonomous improvement requires a protected acceptance contract: the optimizer can improve the agent, but it cannot weaken the definition of success.</p><p>So evaluation cases should normally be immutable during an optimization run. The optimizer can change instructions and knowledge, but not evaluation questions, expected answers, or release thresholds.</p><h3>Another Guardrail: Do Not Prompt Around a Broken Semantic Layer</h3><p>If an evaluation fails because the Lightdash semantic layer incorrectly defines a metric, adding an agent instruction that compensates for the broken metric may make the test green while making the overall architecture worse.</p><p>For this workflow, the coding agent’s autonomous modification surface should stay intentionally small:</p><pre>Allowed:<br>  instructions<br>  knowledge</pre><pre>Escalate:<br>  semantic layer<br>  metric definitions<br>  tags/access scope<br>  permissions</pre><p>If the root cause is outside the allowed scope, the correct behavior is not to invent a workaround. It is to stop and explain what must change.</p><h3>A Green Evaluation Suite Does Not Mean the Agent Is Perfect</h3><p>An LLM is probabilistic, and an evaluation suite represents only the cases we thought to test.</p><pre>100% evaluation pass rate<br>          ≠<br>100% correct for every possible future question</pre><p>It means the current agent satisfies the acceptance contract represented by this evaluation suite for this run. That is still extremely valuable.</p><p>The solution is not to abandon testing. The solution is to continuously improve the test suite as real failures are discovered.</p><h3>Why I Would Not Run an Infinite Autonomous Loop</h3><p>Long-running goal modes make the workflow powerful, but production AgentOps should still be bounded. An agent that cannot make a test pass should not keep changing instructions forever.</p><pre>repeat:<br>    evaluate</pre><pre>    if all_pass:<br>        stop_success</pre><pre>    diagnose failures</pre><pre>    if failure_requires_forbidden_change:<br>        stop_and_escalate</pre><pre>    if no_safe_fix:<br>        stop_and_escalate</pre><pre>    modify instructions or knowledge</pre><pre>until iteration_budget_exhausted</pre><h3>The Bigger Idea: Coding Agents Can Develop Other Agents</h3><p>This is the part I find most interesting.</p><p>Coding agents were originally designed to work on code repositories. But what actually makes them effective is more general: structured state, tools, feedback, constraints, and verifiable completion criteria.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3qcedMCEwB6F3GUL1W0q_Q.png" /></figure><p>Agent developing agent: the coding agent engineers and validates the analytics agent; the analytics agent serves business users.</p><p>The coding agent is not analyzing business data on behalf of the user. Instead, it is engineering the system that will analyze business data on behalf of many users.</p><pre>Coding Agent<br>    ↓ develops<br>Analytics Agent<br>    ↓ serves<br>Business Users</pre><h3>From AgentOps to CI/CD</h3><p>The workflow described in this article is intentionally developer-driven. A person creates or updates the evaluation contract. A coding agent improves the Lightdash agent. The release gate decides whether the resulting state is acceptable.</p><p>But the architecture naturally leads somewhere else.</p><p>From AgentOps to CI/CD: versioned changes can flow through evaluations, coding-agent remediation, release gates, and human approval before promotion.</p><pre>Git change<br>   ↓<br>candidate agent<br>   ↓<br>evaluations<br>   ↓<br>coding-agent remediation<br>   ↓<br>evaluations<br>   ↓<br>all green<br>   ↓<br>human review<br>   ↓<br>promotion</pre><p>At that point, analytics-agent development begins to look remarkably similar to modern software delivery.</p><h3>Conclusion: From Building Agents to Engineering Agents</h3><p>Lightdash AI Agents make natural-language analytics increasingly practical. But making an agent available is not the same as making it reliable.</p><p>If analytics agents are going to become an important interface to enterprise data, we need to apply the same engineering discipline that we already apply to semantic models, data pipelines, infrastructure, and applications.</p><pre>prompt<br>→ try<br>→ manually inspect<br>→ tweak</pre><pre>becomes</pre><pre>desired state<br>→ deploy<br>→ evaluate<br>→ diagnose<br>→ improve<br>→ regress<br>→ gate</pre><p>In this article, we combined Git as the desired-state and knowledge authoring layer; Lightdash as the analytics-agent runtime; lightdash-tools AgentOps bundles for agent configuration and evaluation desired state; the ai-agent-ops MCP profile as the programmable agent-management surface; Lightdash evaluations as acceptance tests; release gates as the definition of done; and coding-agent goal loops as autonomous executors of the development cycle.</p><blockquote><em>Once an analytics agent can be represented as code and evaluated programmatically, a coding agent can develop the analytics agent for us.</em></blockquote><p>This does not eliminate the need for humans. Humans still define business semantics, decide what correct behavior means, and define the evaluation contract and governance boundaries.</p><p>But once we have specified those boundaries, we no longer have to manually perform every iteration inside them.</p><p>That is the transition I find most exciting: <strong>from prompt engineering to AgentOps, and from manually building AI agents to systematically engineering them.</strong></p><p>In the next part, I want to take this one step further and explore what <strong>CI/CD for analytics agents</strong> could look like: automatically evaluating proposed changes, detecting regressions, using coding agents to prepare fixes, and keeping humans in control of the final promotion decision.</p><h3>References</h3><ul><li><a href="https://docs.lightdash.com/agents">Lightdash AI Agents documentation</a></li><li><a href="https://github.com/yu-iskw/lightdash-tools">yu-iskw/lightdash-tools</a></li><li><a href="https://yu-ishikawa.medium.com/lightdash-at-ubie-part-5-from-bi-to-agentic-bi-with-the-official-lightdash-mcp-1584ea27f412">Lightdash at Ubie Part 5</a></li><li><a href="https://yu-ishikawa.medium.com/lightdash-at-ubie-part-6-building-secure-purpose-built-analytics-applications-with-lightdash-b7ce00886a03">Lightdash at Ubie Part 6</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=450bba6d409c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From MCP Prototype to Enterprise Platform: Seven Design Patterns for Internal MCP Servers]]></title>
            <link>https://yu-ishikawa.medium.com/from-mcp-prototype-to-enterprise-platform-seven-design-patterns-for-internal-mcp-servers-d2d666a5904b?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2d666a5904b</guid>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[enterprise-architecture]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-governance]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Mon, 17 Aug 2026 02:34:56 GMT</pubDate>
            <atom:updated>2026-08-17T02:34:56.912Z</atom:updated>
            <content:encoded><![CDATA[<p><em>About a 10-minute read for staff engineers, architects, AI platform engineers, and security-minded developers.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9O_IW2SNvtGlme5ayCuEMg.png" /></figure><p>It is surprisingly easy to build a useful MCP server. Take an internal API, put a handful of operations behind MCP tools, connect an agent, and you have something people can use within an afternoon. That is also where the easy part ends.</p><p>The first version usually inherits the shape of the API behind it. A read endpoint becomes a read tool. A POST becomes a write tool. Authentication gets added at the edge. If the upstream service already has good RBAC, it is tempting to conclude that most of the security problem has been solved.</p><p>For a personal tool, that may be enough. For a company-wide MCP server, it is not.</p><p>An API answers a familiar question: <strong>what is this caller allowed to do?</strong> An agent-facing interface has to answer several more. Which operations should the agent be able to discover at all? Which project is it acting on? When does a write need a person to approve the exact change? What data is safe to put into model context? If something goes wrong next week, can we reconstruct what happened?</p><p>These are different boundaries. Treating them as one authorization problem produces MCP servers that work well in demos and become uncomfortable as soon as they are shared across a company.</p><p>I use the following seven patterns as a reference architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/591/1*ULzbftp0g39dFnsqAy-H4w.png" /><figcaption>Figure 1. Seven-layer enterprise MCP reference architecture.</figcaption></figure><p>The order is deliberate. It starts with things I would expect even from a modest internal deployment and ends with concerns that become much more important as MCP turns into shared infrastructure.</p><p>Before going through them, one version detail matters. This article targets the <strong>MCP 2026–07–28 protocol revision and the v2 TypeScript SDK generation</strong>. The July 2026 revision is a substantial change from the 2025-era protocol: the core protocol is stateless, initialize and Mcp-Session-Id are gone from the modern wire protocol, client context travels per request, server/discover provides optional discovery, and interactive server-to-client work is represented through Multi Round-Trip Requests (MRTR). The v2 TypeScript SDK is the stable SDK line for this revision, although using v2 packages alone does not mean every hand-built client or server automatically speaks the 2026 wire protocol.</p><h3>1. Keep MCP authorization and downstream authorization separate</h3><p>Suppose an internal MCP server fronts a SaaS product.</p><p>The SaaS already supports OAuth. Each employee has a SaaS account. The obvious implementation is to let the employee authenticate, obtain that user’s SaaS token, and forward it through the MCP server.</p><p>There is an appealing property here: the MCP server does not need to recreate the SaaS authorization model. If Alice cannot access a dashboard in the SaaS product, the API still rejects her request.</p><p>I agree with the goal. I would not use the same token for both trust boundaries.</p><p>MCP’s authorization model treats the MCP server as its own protected resource. A token presented to that resource needs to be intended for it. MCP security guidance explicitly forbids accepting a token meant for another service and simply passing it through to a downstream API. The downstream call should use a separate credential issued for that downstream resource.</p><p>The architecture therefore has two authorization legs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PZWXAG0GizKVO5INrRa13Q.png" /><figcaption>Figure 2. Two-leg authorization: MCP resource authorization and downstream delegation.</figcaption></figure><p>This distinction looks fussy until there is more than one client, one downstream system, or one security control. Then it becomes useful very quickly.</p><p>The MCP server can identify and govern access to <em>itself</em>. The SaaS can continue to make the authoritative decision about access to SaaS objects. Revoking MCP access does not have to mean revoking the employee’s ordinary SaaS access. Conversely, a token issued for the SaaS cannot simply be replayed against the MCP resource.</p><p>There is also a practical question: how does the MCP server acquire the user’s downstream authorization without asking the model to carry credentials?</p><p>That is one reason URL-mode elicitation exists. A server can send the user to an external browser flow for an interaction such as third-party OAuth. Credentials stay outside the MCP client’s model-facing interaction, while the MCP server receives and manages the resulting downstream authorization. Form elicitation is not a place to ask for passwords, API keys, or bearer tokens.</p><p>For a Google Cloud integration, the downstream leg can look different. An organization might authenticate the employee to the MCP resource, apply its own policy, and then use short-lived service-account impersonation for a narrowly defined Google Cloud operation. The point is not that impersonation is always preferable. The point is that <strong>MCP identity and downstream execution identity are separate design decisions</strong>.</p><p>At larger organizations, there is another option worth knowing about: MCP’s Enterprise-Managed Authorization extension. It allows an enterprise identity provider to become the central authorization point for access to MCP servers, reducing the per-server OAuth ceremony for employees. That solves an enterprise onboarding and policy problem; it does not remove the downstream authorization boundary described above.</p><p>A useful rule is:</p><blockquote><em>Authenticate the caller to the MCP server first. Delegate to downstream systems separately.</em></blockquote><p>Without this boundary, a small convenience turns into token passthrough, weak audience isolation, and a confused-deputy problem that is difficult to repair after clients depend on it.</p><h3>2. Split one large tool catalog into capability-specific MCP endpoints</h3><p>A SaaS API can easily have hundreds of operations. Mirroring them into one MCP server is technically possible and usually a poor interface for an agent.</p><p>Imagine a BI service exposing all of these through one endpoint:</p><pre>list_projects<br>get_dashboard<br>run_dashboard<br>create_dashboard<br>update_dashboard<br>delete_dashboard<br>purge_dashboard<br>manage_project_access<br>manage_organization_members<br>...</pre><p>An analyst who only wants to answer a question now shares a capability surface with tools for changing and deleting content.</p><p>Traditional API clients cope with large APIs because application code explicitly chooses endpoints. An agent first has to decide <em>which tool to call</em>. Tool discovery is therefore part of the safety boundary, not merely documentation.</p><p>I prefer <strong>profiles</strong>: stable MCP endpoints organized around jobs and risk.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EilDydetxanqAeFGUrGQmw.png" /><figcaption>Figure 3. Capability-specific MCP endpoints hosted by one deployment.</figcaption></figure><p>This does <strong>not</strong> require four deployments. One process can mount several MCP endpoints and share authentication, API clients, logging, schemas, and operational infrastructure.</p><p>The important part is that a profile is a hard capability boundary. If /reader/v1/mcp is read-only, a delete tool should not merely be disabled by a runtime flag. It should not be registered on that endpoint.</p><p>That gives operators another useful control: a production deployment can mount only reader and analyst, while an administrative deployment also mounts governance.</p><p>There is a usability benefit too. A smaller, coherent tool catalog gives the model a cleaner decision problem. The names and descriptions compete with fewer irrelevant operations.</p><p>A common alternative is to expose everything and attach permissions to every call. Authorization still matters, but it solves a different problem. A tool that the agent can discover but is forbidden to execute still affects planning, and a configuration error can turn that latent capability into a real one.</p><p><strong>Do not use authorization as a substitute for reducing the capability surface.</strong></p><h3>3. Treat tool annotations as a risk vocabulary, not a security system</h3><p>Once the catalog is divided into profiles, individual tools still need useful semantics.</p><p>MCP defines four tool annotations that form a small risk vocabulary:</p><ul><li>readOnlyHint</li><li>destructiveHint</li><li>idempotentHint</li><li>openWorldHint</li></ul><p>They matter because an MCP client can use them when deciding how much friction a tool deserves. A read-only lookup and a destructive write should not look identical to the host.</p><p>ToolRead-onlyDestructiveIdempotentOpen worldget_dashboardyes——nocreate_dashboardnonousually nonoupdate_dashboardnopossiblydependsnodelete_dashboardnoyesdependsnosend_messagenononoyes</p><p>The defaults are intentionally conservative when annotations are absent: clients should assume a tool may mutate state, may be destructive, may not be idempotent, and may interact with the open world.</p><p>But annotations are <strong>hints</strong>. They are not authorization.</p><p>That distinction is easy to lose because the annotations sound like policy. Setting destructiveHint: true does not make a destructive operation safe. A client may choose to show a confirmation dialog, or it may use the hint differently. Server-side policy cannot depend on the host doing the right thing.</p><p>I therefore pair protocol annotations with a server-owned exposure model:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_glJIl5g5zNduLGQ7eF34w.png" /><figcaption>Figure 4. Tool risk classification and exposure tiers.</figcaption></figure><p>The last category is important.</p><p>There is a recurring temptation to expose every operation because “the user still has to be authorized.” I would not expose permanent account deletion, irreversible data purge, or similarly catastrophic administrative operations merely to achieve API parity.</p><p>An agent interface does not have to be a complete API interface.</p><p>The API can remain available to deliberate application code or an administrative UI. The MCP surface should be designed around operations we are comfortable allowing an agent to reason about and invoke.</p><h3>4. Narrow the request below the user’s maximum permission</h3><p>Per-user authorization has another subtle failure mode.</p><p>Suppose Alice legitimately has access to 120 projects. She asks an agent to investigate a problem in finance-prod.</p><p>From the upstream service’s perspective, Alice may operate on any of those 120 projects. From the current task’s perspective, 119 of them are irrelevant.</p><p>That difference is worth enforcing.</p><p>I think of effective MCP authority as an intersection:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DpPWf3M59G4AAQWlzCJ-gQ.png" /><figcaption>Figure 5. Effective MCP authority as the intersection of independent controls.</figcaption></figure><p>A deployment might be limited to a particular business unit. A request might additionally be pinned to one project. A tool may require a narrower OAuth scope. The upstream API still has the final say, but the MCP server prevents requests from reaching unrelated parts of the user’s permission set.</p><p>This is especially useful around environments.</p><p>A developer who can legitimately access both staging and production should not have an agent silently infer which one was intended. If a dangerous operation requires an environment and the request does not establish it, failing with something like SCOPE_REQUIRED is better than guessing.</p><p>There is one design detail I care about here: <strong>a ceiling should not become an implicit default</strong>.</p><p>If a deployment is allowed to touch projects A, B, and C, that does not tell us which project the user meant. Allowlist and target are separate concepts.</p><p>The same pattern generalizes well:</p><pre>organization<br>tenant<br>project<br>repository<br>dataset<br>cloud account<br>region<br>environment<br>billing account</pre><p>The current MCP authorization model also supports a useful companion pattern: progressive scope elevation. An MCP request does not need to begin with every OAuth scope the user could possibly grant. When a particular operation needs more authority, the server can indicate insufficient scope and let the client reauthorize with the required scope.</p><p>The principle is simple:</p><blockquote><em>The user’s maximum permission is a ceiling, not the agent’s default operating range.</em></blockquote><p>This reduces the blast radius of both ordinary mistakes and prompt injection.</p><h3>5. For consequential writes, approve a transaction — not a boolean</h3><p>A lot of early agent tooling uses some variation of this:</p><pre>{<br>  &quot;dashboardId&quot;: &quot;123&quot;,<br>  &quot;confirmed&quot;: true<br>}</pre><p>It looks like a safety mechanism. It is not a human approval boundary. The model can set confirmed to true.</p><p>A prompt that says “always ask the user first” is better UX, but it is still a prompt convention.</p><p>MCP has a protocol mechanism for this class of interaction: <strong>elicitation</strong>. In the 2026–07–28 protocol, interactive flows are carried through Multi Round-Trip Requests. A handler can return an input_required result containing an elicitation request. The client gathers the response and retries the original operation, echoing the server&#39;s requestState.</p><p>That gives us the pieces for a much stronger write protocol.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xmZ-Hw_FiE5lee6lET8gaQ.png" /><figcaption>Figure 6. Human-gated destructive transaction using elicitation and protected request state.</figcaption></figure><p>There are several details here that matter more than the confirmation dialog itself.</p><p>First, approval should be bound to the <strong>specific operation and resource</strong>. Approval to delete dashboard A must not be reusable for dashboard B.</p><p>Second, the server should protect requestState as untrusted client-carried data. The v2 TypeScript SDK documentation recommends integrity protection such as HMAC or AEAD, binding the state to the principal, originating operation and parameters, with an expiry. Signed state is not necessarily encrypted; do not put secrets into a merely signed payload.</p><p>Third, re-read the target immediately before applying the mutation. The thing the user approved may have changed while the approval UI was open. For an overwrite, compare a version, ETag, digest, or another precondition.</p><p>Fourth, distinguish <strong>form elicitation</strong> from <strong>URL elicitation</strong>. A form is appropriate for an approval or non-secret structured input. URL mode is for an external interaction such as OAuth or another sensitive browser flow that should not pass through the MCP client.</p><p>And finally, not every write needs this ceremony. Confirmation fatigue is real.</p><p>A reasonable policy might be:</p><pre>read                          -&gt; execute<br>additive, low-risk write      -&gt; execute or lightweight policy<br>overwrite                     -&gt; preview / stronger policy<br>reversible destructive write  -&gt; elicitation<br>irreversible operation        -&gt; keep off MCP</pre><p>Human approval works best when it is reserved for actions where the human decision actually changes the risk.</p><h3>6. Govern what comes back, not only what goes out</h3><p>Security reviews naturally focus on tool inputs: can this user call this operation with these parameters?</p><p>For MCP, the output deserves equal attention.</p><p>A tool result may end up in model context, conversation history, host telemetry, debugging traces, caches, or another agent step. Returning the entire upstream API payload because it is convenient can quietly move far more data than the task requires.</p><p>Consider a project API that returns this:</p><pre>{<br>  &quot;name&quot;: &quot;Finance&quot;,<br>  &quot;members&quot;: [&quot;...&quot;],<br>  &quot;warehouseConnection&quot;: {&quot;...&quot;: &quot;...&quot;},<br>  &quot;scheduler&quot;: {&quot;...&quot;: &quot;...&quot;},<br>  &quot;credentials&quot;: {&quot;...&quot;: &quot;...&quot;},<br>  &quot;internalMetadata&quot;: {&quot;...&quot;: &quot;...&quot;}<br>}</pre><p>The model may only need:</p><pre>{<br>  &quot;id&quot;: &quot;finance&quot;,<br>  &quot;name&quot;: &quot;Finance&quot;<br>}</pre><p>This is not cosmetic response shaping. It is a data-governance boundary.</p><p>I like to classify fields rather than invent a single includeSensitive=true escape hatch.</p><p>Data classMCP defaultPublic / ordinary internal metadatareturnPII needed only occasionallymask or omit; purpose-specific opt-inConfidential operational dataminimizeSecretsnever returnCredentials / tokensnever return</p><p>Purpose-specific controls are safer than a universal bypass. includeEmail=true says something understandable. includeSensitive=true encourages an agent to ask for everything whenever it gets stuck.</p><p>MCP’s tool schema support helps here as well. Define an output schema for the interface you actually want agents to consume rather than treating an upstream JSON response as the contract. In the 2026 revision, tool schemas use full JSON Schema 2020–12 semantics, and structured tool content is no longer limited to object-shaped JSON.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DQolpBa9zQTzzBYZBOcf5A.png" /><figcaption>Figure 7. Response minimization before data enters agent context.</figcaption></figure><p>This pattern also makes upstream API evolution less dangerous. If the SaaS adds a new credential field tomorrow, an explicit MCP response mapper does not automatically start feeding it to models.</p><p>A server can have flawless authentication and still create a serious incident by returning too much data.</p><h3>7. Make policy stateless, observable, and boring to operate</h3><p>The last pattern is mostly invisible when everything is small.</p><p>Then the MCP server gets popular.</p><p>There are several replicas behind a load balancer. Different teams add tools. Security wants audit records. Platform engineering wants rate limits. A write is approved on one replica and retried against another. Somebody asks why an agent changed a production resource three weeks ago.</p><p>This is where MCP 2026–07–28’s stateless core is particularly useful.</p><p>The modern protocol no longer relies on an initialize handshake or Mcp-Session-Id. Each request carries the information needed at the protocol layer, and any request can land on any healthy instance. server/discover is available when a client wants capabilities up front. Streamable HTTP also carries method and tool names in Mcp-Method and Mcp-Name headers, which gives gateways useful routing and policy inputs without parsing the JSON body.</p><p>Application workflows may still be stateful. The difference is that their state should be explicit.</p><p>A query can return a queryId. A browser automation tool can return a browserId. An MRTR interaction can carry protected requestState. A durable workflow can store state in a database. What should disappear is accidental correctness that depends on &quot;the next request reaches the same process.&quot;</p><p>The request path should also become boring and consistent:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KriYDBxTHPdZCS6TRrrHPQ.png" /><figcaption>Figure 8. A consistent stateless policy-enforcement pipeline.</figcaption></figure><p>I would rather have this cross-cutting logic in a small number of shared enforcement layers than rely on every tool author remembering fourteen security rules.</p><p>Audit records should describe the action without becoming a second copy of sensitive application data. A useful event might contain:</p><pre>{<br>  &quot;subject&quot;: &quot;user-123&quot;,<br>  &quot;client&quot;: &quot;coding-agent&quot;,<br>  &quot;profile&quot;: &quot;governance&quot;,<br>  &quot;tool&quot;: &quot;delete_dashboard&quot;,<br>  &quot;project&quot;: &quot;finance-prod&quot;,<br>  &quot;riskClass&quot;: &quot;reversible-destructive&quot;,<br>  &quot;humanApproval&quot;: true,<br>  &quot;result&quot;: &quot;success&quot;,<br>  &quot;correlationId&quot;: &quot;...&quot;<br>}</pre><p>Do not log bearer tokens. Be cautious about logging raw arguments and tool results. An audit system that solves accountability by copying credentials and PII into a log bucket has simply moved the problem.</p><p>The July 2026 specification also documents W3C Trace Context propagation in MCP metadata. That makes it practical to follow one action from an agent host, through the MCP layer, into downstream calls using ordinary distributed tracing.</p><p>This final layer is less glamorous than a new agent feature. It is also the layer that lets an MCP service become normal production infrastructure.</p><h3>The resulting reference architecture</h3><p>Put the seven patterns together and the server stops looking like a thin protocol adapter.</p><p>That is a good thing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FHcbBYkdTzSgvyVMXPMTMw.png" /><figcaption>Figure 9. The complete enterprise MCP service architecture.</figcaption></figure><p>There is one idea underneath all seven patterns:</p><p><strong>Do not collapse authority, capability, intent, and data access into one permission check.</strong></p><p>They answer different questions.</p><ul><li><strong>Identity:</strong> Who is making this request?</li><li><strong>Capability:</strong> What operations should this agent be able to discover?</li><li><strong>Tool safety:</strong> What kind of effect does this operation have?</li><li><strong>Scope:</strong> Where is the agent allowed to act for this task?</li><li><strong>Intent:</strong> Did a person approve this particular consequential change?</li><li><strong>Information governance:</strong> What data is allowed to enter agent context?</li><li><strong>Operations:</strong> Can we enforce these rules consistently and explain the result later?</li></ul><p>A user can be fully authorized to delete a resource while the reader MCP profile quite correctly has no delete tool. A governance profile can expose a reversible delete while requiring human approval. The upstream service can still make the final object-level authorization decision. The response can still omit credentials even when the user is technically allowed to see them elsewhere.</p><p>None of those controls contradict each other. They are layers.</p><p>This is also why I would resist building a giant internal MCP gateway that simply converts every REST operation into a tool. It optimizes the easiest part of the problem — the protocol translation — and centralizes the hardest part: an enormous, heterogeneous capability surface with no opinion about how agents should use it.</p><p>The better abstraction is not <strong>API endpoint → MCP tool</strong>.</p><p>It is:</p><pre>business capability<br>    -&gt; intentionally chosen agent operation<br>    -&gt; risk classification<br>    -&gt; narrow execution context<br>    -&gt; appropriate approval boundary<br>    -&gt; minimal result<br>    -&gt; auditable execution</pre><p>That is more work than wrapping an OpenAPI document.</p><p>For an enterprise MCP server, it is the work that matters.</p><h3>Specification notes and references</h3><p>This article was checked against the MCP documentation available in August 2026, with the <strong>2026–07–28 protocol revision</strong> as the primary target.</p><ol><li><strong>MCP 2026–07–28 release overview</strong> — stateless core, modern removal of initialize/Mcp-Session-Id, server/discover, MRTR, routing headers, caching, authorization changes, and tracing:<br>https://blog.modelcontextprotocol.io/posts/2026-07-28/</li><li><strong>MCP TypeScript SDK v2–2026–07–28 support</strong> — modern serving, requestState, MRTR, integrity protection, and modern/legacy differences:<br>https://ts.sdk.modelcontextprotocol.io/v2/migration/support-2026-07-28</li><li><strong>MCP TypeScript SDK v2 server packages</strong> — v2 is the stable SDK generation implementing the 2026–07–28 specification:<br>https://ts.sdk.modelcontextprotocol.io/v2/api/%40modelcontextprotocol/server/</li><li><strong>MCP authorization and token-passthrough security guidance</strong> — inbound MCP tokens must target the MCP resource; downstream API access uses a separate token:<br>https://modelcontextprotocol.io/docs/tutorials/security/security_best_practices</li><li><strong>Tool annotations</strong> — readOnlyHint, destructiveHint, idempotentHint, and openWorldHint are risk hints rather than authorization controls:<br>https://blog.modelcontextprotocol.io/posts/2026-03-16-tool-annotations/</li><li><strong>Elicitation</strong> — form and URL modes, including out-of-band sensitive interactions:<br>https://modelcontextprotocol.io/specification/draft/client/elicitation</li><li><strong>URL-mode elicitation design</strong> — external authorization and credential acquisition without token passthrough:<br>https://modelcontextprotocol.io/seps/1036-url-mode-elicitation-for-secure-out-of-band-intera</li><li><strong>Enterprise-Managed Authorization</strong> — centralized enterprise IdP control for MCP server authorization:<br>https://modelcontextprotocol.io/extensions/auth/enterprise-managed-authorization</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2d666a5904b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightdash at Ubie Part 6: Building Secure, Purpose-Built Analytics Applications with Lightdash…]]></title>
            <link>https://yu-ishikawa.medium.com/lightdash-at-ubie-part-6-building-secure-purpose-built-analytics-applications-with-lightdash-b7ce00886a03?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/b7ce00886a03</guid>
            <category><![CDATA[mcp-server]]></category>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[analytics-engineering]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 04:51:39 GMT</pubDate>
            <atom:updated>2026-08-15T04:54:34.848Z</atom:updated>
            <content:encoded><![CDATA[<h3>Lightdash at Ubie Part 6: Building Secure, Purpose-Built Analytics Applications with Lightdash OAuth</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZeHSAZbI5YbAt5rLJy6JgA.png" /></figure><p><a href="https://yu-ishikawa.medium.com/lightdash-at-ubie-part-5-from-bi-to-agentic-bi-with-the-official-lightdash-mcp-1584ea27f412?postPublishedType=initial">In Part 5</a>, I wrote about the official Lightdash MCP and how it lets coding agents work directly with our governed analytics layer.</p><p>For most teams that simply want to use Lightdash from an MCP-compatible assistant, I think the official MCP is the right place to start.</p><p>But MCP also points toward a broader change that I find even more interesting.</p><p>Custom software is becoming much cheaper to implement.</p><p>With modern coding agents, a small team — or sometimes one engineer — can prototype a focused internal application surprisingly quickly. That changes the economics behind a decision we have made for decades: whether to adapt a workflow to a general-purpose application or build software tailored to the workflow.</p><p>Analytics is a good example. We might want a specialized application for content development, governance auditing, a particular operational workflow, or an AI agent. A custom MCP server is one possibility, but it is only one possibility.</p><p>The difficult part is not generating another web application or API client. Once a custom application starts accessing organizational analytics on behalf of people, we need to answer a more important question:</p><blockquote><em>Whose permissions should the application have?</em></blockquote><p>This is where Lightdash OAuth applications become an interesting building block.</p><h3>The Identity Problem with Custom Analytics Applications</h3><p>The easiest custom integration often looks like this:</p><pre>Users<br>  |<br>  v<br>Custom application<br>  |<br>  | shared credential<br>  v<br>Lightdash API</pre><p>The credential might be a personal access token or a service account, depending on the use case and available features.</p><p>This architecture is not inherently wrong. Machine identities are appropriate for many automated workloads. A scheduled job that performs one organization-wide administrative task may genuinely be a service rather than a human user.</p><p>But interactive applications have a different question to answer.</p><p>Imagine Alice and Bob use the same internal analytics application. Alice can access a sensitive project; Bob cannot. Alice has permission to edit a dashboard; Bob is read-only.</p><p>If every request from the application reaches Lightdash using one shared identity, the application has to recreate or compensate for those authorization differences itself.</p><p>Conceptually:</p><pre>Alice ----+<br>          |<br>Bob ------+--&gt; Custom app --&gt; Shared identity --&gt; Lightdash<br>          |<br>Carol ----+</pre><p>We have collapsed several human identities into one application identity.</p><p>That becomes especially important with AI applications. A traditional UI exposes a set of buttons and forms. An agent can discover tools, select operations dynamically, chain them, and potentially perform actions the application developer did not anticipate in one fixed screen flow.</p><p>The more capable the application becomes, the less comfortable I am treating identity as an implementation detail.</p><p>For many interactive applications, what I actually want is:</p><pre>Alice --&gt; Custom app --&gt; Alice&#39;s delegated identity --&gt; Lightdash<br>Bob   --&gt; Custom app --&gt; Bob&#39;s delegated identity   --&gt; Lightdash</pre><p>This is the role OAuth can play.</p><h3>Using Lightdash OAuth for Delegated Access</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a1A799m5H72X2L2BPJTl5g.png" /></figure><p>Lightdash Cloud supports custom OAuth applications. A custom application can redirect a user through the Lightdash authorization flow and obtain credentials for accessing Lightdash on that user’s behalf.</p><p>The high-level architecture is familiar:</p><pre>+--------+<br>|  User  |<br>+---+----+<br>    |<br>    | authenticate / authorize<br>    v<br>+--------------------+<br>| Custom application |<br>+---------+----------+<br>          |<br>          | delegated access<br>          v<br>+--------------------+<br>|     Lightdash      |<br>| permissions and    |<br>| access controls    |<br>+---------+----------+<br>          |<br>          v<br>     Lightdash API</pre><p>The important property is not merely “we use OAuth.” The important property is that the custom application can preserve the distinction between end users when it calls the analytics platform.</p><p>That lets Lightdash remain responsible for Lightdash authorization instead of forcing every internal application to build a parallel model of who may access which analytics resources.</p><p>This is a useful architectural boundary:</p><pre>Application responsibility:<br>- its own UI and workflow<br>- application-specific authorization where necessary<br>- safe token/session handling<br>- tool and capability design</pre><pre>Lightdash responsibility:<br>- Lightdash identity<br>- Lightdash roles and permissions<br>- access to projects and content<br>- analytics governance enforced by Lightdash</pre><p>OAuth does not magically make an application secure. We still have to protect client secrets, tokens, redirect URIs, sessions, dependencies, logs, and the application itself. Nor should we assume that every application must use end-user OAuth; service identities remain appropriate for headless workloads.</p><p>But for interactive applications acting on behalf of Lightdash users, delegated identity is a much stronger starting point than casually sharing one highly privileged credential.</p><h3>Managing OAuth Applications as Infrastructure</h3><p>I maintain the open-source Terraform provider for Lightdash. The provider includes a lightdash_oauth_application resource, which means the OAuth application configuration can be managed alongside other Lightdash resources as infrastructure as code.</p><p><a href="https://registry.terraform.io/providers/ubie-oss/lightdash/latest/docs/resources/oauth_application">Terraform Registry</a></p><p>At a high level, I want the lifecycle to look something like this:</p><pre>Terraform<br>   |<br>   | manages<br>   v<br>Lightdash OAuth application<br>   |<br>   | client credentials<br>   v<br>Secret-management boundary<br>   |<br>   v<br>Application runtime</pre><p>For a Google Cloud deployment, Google Cloud Secret Manager is an obvious place to consider for runtime secrets. The exact implementation depends on the deployment model, and teams should be deliberate about how sensitive Terraform values enter state and how secrets are transferred to the runtime. “Managed by Terraform” should not be interpreted as “put every secret in plaintext configuration.”</p><p>Infrastructure as code gives us several useful properties: changes can be reviewed; environments can be reproduced; OAuth application configuration can live near other platform configuration; and the application’s identity setup does not depend entirely on somebody remembering a sequence of clicks in an admin UI.</p><p>The provider is also useful for a broader AgentOps workflow. As of this writing it includes resources for Lightdash project AI agents and their evaluations, allowing parts of agent configuration and offline evaluation definitions to be managed declaratively as well.</p><p>This matters because once analytics agents become production systems, we need to manage more than prompts.</p><h3>Lightdash as a Platform for Purpose-Built Analytics Applications</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SWQiwBLGDiT3CHBc0PllYw.png" /></figure><p>OAuth becomes more interesting when we stop thinking only about MCP.</p><p>A governed analytics platform can sit behind many interfaces:</p><pre>                         +-----------------------+<br>                         | Analytics assistant   |<br>                         +-----------+-----------+<br>                                     |<br>+------+       OAuth      +----------v----------+<br>| User |-----------------&gt;|                     |<br>+------+                  |     Lightdash       |<br>                          |                     |<br>+----------------------+  | semantic layer +    |<br>| Governance/audit app |-&gt;| access control      |<br>+----------------------+  |                     |<br>                          +----------+----------+<br>+----------------------+             ^<br>| Specialized agent    |-------------+<br>+----------------------+</pre><pre>+----------------------+<br>| Custom MCP server    |-------------+<br>+----------------------+</pre><p>The interfaces can be very different while Lightdash remains a common analytics and governance layer.</p><p>This matters because general-purpose software inevitably accumulates compromises. A BI application needs to serve many users and many workflows. A purpose-built application can instead optimize for one job.</p><p>For example, an audit application does not need a dashboard editor. A content-development assistant does not need organization administration. An operational application might expose three business actions instead of a generic data explorer.</p><p>AI-assisted development makes these smaller applications cheaper to prototype and evolve. But there is an important qualification:</p><blockquote><em>Cheap to implement is not the same as cheap to operate securely.</em></blockquote><p>Every additional application creates a security and maintenance surface. It needs dependency updates, observability, testing, incident handling, access reviews, and an owner. We should not create custom applications merely because a coding agent can generate them.</p><p>The better question is whether a specialized workflow creates enough value to justify a specialized interface.</p><h3>A Real Example: lightdash-tools</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ztAPKT980-d7CRwOWXlOeA.png" /></figure><p>One application of this idea is the custom MCP server in my open-source lightdash-tools project.</p><p><a href="https://github.com/yu-iskw/lightdash-tools">GitHub - yu-iskw/lightdash-tools: Lightdash tools (HTTP client, CLI, MCP and more)</a></p><p>I do not view it as a replacement for Lightdash’s official MCP. The official MCP is a much better default when the goal is simply to connect a compatible AI assistant to Lightdash.</p><p>lightdash-tools exists because I have been experimenting with a different problem: what if the MCP capability surface itself is tailored to a particular analytics job?</p><p>The project provides MCP tools backed by various Lightdash APIs and can expose different MCP endpoints from one server using profiles.</p><p>Conceptually:</p><pre>                      Custom MCP server<br>                              |<br>             +----------------+----------------+<br>             |                |                |<br>             v                v                v<br>     content-developer  organization-audit  ai-agent-ops<br>             |                |                |<br>             v                v                v<br>       content work       read-oriented      AI agent<br>                          inspection         operations</pre><p>A content-developer profile can focus on capabilities needed when developing Lightdash content.</p><p>An organization-audit profile can expose a deliberately read-oriented surface for inspecting Lightdash resources. This is useful when the task is to understand or audit an organization rather than change it.</p><p>An ai-agent-ops profile can focus on operating Lightdash AI Agents and workflows such as evaluations.</p><p>The names and exact tool sets can evolve. The design principle is more important than the current list.</p><h3>Designing Tools Around Jobs, Not APIs</h3><p>When implementing an MCP server on top of a large REST API, the most obvious design is mechanical:</p><pre>REST API endpoint<br>       |<br>       v<br>    MCP tool<br>       |<br>       v<br>     Agent</pre><p>Repeat until every API operation has a corresponding tool.</p><p>That can maximize coverage, but maximum coverage is not necessarily the best agent interface.</p><p>I prefer to begin from the job:</p><pre>Job to be done<br>      |<br>      v<br>Capabilities required<br>      |<br>      v<br>Profile / tool surface<br>      |<br>      v<br>    Agent</pre><p>There are several reasons for this.</p><p>First, it makes the intended responsibility of the agent easier to understand. An organization-audit endpoint communicates a very different contract from &quot;here are 150 Lightdash tools.&quot;</p><p>Second, it gives us an application-level opportunity to reduce capabilities. If an audit workflow does not need mutations, we should not expose mutation tools merely because the underlying API has them.</p><p>Third, a smaller and more coherent tool set can make agent behavior easier to evaluate. Tool selection is part of agent behavior; irrelevant tools increase the space of possible actions.</p><p>This is defense in depth, not a substitute for authorization.</p><p>A profile saying “read-only” is not a security boundary if the underlying credential can access an unguarded write path elsewhere. Lightdash permissions, application authorization, MCP tool exposure, and warehouse permissions should reinforce one another rather than relying on a single layer.</p><p>I think this distinction becomes increasingly important as we build agentic systems:</p><blockquote><em>Do not give an agent every capability simply because the API makes it available. Give it the capabilities required for the job.</em></blockquote><h3>From AI Agents to AgentOps</h3><p>Specialized agents also create another requirement: evaluation.</p><p>Traditional deterministic software often has a relationship like this:</p><pre>input -&gt; function -&gt; expected output</pre><p>An analytics agent is different:</p><pre>question<br>   |<br>   v<br>model + instructions + tools + semantic context + current data<br>   |<br>   v<br>response and actions</pre><p>The system is probabilistic, and important parts of its environment change over time. Models change. Semantic definitions change. Dashboards change. Data changes. Tools change.</p><p>So “the agent worked when I tried it” is not enough evidence for a production workflow.</p><p>This is why I have been interested in treating offline evaluations as configuration that can be reviewed and managed. The Lightdash Terraform provider currently includes a resource for project agent evaluations, and lightdash-tools has an ai-agent-ops direction intended for workflows around operating agents.</p><p>There is much more to say about AgentOps — evaluation datasets, graders, regression testing, traces, cost, latency, and production feedback loops — than fits in this article. It probably deserves a separate part of the series.</p><p>The key point here is simpler: once we create purpose-built AI applications, operating and evaluating them becomes part of the analytics platform problem.</p><h3>Official MCP or a Custom Application?</h3><p>I do not think the conclusion is “build your own MCP server.”</p><p>I would choose the simplest architecture that solves the actual workflow.</p><p>RequirementGood starting pointUse Lightdash from Claude Code, Codex, Cursor, or another supported MCP clientOfficial Lightdash MCPGeneral agentic analytics over LightdashOfficial Lightdash MCPPurpose-specific internal UICustom application + OAuthSpecialized business workflowCustom application + OAuth where delegated user identity is appropriateTailored MCP capability surfaceCustom MCP serverRead-oriented governance workflowPurpose-built application or constrained MCP profileHeadless automationService/PAT-style identity where appropriate, with least privilegeSpecialized evaluated analytics agentsCustom application/agent + AgentOps practices</p><p>The official MCP already supports much more than basic analytics, including semantic exploration, queries, content operations, AI Agent context, and permission-aware tool exposure. If those capabilities match the workflow, using the supported integration reduces the amount of custom infrastructure we have to own.</p><p>A custom application becomes interesting when the workflow itself is custom.</p><p>That is a very different reason from “we can implement our own MCP.”</p><h3>From a BI Tool to a Governed Analytics Platform</h3><p>Parts 5 and 6 describe two sides of the same architectural change.</p><p>Part 5 brought Lightdash into coding agents:</p><pre>User -&gt; Coding agent -&gt; Official MCP -&gt; Lightdash</pre><p>Part 6 expands the set of possible interfaces:</p><pre>                              Users<br>                                |<br>             +------------------+------------------+<br>             |                  |                  |<br>             v                  v                  v<br>       Lightdash UI        Coding agents       Custom apps<br>                                |                  |<br>                          Official MCP         OAuth + API<br>                                |                  |<br>             +------------------+------------------+<br>                                |<br>                                v<br>                            Lightdash<br>                                |<br>                      Governed semantic layer<br>                                |<br>                                v<br>                          Data warehouse</pre><p>I think this is an important direction for BI in the AI era.</p><p>The future does not necessarily require one interface to absorb every possible analytics workflow. As the cost of implementing software falls, we can selectively build smaller interfaces and agents around particular jobs.</p><p>But we should not fragment the things that benefit from being shared: metric definitions, semantic context, identity, authorization, and governance.</p><p>That is the combination I find compelling about Lightdash as a foundation for these experiments. The UI remains useful. The official MCP creates a governed interface for general-purpose AI assistants. OAuth and the API let us build purpose-specific applications when we genuinely need them. And infrastructure-as-code and evaluation practices can help us operate those systems more deliberately.</p><p>The interesting shift is therefore not from “BI” to “MCP.”</p><p>It is from thinking of BI as a single user interface to thinking of governed analytics as a platform that can safely support multiple human and agent interfaces.</p><h3>References</h3><ul><li><a href="https://docs.lightdash.com/references/integrations/lightdash-mcp">Lightdash MCP documentation</a></li><li><a href="https://www.lightdash.com/pricing">Lightdash pricing</a></li><li><a href="https://github.com/yu-iskw/terraform-provider-lightdash">Terraform provider for Lightdash</a></li><li><a href="https://registry.terraform.io/providers/ubie-oss/lightdash/latest">Terraform Registry: ubie-oss/lightdash</a></li><li><a href="https://github.com/yu-iskw/lightdash-tools">Lightdash Tools</a></li><li><a href="https://www.npmjs.com/package/@lightdash-tools/mcp">@lightdash-tools/mcp on npm</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b7ce00886a03" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Lightdash at Ubie Part 5: From BI to Agentic BI with the Official Lightdash MCP]]></title>
            <link>https://yu-ishikawa.medium.com/lightdash-at-ubie-part-5-from-bi-to-agentic-bi-with-the-official-lightdash-mcp-1584ea27f412?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/1584ea27f412</guid>
            <category><![CDATA[business-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[mcps]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Sat, 15 Aug 2026 04:43:27 GMT</pubDate>
            <atom:updated>2026-08-15T04:43:27.041Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Oefwg8BhfKyngSnEnFrb8w.png" /></figure><p>In the previous article, I wrote about how Lightdash AI Agents can make business intelligence more conversational. This time, I want to look at the same change from the other direction. Instead of bringing an AI agent into the BI interface, what if we bring our governed BI context into the AI tools where analytics engineers already work?</p><p>That is what the official Lightdash MCP makes possible.</p><p><a href="https://docs.lightdash.com/references/integrations/lightdash-mcp">Lightdash MCP - Lightdash</a></p><p>Today, coding agents such as Claude Code, OpenAI Codex, and Cursor are becoming normal parts of software and analytics engineering. They can inspect a repository, read dbt models, modify YAML, write SQL, run tests, and reason about implementation details. But the codebase is only part of the context an analytics engineer needs. The other part is the analytics system itself: metrics, dimensions, Explores, dashboards, verified content, and the business definitions that our organization has already agreed on.</p><p>The official Lightdash MCP connects those worlds.</p><p>This article is not intended to be a complete MCP setup guide. Lightdash already maintains documentation for that. Instead, I want to focus on why this integration is interesting for analytics engineering, what kinds of workflows it enables, and why I think the semantic layer becomes even more important as AI makes data analysis easier.</p><h3>From BI to Agentic BI</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nxtxe8TKTP_NjcBXHMozbw.png" /></figure><p>Traditionally, BI has a fairly clear interface boundary.</p><pre>Human<br>  |<br>  v<br>BI interface<br>  |<br>  v<br>Lightdash semantic layer<br>  |<br>  v<br>Data warehouse</pre><p>If I am working in a dbt repository and want to investigate a business metric, I often leave my development environment, open the BI tool, find the relevant project or dashboard, explore the data, and then return to the codebase if the investigation points to a modeling problem.</p><p>There is nothing inherently wrong with that workflow. A visual BI interface is still an excellent place to explore data, build dashboards, and communicate results.</p><p>MCP adds another interface:</p><pre>Human<br>  |<br>  v<br>Coding agent<br>  |<br>  v<br>Official Lightdash MCP<br>  |<br>  v<br>Lightdash semantic layer<br>  |<br>  v<br>Data warehouse</pre><p>The important change is not simply that an LLM can call an API. We have been able to build API integrations for a long time. The interesting change is that an AI assistant can combine Lightdash’s analytics context with the development context it already has.</p><p>For an analytics engineer, this can connect two previously separate loops: investigating what is happening in the business and investigating how that information is modeled in code.</p><h3>Connecting Coding Agents to Lightdash</h3><p>Lightdash exposes an official MCP endpoint for Lightdash Cloud. As of August 2026, Lightdash documents integrations for clients including Claude, OpenAI Codex, Claude Code, and Cursor. Interactive integrations use OAuth, and Lightdash states that MCP respects the permissions and user attributes of the authenticated Lightdash user.</p><p>The current endpoint has the following shape:</p><pre>https://&lt;your-instance&gt;.lightdash.cloud/api/v1/mc</pre><p>For example, Claude Code can register the HTTP MCP endpoint and then complete the Lightdash OAuth flow. Cursor and Codex can similarly connect to the Streamable HTTP endpoint. The exact client configuration will continue to evolve, so I recommend using the official documentation rather than copying a configuration from an article months later.</p><p>What matters architecturally is what happens after authentication:</p><pre>Coding agent<br>     |<br>     | OAuth session<br>     v<br>Lightdash MCP<br>     |<br>     | Lightdash permissions + user attributes<br>     v<br>Projects / semantic layer / content / queries</pre><p>This is important. Connecting AI to analytics should not mean creating a new, ungoverned route around the BI platform.</p><p>Lightdash’s official MCP currently exposes a surprisingly broad set of capabilities. An assistant can discover projects, Explores, metrics and dimensions; find existing and verified content; execute semantic-layer metric queries; render results; work with charts and dashboards through content-as-code operations; use Lightdash AI Agent context; and more. Lightdash also gates capabilities using the caller’s permissions, and administrators can disable MCP content writes organization-wide.</p><p>In other words, this is much more interesting than a run_query tool.</p><h3>A Day-to-Day Analytics Engineering Workflow</h3><p>Consider a simplified question:</p><blockquote><em>Why did weekly active users decrease last week?</em></blockquote><p>A coding agent with warehouse access could immediately generate SQL. That can be useful, but I don’t necessarily want SQL generation to be the first step.</p><p>I would rather begin from the analytics concepts our organization already uses.</p><p>A workflow might look like this:</p><pre>Business question<br>      |<br>      v<br>Ask the coding agent<br>      |<br>      v<br>Discover the relevant Lightdash metric<br>      |<br>      v<br>Query and break down the metric<br>      |<br>      v<br>Identify an unexpected segment or period<br>      |<br>      v<br>Inspect the corresponding dbt models in the repository<br>      |<br>      v<br>Form and test hypotheses<br>      |<br>      v<br>Validate the result through Lightdash</pre><p>The conversation could start with prompts such as:</p><pre>Find the metric we use for weekly active users and show its trend for the<br>last eight weeks. Prefer verified Lightdash content where it exists.</pre><p>Then:</p><pre>Break down the decrease by the dimensions that are most likely to explain<br>it. Tell me which change contributes most to the week-over-week decline.</pre><p>Suppose the agent discovers that most of the decline is concentrated in one product surface. At that point, the investigation can cross the boundary into the repository:</p><pre>Find the dbt models that produce the fields used in this Lightdash Explore.<br>Check recent changes that could affect this segment, but don&#39;t modify<br>anything yet.</pre><p>Now the agent can reason over two kinds of context at once.</p><p>Analytics context tells it which metric the organization actually uses, which dimensions are available, which content is considered trustworthy, and what the data currently shows.</p><p>Engineering context tells it how those concepts are implemented, which models depend on one another, what changed in Git, and which tests or definitions may be relevant.</p><p>That combination is where I find MCP particularly compelling for analytics engineering.</p><p>It is not just “chat with the database.” It can become a bridge between the semantic representation of the business and the code that implements it.</p><h3>Why the Semantic Layer Matters More in the AI Era</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GgGKT0BOuArWJ9f17zuRJw.png" /></figure><p>A natural question is: if coding agents are increasingly capable of writing SQL, why involve Lightdash at all?</p><p>There are situations where direct warehouse access is exactly what we want. Lightdash itself even exposes a permission-gated SQL execution capability through its MCP. But arbitrary SQL and governed semantic queries solve different problems.</p><p>Compare these two paths:</p><pre>Agent -&gt; warehouse</pre><p>and:</p><pre>Agent -&gt; Lightdash -&gt; governed semantic layer -&gt; warehouse</pre><p>In the first path, the agent needs to infer business meaning from tables, columns, SQL, documentation, and whatever other context we provide. A column called revenue does not tell us whether it represents gross revenue, net revenue, booked revenue, recognized revenue, or some historical definition that should no longer be used.</p><p>The hard part of analytics has never been only SQL syntax.</p><p>Organizations invest in semantic layers because they want shared definitions of metrics and dimensions, reusable modeling, discoverability, governance, and a common vocabulary between producers and consumers of data. Those properties do not become obsolete because an LLM can write a query in seconds.</p><p>I think the opposite is likely to happen: semantic governance becomes more valuable as the marginal cost of asking questions approaches zero.</p><p>When humans manually build queries, the effort itself limits how many analytical paths we explore. Agents can generate and execute many more operations. That increases the importance of grounding those operations in the right definitions and applying the right permissions.</p><p>The question gradually shifts from:</p><blockquote><em>Can we generate the SQL for this analysis?</em></blockquote><p>toward:</p><blockquote><em>Is the agent using the right organizational definition, the right context, and the right access boundary?</em></blockquote><p>This is also why verified content is interesting in an agentic workflow. Lightdash’s MCP can discover content that administrators have marked as verified. An agent can therefore be instructed to prefer canonical dashboards or charts when looking for patterns or building new content instead of treating every artifact as equally authoritative.</p><h3>One Analytics Layer, Multiple Interfaces</h3><p>MCP does not make the Lightdash UI obsolete. It gives the same analytics platform another useful interface.</p><p>InterfaceParticularly useful forLightdash UIInteractive exploration, visualization, dashboards, and sharingLightdash AI AgentsConversational analytics with domain-specific agent contextOfficial Lightdash MCPExternal assistants and coding-agent workflowsLightdash APIProgrammatic integrations and purpose-built applications</p><p>I like thinking about this as separation between the analytics layer and its interfaces.</p><pre>                  +----------------+<br>                  | Lightdash UI   |<br>                  +-------+--------+<br>                          |<br>+--------------+    +-----v-------+    +----------------+<br>| Coding agent |---&gt;| Lightdash   |&lt;---| Custom app     |<br>| via MCP      |    | analytics   |    | via API        |<br>+--------------+    | layer       |    +----------------+<br>                    +-----+-------+<br>                          |<br>                          v<br>                    Data warehouse</pre><p>Different users and workflows do not need to interact with analytics in exactly the same way.</p><p>A business user may prefer a dashboard. An analyst may prefer the Explorer. An analytics engineer debugging a model may prefer Claude Code or Cursor. An internal application may need a focused UI that exposes only one workflow.</p><p>What should remain consistent is the underlying definition and governance of the data.</p><h3>Where the Official MCP Fits</h3><p>If your requirement is simply:</p><blockquote><em>I want my MCP-compatible assistant or coding agent to work with Lightdash.</em></blockquote><p>I would start with the official MCP.</p><p>Lightdash has already implemented OAuth authentication, permission-aware tool exposure, semantic-layer discovery, query execution, content operations, project context, AI Agent context, and client integrations. Reimplementing those capabilities creates code that somebody has to secure, test, deploy, observe, upgrade, and maintain.</p><p>There are also meaningful safety controls in the official implementation. For example, Lightdash documents that content creation and editing are permission-gated and can be disabled across MCP clients by an organization administrator. Its documentation also recommends read-only warehouse credentials because the MCP’s SQL capability can execute arbitrary SQL for users who have the required SQL Runner permission.</p><p>That is an important reminder: “the agent is read-only” is not something we should assume merely because our intended use case is analysis. We need to inspect the effective permissions of every layer, including the warehouse connection.</p><p>The official MCP should therefore be viewed as a strong default, not as a stepping stone that everyone eventually needs to replace.</p><p>But it also raises another possibility.</p><h3>From Agentic BI to Programmable BI</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*O15kK6Mar-f0ncU0Fa4Tcw.png" /></figure><p>Once we get used to accessing Lightdash from a coding agent, it becomes natural to ask a broader question:</p><blockquote><em>What if the interface itself were designed for one particular analytics job?</em></blockquote><p>Maybe we want a small governance application that only audits organizational resources. Maybe we want an internal application for a very specific business workflow. Maybe we want a specialized agent with a deliberately narrow tool surface. Or perhaps we do want a custom MCP server because we need API capabilities or operational behavior tailored to our environment.</p><p>Historically, many of these ideas would fail a simple cost-benefit test. Building and maintaining a custom application for a narrow internal workflow was expensive.</p><p>AI-assisted development is changing the implementation cost. It does not remove the cost of security, operations, testing, or maintenance, but it makes purpose-built software worth reconsidering in places where we previously defaulted to a single generalized interface.</p><p>That leads to a more fundamental problem than MCP tool design: identity.</p><p>If Alice and Bob use the same custom analytics application, should the application access Lightdash as one shared machine identity? Or should Lightdash still know that one request is being made on behalf of Alice and another on behalf of Bob, and apply their respective permissions?</p><p>For interactive custom applications, that distinction matters a lot.</p><p>Fortunately, Lightdash Cloud provides another useful building block: OAuth applications. That is the topic of Part 6.</p><h3>References</h3><ul><li><a href="https://docs.lightdash.com/references/integrations/lightdash-mcp">Lightdash MCP documentation</a></li><li><a href="https://www.lightdash.com/pricing">Lightdash pricing</a></li><li><a href="https://www.lightdash.com/privacy-policy">Lightdash privacy policy</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1584ea27f412" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Agents Need a Harness for Complexity of dbt Models]]></title>
            <link>https://yu-ishikawa.medium.com/ai-agents-need-a-harness-for-complexity-of-dbt-models-e171f6144651?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/e171f6144651</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[analytics-engineering]]></category>
            <category><![CDATA[agentic-engineering]]></category>
            <category><![CDATA[dbt]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Fri, 01 May 2026 12:46:39 GMT</pubDate>
            <atom:updated>2026-05-01T12:46:39.986Z</atom:updated>
            <content:encoded><![CDATA[<p>How a custom SQLFluff plugin can give coding agents feedback before complex SQL reaches human review.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*JzJ6vkLIYl5-37oI.png" /></figure><p>AI coding agents can generate dbt models in seconds. That is useful, but it also moves the bottleneck: the hard part is no longer writing SQL, it is reviewing whether the SQL should exist in that shape.</p><p>A model can compile, pass tests, and still be too complex for a human reviewer to trust.</p><p>That is the problem I kept running into in agent-assisted dbt work. The agent could produce runnable SQL quickly, but without a tight feedback loop it would often hand back a model that was technically valid and practically hard to review: too many CTEs, too many joins, dense CASE logic, or predicates that made the real business rule difficult to see.</p><p>This is where I think analytics engineering needs a different framing for static analysis. Linting is not only style enforcement anymore. In an agent workflow, static analysis can become part of the harness around the agent.</p><h3>What I Mean By An Agent Harness</h3><p>By agent harness, I mean the feedback loop around an AI coding agent: the checks, tools, constraints, and signals that tell the agent whether its output is acceptable before a person reviews it.</p><p>For dbt and SQL work, that harness might include dbt compilation, unit tests, model contracts, SQLFluff style rules, warehouse-specific checks, documentation checks, and review conventions. None of those replace a human analytics engineer. They give the agent a better way to notice when its work is drifting away from what the team can maintain.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ifI1hbMjE4YhnTCw.png" /></figure><p>The important part is the loop:</p><ol><li>A person asks an agent to build or refactor a model.</li><li>The agent writes SQL.</li><li>The harness checks the result.</li><li>The agent receives actionable feedback.</li><li>The agent simplifies the model before asking for review.</li></ol><p>That changes the role of tooling. A check that used to say “this file fails lint” can now say “this is the next concrete thing the agent should fix.”</p><h3>Valid SQL Is Not Enough</h3><p>SQL complexity is not a theoretical concern in dbt projects. It shows up in everyday review:</p><ul><li>A chain of CTEs hides which transformation stages deserve names.</li><li>A large join fan-in makes lineage and row multiplication harder to reason about.</li><li>Nested subqueries make it difficult to see the actual shape of the data.</li><li>Dense CASE expressions embed business rules directly in one file.</li><li>Large boolean predicates are easy to misread and hard to test by inspection.</li><li>Window-heavy statements can pack several analytic ideas into one model.</li></ul><p>Humans can learn a team’s taste for these tradeoffs. Agents need the taste encoded as feedback.</p><p>That does not mean every complex query is wrong. Sometimes complexity is intentional. The goal is not to ban it; the goal is to make complexity visible early enough that the author, whether human or agent, has to justify it.</p><h3>The Case Study: sqlfluff-complexity</h3><p><a href="https://github.com/yu-iskw/sqlfluff-complexity">sqlfluff-complexity</a> is a SQLFluff plugin for finding SQL and dbt models that are too complex to review comfortably. It adds CPX rules for CTE count, join count, nested subquery depth, CASE expressions, boolean predicates, window functions, and an aggregate weighted complexity score.</p><p><a href="https://github.com/yu-iskw/sqlfluff-complexity">GitHub - yu-iskw/sqlfluff-complexity: A custom SQLfluff plugin to calculate SQL complexity</a></p><p>The plugin works through SQLFluff’s parser. In dbt projects, that means it fits into the existing SQLFluff/dbt templater workflow and measures SQL that SQLFluff can parse. It does not try to score the dbt DAG, read manifest.json directly, or understand business meaning. Those are different problems.</p><p>The useful thing for an agent is that CPX feedback is specific:</p><pre>CPX_C102: join count 12 exceeds max_joins=8.<br>CPX_C201: aggregate complexity score 74 exceeds max_complexity_score=60.</pre><p>That gives the agent a concrete next move. Split the model. Extract an intermediate relation. Name the transformation stage. Move a classification rule into a mapping model. Or explain why the complexity is intentional.</p><h3>A Copyable SQLFluff Configuration</h3><p>Here is a small .sqlfluff example for a dbt project. The thresholds are deliberately ordinary starting points, not universal limits.</p><pre>[sqlfluff]<br>dialect = postgres<br>templater = dbt<br>rules = CPX_C101,CPX_C102,CPX_C103,CPX_C104,CPX_C105,CPX_C106,CPX_C201<br><br>[sqlfluff:templater:dbt]<br>project_dir = .<br>profiles_dir = ~/.dbt<br>profile = my_project<br>target = dev<br>dbt_skip_compilation_error = False<br><br>[sqlfluff:rules:CPX_C101]<br>max_ctes = 8<br>[sqlfluff:rules:CPX_C102]<br>max_joins = 8<br>[sqlfluff:rules:CPX_C103]<br>max_subquery_depth = 3<br>[sqlfluff:rules:CPX_C104]<br>max_case_expressions = 10<br>[sqlfluff:rules:CPX_C105]<br>max_boolean_operators = 20<br>[sqlfluff:rules:CPX_C106]<br>max_window_functions = 10<br>[sqlfluff:rules:CPX_C201]<br>max_complexity_score = 60<br>complexity_weights = ctes:2,joins:2,subquery_depth:4,case_expressions:2,boolean_operators:1,window_functions:2<br>mode = enforce</pre><p>Then run the normal SQLFluff command:</p><pre>sqlfluff lint models/</pre><p>For teams that want to calibrate before enforcing, the companion report command can be used as a non-blocking signal:</p><pre>sqlfluff-complexity report --dialect postgres --config .sqlfluff models/</pre><p>The project docs include more detail on <a href="https://github.com/yu-iskw/sqlfluff-complexity/blob/preview/docs/configuration.md">configuration</a>, <a href="https://github.com/yu-iskw/sqlfluff-complexity/blob/preview/docs/dbt.md">dbt usage</a>, <a href="https://github.com/yu-iskw/sqlfluff-complexity/blob/preview/docs/rules.md">rule meanings</a>, and <a href="https://github.com/yu-iskw/sqlfluff-complexity/blob/preview/docs/reporting.md">report mode</a>.</p><h3>What The Signals Mean</h3><p>The plugin does not try to understand whether a metric is good or bad in isolation. It asks review-oriented questions:</p><ul><li>How many CTEs does this statement contain?</li><li>How many joins does it perform?</li><li>How deeply are subqueries nested?</li><li>How much CASE and boolean logic is embedded inline?</li><li>How many window functions are packed into one statement?</li><li>What is the aggregate complexity score?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*pFE6Xa4zxu0jlGf6.png" /></figure><p>Those questions are useful because they are close to how analytics engineers review SQL. A reviewer may not care that the score is exactly 74. They care that the score points to a model where several sources of review difficulty have accumulated.</p><h3>The Agent Workflow</h3><p>Here is the workflow I want from an AI coding agent working on dbt models:</p><ol><li>Make the requested change.</li><li>Run SQLFluff with CPX rules enabled.</li><li>Treat CPX failures as refactoring feedback.</li><li>Re-run the check.</li><li>Hand off the pull request with either simpler SQL or a clear explanation of why the remaining complexity is intentional.</li></ol><p>This can be turned into a short agent instruction:</p><pre>Before handing off dbt model changes, run SQLFluff with CPX rules enabled.<br>If any CPX rule fails, simplify the model or explain why the complexity is intentional.<br>Prefer extracting named intermediate models over leaving large joins, deep subqueries,<br>or dense business logic in a single SQL file.</pre><p>That instruction is intentionally small. The harness should make the right behavior easy, not bury the agent in a policy document.</p><h3>The Gap Is Narrow, But Real</h3><p>I did not build this because SQL tooling is empty. SQLFluff, dbt, warehouse query analyzers, semantic-layer tooling, and CI checks already solve important problems.</p><p>The gap I kept running into was narrower: I wanted a dbt-friendly static feedback signal that an AI coding agent could use to notice when a model was becoming too complex to review.</p><p>That distinction matters. sqlfluff-complexity is not a replacement for dbt tests, model contracts, code review, observability, or good modeling judgment. It is a reviewability signal. It gives both agents and humans a shared vocabulary for one question:</p><blockquote><em>Is this SQL still small enough to reason about?</em></blockquote><h3>How I Would Roll This Out</h3><p>I would not start by failing every existing model in CI.</p><p>Start in report mode. Look at the models that already feel painful to review and see whether the signals match your intuition. Set generous thresholds first. Tighten rules where ownership is mature. Use path overrides if staging models and marts deserve different budgets. Let exceptions be explicit.</p><p>Most importantly, make the agent run the check before handoff. The best time to simplify generated SQL is before a reviewer has already spent twenty minutes reconstructing the model in their head.</p><h3>Faster SQL Needs Faster Feedback</h3><p>The future of analytics engineering is not just faster SQL generation. It is faster feedback.</p><p>If agents are going to write dbt models with us, they need the same review signals we wish every human author had before opening a pull request. Static analysis is one practical way to provide those signals. <a href="https://github.com/yu-iskw/sqlfluff-complexity">sqlfluff-complexity</a> is my attempt to make one part of that harness concrete.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e171f6144651" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Don’t Just Prompt, Orchestrate: The Architecture of Multi-Agent Team Management]]></title>
            <link>https://yu-ishikawa.medium.com/dont-just-prompt-orchestrate-the-architecture-of-multi-agent-team-management-dd326b3ceab5?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/dd326b3ceab5</guid>
            <category><![CDATA[project-management]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[product-management]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Sat, 31 Jan 2026 00:48:07 GMT</pubDate>
            <atom:updated>2026-01-31T00:48:07.812Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Moving beyond “chatting with code” to managing autonomous sub-agent swarms.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IEIUh1QOI5xtqLttX93msg@2x.jpeg" /></figure><h3>The Promotion: From Coder to Fleet Commander</h3><p>We are currently witnessing a silent promotion in the software engineering career track. Until recently, using AI meant being an “individual contributor” – you typed a prompt, and the AI wrote some code. You were still the one doing the typing, just faster.</p><p>But with the rise of Sub-Agent Architectures (like those in Claude Code, Gemini CLI, and Cursor), the role has fundamentally changed. We are no longer just “coding with AI.” We are acting as Engineering Managers for synthetic teams.</p><p>The bottleneck isn’t the AI’s intelligence anymore; it’s the management architecture. Just as a human manager fails if they micromanage every line of code, an AI engineer fails if they try to “prompt” every single action. To build at scale, we must stop prompting and start orchestrating.</p><p>We built Vibe to demonstrate this shift: a system where you don’t talk to a chatbot – you direct a hierarchy of specialized agent teams.</p><h4>The New Organizational Chart</h4><p>In a traditional “Copilot” workflow, you have a 1:1 relationship with an AI. In the Vibe Architecture, we treat the AI as a Multi-Agent Team. We structure our work not as a series of chat messages, but as a delegation hierarchy.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RuXVXE3SfOut8PCX3poi2g@2x.jpeg" /><figcaption>hi</figcaption></figure><p>The human user acts as the Director of Intent, defining the what and why. Below them sits the Orchestrator (your Staff Architect), and below that, the parallel Execution Swarms.</p><h3>The Management Interface: Kanban as Protocol</h3><ol><li>If you have five sub-agents working in parallel, a chat window is the wrong interface. You need a management dashboard.</li><li>We use Vibe Kanban (powered by the Model Context Protocol) as the “Neural Link” between the human manager and the agent teams. It’s where strategy is converted into executable tasks.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*emt9jsiwd69nOAILTuX8Gw@2x.jpeg" /></figure><p>Your job is to review the “Backlog” and “To Do” columns. You don’t check code syntax; you check the plan. “Are we missing a migration?” “Is this the right library?” Once you approve, the tickets are broadcast to the sub-agent swarms for execution.</p><h3>2. The “Plan, Then Swarm” Workflow</h3><p>This architecture enables a fundamentally different workflow. Instead of a linear back-and-forth conversation, you have a “plan and fan-out” process.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5CLG58iXbzC7sGVro-Gjdw@2x.jpeg" /></figure><p>Drafting: The Orchestrator analyzes your high-level goal and researches the codebase.</p><p>Approval: It presents a plan (often with an ADR – Architectural Decision Record) for your sign-off.</p><p>Parallel Execution: Once approved, it decomposes the plan into atomic tickets on the Kanban board. These are then dispatched to specialized sub-agents (Builder, Verifier) who work in parallel using ephemeral Git worktrees.</p><h3>3. Engineering Culture for Robots: “Hygiene as Code”</h3><p>A human team needs a handbook to maintain quality. An AI team needs “Hygiene as Code.”</p><p>If we let sub-agents run wild, we get “AI Debt” – unmaintainable code generated at lightning speed. To prevent this, we inject “Agent Skills” that force our synthetic teams to follow strict protocols.</p><p>Mandatory ADRs: Before writing significant code, the Orchestrator is constrained to write an ADR, documenting why it chose a specific path. This creates a persistent “Architectural Memory.”</p><p>Automated Changelogs: We force agents to use a manage-changelog skill. Every task completion requires a structured entry, turning execution chaos into a clean, human-readable release note.</p><h3>The Future: The Rise of the AI Architect</h3><p>The “Prompt Engineer” is a transitional role. The future belongs to the AI Architect and Team Manager.</p><p>Your value is no longer defined by how well you can craft a sentence to trick an LLM into writing code. It is defined by your ability to:</p><p>Design the topology of your sub-agent teams.</p><p>Define the “Rules of Engagement” (hygiene skills) they must follow.</p><p>Orchestrate the flow of work through a Kanban-driven system.</p><p>We are entering the era of Scalable Management. You are no longer the lone wolf coding in the dark; you are the commander of a fleet. The only question is: are you ready to lead?</p><p>Want to inspect our team architecture? Check out <a href="https://github.com/yu-iskw/vibe-team-coding-demo">the GitHub repository</a> and look at our .cursor/rules and sub-agent definitions to see how we structure our digital workforce.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dd326b3ceab5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Agentic Makefile: Why Every Repository Needs a Self-Describing AI Layer]]></title>
            <link>https://yu-ishikawa.medium.com/the-agentic-makefile-why-every-repository-needs-a-self-describing-ai-layer-b772d9fac440?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/b772d9fac440</guid>
            <category><![CDATA[claude-code]]></category>
            <category><![CDATA[vibe-coding]]></category>
            <category><![CDATA[governance-and-tech]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[agent-skills]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Mon, 26 Jan 2026 05:58:25 GMT</pubDate>
            <atom:updated>2026-01-26T05:58:25.892Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’ve started using AI agents like Claude Code or Cursor lately, you’ve likely encountered a new kind of friction.</p><p>Every time you open a new repository, the dance begins. You ask the agent to run tests, and it fails because it doesn’t know you’re using poetry instead of pip. You ask it to fix lint errors, and it tries to run eslint when your project is strictly ruff.</p><p>You spend the first ten minutes of every session “teaching” the agent your project’s specific commands. It’s a recurring tax on productivity — a Developer’s “Groundhog Day” where the local context is lost the moment you switch contexts.</p><p>We’ve reached a point where our agents are brilliant, but they are also “repository-blind.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WMygUnDZt-si9hjwRvvBwA.png" /></figure><h3>The Initialization Tax: A Developer’s New “Groundhog Day”</h3><p>At an organizational level, this “tax” isn’t just an individual annoyance — it’s a management bottleneck.</p><p>As teams adopt AI agents across dozens or hundreds of repositories, the fragmentation of tech stacks (Python, TypeScript, Go) and tooling (pip vs. poetry, npm vs. yarn) becomes a major hurdle. Each project requires manual effort to “teach” the agent how to run tests or apply linting. This leads to several critical issues:</p><ul><li><strong>Inconsistency</strong>: Agent instructions across different repositories are often implemented in varied ways, making the developer experience unpredictable.</li><li><strong>Maintenance Burden</strong>: Manually maintaining and synchronizing specialized skill files across hundreds of repositories is nearly impossible. As tools change, the agentic instructions quickly become stale and “broken.”</li><li><strong>Wasted Effort</strong>: Instead of focusing on building features, developers and platform engineers spend valuable time implementing and troubleshooting the same boilerplate agent configurations.</li></ul><p>Without a centralized way to standardize these workflows, the promise of “agentic productivity” is drowned in a sea of manual configuration.</p><h3>The Vision: The “Agentic Makefile”</h3><p>Decades ago, we solved this for humans with the Makefile. Before make, a developer had to read a README (if they were lucky) to find the specific compiler flags and linker paths. make build and make test became the universal interface. It didn&#39;t matter if the underlying code was C, Fortran, or Go—the interface was the same.</p><p>In 2026, we need a <strong>Makefile for Agents</strong>.</p><p>We need a standardized, self-describing layer that tells an agent: “In this repository, this is how you test, this is how you lint, and this is how you build.”</p><p>This is where the <strong>Agent Skill</strong> framework comes in, and more specifically, the <strong>Meta-Agent Skill</strong> (meta-agent-skills).</p><p><a href="https://github.com/yu-iskw/meta-agent-skills">GitHub - yu-iskw/meta-agent-skills</a></p><h3>Meet the Meta-Agent Skill: A Factory for Capabilities</h3><p>The meta-agent-skills is what we call a &quot;Meta-Skill.&quot; It isn&#39;t just a single capability like &quot;run tests&quot;; it is a <strong>skill factory</strong>.</p><p>Instead of manually writing .claude/skills/test-unit/SKILL.md for every repo, you simply hand the agent the Meta-Skill. This single &quot;point of truth&quot; eliminates the need to implement hundreds of individual skills across your organization. The agent then performs a three-step bootstrap:</p><ol><li><strong>Detect</strong>: It scans your codebase for “signals.” It looks for pyproject.toml, package.json, go.mod, or Makefile. It distinguishes between pytest, mocha, and go test.</li><li><strong>Verify</strong>: Crucially, it doesn’t just guess. The agent proactively verifies its findings. It runs poetry --version or npm run lint -- --help to ensure the commands it&#39;s about to &quot;learn&quot; actually work in your environment.</li><li><strong>Instantiate</strong>: Finally, it reads markdown templates and fills in the blanks. It generates specialized, ready-to-use Agent Skills (like lint-fix, test-unit, security-scan) and even specialized Subagents (like a QA Engineer or Security Auditor) tailored specifically to your tech stack.</li></ol><h3>Architecture: No Scripts, Just Brains</h3><p>One of the most radical departures in this implementation is that there are <strong>no Python or Bash scripts</strong> driving the generation logic.</p><p>Traditional automation relies on rigid if/else logic. If package.json exists, then assume npm. But what if a project has both npm and pip? What if it uses bun?</p><p>The Meta-Skill leverages the agent’s <strong>semantic understanding</strong>. By using a prompt-driven approach (Instructions + Templates), the agent can handle edge cases that would break a hard-coded script. It reads your project’s CONTRIBUTING.md and <em>understands</em> that you prefer yarn over npm, then configures the lint-fix skill accordingly.</p><p>We’ve swapped fragile regex-based detection for LLM-based reasoning.</p><h3>Polyglot Power: Consistency Across the Stack</h3><p>The result of this “Agentic Makefile” is a unified experience across any language. Whether you are in a Go microservice or a Next.js frontend, your interaction with the AI agent remains identical.</p><h3>Example: The lint-fix Skill</h3><p>In the background, the Meta-Skill uses a template like this:</p><pre># Lint Fix<br><br>1. Run Linter: `{{ lint_fix_command }}`<br>2. Run Type Checker: `{{ type_check_command }}`</pre><p>The AI agent fills those placeholders dynamically:</p><ul><li><strong>In Python</strong>: It becomes ruff check --fix and mypy ..</li><li><strong>In TypeScript</strong>: It becomes eslint --fix and tsc --noEmit.</li><li><strong>In Go</strong>: It becomes golangci-lint run --fix and go vet ./....</li></ul><p>The developer doesn’t care <em>how</em> it’s configured; they just know that if they say “fix the lint errors,” the agent has the correct, verified instructions to do so.</p><h3>Beyond Skills: The Rise of Subagents</h3><p>It doesn’t stop at atomic skills. The Meta-Skill also generates specialized <strong>Subagents</strong>.</p><p>By analyzing the repo, it can instantiate a QA Engineer agent that knows exactly how to run your Cypress or Playwright tests, or a Security Auditor that knows how to interpret trivy scans. You aren&#39;t just getting commands; you&#39;re getting a team of virtual experts that understand your specific tooling.</p><h3>Conclusion: Toward an “Agent-Ready” Standard</h3><p>The “Agentic Makefile” concept isn’t just about saving ten minutes of setup. It’s about building a future where repositories are <strong>“Agent-Ready”</strong> by default, and organizations can scale their agentic workflows without the overhead of manual configuration.</p><p>As AI agents become a standard part of our IDEs and CI/CD pipelines, the way we describe our projects needs to evolve. We shouldn’t just write documentation for humans; we should provide a machine-readable (and agent-understandable) layer that defines our project’s operational boundaries.</p><p>By adopting a meta-skill approach, we stop fighting with our tools and start collaborating with them.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b772d9fac440" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Beyond the Data Mesh: The Shift to Human-to-AI-to-Human Enablement in Data Platform Engineering]]></title>
            <link>https://yu-ishikawa.medium.com/beyond-the-data-mesh-the-shift-to-human-to-ai-to-human-enablement-in-data-platform-engineering-4035796770ce?source=rss-11f93f9d3bca------2</link>
            <guid isPermaLink="false">https://medium.com/p/4035796770ce</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-governance]]></category>
            <category><![CDATA[platform-engineering]]></category>
            <dc:creator><![CDATA[Yu Ishikawa]]></dc:creator>
            <pubDate>Thu, 22 Jan 2026 04:10:00 GMT</pubDate>
            <atom:updated>2026-01-22T04:10:00.595Z</atom:updated>
            <content:encoded><![CDATA[<p>For the last few years, Data Platform Engineering has been chasing the dream of “Self-Service Analytics” and the “Data Mesh.” We built robust platforms using tools like dbt and Airflow, set up Data Catalogs, and wrote endless documentation on “How to define a metric.” The goal was the same as in software engineering: reduce the bottleneck so Analysts and Analytics Engineers could move fast.</p><p>But let’s be honest about the cost. Traditionally, enabling a data team meant <strong>Human-to-Human </strong>communication. You write a confluence page on “Naming Conventions for dbt Models”; a new Analytics Engineer reads it (maybe). You hold a meeting to explain why a specific SQL query is blowing up the warehouse costs; they listen (hopefully). When the data warehouse pricing model changes or we migrate from Legacy SQL to Standard SQL, we have to repeat the cycle.</p><p>We are hitting the limits of what human-to-human enablement can scale efficiently. The “Data Democratization” we strove for has often resulted in “Data Chaos” — duplicated logic, unoptimized queries, and a governance nightmare that requires <em>more</em> humans to police.</p><p>We need a paradigm shift. We must move from <strong>Human-to-Human</strong> enablement to <strong>Human-to-AI-to-Human</strong> enablement.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ShsYT0P0AgHDLIFv_sr51Q.png" /></figure><h3>The New Enablement Model: Human-to-AI-to-Human</h3><p>In this new model, the primary job of the Data Platform Engineer is no longer just maintaining the infrastructure or writing docs for humans, but to write <strong>context and skills for AI Agents</strong>.</p><ol><li><strong>Human (Data Platform Engineer):</strong> Codifies governance rules, query patterns, and architectural standards into AI-consumable formats (Sub-agents, Skills, MCP servers).</li><li><strong>AI (The Agent):</strong> Absorbs this context instantly — it knows your schema, your semantic layer, and your cost constraints.</li><li><strong>Human (Analytics Engineer/Analyst):</strong> Interacts with the AI. The AI guides them to the right table, writes the boilerplate dbt YAML, and optimizes their SQL before it ever hits the warehouse.</li></ol><p>This is <strong>Governance as Code</strong>, but actually usable.</p><p>When you write a Wiki page on “Optimizing for Partition Pruning,” you rely on an analyst remembering to check the partition column. When you build a <strong>Query Optimizer Sub-agent</strong>, the analyst simply asks, <em>“Write a query to get last month’s revenue,”</em> and the agent automatically applies the correct partition filters because <em>you</em> taught it to.</p><p>If the “manner” of the technology changes — say, we migrate our orchestration from Airflow to Prefect, or our transformation logic from stored procedures to dbt — you don’t need to retrain the entire analytics organization. You update the orchestrator-agent&#39;s underlying Skill. To the analyst, the interface remains the same: they ask the agent to &quot;Schedule this job,&quot; and the new logic handles the rest.</p><h3>The Technical Implementation: Agents, Skills, and Context in Data</h3><p>To make this practical for Data Platforms, we need to map emerging AI standards to our data stack. The “Data Catalog” of the future isn’t just a searchable website; it’s an active Agent interface.</p><h3>1. Context as Code: The Semantic “README”</h3><p>We are seeing the rise of standardized context files like AGENTS.md. In a data repo, this is critical. A human README.md often fails to capture the nuance of &quot;Which users table is the <em>real</em> one?&quot;</p><ul><li><strong>Traditional way:</strong> A new Data Analyst joins and spends 2 weeks asking, “Is dim_users or users_snapshot the source of truth?&quot;</li><li><strong>New way:</strong> The Data Platform team maintains an AGENTS.md (or leverages the Semantic Layer). When the analyst opens their IDE (Cursor, VS Code), the agent reads this context. It knows that dim_users is the gold standard and that users_snapshot is deprecated <em>before the analyst writes their first join</em>.</li></ul><h3>2. Capabilities as Code: Data Skills</h3><p>A major bottleneck in Data Platform Engineering is “Support Tickets.” Platform teams are constantly asked to grant permissions, debug failed pipelines, or explain why a test failed.</p><p>Enter <a href="https://agentskills.io/"><strong>Agent Skills</strong></a>. Agent Skills are portable bundles of expertise. Instead of teaching every analyst how to parse a complex JSON log in BigQuery or Snowflake, the Platform team builds a log-parsing-skill. The analyst simply asks, <em>&quot;Extract the error messages from the raw event logs,&quot;</em> and the agent executes the correct, optimized SQL function.</p><p>This decouples the <strong>what</strong> (business logic) from the <strong>how</strong> (SQL dialect). If you switch your warehouse backend, you update the Skill, and the analyst’s workflow (asking the agent) remains uninterrupted.</p><h3>3. The Workforce: Specialized Data Sub-agents</h3><p>Perhaps the most powerful shift is the concept of <strong>Sub-agents</strong> tailored for the data lifecycle.</p><ul><li><strong>The “Governance Steward” Agent:</strong> This agent doesn’t just check code style; it checks <em>meaning</em>. It can be configured to flag any query that accesses PII (Personally Identifiable Information) without masking, or strictly enforce that all new dbt models must have a description and unique test defined in YAML.</li><li><strong>The “Migration” Agent:</strong> If you are moving from a legacy system (like Hive or proprietary stored procs) to a modern stack (dbt/Cloud DW), this agent specializes in translating the logic, not just the syntax. It knows your specific mapping rules.</li></ul><p>By implementing these Sub-agents, you scale your Platform team’s “Senior Engineer” intuition to every junior analyst in the organization.</p><h3>The Strategic Advantage: Why This Matters for Data Executives</h3><p>For managers and executives, this shift addresses the core tension of Data Engineering: <strong>Self-Service usually leads to Tech Debt.</strong></p><p>Building a “Golden Path” for data often turns into a “Golden Handcuff” where changing the platform breaks thousands of user-created dashboards and queries.</p><p>By leveraging AI agents:</p><ol><li><strong>You Decouple Business Logic from Implementation Details:</strong> You can refactor your core data models or change your data quality tool (e.g., from Great Expectations to dbt Tests) without breaking the analysts’ workflow. You just update the Agents.</li><li><strong>You Scale Cost Optimization:</strong> You don’t need to manually review every PR for expensive joins. The Optimizer-Agent catches full-table scans instantly, explaining the cost impact to the user and rewriting the query to be efficient.</li><li><strong>You Focus on High-Value Infrastructure:</strong> Your Data Platform team stops answering “Where is this data?” questions. They focus on <em>Reliability, Scalability, and Architecture</em> — ensuring the pipes are clean so the agents can serve the water.</li></ol><h3>Urgent Movements: What We Must Do Now</h3><p>We are at an inflection point in Data &amp; Analytics. The tools are here, but the workflow is stuck in 2020. Here is the urgent agenda for Data Platform leaders:</p><ol><li><strong>Standardize Metadata for Agents (Adopt AGENTS.md):</strong> Ensure your data dictionary and glossary are accessible to AI. Don’t lock them in a GUI-only Data Catalog. Expose them via text files or APIs that agents can read.</li><li><strong>Build “Agent-Ready” Data Tools:</strong> Stop relying solely on GUI-based ETL tools that hide logic. Prefer code-first approaches (like dbt or Dataform) or tools that expose readable configs (YAML/JSON) so Agent Skills can interact with them.</li><li><strong>Design for “Sub-agent” Architecture:</strong> Don’t just give your analysts generic ChatGPT. Curate a library of Sub-agents that know <em>your</em> specific schemas (“The Marketing Data Expert,” “The Finance Data Expert”).</li><li><strong>Embrace MCP (Model Context Protocol):</strong> This is the future of connecting AI to your Data Warehouse context. If you use tools like DataHub or Amundsen, think about how to expose that metadata via MCP so an agent in the IDE knows who owns a dataset immediately.</li></ol><h3>Conclusion</h3><p>The era of the “Data Dictionary” wiki page is ending. The future is an active, intelligent layer between the data consumer and the data platform.</p><p>By shifting to a <strong>Human-to-AI-to-Human</strong> model, we acknowledge a simple truth: The data landscape is too complex for everyone to know everything. We cannot afford to manually teach every analyst the nuances of every table and every tool.</p><p>Instead, we teach the Agents. And the Agents enable the humans.</p><p>The Data Platform teams that succeed in the next 5 years won’t be the ones with the strictest permissions or the most meetings. They will be the ones who have built the most capable fleet of AI agents to guide their organization through the data.</p><h3>References &amp; Further Reading</h3><ul><li><strong>Agent Skills Standard:</strong> <a href="https://agentskills.io/home">https://agentskills.io/home</a></li><li><strong>Claude Code Sub-agents:</strong> <a href="https://code.claude.com/docs/en/sub-agents">https://code.claude.com/docs/en/sub-agents</a></li><li><strong>Cursor Sub-agents &amp; Context:</strong> <a href="https://cursor.com/docs/context/subagents">https://cursor.com/docs/context/subagents</a></li><li><strong>AGENTS.md Specification:</strong> <a href="https://agents.md/">https://agents.md/</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4035796770ce" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>