<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[paradime.io - Medium]]></title>
        <description><![CDATA[Paradime is the Cursor for Data platform for agentic data engineering. Data engineers and analysts use Paradime to code, ship, fix, and scale data pipelines 10x faster. - Medium]]></description>
        <link>https://medium.com/paradime-labs?source=rss----cd7e09652475---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>paradime.io - Medium</title>
            <link>https://medium.com/paradime-labs?source=rss----cd7e09652475---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 09:52:57 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/paradime-labs" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[How to migrate from dbt Core™ v1 to dbt™ v2?]]></title>
            <link>https://medium.com/paradime-labs/how-to-migrate-from-dbt-core-v1-to-dbt-v2-4d8264c03440?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/4d8264c03440</guid>
            <category><![CDATA[dbt]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Thu, 01 Oct 2026 12:24:46 GMT</pubDate>
            <atom:updated>2026-10-01T12:24:46.793Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rb8OfjdLksxbEdfJQ1iedw.png" /></figure><p>dbt™ v2 is the current major era of dbt™: a Rust rewrite (often called Fusion) that ships as a self-contained binary, connects via ADBC drivers, and is stricter at parse time while keeping familiar project language and DAG semantics. Official docs (2026) treat the move as prepare on latest v1 → fix deprecations → install v2 → validate. You can keep v1.x if packages or tooling are not ready; new capabilities land in v2 only over time.</p><h3>Prepare on the latest v1 track</h3><p>Upgrade environments and jobs to the v1 latest release track first. Resolve every deprecation warning while still on v1 — warnings become errors in v2.</p><ul><li>Use dbt-autofix (<a href="https://docs.paradime.io/products/code-ide">Paradime Code IDE</a>, VS Code/Cursor extension, or the autofix GitHub repo) to move arbitrary configs into meta, upgrade packages, and clear common YAML issues.</li><li>Remove any flags: overrides in dbt_project.yml that opt out of behavior-change flags; v2 forcibly enables them.</li><li>On dbt Core™ v1.12, dry-run the Rust parser without switching engines:</li></ul><pre>dbt parse - use-v2-parser</pre><p>Confirm packages list require-dbt-version including 2.0.0 (or higher) on the dbt™ package hub. Popular dbt-labs packages (dbt_utils, dbt_project_evaluator, audit_helper, dbt_external_tables) are already v2-compatible; bump others or temporarily remove unready ones.</p><h3>Install v2 and verify</h3><p>Upgrading locally is an install step. Recommended distribution is the dbt package (v2 by default). For Apache 2.0-only orgs, use dbt-oss. Stay on v1.x only if you must.</p><pre>python -m pip install dbt<br><br># or later:<br>python -m pip install - upgrade dbt<br>dbt - version<br>dbt login status # optional; needed for static_analysis: strict</pre><p>Other installers: Homebrew, curl install script, or winget install --id dbtLabs.dbt --exact.</p><ul><li>v2 adapters are bundled — no separate pip install dbt-&lt;adapter&gt;.</li><li>On first run, dbt™ downloads ADBC drivers from the dbt Labs CDN (public.cdn.getdbt.com) and caches them; use dbt system install-drivers to pre-cache.</li><li>GA adapters include Snowflake, BigQuery, Redshift, and Databricks; ClickHouse (private beta), Spark (CLI beta), and DuckDB (CLI) are also listed. Adapter GA timing can differ between dbt™ platform and local CLI.</li></ul><h3>Profiles, project config, and side-by-side runs</h3><p>CLI still uses profiles.yml. Search order: --profiles-dir, then project root, then ~/.dbt/ (recommended). Create or refresh with dbt init, then dbt debug. Profile type remains the adapter name (snowflake, bigquery, databricks, redshift, etc.); credentials stay in the profile/target as before.</p><p>In dbt_project.yml:</p><ul><li>Drop opt-out flags:</li><li>Fix unused/unknown config paths (v2 errors on them)</li><li>Prefer explicit static analysis:</li></ul><pre>models:<br>  your_project:<br>    +static_analysis: baseline # default; warnings only<br>    # +static_analysis: strict # needs dbt login; column lineage / type checks</pre><p>Move standalone YAML anchors under a top-level anchors: key; self-referential recursive anchors are unsupported. Access custom meta configs with config.meta_get() / config.meta_require(), not config.get() / config.require().</p><p>v2 writes a v12 manifest compatible with v1 (extra v2-only fields are ignored by v1), so you can run v2 and v1 side by side; state:modified, --defer, and cross-environment dbt docs generate work across mixed engines.</p><h3>Scripts, jobs, and known breaking changes</h3><ul><li>Replace --models / --model / -m with --select / -s (v2 errors on the old flags).</li><li>Remove --partial-parse / --no-partial-parse from job commands (warning dbt1700).</li><li>Many historic flags (--print, --single-threaded, --use-experimental-parser, etc.) become no-ops with warnings—clean them from CI.</li><li>Parse fails earlier than v1 on missing macros, adapter methods, generic tests, and undefined var()s.</li><li>Conflicting package version pins between root and local packages error at dbt deps.</li><li>dbt clean no longer deletes resource-path files or paths outside the project.</li><li>Unit tests in dbt build run first, then the rest of the DAG.</li><li>Threading defaults differ by adapter (Snowflake/Databricks auto-manage parallelism; BigQuery/Redshift respect --threads, with 0/omit for dynamic).</li></ul><p>After install, validate:</p><pre>dbt deps<br>dbt parse<br>dbt compile<br>dbt build<br>dbt test<br><br># optional v2 extras:<br>dbt lint<br>dbt freshness<br>dbt docs generate</pre><p>On dbt™ platform: prepare on v1 Latest → upgrade assistant for dev → staging → prod, selecting v2 Stable; rollback to v1 Latest only for production-critical issues.</p><p>Full reference: <a href="https://docs.getdbt.com/docs/dbt-versions/core-upgrade/upgrading-to-v2">Upgrading to v2</a> and the <a href="https://docs.getdbt.com/docs/dbt/dbt-readiness">v2 readiness checklist</a>.</p><h3>Automate the migration with Paradime DinoAI agents</h3><p>The checklist above is mostly mechanical: find deprecations, rewrite flags, bump packages, fix YAML, then re-parse and rebuild. That is exactly what <a href="https://docs.paradime.io/dino-ai">Paradime DinoAI</a> is built for — warehouse-aware agents that can read your dbt™ project, run terminal commands, open PRs, and heal failed pipelines.</p><h4>1. Clear deprecations in the IDE with Copilot</h4><p>Open the Paradime Code IDE and use DinoAI Copilot in Agent mode. Attach terminal context from dbt parse --use-v2-parser (or a failing dbt parse after you switch engines), then prompt for a migration pass, for example:</p><blockquote><em>Resolve every v1→v2 deprecation in this project: move arbitrary configs into </em><em>meta, replace </em><em>--models/</em><em>-m with </em><em>--select/</em><em>-s, drop opt-out </em><em>flags: and unknown config paths, relocate YAML anchors under </em><em>anchors:, and bump packages whose </em><em>require-dbt-version excludes 2.0.</em></blockquote><p>Review the diff, run parse again, and open a PR from the panel. Commit a .dinorules (and optional .dinoprompts) so every later agent run follows the same v2 conventions.</p><h4>2. Codify a version-controlled migration agent</h4><p>For a repeatable, unattended pass, define a Programmable Agent under .dinoai/agents/ (or build it in the Agents app and deploy as YAML). Give it a fixed role/goal such as “dbt™ v2 migration fixer,” allowlist tools for file edits, code search, terminal (dbt parse, dbt deps, dbt build), and GitHub PRs, then trigger it from a Bolt schedule or the Paradime API / Python SDK.</p><ul><li>Attended runs (Copilot, Slack) can ask clarifying questions.</li><li>API and Bolt runs assume and continue — ideal for overnight remediation across large repos.</li><li>Pin the agent to an environment that still runs v1.12 while you prepare, then retarget a v2 environment once packages and CI are clean (Paradime supports per-environment dbt™ versions).</li></ul><h4>3. Validate on Bolt, then Fix with DinoAI</h4><p>Point a Bolt schedule at the upgraded project (or a side-by-side v2 environment) and run dbt deps → dbt parse → dbt build. When something fails, open the run’s DinoAI tab and use Fix with DinoAI: it summarises the log, proposes SQL/YAML/config edits, and opens a PR for review. Turn on Self-Healing if you want AutoPilot to attempt the fix and re-run without waiting. After you are on dbt™ 2.0, Paradime State can skip or clone unchanged models so migration validation builds only what changed.</p><p>Start from the <a href="https://docs.paradime.io/dino-ai">DinoAI docs</a> — Copilot for interactive fixes, Programmable Agents for automated sweeps, and the Bolt Pipeline Agent for production failures after the cutover.</p><p><a href="https://www.paradime.io/"><em>Paradime</em></a><em> automates data engineering, DataOps, and ITOps using AI agents. Anyone can use Paradime to build agents and run data pipelines together in a single place. Customers like Braze, Tide, and FTSE100 enterprises use Paradime to run their data operations from a single place. This publication is managed and maintained by the Paradime Team.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4d8264c03440" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/how-to-migrate-from-dbt-core-v1-to-dbt-v2-4d8264c03440">How to migrate from dbt Core™ v1 to dbt™ v2?</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What does the #1 harness for data and dbt™ work look like?]]></title>
            <link>https://medium.com/paradime-labs/what-does-the-1-harness-for-data-and-dbt-work-look-like-6c571658e6e8?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/6c571658e6e8</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Wed, 30 Sep 2026 18:01:43 GMT</pubDate>
            <atom:updated>2026-09-30T18:03:11.738Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ClrSwQGc4Qbxetn8iGfdiQ.png" /></figure><p>In the world of data engineering, not many are attempting to automate it. A few that are, most of them would rather ship a shell and get you to bring your own token, build your own harness so they don’t have to build the intelligence for actually getting data work done efficiently using AI agents. That’s the problem we are solving for our customers. We want our customers to be able to go from POC to production, while most of our smaller competitors today are catering to the hobbyists.</p><p>When you build your own AI agent and direct it to solve data engineering problems, those agents will tell you they fixed even when the changes do not compile. This is because LLM models are uniquely biased to reach a positive result. As users, you find this out in CI, or later in production.</p><p>We have watched this happen enough times that we no longer treat it as a quirk. This happens because verification is optional when you are using a frontier model for agentic workflows.</p><p>For us the interesting question in agentic data work is can the agent platform ensure formal verification of the work is done when the work is completed. That will increase trust, accuracy, and enable users to extend to production use cases.</p><p>Today, we have customers using self-healing agents on top of their production data pipelines running on Bolt, or in their own cloud on Airflow or Dagster. For us, this is a problem that needs to be solved.</p><p>The first generation of the DinoAI harness we built, excelled at interactive co-pilot use cases. However, as more and more users started using DinoAI agents for background tasks, that harness started to fall short. It needed a fundamentally new design.</p><p>So, at Paradime, we completely rebuilt the DinoAI harness so that it works for both co-pilot and background agentic tasks. Three outcomes came out of the work, that we measured on live infrastructure:</p><ol><li><strong>120X spin up time improvement</strong>. Agent sessions now in almost a second, compared to 2 mins before.</li><li><strong>Up to 88% lower AI cost per task.</strong> Users saw the same work completed for a fraction of the spend.</li><li><strong>Runaway sessions eliminated.</strong> We have bounded context so any particular session will not go out of whack.</li><li><strong>Built in verifier.</strong> DinoAI has to prove its data work is correct before it is allowed to say “done.” No other AI data agent enforces this out of the box.</li></ol><h3>Agent Spins up 60X faster</h3><p>Paradime agents spin up was slow and almost always took somewhere between one and three minutes. We have now fixed that. We have completely re-architected our sandbox and now the sandbox spins up in less 2 seconds, almost instantly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*UJkQouEe3K2uklJn.png" /></figure><h3>“Memory Sidecar” for agents</h3><p>Running agents consume tokens and tokens are consumed at every step and turn. Any context that is added to the agent is carried through each turn. Now, if an user added a huge file or a giant search result in a turn, that context then would carry forward through each step and turn and the context size would keep monotonically increasing consuming tokens again and again.</p><p>So we fixed with the concept of <strong>“memory sidecar”</strong>.</p><p>We park big artefacts on a memory layer that is attached as a sidecar to the agent session. It’s like side notes that an analyst keeps while doing critical work. We fetch this information when we want and discard it when we don’t need it anymore.</p><p>This keeps the context window range bound and prevents it from exploding. This also helps us manage unstructured context from Jira, Linear, Confluence, Notion, Google Drive, and more range bound and manageable.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*ySgA-K0v7-BrMAJ7mWJ5fg.png" /></figure><p>One real runaway task used to burn about 600K tokens. It now finishes in about 95K. Same result, roughly 6× cheaper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jqrIAXX2uJ_DgPx5.png" /></figure><h3>Reduced Context Rot</h3><p>We also stopped stuffing the context with tool noise coming from vanilla MCP like wrappers. When DinoAI reads a Jira ticket, a Confluence page, or a Linear issue, those APIs return a lot of clutter. We now distill and remove that clutter at the source. For example, on Jira, we keep sprints, story points, owners, and dates while we discard the rest.</p><p>As a real-life measurement Jira ticket: <strong>64,000 characters → 4,200.</strong> We reduced context rot by <strong>93%</strong>, which keeps more of the useful fields. We also architected in a way that the full raw response is also available through a separate request if the agent needs it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ibNtE2okeMkNGeOt.png" /></figure><p>This is where a lot of “we connected MCP” anecdotes fall apart. If you bolt a vanilla MCP server onto a general coding agent, you typically get the raw API or tool payload dumped straight into context like the full Jira / Confluence / Linear JSON, comments, metadata, nested fields, the lot.</p><p>However, the agent and the task does not need that. Instead of pulling all that into context and then reasoning on it, it’s faster and more accurate to pull what is needed. Context stays clean and outcomes are more predictable.</p><p>We built this into DinoAI where the context is filtered at source, before it’s sent to the LLM models. Same Jira ticket, using DinoAI has 93% less characters in the working context.</p><p>Bolting MCP onto a general agent does not give you the same cost or quality. You get a wrapper that is bringing the whole API output like a firehose into your context. We instead chose to reduce the clutter.</p><p>When we measured this on a real task, the AI cost per task went down by a humungous 88%.</p><h3>In-built Verification</h3><p>Every AI agent on the market claims it’s done the job that is not real. Most tools ask the model to double-check, build this into their system prompt and hope the model listens. We have seen with newer models like Opus 4.6+, Fable series, that these models are quite poor at following system prompts because their underlying system prompts have been laid so bar that any rules you give them, they are pretty bad at following.</p><p>So, we decided that was not good enough for production data engineering work including building dbt™ pipelines, so we stopped leaving verification to the model.</p><p>We have now built-in a verifier into the DinoAI harness so that the agent workflows <strong>can independently verify the files it changed.</strong> If the work is broken, the agent figures out the exact error, and fixes it. It cannot skip the verification.</p><p>The verification is adaptive and is tuned to the change:</p><ul><li><strong>Comment-only edit</strong> → quick syntax check. Your warehouse is never touched.</li><li><strong>Real logic change</strong> → the warehouse validates the SQL in seconds, with near-zero cost.</li><li><strong>Model with data-quality tests</strong> → build + tests run before DinoAI is allowed to complete.</li></ul><p>In real-life examples, we told an agent to break a model and immediately claim success. The harness caught it, blocked the claim, and the agent fixed its own mistake. <strong>25 seconds from bad edit to verified fix.</strong> In eval runs the gate also caught accidental mistakes twice and forced corrections before anyone saw them.</p><p>When something genuinely cannot be fixed, DinoAI says so. Retries are bounded, outcomes are transparent, and there are no silent failures or infinite loops. Broken code is caught, blocked, and fixed before it reaches the user.</p><h3>Prompted checks ≠ enforced gate</h3><p>A lot of agent products put validation tools next to the model e.g. compile, build, run tests, and call that reliability. This control model is problematic: if the agent chooses whether to call the check means this control is not deterministic.</p><p><strong>We have made control deterministic in our agents.</strong></p><p>We are shipping a <strong>pre-delivery gate</strong> that blocks “done” on unverified data work, with risk-based escalation so you never pay for warehouse checks when the change is as simple as a comment edit.</p><p>The verification layer is <strong>deterministic platform code</strong>, not another AI judging an AI. It is auditable, loggable, and fails safe.</p><p><strong>No other AI data agent enforces this.</strong></p><h3>How we compare</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/823/1*FAjUQxPPdEF1rFiSh8u6aw.png" /></figure><p>The industry’s own published results keep showing the same failure: even well-tuned agents still fail in the confidently-wrong way when verification is left to the model’s discretion. We have made it a platform guarantee, instead of a prompt suggestion.</p><h3>Provider by provider Comparison</h3><p><em>Based on public repos, docs, and our own hands-on analysis as of September 2026.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/559/1*LXVsi1wjYCRJilXq7SHVTw.png" /></figure><h3>State-Aware-Orchestration in the Harness</h3><p>Once the DinoAI agents finish a task, then the next step is running pipelines. Paradime State is how the harness decides what actually needs to run.</p><p>This week, we shipped <strong>Paradime State</strong> — state-aware orchestration for humans <em>and</em> DinoAI agents inside Paradime Bolt. Before a model is built, Paradime State is the Decision Engine on top of dbt Core™ v2.0, and chooses to <strong>SKIP</strong>, <strong>CLONE</strong>, or <strong>BUILD it</strong> from SQL logic, source freshness, object reuse across jobs, and column-level lineage. You only burn warehouse compute on what changed.</p><p>Combined with the harness, this is extremely powerful. Agents query and run pipelines differently compared to humans. So when an organization fires of a swarm of agents to fix all the tech debt and Jira tickets, having a cached state means the work will be faster and warehouse credits consumed will be much lower.</p><p><strong>Paradime is now the only company in the world that supports state-aware orchestration in the harness — for humans and for agents.</strong></p><p>You can read more and how it compares to state:modified+ here: <a href="https://www.paradime.io/blog/state-aware-orchestration-for-humans-and-agents">State-aware orchestration for humans and agents</a>.</p><h3>What this means for your team</h3><p><strong>Data leaders:</strong> AI spend per task drops and becomes predictable. No runaway bills. Agent ships deterministically verified work and hence the agents can be rolled out into production.</p><p><strong>Analytics engineers:</strong> DinoAI stops being a fast intern you have to babysit. Changes arrive pre-compiled and pre-tested, so the first pass is more often correct, and review burden goes down.</p><h3>Conclusion</h3><p>This update to the new generation of the Paradime harness results in radically faster spin up times, faster results using memory sidecar and accurate context, and finally in-built verification for accurate outcomes.</p><p>We will soon publish benchmark results we achieved with the new harness, but believe me when I say that this is gonna blow your mind. I have seen teams satisfied with having just a few skills in their repo and then how they got burned trying to wrangle LLMs with newer models. Eventually, they missed deadlines on getting AI into production.</p><p>On the other hand, with Paradime we are giving users the picks and shovels, the infra to build those agents and ship them into production.</p><p>If you want to know more, sign up for free at <a href="https://app.paradime.io/">https://app.paradime.io</a> and reach out to us.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c571658e6e8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/what-does-the-1-harness-for-data-and-dbt-work-look-like-6c571658e6e8">What does the #1 harness for data and dbt™ work look like?</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Your dbt™ Project Has a Testing Problem That Won’t Fix Itself]]></title>
            <link>https://medium.com/paradime-labs/your-dbt-project-has-a-testing-problem-that-wont-fix-itself-01e7c7a5708c?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/01e7c7a5708c</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Wed, 16 Sep 2026 23:57:41 GMT</pubDate>
            <atom:updated>2026-09-16T23:57:42.348Z</atom:updated>
            <content:encoded><![CDATA[<p>Every dbt™ project starts disciplined. Models are clean, tests are written, schema.yml files are tidy. Then the project hits 200 models with a dozen contributors, and something gives. Tests go unwritten on new models because there&#39;s always a more urgent feature. Existing tests break when columns get renamed upstream, and nobody notices until a dashboard goes wrong in production. Coverage concentrates on a handful of critical marts while the long tail of staging and intermediate models goes unchecked.</p><p>This is test debt, and it compounds faster than most teams realize.</p><p>If your team ships 10 new models per sprint and writes tests for 6 of them, you accumulate 4 untested models every two weeks. After a year, that’s roughly 100 models with zero coverage. According to dbt™ Labs, only about 40% of dbt™ projects run tests each week. Among those that do, coverage is often clustered on a few high-profile models. The 2024 State of Analytics Engineering survey found that 57% of data teams cite data quality as their biggest problem. That number and the test-debt math are connected.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hcNubph5FvzKuHMgHe6v5Q.png" /></figure><p>The standard fix — tell engineers to write more tests — doesn’t scale. The standard tooling fix — a linter that yells about missing tests in CI — creates busywork without ensuring the tests are any good. What actually works is shifting test authorship from manual to automated, with engineers reviewing rather than writing from scratch.</p><p>That’s what a test maintainer agent does.</p><h3>What the agent sees before writing anything</h3><p>A test maintainer agent reads three layers of context before generating a single line of YAML.</p><p>First, code context: your .sql model files, schema.yml definitions, source configs, macros, and dbt_project.yml. The agent traces ref() and source() calls to understand the dependency graph.</p><p>Second, warehouse context: the agent queries your actual data warehouse — Snowflake, BigQuery, Databricks, Redshift — to inspect column types, value distributions, and row counts. This grounds test generation in what the data looks like, not just what the SQL says it should look like.</p><p>Third, run logs: historical test results, execution times, failure patterns. If a test has been flapping for two weeks, the agent knows.</p><p>From this context, the agent infers test candidates using structural patterns. A column ending in _id or _key gets unique and not_null. A column used in a downstream JOIN gets not_null. A CASE WHEN with a finite value set gets accepted_values. A column referencing a ref() or source() gets a relationships test. Anything with ambiguous or complex logic gets a custom SQL assertion with a # TODO: confirm comment for the engineer to review.</p><p>The result looks like a schema.yml that an experienced engineer would write — because the inference rules mirror what experienced engineers do, just applied consistently across every model in the project.</p><h3>The self-validation loop</h3><p>Trust is the obvious obstacle. If you don’t trust the generated tests, you’ll review each one as carefully as if you’d written it yourself, and the time savings disappear.</p><p>The agent earns trust by validating its own output. After generating tests, it runs dbt test --select &lt;model&gt; against your development or staging environment. If a proposed accepted_values test fails because the inferred value list is incomplete, the agent queries the warehouse for the actual distinct values, updates the test, and re-runs. It iterates until the tests pass.</p><p>If the data itself is the problem — the test is correct but the data violates it — the agent marks the issue with a # DATA ISSUE comment and sets severity: warn so the team can investigate. The hard guardrail: it never commits a test that fails at severity: error. Every test that lands in your PR has been verified to pass in your environment.</p><p>This means the engineer’s review shifts from “does this test work?” to “does this test make sense?” The former is tedious; the latter is the kind of judgment call engineers should spend time on.</p><h3>Self-healing in production</h3><p>Test generation happens at development time. But production data changes independently of code. A source table renames a column from amount to order_amount. A vendor changes their data format. A backfill introduces duplicates. None of these involve a code change, so CI won&#39;t catch them.</p><p>A self-healing agent monitors production runs and responds when tests fail. The workflow: detect the failure, parse the run logs to identify root cause, generate a targeted fix, validate the fix in a sandbox, and open a pull request. All of this happens before the on-call engineer opens their laptop.</p><p>The fix depends on the failure type. A schema mismatch (renamed column) gets a model SQL update and corresponding test adjustment. An incomplete accepted_values list gets new values added. A freshness SLA violation gets escalated with the specific lag measured. A genuinely ambiguous failure gets flagged for human review rather than guessed at.</p><p>This compresses mean-time-to-repair dramatically. The usual 3 AM debugging cycle goes: get paged, open the alerting tool, find the failing test, find the model, find the upstream source, read the run logs, read the recent code changes, check downstream impact, write a fix, test it, open a PR. Most of that time is context gathering. When the agent has already gathered the context, identified the cause, and opened a PR with a fix, the engineer’s job shrinks to reading the PR description and clicking merge.</p><h3>Why inconsistency is the real problem</h3><p>Different engineers write tests differently, and this creates unpredictable failure surfaces. A senior engineer might add unique, not_null, accepted_values, and a couple of custom assertions. A junior engineer might add a single not_null on the primary key and move on. Without enforced standards, test quality becomes a function of who happened to build the model.</p><p>Some models end up over-tested — 50+ tests generating alert fatigue, where people start ignoring failures. Others end up under-tested — zero coverage on columns feeding executive dashboards. Both extremes are bad.</p><p>An agent applies the same rules everywhere. You define those rules in a configuration file (Paradime uses .dinorules, a plain-text file committed to your repo root), and the agent follows them for every model it touches. If your rules say &quot;every mart-layer model needs unique and not_null on its primary key, relationships tests on all foreign keys, and accepted_values on status columns,&quot; that&#39;s what every mart gets. The senior engineer&#39;s thoroughness becomes the baseline instead of the exception.</p><h3>Prioritizing by business impact</h3><p>When multiple tests fail in the same run, the agent uses lineage information to triage. A unique failure on the primary key of a model feeding the CEO&#39;s revenue dashboard is critical. A not_null failure on a staging model used only in development is low priority.</p><p>The prioritization determines alert routing — critical failures go to PagerDuty, medium ones to Slack, low ones to the log — and fix ordering. This replaces the typical Slack alert that says “Job XYZ failed” with a structured incident report: the failing test, the root cause, the proposed fix, and which dashboards and reports are affected downstream.</p><h3>Where it fits in the workflow</h3><p>The most natural integration point is your pull request workflow. When an engineer opens a PR that touches a .sql file, the test maintainer agent triggers via GitHub Actions. It reads each changed model, infers tests, writes YAML, runs dbt test, iterates until green, and commits the results to the PR branch. The engineer sees the generated tests alongside their model changes in the same review they&#39;d do anyway.</p><p>For production, the agent plugs into your scheduler. When a scheduled dbt build fails, the agent gets triggered by the failure event, runs its diagnosis-and-fix cycle, and opens a PR. Teams configure whether fixes auto-merge or require approval. Most start with mandatory review and relax controls as confidence builds. Either way, every change is a Git commit with a clear author identity, full diff, and rollback path. No black-box modifications.</p><h3>Measuring whether it’s working</h3><p>Three metrics tell you if the investment is paying off.</p><p>MTTR: measure the time from alert to production fix, before and after. The context-gathering phase is what the agent eliminates, and that’s usually 70–80% of the total debugging time.</p><p>Coverage growth: track the percentage of models with at least one test, broken down by layer (staging, intermediate, marts). The agent should increase this number monotonically without adding manual work. If it plateaus, check whether new models are being excluded from the agent’s trigger paths.</p><p>Engineering hours: measure time spent on test-related work per sprint. Writing tests typically runs 4–8 hours per engineer per sprint. With an agent handling generation, that drops to 30–60 minutes of review time. The freed hours go to designing business logic, building new models, and the kind of work that actually moves the business forward.</p><h3>The shift</h3><p>Manual dbt™ testing works at small scale and degrades predictably as projects grow. The degradation isn’t anyone’s fault — it’s structural. There are always more models to build than tests to write, and the incentive is always to ship the model now and backfill the tests later. “Later” rarely arrives.</p><p>An automated testing agent changes the default. Instead of models shipping without tests unless someone remembers, models ship with validated tests unless someone opts out. That inversion is the whole point: quality becomes the path of least resistance rather than the thing that requires extra effort.</p><p><em>We built this pattern into </em><a href="https://www.paradime.io/programmable-agents"><em>Paradime’s DinoAI</em></a><em>. The Test Maintainer agent handles test generation in CI, and </em><a href="https://www.paradime.io/bolt-ci-cd-run-dbt"><em>Bolt</em></a><em> handles self-healing in production. Both are free to try.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=01e7c7a5708c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/your-dbt-project-has-a-testing-problem-that-wont-fix-itself-01e7c7a5708c">Your dbt™ Project Has a Testing Problem That Won’t Fix Itself</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replace your Data Team’s Intake Form With an AI Interviewer]]></title>
            <link>https://medium.com/paradime-labs/replace-your-data-teams-intake-form-with-an-ai-interviewer-a879be6706ad?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/a879be6706ad</guid>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Wed, 16 Sep 2026 09:56:04 GMT</pubDate>
            <atom:updated>2026-09-16T09:56:04.303Z</atom:updated>
            <content:encoded><![CDATA[<p>Every data team has the same intake problem. Someone in marketing Slacks you: “hey, can we get churn rate on the customer dashboard?” That message is missing the business context, the urgency, who needs it, what breaks without it, and whether someone already built it six months ago. So you ask follow-up questions. They answer half of them. You file a ticket with gaps. The ticket sits in the backlog, and when someone picks it up three weeks later, they have to re-interview the requester anyway.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1et_1_enyIHj6o_Wb_D3ww.jpeg" /></figure><p>Forms don’t fix this. People skip optional fields, write “see above” in required ones, and pick “High” priority for everything. The fundamental issue is that a form asks the requester to do the translation work — turn a business need into an engineering spec — and that’s exactly the part they can’t do.</p><p>We built something different: a Slack bot that conducts a short intake interview in plain language, resolves vague references against the codebase on its own, and files a well-formed Jira ticket at the end. The requester never sees a priority dropdown, a project key, or a custom field. They just answer questions about what they need and why.</p><p>This pattern came out of a production deployment at a Paradime customer. Here’s how it works and how to set one up yourself.</p><h3>The interview, not the form</h3><p>The bot asks one question at a time. This sounds obvious, but it’s the single decision that makes the whole thing work. Dump a five-question form into Slack and people abandon it. Ask “what decision does this enable?” and wait, and they actually think about it.</p><p>The questions split into two layers. Core questions hit every request: what type of work is this, what business question does it answer, who consumes the output, and how urgent is it. Branch questions adapt to the request type — a data-quality bug needs “expected vs. actual values and when it started,” while a new data source needs “where does this data live and who owns it.”</p><p>Two rules keep the interview usable for non-technical people. First, the bot only asks what the requester can reasonably know. Column names, model layers, refresh cadence, backfill strategy — none of that is their call, so the bot never asks. If they volunteer technical details, it captures them. Otherwise it moves on. Second, urgency is asked in plain language (“how soon do you need this, and is it blocking something?”) and the bot infers the Jira priority itself. Nobody picks from a four-point scale they’ve never calibrated.</p><h3>Resolving “the revenue dashboard”</h3><p>Requesters speak in loose references. “The revenue dashboard.” “That monthly report finance uses.” “The table with customer segments.” The bot needs to turn those into real model names to write a useful ticket and to check whether someone already filed the same request last week.</p><p>The key design choice here is <em>where</em> that resolution happens. Running repository searches directly in the Slack conversation dumps large result sets into the chat history, and in a Slack-based agent, everything in the conversation gets re-sent to the model with every subsequent message. Ten search results on message three means you’re paying for those ten results again on messages four, five, six, and seven.</p><p>Instead, the bot delegates each lookup to a sub-agent. “Which dbt model feeds the revenue dashboard, and does it already contain a discount rate field?” The sub-agent runs the search in its own short-lived context, summarizes the findings, and hands back a sentence or two. Same answer, a fraction of the token cost.</p><p>The bot uses this same mechanism for duplicate detection. Before filing a ticket, it checks the existing backlog. If something similar already exists, it surfaces the match and asks the requester whether to proceed or link to the existing ticket instead.</p><h3>One conversation, one ticket</h3><p>This constraint is about cost discipline. Every message in a Slack thread includes the full conversation history as input to the model. A ten-message intake interview is cheap. A ten-message interview followed by fifteen more messages filing three additional tickets in the same thread is expensive, because messages sixteen through twenty-five carry the full weight of everything before them.</p><p>The bot files one ticket per conversation and then nudges the requester to start a new chat for their next request. The nudge includes the piece that makes it painless: mention a ticket key in the new chat and the bot can pull up the full context itself using the Jira API. No conversation history needed. In production, this pattern is the difference between tickets that cost hundreds of credits and tickets that cost hundreds of thousands.</p><h3>Setting it up</h3><p>The implementation uses Paradime’s programmable agents. You create an agent named dinoai-slack-bot (this specific name binds it to Slack DMs and @-mentions), give it a role, goal, and backstory that encode the interview flow, and deploy it to your repository.</p><p>The <strong>role</strong> tells it what it is: a data request intake specialist that gathers requirements and files one good ticket. No code, no pipelines, no SQL.</p><p>The <strong>goal</strong> encodes the interview structure: core questions in order, branch questions by request type, a confirmation step where the requester sees exactly what will be filed, and then ticket creation. The goal also carries the conversation hygiene rules — one ticket per thread, the nudge to start fresh.</p><p>The <strong>backstory</strong> carries the operational details: how to resolve loose references via sub-agents, how to infer priority from plain-language urgency answers, the Jira field mapping (project key, issue type, custom fields), and the description format for tickets. You’ll need to swap in your own project key, issue types, and any required custom fields your Jira project demands. A missing required custom field is the most common reason ticket creation fails on first setup.</p><p>The tools allowlist is deliberately tight: Jira tools for reading and writing tickets, and file/search tools for repository lookups. The bot can’t run SQL, write code, or trigger pipelines. Scoping it to intake keeps it from trying to solve the request itself.</p><p>Once deployed to your repo’s default branch, the agent definition replaces the default Slack bot personality. DM the bot something vague — “hey, can we get churn rate added to the customer dashboard?” — and it should walk you through the interview one question at a time, resolve “the customer dashboard” against your repo, present a confirmation summary, and file the ticket only after you approve.</p><h3>What the ticket looks like</h3><p>The description follows a fixed format:</p><blockquote><em>Add churn rate calculation to the Customer Health dashboard (model: customer_health_dashboard).</em></blockquote><blockquote><em>Context: Marketing needs churn rate alongside retention metrics for the quarterly business review. Consumer: Marketing analytics team, viewed in Looker.</em></blockquote><blockquote><em>Acceptance Criteria:<br></em>- Churn rate field added to the customer health dashboard<br>- Calculation matches the finance team’s definition (monthly customer churn)<br>- Field is available in Looker by end of sprint</blockquote><p>Priority is set automatically. Labels include intake-bot so you can filter for bot-filed tickets. Acceptance criteria come directly from what the requester said — the bot doesn&#39;t invent requirements.</p><h3>Closing the loop</h3><p>The intake bot fills the backlog in a structured, consistent way. A natural next step is to pair it with an execution agent that picks up tickets labeled agent-ready and delivers the dbt changes as pull requests. Intake handles the front door; a backlog agent handles the work.</p><p>The broader pattern transfers beyond Jira, too. The same interview-and-file structure works with Linear, Asana, or any project tracker with an API. The three design principles stay the same: ask only what the requester can know, keep research out of the conversation context, and scope each thread to one deliverable.</p><p><em>Built with </em><a href="https://www.paradime.io/"><em>Paradime</em></a><em>. The Jira intake agent is one of several pre-built agent patterns available in the programmable agents framework.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a879be6706ad" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/replace-your-data-teams-intake-form-with-an-ai-interviewer-a879be6706ad">Replace your Data Team’s Intake Form With an AI Interviewer</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is an Agent Studio?]]></title>
            <link>https://medium.com/paradime-labs/what-is-an-agent-studio-ff9798d98524?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/ff9798d98524</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Mon, 14 Sep 2026 12:48:38 GMT</pubDate>
            <atom:updated>2026-09-14T12:48:38.418Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>At <a href="https://www.paradime.io/">Paradime</a>, we believe the future data engineer will not hand code data pipelines. They will role out agents that actually do the work of coding, deploying, monitoring, and fixing data pipelines. They will build agents that manage the entire lifecycle of data movement.</p><p>As a result, these engineers and teams will need fundamentally different tooling to what exists today. I can go onto granular features that should make up this new kind of tooling but then I would digress. Instead, what we believe is users should have a platform where the substrate is what a mid-senior data engineer would need to do their work, we will call this the harness. And then built on top of this harness are the tools that make data engineering with agents work.</p><h3>The Future Data Platform</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FOcJRQEzDApspxZjggM86g.png" /></figure><p>We envision a platform, where the Paradime Context Graph acts as the backbone context graph assimilating information from 50+ integrations and a combination of inbuilt context (e.g. column-level lineage) and pre-built context (e.g. BI tooling).</p><p>The context graph than feeds into DinoAI that comprises of the Harness, model routing, and a durable runtime.</p><p>Finally, these lead to a set of user-facing products as follows:</p><ul><li>Agent Studio for building agents and agent fleets while the IDE is for human work</li><li>Orchestration for running pipelines, and Agent Runtime for durable, long-running agents</li><li>Observability of logs and agent traces for pipelines and agents.</li></ul><p>The whole platform today is now accessible from anywhere — Slack, MS Teams, Claude Code, CoCo, Cursor, Kiro, and API endpoints too — anywhere people work.</p><p>Application features like Column-level lineage will cease to be a product but will become a feature that agents leverage to get data work done.</p><h3>Introducing Agent Studio</h3><p>We will be sharing more details on each and every component of the future data platform. But today will start with the Agent Studio.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OfInGFCYZK9uGqqOQW4Ttw.png" /><figcaption>The Agent Studio UI</figcaption></figure><p>Paradime context graph today handles more than 50+ integrations on top of pre-built and in-built context layers. Users looking to build AI agents on top of this information need an UI where to build, test and fine tune their agents. The opposite approach of build an agent as a black box and then rolling it out is a high risk strategy. The agents might produce wrong outcomes, or behave erratically or worse lose trust of the end users. Hence, data engineers looking to roll out agents, should have a place to test their agents.</p><p>Paradime Agent Studio fills that gap in today’s tooling stack among data engineers.</p><p>You can start a conversation with the context of your warehouse, code base, and any additional context added through integrations on any data problem you are facing.</p><p>The agent studio comes with persistent state so you can start in a Slack thread, and then continue on Paradime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qm9xAW-23xYSEA8FtvAn9A.png" /><figcaption>Agents with a persistent state — start in Slack, continue in Paradime</figcaption></figure><h3>Agent Templates</h3><p>With the Agent Studio, we are rolling in a set of pre-built agent templates to get you started super fast on use cases like self-healing data pipelines, automated warehouse cost optimization, and close gaps in dbt docs and tests.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*knw1dporRFcNVJQPOXPXyQ.png" /><figcaption>Agent templates</figcaption></figure><h3>Model Families for Agents</h3><p>When you are building agents using the Agent Studio, we give users the ability to add the goal and role the agent should play. But also on top, a unique capability we provide is <strong>Model Families</strong>. These are pre-built model baskets for agents that are optimized for speed, deep work, or open weights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XXH7ylAvIVYaWWc8v7a4Lw.png" /><figcaption>Agents with Model Families — optimize effort vs cost</figcaption></figure><p>The idea here is to give end users the control to optimize for the best performance.</p><h3>Agent Governance and Control</h3><p>The next standout feature when building agents are the levers for governance and control. Users can set both an allow-list and deny-list of tools for each agent so to set the right guardrails for agents to operate within its intended capabilities.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DJWD_gwd5MVjTOuw1qk8hQ.png" /><figcaption>Agent guardrails and control</figcaption></figure><p>Tucked nicely beneath the tools section is the output channels and squads. Output channels define where the agent should stream its output when its working like Slack, MS Teams or just in the app.</p><h3>Agent Squads</h3><p><strong>Squads</strong> is a pretty powerful feature to run agent squads where you can run a team or a fleet of agents. A classic example is to build a data team using DinoAI agents like in the example below where a master agent (aka. Data Team Lead) runs a squad of data engineers and data analysts.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nn4sCe6_ZCuf3MPN9wclTg.png" /><figcaption>Data Team Agent Squad</figcaption></figure><h3>What’s Next</h3><p>This is a very active area of development at Paradime today. We are building the data platform and the harness and the agent layer on top at the same time. That way we make sure that while the harness is continuously improving with new features and capabilities, the interaction layer is also evolving.</p><p>We ultimately think that data engineers should become AI engineers. To be able to do that they can’t be stuck in legacy tools of the MDS stack with a narrow feature set. These tools typically just expose an MCP server so end users can interact with them during interactive Claude or ChatGPT sessions. But these tools are not built for agents that need high quality context operating at a high performance bar.</p><p>That’s where the context engine and the harness underneath we are building at Paradime comes in. There is a lot of exciting work we are doing around security, speed, integrations, model families, and interoperability with other coding agents while bringing in new capabilities to the harness itself.</p><h3>Conclusion</h3><p>If you or your organization is currently experimenting with building agents for data operations, IT operations, or revenue operations, we would love to help. We can set up an initial consultation to see if we can help and we can architect a solution for you that meets your needs. The Paradime platform today serves startups to complex FTSE100, and Global 2000 enterprises.</p><p>DM me at <a href="/service/mailto:kaustav@paradimehq.com">kaustav@paradimehq.com</a> or signup for a trial at <a href="https://app.paradime.io/">https://app.paradime.io</a> and we will help you get started.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ff9798d98524" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/what-is-an-agent-studio-ff9798d98524">What is an Agent Studio?</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DinoAI v3.0: The ADE-Bench Report]]></title>
            <link>https://medium.com/paradime-labs/dinoai-v3-0-the-ade-bench-report-04c31b2f47af?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/04c31b2f47af</guid>
            <category><![CDATA[agentic-ai]]></category>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Thu, 09 Apr 2026 12:27:07 GMT</pubDate>
            <atom:updated>2026-04-09T12:27:07.452Z</atom:updated>
            <content:encoded><![CDATA[<p>When we started building DinoAI v3.0, we made a bet: that a specialist data engineering agent, given the right context and the right tooling, could decisively outperform general-purpose coding agents on real analytics work.</p><p>This report is the result of testing that hypothesis against the hardest public benchmark we could find — and then publishing every number, every failure, and every link so you can verify it yourself.</p><h3>The headline result</h3><p>On dbt Labs’ <strong>ADE-Bench</strong> — a 43-task, 258-test benchmark of real-world analytics engineering challenges — Paradime DinoAI with Claude Sonnet 4.6 resolved <strong>38 of 43 tasks (88.37%)</strong> and passed <strong>254 of 258 individual tests (98.45%)</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wDI84VEQQjW2rZrq.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zY2FWHKsOusYtcH2gnTY7g.png" /></figure><p><em>Benchmark run: April 1, 2026. Sources and raw data are linked in the Reproducibility section below.</em></p><h3>What is ADE-Bench?</h3><p>ADE-Bench is a public benchmark created and maintained by dbt Labs to measure how well AI agents perform on real analytics engineering tasks. Unlike toy coding benchmarks — write a function, pass a couple of unit tests — ADE-Bench is designed to reflect the messiness of actual data work: operating inside an existing codebase, reasoning over multiple models and their dependencies, handling schema changes, and producing changes that pass an integration test suite.</p><p>The benchmark spans (at the time of testing) <strong>43 tasks across 8 real-world domains</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bYWfMdBD3i1JAKab4lI33Q.png" /></figure><p>Each task ships with a set of integration tests — <strong>258 tests in total across the 43 tasks</strong>. A task is marked <strong>resolved</strong> only if the agent’s final code passes every single test. Partial credit isn’t a thing. If one test fails, the whole task is unresolved.</p><p>That makes ADE-Bench a hard, honest benchmark. There’s no hiding behind “mostly right” — either the pipeline works end-to-end, or it doesn’t.</p><h3>How to read the results</h3><p>Two metrics, two different things they tell you.</p><p><strong>Tasks resolved</strong> is the headline metric. It measures how many of the 43 tasks the agent shipped correctly. DinoAI resolved 38 of 43, which is 88.37%.</p><p><strong>Test pass rate</strong> is a more granular view. It counts individual tests across all 258 tests in the benchmark. DinoAI passed 254 of 258, which is 98.45%.</p><p>That gap between 88.37% and 98.45% is worth pausing on. It tells you something important about how DinoAI fails: <strong>when it fails, it fails tight</strong>. On the tasks DinoAI didn’t resolve, it was typically one or two tests shy of a full pass — not catastrophically wrong. In the 5 tasks marked unresolved or skipped, DinoAI still passed 12 of the 26 tests they contained. That matters because in real-world data engineering, “90% of the way there” is reviewable. “Completely off the rails” isn’t.</p><h3>Category breakdown: where DinoAI shines, where it struggles</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9BkjH2gHEy12PQvNV8TjhQ.png" /></figure><p>DinoAI went <strong>5-for-5 on the categories that represent the bulk of day-to-day analytics engineering work</strong>: marketplace data, core dbt patterns, project tracking, financial modeling, and basic transformations. On the 28 tasks across those five categories, DinoAI resolved all 28.</p><p>Where did it struggle? Motorsport analytics (f1) and customer support data (intercom). Both categories involve complex time-based aggregations, multi-table joins with ordering edge cases, and domain-specific conventions that pushed the agent’s reasoning harder. And we were unable to complete the workday category due to an environment issue during the benchmark run — more on that below.</p><h3>The failures, in detail</h3><p>No hedging, no spin. Here’s exactly what went wrong on the 5 tasks DinoAI didn’t resolve.</p><p><strong>f1002–9 of 10 tests passed.</strong></p><p>A multi-model stats aggregation over an F1 driver career dataset. DinoAI’s final models passed 9 of 10 tests; one schema mismatch in the most_podiums model caused the final test to fail. The work was almost entirely correct — it produced accurate data values for all 20 ranked drivers, but included extra breakdown columns (p1, p2, p3) that the spec did not ask for. The equality test compares column schemas before comparing data, so the additional columns caused an immediate schema mismatch failure even though the underlying numbers were perfect.</p><p><strong>f1006–2 of 4 tests passed.</strong></p><p>DinoAI passed 2 of 4 tests. Both equality tests failed due to a single logic error: SUM(points) was used on standings tables that store cumulative season-to-date totals, not per-race points. This caused every row in constructor_points and driver_points to be inflated ~10–12× (e.g. Red Bull 2023: 10,158 pts instead of 860). The tables were built and populated correctly — the data just held the wrong numbers. The fix is a one-word change per model: SUM → MAX.</p><p><strong>f1008 — skipped.</strong></p><p>This task does not include a meaningful reference output for validation. The solution seed is a placeholder (foo) rather than a real expected result for the constructor models. Because we cannot reliably verify whether the task was completed correctly, we chose not to run it. Rather than reporting a misleading pass or fail, we marked it as skipped.</p><p><strong>intercom003–1 of 2 tests passed.</strong></p><p>The model was built correctly — all 19 columns, correct logic, correct values. The test failed due to a <strong>data mismatch on the primary key (</strong>conversation_id<strong>)</strong>, caused by a bug in the benchmark&#39;s own answer key.The benchmark&#39;s expected output stored conversation_id as a 32-bit integer. The actual source IDs are 15-digit numbers (156643300001311) that silently overflow a 32-bit integer, producing a completely different value (300001311). DinoAI&#39;s model used the correct full-length IDs from the source data — but those didn&#39;t match the overflowed values frozen into the answer key.</p><p><strong>workday001 — skipped (9 tests).</strong></p><p>This task is an incomplete stub and was not run in the benchmark. The prompt instructs the agent to do nothing, and the solution seed is a placeholder (foo) with no real expected output. There is no meaningful work to evaluate or validate. Rather than reporting a misleading result, we marked it as skipped. The task needs a proper prompt, a well-defined goal, and a real ground-truth seed before it can be included in a future benchmark run.</p><p>If you want to get strict about it — excluding the two tasks skipped for environment reasons — DinoAI’s “effective” task resolution rate is <strong>3 failures out of 41 eligible tasks, or 92.68%</strong>. We’re reporting the full 88.37% number anyway because environment issues are part of the job, and we’d rather undersell than oversell.</p><h3>Why DinoAI wins (and why generalist agents lose)</h3><p>Here’s the part that should catch your attention.</p><p>The same frontier model — Anthropic’s Claude Sonnet 4.6 — is powering both Claude Code (39.53%) and DinoAI (88.37%) in this benchmark. Same underlying model. <strong>49-point gap in task resolution.</strong></p><p>How does identical model intelligence produce such different results? Three reasons.</p><h3>1. Context graph, not context dump</h3><p>Claude Code, when given access to a codebase, treats every file as roughly equal weight. Point it at a large repo and it’ll grep, read, and pull context across files without strong priors about which ones matter for the task at hand. It works, but it’s expensive and it dilutes reasoning on harder tasks.</p><p>DinoAI builds a <strong>context graph</strong> during setup. It understands the dependency relationships between models, which ones feed which, which tests guard which pipelines, which lineage edges matter for which transformations, and which docs correspond to which business logic. When you ask DinoAI to fix a schema change, it doesn’t grep the whole repo. It walks the dependency graph, identifies exactly the models affected, pulls only the relevant tests, and operates on a targeted context window.</p><p>The practical result: DinoAI uses a fraction of the context Claude Code would use for the same task, and the context it does pull is directly relevant to the work.</p><h3>2. Multi-model routing</h3><p>DinoAI doesn’t use one model for everything. Reasoning-heavy work (planning multi-step refactors, debugging complex failures) goes to Claude Opus. Straightforward SQL edits and targeted file changes go to Haiku. Structured intermediate steps go to Sonnet.</p><p>Generalist agents run every task through the same frontier model. That means you’re either overpaying for simple tasks (burning Opus tokens on a rename) or underpowering hard ones (forcing Haiku to reason through a multi-model refactor). DinoAI routes per-task, which lets it spend reasoning budget where it actually matters.</p><h3>3. A specialist test loop</h3><p>When DinoAI starts a task, it spins up an isolated sandbox, clones the relevant code, wires up the correct warehouse schema, and lineage context, makes its changes, and iterates running the dbt™ CLI until they pass — or until it can explain why they won’t.</p><p>Claude Code <em>can</em> do this if you prompt it carefully and wire up the right tool access. But it’s not the default loop. For DinoAI, it’s the only loop. Every single task starts with “what does the data problem look like?” and ends with “do the tests pass to solve the problem at hand?” That’s an enormous alignment advantage on a benchmark whose entire scoring criterion is “did the tests pass?”</p><h3>The kicker: we beat dbt Labs on their own benchmark</h3><p>dbt Labs built ADE-Bench. They designed the tasks, wrote the test suites, and know the benchmark intimately. They also ran it themselves and published the results.</p><p><strong>dbt Labs’ own ADE-Bench score: 58.14%</strong> (25 of 43 tasks resolved).</p><p><strong>Paradime DinoAI’s score: 88.37%</strong> (38 of 43 tasks resolved).</p><p>We beat the team that built the benchmark by <strong>more than 30 points on their own test</strong>.</p><p>We’re not flagging this to be smug — we’re flagging it because it’s the strongest stress test a benchmark can get. When the team that designed a test performs worse than an outside contestant, you’re looking at one of two possibilities:</p><ol><li>The contestant has genuinely better tooling for the task.</li><li>The test is gameable and the outside contestant optimized for the test itself.</li></ol><p>ADE-Bench isn’t gameable. It’s a public, versioned benchmark with real integration tests that you either pass or you don’t. Which means the gap between DinoAI and dbt Labs reflects real tooling advantages — the context graph, the multi-model routing, the sandboxed test loop we just walked through above.</p><p>And to be clear: <strong>we have enormous respect for dbt Labs</strong>. They built the tool that defined the modern analytics engineering workflow. They built the benchmark we’re using to measure our own work. And they ran that benchmark honestly and published their own score — a move very few companies would have made. We’re reporting our delta because the community deserves to see both numbers side by side.</p><h3>Reproducibility</h3><p>A benchmark only matters if you can reproduce it. Here’s everything you need to verify our results independently.</p><h3>Our raw data</h3><p>Every task, its status, test counts, and the per-test pass/fail breakdowns are available as CSV.</p><ul><li><a href="https://docs.google.com/spreadsheets/d/17zKWhgu59msTq_2fLKQe0ziFfu_2norL3KwE9hxP0LY/edit?usp=sharing">Download ADE-Bench raw data (per-task)</a></li></ul><h3>Competitor benchmark sources</h3><p>We’re publishing the exact URLs for every competitor’s own benchmark page. Go read them. We’d rather you compare the full context than take our word for it.</p><ul><li>Altimate AI — <a href="http://altimate.sh/benchmarks">altimate.sh/benchmarks</a></li><li>Cortex Code CLI — <a href="http://snowflake.com/en/blog/cortex-code-cli-expands-support">snowflake.com/en/blog/cortex-code-cli-expands-support</a></li><li>dbt Labs — <a href="http://getdbt.com/blog/ade-bench-dbt-data-benchmarking">getdbt.com/blog/ade-bench-dbt-data-benchmarking</a></li><li>Claude Code — Baseline run of Anthropic Claude Code against the same task set with no custom tooling</li></ul><h3>How to run ADE-Bench yourself</h3><p>ADE-Bench is publicly maintained by dbt Labs. Clone the repo, configure the agent of your choice, and run the test harness against the 43 tasks. If you run it and your numbers differ from ours, email us — we want to understand why.</p><h3>Our commitment</h3><p>We will <strong>re-run this benchmark every month</strong>, and we will publish the updated results regardless of whether DinoAI’s lead shrinks or grows. If a new frontier model ships, we re-run. If a new competing agent ships, we re-run. Transparency is the only way this kind of benchmarking stays useful for the community.</p><h3>What’s next</h3><p>A few things already on the roadmap:</p><ol><li><strong>Monthly re-runs</strong> of the full ADE-Bench suite against the latest frontier models and competing agents. We’ll publish every run.</li><li><strong>Category-specific deep dives</strong> — starting with a full write-up of how DinoAI handles the f1 category and what it would take to close the gap on motorsport-style time-series aggregations.</li><li><strong>Expanding to additional public benchmarks</strong> as they emerge in the data engineering agent space. ADE-Bench is the best public benchmark we have today, but we expect more to appear, and we want DinoAI measured against all of them.</li></ol><h3>Try it yourself</h3><p>ADE-Bench measures what DinoAI can do on a standardized test. What matters for your team is what it can do on <strong>your</strong> data, <strong>your</strong> workflows, and <strong>your</strong> backlog.</p><p>We’re running a <strong>30-day open beta</strong> of DinoAI v3.0 starting today. No usage limits. No seat charges. No long-term commitment. Connect your warehouse, your repo, and your Slack workspace, and see what an 88.37%-on-ADE-Bench agent can do against your real tickets.</p><p><a href="https://paradime.io/dino-ai"><strong>Start the 30-day beta →</strong></a></p><p><a href="https://www.paradime.io/blog/introducing-dinoai-v3.0-the-first-end-to-end-data-engineering-agent"><strong>Read the DinoAI v3.0 launch announcement →</strong></a></p><p><em>Questions about methodology, the raw data, or reproducibility? Email </em><a href="/service/mailto:benchmarks@paradime.io"><em>benchmarks@paradime.io</em></a><em>. We read every message and we’ll answer technical inquiries in detail — especially from anyone attempting to reproduce the benchmark independently.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=04c31b2f47af" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/dinoai-v3-0-the-ade-bench-report-04c31b2f47af">DinoAI v3.0: The ADE-Bench Report</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introducing DinoAI v3.0: The First End-to-End Data Engineering Agent]]></title>
            <link>https://medium.com/paradime-labs/introducing-dinoai-v3-0-the-first-end-to-end-data-engineering-agent-da0372ac5e1f?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/da0372ac5e1f</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[data-engineering]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Thu, 09 Apr 2026 12:21:28 GMT</pubDate>
            <atom:updated>2026-04-09T12:21:29.276Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*thj9TzFRL95y0oBPgazfKw.png" /></figure><p>Today, we’re launching <strong>DinoAI v3.0</strong> — our most advanced release yet, and a fundamental shift in what a data engineering platform can be.</p><p>If you’ve been following Paradime, you know the journey. In 2024, we launched the first version of DinoAI — a chatbot you could ask questions about your data stack. In 2025, we shipped agent mode — a copilot that wrote code alongside you. With v3.0, we’re going much further: DinoAI is now a <strong>true background agent</strong> that works as a skilled analytics engineer and data engineer on your team. With this launch, Paradime becomes a fully agentic operating system for data engineering.</p><h3>The bottleneck nobody talks about</h3><p>Over the past year, we’ve watched hundreds of teams use DinoAI to code, build models, triage pipelines, and fix issues. And we learned something important: <strong>coding speed alone isn’t the bottleneck</strong>. The real bottleneck is the lead time to actually get work done.</p><p>Think about how most data teams operate today. A request comes in from a stakeholder. It gets translated into a ticket. The ticket gets prioritized into a sprint. If it doesn’t make the cut, it waits for the next one. Each sprint is roughly two weeks, so a simple ticket — something as straightforward as adding a few columns to a report — can easily take <strong>ten to twenty days</strong> to ship in a medium-to-large enterprise.</p><p>From the business’s perspective, everything is delayed. In a normal quarter, you get maybe ten working weeks. If a finance or sales stakeholder requests a dashboard update that takes ten to twenty days to deliver, <strong>you’ve just lost a third of the quarter</strong> waiting on a single piece of work.</p><p>That’s what we’re changing. Businesses should move at the speed of light — unconstrained by sprint cycles, ticket queues, or the traditional SDLC.</p><h3>What DinoAI v3.0 actually is</h3><p>DinoAI v3.0 is a <strong>true background agent that lives where you work</strong>. You can invoke it from Slack today. You can invoke it from Microsoft Teams today. And coming very soon, you’ll be able to invoke it from Claude, Claude Code, ChatGPT, inside Paradime, and from your phone. We’re removing every barrier to using DinoAI from any system your team already spends time in.</p><p>Here’s what that unlocks: <strong>what used to take ten to twenty days now takes minutes.</strong></p><p>Imagine it’s the week before month-end close. Someone in finance needs a few new columns added to a dashboard report. Traditionally, that creates hours of stress, coordination across teams, and — if it isn’t planned perfectly — days or weeks of waiting.</p><p>With DinoAI v3.0, that finance team member doesn’t even need access to Paradime. They go into a Slack channel where the DinoAI agent is installed, describe the problem in plain language, follow the agent’s chain of thought, verify the output, and go from ticket to pull request in <strong>less than five minutes</strong>.</p><p>You’re compressing what used to take ten days into five minutes. That’s a <strong>90–99% improvement in efficiency</strong> — an unprecedented level of velocity and time compression for a data organization.</p><h3>Under the hood</h3><p>Inside Paradime, DinoAI now gets a <strong>separate entity of its own</strong>. There’s a dedicated settings panel for the agent, where you connect its code repository, data warehouse, and all the integrations it should have access to — GitHub, your cloud warehouse, column-level lineage across dbt™, Looker, Tableau, and Power BI, and documentation from Confluence, Jira, Linear, Google Drive, Notion, and more.</p><p>On the back of that, the agent builds its own <strong>context graph</strong>. Whenever anyone invokes DinoAI, it automatically pulls from that graph — choosing the right information and the right context to get the work done.</p><p>When you trigger a task from Slack, Paradime spins up a <strong>secure, isolated sandbox</strong> on the cloud inside your Paradime infrastructure. All the work happens within that sandbox. When the task is complete, the sandbox terminates. Multiple people across your organization can work with DinoAI simultaneously — each in their own thread, each in their own sandbox — receiving tabular output, code diffs, and the final pull request right inside the Slack thread.</p><h3>“Why not just use Claude Code?”</h3><p>It’s the question we get asked more than any other. We’ve spent months benchmarking it, and here’s what the data shows.</p><p>One of the metrics we track most closely at Paradime is <strong>acceptance rate</strong> — the percentage of AI-generated output a user actually accepts as work they’d ship.</p><p>On pure analytics and data engineering tasks:</p><ul><li><strong>Vanilla Claude:</strong> ~30–35% acceptance rate</li><li><strong>Claude with skills layered on top:</strong> ~60–65%</li><li><strong>DinoAI (specialist data engineering agent):</strong> <strong>~94%</strong></li></ul><p>Why such a dramatic gap? Because DinoAI knows <strong>when</strong> to bring in which context, and how to use it optimally. To a general-purpose LLM, a Confluence page, a GitHub repo, and a Jira ticket all have equal weight — so it pulls from all of them, inflating the context window, driving up cost, and producing suboptimal output.</p><p>DinoAI understands which context matters for which task. It efficiently manages the context window and routes intelligently across models — reasoning-heavy work goes to Opus, straightforward tasks go to Haiku. You get better results, faster, at the same cost per token.</p><h3>Benchmarked against the industry</h3><p>And if you strip enterprise context out of the equation entirely and run a pure benchmark — dbt Labs’ <strong>ADE-Bench</strong> — the results are just as clear:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sPv76eQjla-WNYIj.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zY2FWHKsOusYtcH2gnTY7g.png" /></figure><p>DinoAI is the <strong>undisputed leader</strong> for completing analytics and data engineering tasks. And here’s the kicker: <strong>dbt Labs themselves — the team that built the benchmark — scored 58.14% on their own test</strong>. DinoAI beat them by more than 30 points.</p><p>Zoom in further and the picture gets even clearer. Across the 258 individual tests that make up the 43 tasks, DinoAI passed <strong>254 of them</strong> — a <strong>98.45% test pass rate</strong>. It’s like having a mid-senior analytics engineer on your team — the kind that would cost more than $100,000 a year — available 24/7, at a fraction of the cost.</p><p><a href="https://www.paradime.io/blog/dinoai-ade-benchmark-report">Read the full benchmark report →</a></p><h3>Every workflow. Every warehouse.</h3><p>With this launch, we’ve gone way beyond our traditional focus on dbt™. DinoAI v3.0 now supports:</p><ul><li><strong>dbt Core</strong>™ and <strong>dbt Cloud</strong>™</li><li><strong>Apache Spark</strong></li><li><strong>Stored procedures</strong></li><li><strong>SQL</strong></li><li><strong>Python</strong></li><li><strong>Airflow, Dagster, and Prefect</strong> workflows</li></ul><p>Essentially, every possible data engineering workflow that exists in any organization, anywhere in the world.</p><p>On the warehouse side, DinoAI works with <strong>Snowflake, BigQuery, Redshift, Databricks, Trino, Starburst, ClickHouse, SQL Server, Microsoft Fabric, DuckDB, MotherDuck,</strong> and more. You don’t need to be on a bleeding-edge warehouse to take advantage of agentic features — as long as you have a code repository and a warehouse we support, the agent works for you.</p><h3>What this means for data teams</h3><p>DinoAI v3.0 fundamentally changes what your data team spends its time on.</p><p>You’re no longer constrained by a queue of low-value data requests. DinoAI handles those completely. Your team is freed up to focus on <strong>high-value, strategic work</strong> — learning new skills, expanding impact beyond analytics into ML, experimentation, forecasting, and all the things business functions desperately need but your team never had time for.</p><p>And it means something even bigger: <strong>everyone in the organization can now be a data engineer</strong>. It’s not a specialist skill anymore. Technical and non-technical users alike can take full advantage of the data platform — without burdening the data team.</p><h3>Getting started</h3><p><strong>For existing customers</strong>, the agent is available in your Paradime instance from tomorrow. Head to the new agent settings panel under your data warehouse connections, set up which warehouse and repository the agent should use, install the updated Slack app, and you’re ready to go.</p><p><strong>For new customers</strong>, it’s just four steps:</p><ol><li>Sign up for Paradime</li><li>Set up your development environment</li><li>Set up your agent environment</li><li>Start using DinoAI from Slack</li></ol><h3>Pricing</h3><p>We haven’t finalized the exact mechanics yet, but one thing is certain: pricing will be <strong>usage-based, not per-seat</strong>. We want you to experience the product before making a long-term commitment.</p><p>For the next <strong>30 days</strong>, we’re running an open beta — <strong>no usage limits, no minimum commitment</strong>. Sign up and start using it straight away.</p><h3>What’s next</h3><p>A few things already on the roadmap:</p><ol><li><strong>Expanded invocation surfaces</strong> — use DinoAI from Microsoft Teams, ChatGPT, Claude, Claude Code, and more</li><li><strong>Activity logs &amp; audit trails</strong> — a full activity log of every agent action, available within Paradime for enterprise customers</li><li><strong>Multi-repository support</strong> — the agent will be able to work across multiple code repositories in a single environment</li></ol><h3>Final thoughts</h3><p>With the advent of tools like Claude Code, the cost of generating code is effectively near zero. What truly differentiates teams now isn’t the code they generate — it’s the <strong>quality</strong> of that code, <strong>how quickly</strong> they get to that quality, and how quickly they can unlock and optimize organizational processes around data.</p><p>DinoAI v3.0 is our answer. It’s a background agent available 24/7 — so for global teams, data team availability is no longer a blocker. If you need something done, you trigger it whenever you need it.</p><p>At Paradime, we’re thrilled to be the first company building a complete, end-to-end data engineering agent — one that unburdens data teams while increasing the productivity of the entire business by orders of magnitude.</p><p>We can’t wait to see what you build with it.</p><p><strong>Ready to try DinoAI v3.0?</strong> <a href="https://paradime.io/dino-ai">Sign up for the 30-day open beta →</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=da0372ac5e1f" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/introducing-dinoai-v3-0-the-first-end-to-end-data-engineering-agent-da0372ac5e1f">Introducing DinoAI v3.0: The First End-to-End Data Engineering Agent</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Software Development Lifecycle (SDLC) Is Collapsing in Data Engineering]]></title>
            <link>https://medium.com/paradime-labs/the-software-development-lifecycle-sdlc-is-collapsing-in-data-engineering-0d04971c6889?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/0d04971c6889</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Fri, 27 Mar 2026 20:18:07 GMT</pubDate>
            <atom:updated>2026-03-27T20:18:08.522Z</atom:updated>
            <content:encoded><![CDATA[<p>There’s a mental model that every analytics engineering team carries around, even if they’ve never drawn it on a whiteboard. It goes like this: someone writes a ticket. Someone else picks it up. They read the spec, open the IDE, write the SQL, test it locally, push a branch, open a PR, wait for review, get feedback, iterate, merge, deploy, monitor. Each stage has its own tool, its own context switch, its own latency. The whole thing takes days. Sometimes weeks. And everyone involved treats it as normal — the way things have always been done.It’s not normal. It’s a historical accident. And it’s ending.</p><h3>Four Eras in Ninety Seconds</h3><p>If you compress the history of data engineering into a timeline, you get four distinct phases, each shorter than the last.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*iJLEiGQwFJ2dCCv_.png" /></figure><p>We are at the threshold of Era 4. Most teams are still operating in Era 3. That gap is about to become a canyon.</p><h3>The SDLC Didn’t Shrink. It Collapsed.</h3><p>Here’s what the traditional data engineering workflow looks like if you’re honest about it — eleven discrete stages, each involving a context switch, each costing between fifteen and thirty minutes of cognitive recovery time. A single dbt model change can touch six different tools and take two days to ship.</p><p>Now here’s what happens when the ticket <strong>is</strong> the prompt:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*y1H8C32kWZWlYnfe.png" /></figure><p>Six stages collapsed. But more importantly, the three most time-consuming stages — context-gathering, development, and self-testing — are now handled by a system that doesn’t lose context, doesn’t need coffee, and doesn’t get pulled into a Slack thread about the office kitchen.</p><p>The bottleneck moved. It used to be “how fast can a human write code.” Now it’s “how fast can a human verify that code is correct.” And that second question has a very different answer, because verification is cognitively cheaper than creation.</p><h3>The Ticket-as-Prompt Isn’t a Metaphor. It’s an Architecture.</h3><p>This is where most “AI for data” conversations go sideways. People treat the idea of a ticket becoming a prompt as a cute analogy. It’s not. It’s a literal description of how autonomous agents work when they’re properly integrated.</p><p>Consider what happens when you connect an agent to your project management tool — Linear, Jira, Asana, whatever your team uses. The ticket contains structured information: a title, a description, acceptance criteria, sometimes linked specs or Notion pages. That’s not just context for a human. It’s a specification that a sufficiently capable agent can execute against.</p><p>The agent reads the ticket. It reads the linked Notion page with the technical spec. It examines the data warehouse schema. It checks the existing dbt project structure, naming conventions, and testing patterns. It builds the model, generates the YAML, writes the tests, validates everything against the warehouse, and opens a pull request — all without a human touching a keyboard.</p><p>This isn’t science fiction. This is documentation-driven development where the documentation layer and the execution layer share the same context. The information loss between “what we want” and “what we built” approaches zero.</p><h3>Why Most “AI-Assisted” Tools Miss the Point</h3><p>The market is flooded with AI coding assistants. Every IDE has a chat panel now. You can ask a question, get a snippet, paste it in. That’s Era 3. It’s useful. It’s also table stakes.</p><p>The problem with Era 3 tools is that they optimize the wrong thing. They make <strong>writing</strong> faster. But writing was never the bottleneck in analytics engineering. The bottleneck is the space between systems — the gap between the Notion spec and the IDE, between the Jira ticket and the branch, between the Slack conversation where someone said “we need to add region to the revenue model” and the PR where that change actually ships.</p><p>Era 4 tools don’t optimize writing speed. They eliminate the translation step entirely. They operate across the surfaces where work actually happens — your project tracker, your documentation platform, your messaging app, your version control system — and they treat all of those as a single, continuous context.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mZBG989U-KQBTfy-.png" /></figure><p>This is the difference between a tool that helps you write code faster and a platform that turns intent into deployed code. One is a feature. The other is an architecture.</p><h3>What the Human Actually Does Now</h3><p>If the agent handles implementation, what’s left for the engineer?</p><p>More than you’d think. And arguably more valuable than what they were doing before.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Myx6HzIHsv0voMTO.png" /></figure><p>The net effect is that the human moves up the stack. Less time fighting syntax. More time thinking about whether the system is <strong>right</strong>.</p><h3>The Composition Problem</h3><p>Here’s the part that separates the serious platforms from the demos.</p><p>Anyone can wire up an LLM to generate a dbt model. That’s a weekend project. The hard part — the part that determines whether you’re building a Prius or an F1 car — is the <strong>composition</strong> of tooling around the generation step.</p><p>Can the agent read your warehouse schema and understand which tables are sources, which are staging, and which are marts? Can it respect your team’s naming conventions without being told every time? Can it pull context from your Notion wiki, your Jira board, and your existing codebase simultaneously? Can it open a properly formatted PR with a description that references the original ticket? Can it run the build, check for failures, and iterate before asking for your review?</p><p>Each of those capabilities is individually unimpressive. Together, they’re the difference between an agent that produces throwaway snippets and one that ships production code.</p><p>And the composition has to extend beyond the IDE. Analytics engineering doesn’t happen in one place. Conversations happen in Slack. Specs live in Notion or Confluence. Tickets live in Linear or Jira. Code lives in GitHub. The warehouse is Snowflake or BigQuery or Databricks. A truly autonomous agent operates across all of these surfaces, because that’s where the context lives.</p><blockquote><em>An agent that only works inside the IDE is an Era 3 tool wearing an Era 4 costume. The real shift happens when the agent meets you where you work.</em></blockquote><p>In Slack when a pipeline breaks at 3 AM. On your phone when you need to approve a deployment. In your project tracker when a ticket is assigned.</p><h3>The Acceptance Rate Is the Only Metric That Matters</h3><p>When everyone has access to the same foundational models — and increasingly, everyone does — the differentiator isn’t generation speed. It’s acceptance rate.</p><p>How much of what the agent produces do you actually keep?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*jCe8KAseNieslJdQ.png" /></figure><p>A 30% acceptance rate means the agent is generating noise that you have to sift through. You’re spending more time rejecting bad output than you would have spent writing the code yourself. That’s a net negative.</p><p>A 90% acceptance rate means the agent understands your context deeply enough to produce work that’s right the first time. Your job becomes a quick review pass, not a rewrite. That’s a 10x improvement.</p><p>The acceptance rate is a function of context density. The more the agent knows about your project — your warehouse schema, your coding conventions, your documentation standards, your business logic, your team’s review preferences — the higher the acceptance rate. And context density is a function of integration depth. An agent that only sees your codebase has thin context. An agent that also sees your Notion specs, your Linear tickets, your Slack conversations, and your warehouse metadata has rich context.</p><p>This is why composition matters more than raw model capability. A frontier LLM with thin context will produce worse output than a capable LLM with rich, domain-specific context. The model is table salt. The context is the Maldon flakes.</p><h3>The Window Is Open. It Won’t Stay Open.</h3><p>There’s a pattern in technology adoption that plays out with uncomfortable regularity. A new capability emerges. Early adopters gain a compounding advantage. The majority waits for consensus. By the time consensus forms, the early adopters have built such a lead that catching up requires 10x the effort.</p><p>We’re in the early-adopter window for autonomous data agents right now. The teams that learn to work with agents — that develop the muscle memory for writing good specifications, that build the guardrails and standards that make agents reliable, that restructure their workflows around human-as-reviewer instead of human-as-author — those teams will compound their advantage every quarter.</p><p>The teams that wait will find themselves in the position of the company that was still deploying manually when everyone else had CI/CD. Not dead. Just permanently behind.</p><blockquote><em>The software development lifecycle is collapsing. The question isn’t whether your team will adopt autonomous agents. It’s whether you’ll be the one setting the pace or the one trying to catch up.</em></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0d04971c6889" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/the-software-development-lifecycle-sdlc-is-collapsing-in-data-engineering-0d04971c6889">The Software Development Lifecycle (SDLC) Is Collapsing in Data Engineering</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The F1 Problem: When Everyone Has a Car, Who Needs a Racing Team?]]></title>
            <link>https://medium.com/paradime-labs/the-f1-problem-when-everyone-has-a-car-who-needs-a-racing-team-69106cd4386a?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/69106cd4386a</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[ai]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Mon, 16 Mar 2026 16:44:27 GMT</pubDate>
            <atom:updated>2026-03-16T16:44:26.887Z</atom:updated>
            <content:encoded><![CDATA[<p>When AI makes code writing a commodity, the gap between casual users and elite teams doesn’t shrink. It becomes enormous.</p><p>Salt is arguably the most commoditized substance on Earth. Every kitchen has it. Every restaurant. Every gas station snack bag. It costs almost nothing — a kilo of table salt runs you less than a dollar in most countries. And yet, Himalayan pink salt commands ten to thirty times that price. Fleur de sel? Even more. Maldon flakes sit on the shelves of specialty grocers with the quiet confidence of a product that knows its worth. Same molecule. Same NaCl. Wildly different value perception.</p><p>This is the story of AI coding tools right now. And most people are missing it completely.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EvPCgyqPrTtEs_au.png" /></figure><h3>The Cost of Code Just Hit Zero. Now What?</h3><p>If you spend any time on LinkedIn right now, you’ll encounter a particular breed of post. Someone fires up Claude Code, generates a hundred dbt™ models in an afternoon, and writes a breathless update about how “the game has changed.” The comment section erupts. Heart reacts pile up. Someone calls it the future of data engineering.</p><p>And they’re not wrong, exactly. They’re just not seeing the whole picture. Because the act of writing code is now super cheap — effectively approaching zero marginal cost. But the value of <strong>the right code, in the right context, shipping reliably at scale</strong>? That’s never been more expensive to achieve.</p><p>The cost of writing code going to zero has an unintuitive consequence. If code is essentially free to produce, then writing velocity is no longer the bottleneck. <strong>Writing performance is.</strong> The question isn’t “how fast can you generate code” — it’s “how good is the code you accept, and how quickly does it get you to the outcome?”</p><blockquote><em>“If the cost of writing code goes to zero, nobody cares about writing more code. Writing performance becomes more important than writing velocity.”</em></blockquote><p>This is a distinction that gets lost in the LinkedIn demo reel. When you see someone generate a hundred dbt™ models with an AI tool, you’re watching them drive a Prius. It’s functional. It’s efficient. It gets you to the supermarket and back. But nobody is confusing a Prius with a Formula 1 car.</p><h3>Prius Mode vs. Pit Lane</h3><p>Millions of people drive cars every day and believe they have a good car. They do. A modern sedan is an engineering marvel — airbags, fuel injection, GPS, lane assist. It’s the product of a century of innovation trickling down into commodity. But then you see a Formula 1 car, and you realize you’re looking at an entirely different species. Same four wheels. Same internal combustion principles. A completely different universe of engineering precision.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wj6ftwCHenQCtTOu.png" /></figure><p>This is exactly the dynamic playing out in AI-assisted development. The LinkedIn influencers demoing a hundred dbt™ models with Claude Code are driving the Prius. It works. It’s impressive relative to where we were three years ago. But enterprise data teams — the ones running a thousand interconnected models across multiple warehouses, with FP&amp;A teams downstream, with regulatory requirements, with SLAs that actually matter — those teams need the F1 car.</p><p>And here’s the beautiful irony that makes this analogy more than metaphor: the Prius literally descends from F1 technology. Toyota’s hybrid system — KERS, the kinetic energy recovery system — was born on the Formula 1 track. They experimented at the extreme. They absorbed the development cost. Then they deployed it at commodity scale. That’s how elite innovation becomes everyday reality. It trickles down. But the people driving the Prius don’t get to claim they’re racing in the Grand Prix.</p><h3>What “Premium” Actually Means</h3><p>So if everyone has access to the same AI coding tools — and they increasingly do — where does the premium emerge? It’s not in the code generation itself. That’s table salt. The premium is in three things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*m3pe-LJ0HNhkFL3_.png" /></figure><p><strong>Acceptance rate:</strong> How much of the AI-generated output do you actually keep? The people posting LinkedIn demos don’t talk about this because, in a demo, you accept everything. In production, the acceptance rate is the entire game. An elite team doesn’t just generate code — they have the architecture, the context-awareness, and the prompt engineering to generate code that’s <em>right the first time</em>. Measuring acceptance rate is the difference between having a speedometer and just feeling like you’re going fast.</p><p><strong>Time to completion:</strong> Not time to generation — time to <em>completion</em>. How long from intent to merged, tested, deployed, monitored code? This is where the composition of your tooling stack matters enormously. The AI experience isn’t just the LLM response. It’s the amalgam of tooling around it — the CI pipelines, the testing frameworks, the governance layers, the deployment automation — that turns a generated snippet into a production asset.</p><p><strong>Composition of tooling:</strong> Elite teams aren’t just using one tool. They’re composing entire ecosystems — agents that understand their data lineage, tools that know their schema conventions, frameworks tuned to their specific domain. The ticket becomes the prompt. The Linear issue <em>is</em> the specification that gets executed. There’s no loss of information between intent and implementation. That integration density is the Himalayan salt of the development world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J7NHRIwFC59qziDV.png" /></figure><h3>The Cambrian Moment</h3><p>We are living in what feels like a Cambrian moment for developer tooling. Things are changing so quickly that a partnership announced two months ago can be made obsolete by a feature launch today. The velocity is disorienting. And in that disorientation, a lot of people are mistaking access for mastery.</p><p>They’re not the same thing. Access is the Prius. Mastery is the racing team — the pit crew, the telemetry engineers, the aerodynamicists, the strategists who know when to pit and when to push. Access gets you on the road. Mastery gets you on the podium.</p><p>And here’s the thing about the asymptote we’re all climbing: we’ve reached the flat part. The early gains from AI tools were dramatic — a 10x improvement was genuinely available to almost anyone. But now, squeezing out that last 1% of edge requires 99% of the effort. That’s where the real competition lives. That’s where the F1 teams operate.</p><blockquote><em>The people who grab that experience early — who learn to operate at the frontier, not just at the surface — will take a disproportionate share of what comes next.</em></blockquote><p>So the next time you see a LinkedIn post celebrating a hundred AI-generated models, appreciate it for what it is: a Prius driving smoothly down a well-paved road. Then ask yourself what the team with a thousand models, a 90% acceptance rate, and a four-minute time-to-completion is doing. Because that’s the F1 car. And that’s where the race is actually being won.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=69106cd4386a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/the-f1-problem-when-everyone-has-a-car-who-needs-a-racing-team-69106cd4386a">The F1 Problem: When Everyone Has a Car, Who Needs a Racing Team?</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Google Workspace to dbt™ Code: AI Workflows with DinoAI]]></title>
            <link>https://medium.com/paradime-labs/from-google-workspace-to-dbt-code-ai-workflows-with-dinoai-492599b5f439?source=rss----cd7e09652475---4</link>
            <guid isPermaLink="false">https://medium.com/p/492599b5f439</guid>
            <category><![CDATA[ai-agent]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[context-engineering]]></category>
            <dc:creator><![CDATA[Kaustav Mitra]]></dc:creator>
            <pubDate>Sun, 01 Mar 2026 20:15:38 GMT</pubDate>
            <atom:updated>2026-03-01T20:15:38.057Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FBqjw_zQ0rxA8xCD_nTVYw.avif" /></figure><h3>Introduction</h3><p>Your dbt™ project’s biggest problem isn’t the SQL — it’s the context that never makes it in. The spec doc in Drive. The “official” definitions in a Sheet. The data contract nobody remembers to enforce. DinoAI’s new Google Workspace integrations pull that context directly into your development workflow, so you can stop translating and start shipping.</p><p>Three new tools: <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-docs-tool">Google Docs</a>, <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-drive-search-tool">Google Drive Search</a>, and <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-sheets-tool">Google Sheets</a>. They’re most useful when chained together with DinoAI’s action tools (Terminal, File System, GitHub PR) inside a single <a href="https://docs.paradime.io/app-help/documentation/dino-ai/agent-mode">Agent Mode</a> workflow.</p><h3>Workflow 1: Spec Doc → dbt™ Models and Tests</h3><p>A product manager drops a Google Doc with requirements. Instead of reading it in one tab and building in another, point DinoAI at it directly.</p><pre>Read this Google Doc: &lt;https://docs.google.com/document/d/1abc...xyz&gt;<br>Generate dbt source definitions, a staging model, a marts model,<br>and a schema.yml with column descriptions and tests.<br>Then run dbt compile to validate.</pre><p>DinoAI extracts the business logic, scaffolds the assets, and validates them. If dbt compile surfaces ref errors, it fixes them before returning results to you.</p><h3>Workflow 2: Data Contract → Enforced Source Definitions</h3><p>Contracts live in Drive. Models get built without them. DinoAI can close that loop.</p><pre>Search Google Drive for the latest &#39;orders data contract&#39; modified this month.<br>Read it and generate a dbt source definition with column-level tests —<br>unique, not_null, accepted_values where specified.<br>Embed the contract doc URL in the meta block.</pre><p>The modification date filter ensures you’re working from the current version, not a stale copy. The embedded URL in meta makes the source definition traceable back to the document that mandated it. When the contract updates, re-run the same prompt.</p><h3>Workflow 3: Google Sheet → dbt™ Seed</h3><p>Reference data living in Sheets — territory mappings, cost center codes, fiscal calendars — should be in your dbt™ project as tested, documented seeds. Now it can be, in one prompt.</p><pre>Read the Google Sheet 1uXabc... (sheet: territory_mapping).<br>Create seeds/territory_mapping.csv.<br>Generate a schema.yml with not_null tests on territory_id<br>and accepted_values for region based on what&#39;s in the sheet.<br>Run dbt seed and dbt test --select territory_mapping.</pre><p>DinoAI infers types, maps distinct values to accepted_values tests automatically, writes the seed, and runs a full validation cycle. For multi-tab workbooks, specify the sheet name or GID — it removes ambiguity.</p><h3>Workflow 4: Drive Content → dbt™ Documentation</h3><p>Most undocumented models aren’t undocumented because the knowledge doesn’t exist — it’s because nobody has time to find it and write YAML. DinoAI can do that automatically.</p><pre>Search Drive for docs mentioning &#39;customer lifetime value&#39; or &#39;LTV methodology&#39;.<br>Read the most relevant one and generate column-level descriptions<br>for models/finance/fct_customer_ltv.sql.<br>Match the language in the doc — don&#39;t use generic descriptions.</pre><p>The “match the language” instruction matters — it keeps descriptions grounded in your team’s actual definitions rather than generic boilerplate. This scales: point DinoAI at a Drive folder and a models directory and let it run.</p><h3>Workflow 5: Spec → Pull Request</h3><p>The full loop — from Google Doc to committed, PR-ready code.</p><pre>Read the spec at &lt;https://docs.google.com/document/d/1xyz&gt;...<br>Create a feature branch called feature/refunds-model.<br>Scaffold the refunds staging and marts models from the spec.<br>Run dbt build --select refunds+.<br>If tests pass, commit and open a PR linking back to the spec doc.</pre><p>DinoAI chains Docs → File System → Terminal → GitHub PR in one workflow. The PR description includes a spec summary and a direct link to the source document — code and requirements stay connected.</p><h3>Workflow 6: Multi-Sheet Dimensional Modeling</h3><p>When business definitions are spread across multiple spreadsheets, DinoAI can read them all and reason across them.</p><pre>Read three Google Sheets:<br>1. abc... (sheet: products) — product dimension definitions<br>2. def... (sheet: channels) — channel taxonomy<br>3. ghi... (sheet: fiscal_calendar) — fiscal week offsets</pre><pre>Generate dim_products, dim_channels, and a fiscal_calendar seed.<br>Create a schema.yml covering all three with appropriate tests.<br>Run dbt build --select dim_products dim_channels fiscal_calendar.</pre><p>Multi-source reconciliation that normally takes an afternoon happens in a single prompt.</p><h3>A Few Things Worth Knowing</h3><p><strong>Filter Drive searches by date.</strong> Always include a modification date in Drive Search prompts when looking for contracts or specs. Stale docs produce wrong output.</p><p><strong>Anchor to source URLs.</strong> Ask DinoAI to embed doc URLs in meta blocks and PR descriptions. Traceability from dbt™ asset back to source document is the point.</p><p><strong>Let DinoAI handle the debug loop.</strong> When a workflow includes dbt build or dbt test, DinoAI reads error output and iterates. Don&#39;t interrupt it at the first failure.</p><p><strong>Keep sheets clean.</strong> Headers in row 1, consistent column types. The Sheets Tool performs significantly better with well-structured data.</p><p>All three tools require a connected Google account in Paradime. Setup: <a href="https://docs.paradime.io/app-help/documentation/integrations/code-ide/google-docs">Google Docs</a> · <a href="https://docs.paradime.io/app-help/documentation/integrations/code-ide/google-drive">Google Drive</a> · <a href="https://docs.paradime.io/app-help/documentation/integrations/code-ide/google-sheets">Google Sheets</a></p><p>Full tool docs: <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-docs-tool">Google Docs Tool</a> · <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-drive-search-tool">Google Drive Search Tool</a> · <a href="https://docs.paradime.io/app-help/documentation/dino-ai/tools-and-features/google-sheets-tool">Google Sheets Tool</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=492599b5f439" width="1" height="1" alt=""><hr><p><a href="https://medium.com/paradime-labs/from-google-workspace-to-dbt-code-ai-workflows-with-dinoai-492599b5f439">From Google Workspace to dbt™ Code: AI Workflows with DinoAI</a> was originally published in <a href="https://medium.com/paradime-labs">paradime.io</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>