<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jason Lee on Medium]]></title>
        <description><![CDATA[Stories by Jason Lee on Medium]]></description>
        <link>https://medium.com/@jasonlee_82633?source=rss-1d80ad648c------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*cwRQ1Y1owMoYSnQvoEEHbQ.jpeg</url>
            <title>Stories by Jason Lee on Medium</title>
            <link>https://medium.com/@jasonlee_82633?source=rss-1d80ad648c------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 10:52:21 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jasonlee_82633/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Analytics in the Age of AI]]></title>
            <link>https://engineering.instawork.com/analytics-in-the-age-of-ai-864a4fb980e0?source=rss-1d80ad648c------2</link>
            <guid isPermaLink="false">https://medium.com/p/864a4fb980e0</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[business-strategy]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Jason Lee]]></dc:creator>
            <pubDate>Tue, 24 Mar 2026 05:12:55 GMT</pubDate>
            <atom:updated>2026-03-25T15:57:14.079Z</atom:updated>
            <content:encoded><![CDATA[<h3>The bottleneck isn’t analysis anymore</h3><p>Here’s something we didn’t expect to be writing: the biggest challenge facing our analytics team right now isn’t doing analysis. It’s making sure everyone else’s analysis is trustworthy.</p><p>Six months ago, we were a typical under-resourced analytics team. Backlog growing faster than we could chip away at it. Long SLAs for data requests. Functional partners waiting weeks for answers to questions that felt urgent to them. The usual story that most analytics leaders know well.</p><p>Today, almost 1 in 3 of non-engineers across the company actively using Cursor to query data, build reports, and run their own analyses. Our Head of Marketing writes SQL. Our Chief Legal Officer builds dashboards. And our analytics team spends more time curating context and quality-assuring other people’s work than writing queries themselves.</p><p>This is the story of how that happened at Instawork — and what it taught us about where analytics is actually heading.</p><h3>Where we started</h3><p>Instawork is a Series D marketplace for work. Our analytics team is small but operates across both Product and GTM — the classic setup where a handful of people are trying to serve an entire company’s data needs.</p><p>We dealt with the problems most analytics teams deal with. Limited bandwidth meant not every function got the support they wanted. Our backlog kept growing. Some teams waited weeks for relatively straightforward analyses. And a lot of valuable data was locked in formats we hadn’t gotten around to structuring — sales call recordings, agent &lt;&gt; user transcripts, and contract details sitting in PDFs.</p><p>We were constrained in the way almost every analytics team is constrained: too many questions, not enough people to answer them.</p><h3>The unlock: giving everyone an IDE</h3><p>We started by getting our entire analytics team onto Cursor. The initial use case was narrow — building dbt models, mostly. But it expanded quickly. Within weeks, analysts were using it for SQL querying, ad hoc analysis, and automating reporting workflows.</p><p>We were partly inspired by what the team at Faire had done with AI-assisted query writing. Their experience — and their honesty about the onboarding challenges — gave us the confidence to try it ourselves.</p><p>Within a month, the use cases had multiplied: writing SQL, automated reporting, full analysis write-ups, and workflow automation. Things that used to take a week were happening in minutes.</p><p>For example, reporting is the silent drain on every analytics team. Stakeholders want regular updates, breakdowns, and explanations for what changed — all valuable, but time-consuming. Instead of servicing these manually, we built a “reporting skill.” It runs the SQL, analyzes metric changes, generates a clean write-up, and sends it on a schedule. We teach it once, and it runs every time. In practice, these reporting automations can free up ~20% of the team’s time — shifting effort from reporting to actual problem solving.</p><h3>What surprised us</h3><p>While the productivity gains were expected, the second-order effects were not. Here are the four things that caught us off guard.</p><h3>Everyone became an analyst</h3><p>Two months after we started, we have 2x more non-Tech Cursor users than Product &amp; Eng combined. The power users weren’t who we expected. Product managers and data-adjacent people picked it up. But so did our Head of Marketing; Head of Comms, our RevOps team, and our Product Researcher.</p><p>One of the our most surprising use cases is from our Chief Legal Officer (CLO) in dealing with accident claims management. At Instawork, supporting Pros through claims used to require stitching together data from multiple systems — shift details, hours worked, incident reports — often taking days and slowing down response times. Our CLO spent a weekend building a dynamic dashboard — all through natural language with Cursor — to access all relevant data in one place. Claims can now be triaged and prioritized in minutes instead of days, with far less manual back-and-forth. We can also proactively spot where claims are increasing — by partner, role, or shift pattern — so we can intervene early with staffing, training, or operational changes before issues escalate. That speed means Pros hear back faster, get routed to the right support sooner, and feel supported in real time — while Partners benefit from quicker resolution, fewer incidents, and lower overall claim costs.</p><p>At Instawork, our Head of Comms — who has never written a SQL query before — now runs monthly, media-ready wage analyses directly using AI. Instead of waiting on analysts, she can pull and validate insights across roles and regions in minutes. That speed lets us respond to press in real time and share more transparent, data-backed stories about Pro earnings.</p><h3>The analytics team’s job flipped</h3><p>As more people started self-serving on data, the ad hoc requests that used to dominate our team’s time started drying up. Last-minute, urgent requests went from roughly 30% of our team’s time to less than 10%.</p><p>Our analytics team now spends as much time enabling others as doing analysis ourselves. The job shifted from “answer this question” to “make sure the people answering their own questions have what they need to get it right.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*646gnqd0zQOuf5R6O9Onuw.png" /></figure><h3>We accidentally created an insight firehose</h3><p>When everyone can produce analysis, everyone does. The volume of insights being generated across the company went up dramatically. And that sounds great until you realize that humans can’t process or prioritize all of it.</p><p>The constraint flipped. It was no longer about the speed of analysis execution — it was about the quality of the questions being asked. Turns out, the scarce resource in a world of abundant analysis isn’t SQL skills. It’s the ability to frame sharp, meaningful problem statements and reason clearly about the business.</p><h3>Trust became the whole game</h3><p>When you democratize analysis, you also democratize the potential for mistakes. Quality assurance at scale became our single biggest challenge — making sure AI-generated analyses were accurate and that hallucinated numbers weren’t making their way into decisions.</p><p>I’ll use myself as the example. I ran a sales productivity analysis — over a dozen cuts asking where ROI was highest across segments. Produced it all in one sitting. I was thrilled. But I’d missed something: the query only included leads that had already completed a demo, not all leads worked. That inflated ROI across the board, because it ignored all the effort that never converted. Left unchecked, it could have justified investing headcount in the wrong places. An analyst on the team caught it. The error wasn’t in the SQL — it was in the logic, the kind of thing you’d naturally pressure-test when you’re building an analysis over a week. When you produce it in minutes, you’re reviewing an output, not constructing an argument. And that’s a very different kind of quality check.</p><p>This forced us to build QA infrastructure we hadn’t anticipated. We started building QA agents to catch common errors. We iterated heavily on our context documentation to reduce the frequency of mistakes. It’s an ongoing effort, and we’re not done — but recognizing that trust is the core problem was the crucial shift.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_HfK_TE78gFCqkI7mgMcfQ.png" /></figure><h3>The new bottleneck: context infrastructure</h3><p>When analysis execution is cheap, the expensive thing is the knowledge that makes analysis trustworthy. Think about what your best analyst knows about a given topic. They know which tables to use and which ones are stale. They know the edge cases — that one metric that looks wrong every quarter because of seasonality, or that a particular segment needs to be filtered out of certain comparisons. They know what “normal” looks like for a given number, and they can spot when something is off.</p><p>All of that context lives in people’s heads. And when AI is doing the analysis, it doesn’t have any of it — unless you build the infrastructure to transfer it.</p><p>That’s become the primary focus for our analytics team. We’re building what I’d call context infrastructure, and it falls into a few categories.</p><ul><li><strong>Business context</strong> is the foundational layer: metric definitions and expected ranges, documentation of which tables and datasets to use for which questions, edge cases and gotchas that trip people up, and what we call “golden queries” — curated, validated query patterns that serve as trusted starting points for common analyses.</li><li><strong>Skills</strong> sit on top of that: frameworks for reporting, standardized cohort analysis methodology, metric decomposition approaches. The kind of reusable analytical logic that our best analysts apply intuitively but that needs to be made explicit for an AI to use well.</li></ul><p>The other frontier is unlocking unstructured data. When bandwidth to do analysis is no longer the constraint, data quality and data availability become the next bottleneck. We have a wealth of potential insights locked in unstructured formats — sales and customer support call recordings, agent interactions, research inputs — that we’re only beginning to tap.</p><h3>What this means if you lead an analytics team</h3><p>I think we’re seeing the early contours of a real shift in what analytics teams look like and what they do:</p><p><strong>The team composition changes.</strong> Fewer traditional analysts doing bespoke, one-off work. More analytics engineers focused on data quality, semantic layers, context curation, and reusable logic. Analytics headcount probably grows more slowly — or even shrinks in some organizations. But the team’s leverage and strategic impact goes up. It’s not about doing less; it’s about doing fundamentally different, higher-leverage work.</p><p><strong>The skills that matter shift.</strong> Technical skills like SQL become less important when the AI writes it for you. What matters more is intuition for QA and logical reasoning — can you look at an output and tell whether it makes sense? Can you frame a problem clearly enough that an AI (or a non-analyst colleague) can actually investigate it well? When everyone can produce analysis, the job becomes maximizing signal-to-noise and minimizing slop.</p><h3>Creating the magic moment</h3><p>If you’re thinking about rolling something like this out, here’s my practical advice: it’s all about creating that first magic moment. There will be resistance, particularly from non-technical people who look at Cursor/Claude Code and assume it’s not for them.</p><p>We learned what doesn’t work: giving people dedicated time to experiment and hoping they figure it out. Our AI-native people used that time well, but the people who weren’t already comfortable with technical tools struggled to get started on their own. Unstructured exploration doesn’t help someone who doesn’t know what to explore.</p><p>What did work was facilitated onboarding — structured sessions where we walked people through Cursor hands-on, followed by office hours for 1:1 attention. The goal was always to drive to the magic moment as fast as possible. For some people, that was seeing data come back from their first successful query. For others, it was publishing an analysis or creating their first dashboard. The specific moment varied, but the pattern was the same: once someone saw the tool do something they couldn’t do before — or something that would have taken them hours — they were hooked.</p><h3>Where this is going</h3><p>We’re moving toward a world where every function in a company can self-serve on data. Where the analytics team is a small, high-leverage group focused on context, quality, and the hardest problems — not on fielding a never-ending queue of ad hoc requests. Where the time from question to insight collapses from days to minutes, not just for analysts, but for the Head of Marketing, the CLO, the finance lead, and the ops manager.</p><p>The bottleneck isn’t analysis anymore. It’s context. And the teams that figure that out first are going to have a serious edge.</p><p><a href="https://medium.com/@sifanliu/from-writing-sql-to-automating-insights-e77efe25fa57">See Part II of this article to learn more about our approach to agentic analytic</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=864a4fb980e0" width="1" height="1" alt=""><hr><p><a href="https://engineering.instawork.com/analytics-in-the-age-of-ai-864a4fb980e0">Analytics in the Age of AI</a> was originally published in <a href="https://engineering.instawork.com/">Instawork Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>