AI assisted In-house Product Design — Extracting Design Insights from Massive User Feedback
In the first half of 2026, the conversation around AI-assisted design has been overwhelmingly focused on how to use tools like Stitch, Design.md, and Claude Design to rapidly mass-produce mockups and generate near-deliverable design files.
But for me right now, these technologies still need more exploration before they can realistically connect to a designer's daily work — because I'm an in-house designer, and twenty-some of us share a massive design system. I still can't imagine how to convey it correctly to an AI.
Conversely, what has actually helped my work this past half-year is having AI help me research user feedback — and that's what this article aims to document.
“Having Data” ≠ “Being Able to Use It Right Now”
My company has abundant user feedback data, including thousands of NPS (Net Promoter Score) responses and App Store reviews every month. This data should be a precious mine for understanding users and validating design decisions. But in practice, it often goes unused by designers in the moment — never realizing its potential value.
The problem isn’t a lack of data. It’s the “extractability”:
- Tree-structured data, deeply buried:
Our monthly UXR reports organize NPS and App Reviews by category, forming a tree structure. When a designer needs to find feedback about a specific feature, they must manually drill down through layers of categories — time-consuming and inefficient. - UX Friction gets drowned out:
In the data, the majority of user feedback is about transactions and bonuses — things tied to business decisions and company generosity. The UX friction that designers are better positioned to address — the kind that impacts conversion rates — often doesn’t make up the majority. It gets buried under the louder noise of benefit-related hot topics. - Hours of search cost:
For example, when I was designing a deposit-related feature, finding cross-month user pain points about deposits required hours of my time — filtering month by month, category by category, then manually judging which feedback was relevant to my design context. This enormous time cost makes the opportunity for designers to proactively leverage this data in daily work nearly nonexistent.
— And this is exactly the kind of tedious, mechanical work that can be handed off to AI!
AI as My Data Indexer
In mid-2025, I started learning how to use AI to solve the problem of needing to sift through large volumes of data.
My first attempt was feeding this massive data (about 8,000 records per month) into the then-mainstream ChatGPT and Google Gemini. I hit bottlenecks quickly:
- Context window limits:
Large amounts of data would rapidly exceed these models’ context windows, causing them to take too long to read. - Frequent hallucinations:
Worse, when models processed too much information, they began producing massive hallucinations — answers that were simply unreliable and unusable as design evidence.
These attempts made me realize that simply “feeding data” wasn’t enough. So I started exploring a more complex path: building an Agent + RAG (Retrieval Augmented Generation) architecture using n8n. Theoretically viable, but brutally difficult in practice. It didn’t just require deep technical knowledge — more importantly, its complexity made it nearly impossible for non-technical colleagues to replicate or adopt.
Just as I was stuck on this, I discovered a breakthrough tool — Google NotebookLM.
NotebookLM’s biggest draw is its specially trained model that strictly adheres to “mandatory fact-based answering.” This means it will strictly base its responses on the raw data I provide — and most critically, it attaches citations linking back to the original data, letting me jump directly to the source feedback for verification.
This was exactly what I needed!
NotebookLM — Massive Input, Fuzzy Search
Before, I had to manually follow the tree structure pre-classified by our Researchers, drilling down layer by layer to find the information I needed. Now, NotebookLM provides a full-text, fuzzy search capability across the entire range of raw data. When I want to find a specific fact or insight, the time cost drops from hours to minutes.
I just type: “What pain points exist around deposits?” NotebookLM quickly and accurately extracts all relevant user feedback from months of data, with verifiable source links attached.
Discovering Hidden UX Friction: “Merge Orders”
Through NotebookLM’s search, I uncovered UX friction that was easily buried in quantitative stats but deeply impactful.
Get Getter Chen’s stories in your inbox
Join Medium for free to get updates from this writer.
For example: in the 2024–25 data, NotebookLM helped me find 5 pieces of user feedback mentioning a desire for a “merge orders” feature.
The interesting part? We already had this feature! But from the context of user feedback, they either couldn’t find it or didn’t realize how to activate it in our system — it wasn’t missing functionality.
Users said they wanted to “merge” or “combine” orders, while our existing feature was labeled “retain.” This vocabulary gap was the root cause of the user perception gap.
This case fully demonstrates NotebookLM’s value as an “evidence extractor.” It lets those seemingly unimportant voices — invisible in aggregate data — be clearly captured and transformed into actionable design insights.
Complementary Strategy: NotebookLM as the “Evidence Foundation”
NotebookLM has its limits — you can’t rely on it alone for product design. For example:
- It can’t browse the web for real-time information.
- It lacks multimodal capability — it can’t directly read my design proposal mockups or understand which visual context a user complaint refers to on a page.
So for deeper idea discussions, concept validation, or analysis that requires visual information, I go back to the more full-featured Google Gemini or ChatGPT. For instance, I use NotebookLM to find user feedback that supports or contradicts a design hypothesis, then bring that “evidence” into Gemini or GPT for broader brainstorming and feasibility analysis.
Strategically using NotebookLM as the “fact foundation,” then pairing it with other LLMs for “divergent thinking” — this lets me balance rigor and creativity simultaneously in my design process.
Data Prep: A “Rough-and-Ready” Semi-Automated Workflow
Thousands of records a month is no small amount, and every company stores raw data differently. My current data prep process is semi-automated, taking about half an hour per month:
1. Export: I export the monthly NPS and App Review data from the company database as CSV.
2. Preprocessing (Custom GPT-assisted): I use a custom GPT I built to handle these CSV files. It automatically reorders columns and splits the raw file into smaller PDFs of 2,000 records each.
3. Import into NotebookLM: Import the split PDFs into NotebookLM.
Why 2,000 records? Why reorder columns and convert to PDF tables? This is just my personal experience using NotebookLM —
- 2,000 records was the sweet spot where NotebookLM 1.0 indexed fast enough without creating too many separate files.
- PDF table format beats CSV: I found that raw data in PDF table format is the easiest to hover-read original feedback inside NotebookLM. Importing an entire CSV directly makes the long-scrolling table a terrible reading experience, drastically lowering review efficiency.
※ This preprocessing workflow needs to be designed independently based on your company and how your data is stored/exported — it’s the least automatable part.
Impact: From “Uncertain” to “Evidence-Based” Design Proposals
In the roughly half year since I built and promoted this NotebookLM-centered user feedback analysis mechanism, I’ve integrated these qualitative feedback insights into three self-initiated design proposals, giving my design decisions solid grounding. I can ask AI whether my idea has corresponding feedback, or describe quantitative data I’m seeing and ask AI whether there’s a matching explanation.
One feature developed from these qual/quant insights has already gone live, achieving CTR 49% → 86%, CVR 76% → 89% in A/B testing — proving this method’s effectiveness and setting a precedent for more similar projects going forward.
I believe that as more success cases of user-feedback-driven design accumulate internally, other designers will be able to use the NotebookLM I built more fully and eagerly — internalizing more user voices into the core engine of our product iteration.
In this case, AI helped me not with output volume, but with decision quality — it lets me luxuriously explore the qualitative feedback database for every design decision. The data prep approach may not transfer directly to your context, but I hope this mindset of indexing massive data can help you too.
