<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Adam Malamis on Medium]]></title>
        <description><![CDATA[Stories by Adam Malamis on Medium]]></description>
        <link>https://medium.com/@AdamMalamis?source=rss-66797233241b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1P_gURbotf7CmYrphiFIPQ.jpeg</url>
            <title>Stories by Adam Malamis on Medium</title>
            <link>https://medium.com/@AdamMalamis?source=rss-66797233241b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 08:06:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@AdamMalamis/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Dissecting Rand Fishkin’s Research on AI Visibility]]></title>
            <link>https://medium.com/@AdamMalamis/dissecting-rand-fishkins-research-on-ai-visibility-c6cfd0dc468c?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c6cfd0dc468c</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[generative-ai-tools]]></category>
            <category><![CDATA[ai-search]]></category>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Mon, 09 Mar 2026 03:30:29 GMT</pubDate>
            <atom:updated>2026-03-09T13:38:12.409Z</atom:updated>
            <content:encoded><![CDATA[<h3>Dissecting Rand Fishkin’s AI Visibility Research</h3><p><strong>Disclaimer</strong>: Hi, I’m <a href="https://www.linkedin.com/in/adammalamis/">Adam</a>, head of product for an AI analytics platform called <a href="https://gander.li/gYWYajV">Gander</a>. Given my current role, it’s safe to assume there’s bias in my response. I’ve tried to eliminate it as much as possible. If you think what I’m stating here is wrong, that my arguments are weak, or if I’ve missed the mark, I want to hear from you.</p><h3>Summary</h3><p>The industry is touting a <a href="https://sparktoro.com/blog/new-research-ais-are-highly-inconsistent-when-recommending-brands-or-products-marketers-should-take-care-when-tracking-ai-visibility/">recent SparkToro article</a> by SEO OG,<a href="https://www.linkedin.com/in/randfishkin/"> Rand Fishkin</a>, describing AI tracking tools as “selling a boondoggle of snake oil chicanery”. It was a solid read; an absolute masterclass in knowing one’s audience and writing a compelling advertorial. It took a minute to process, but I was in awe. He took two of my favourite things, rhetoric and research, and executed them like the seasoned marketing pro that he is.</p><p>Each argument and corresponding statement was beautifully presented and reinforced — some by data, others by well-constructed rhetorical devices. Here’s the crux of the article:</p><ul><li>Popular AI visibility tools are an expensive waste of budget. Anyone using these can’t be a serious marketer.</li><li>Because LLMs, like ChatGPT, are wildly inconsistent when presenting results.</li><li>We should probably track visibility, though… since it’s the only meaningful metric we currently have.</li><li>So try my friend Patrick’s tool instead. It has a <em>free</em> plan. LOL.</li></ul><p>Ok, you caught me. I added the LOL in for effect.</p><p>Let’s break this down further.</p><h3>Know your audience</h3><p>The majority of search professionals would agree the industry is in the process of a seismic shift. Google, the Galactus of search, has had a stranglehold on your marketing dollars for more than two decades. Bing, its closest competitor, took over 10 years to<a href="https://en.wikipedia.org/wiki/Search_engine?utm_source=chatgpt.com#Market_share"> pry 4% of marketshare</a>.</p><p>While the numbers I’ve read vary, an aggressive estimate would say that around 20% — or a whopping 1 in 5 — searches now happen outside of Google’s ecosystem. This number is even more impressive when you factor in that<a href="https://ppc.land/chatgpt-holds-two-thirds-market-share-as-gemini-and-grok-gain-ground"> Gemini is ChatGPT’s closest competitor</a>.</p><p>As early as April 2025, I saw purported GEO (or AIO, AEO, LLMO) experts emphasizing the end of SEO. On the other side of the spectrum, SEOs have been arguing that GEO=SEO. While both claims are incorrect, the latter has<a href="https://www.gsqi.com/marketing-blog/straight-from-the-ai-source-is-aeo-geo-different-than-seo/?utm=GanderAIAnalytics"> some traction</a>. There<a href="https://www.seroundtable.com/structured-data-schema-ai-search-visibility-40099.html"> was</a>, and is, a lot of misinformation going around, which is to be expected at the onset of an evolving discipline. The organic search community is rattled, and understandably so. Change can be daunting and most service-based professionals feel that there is an AI-shaped target on their backs.</p><p>The pervasive thought amongst SEOs practitioners was that there was nothing to see here. GEO is merely a shiny new acronym, an unnecessary polish on an existing practice only to be used by grifters and wannabe pundits.</p><p>Are there grifters? Yes. Absolutely. It’s part of the reason we started Gander.</p><p>Rand’s tone and position capitalize on the community’s existing sentiment of skepticism, anxiety, and FOMO. If the Wizard of Moz says them fancy GEO tools is a big ol’ boondoggle, then its gots to be so? Right?</p><p>Let’s pull back the curtain a little.</p><h3>Conflict of Interest</h3><p>At the beginning of the article, Rand calls out a <em>potential</em> conflict of interest:</p><blockquote>“Conflict of interest? Well, maybe, but given my starting point as a<a href="https://en.wikipedia.org/wiki/Procatalepsis"> devout skeptic</a>, already convinced that AI tracking was a boondoggle, Patrick’s calm demeanor, access to millions of AI response data points, and willingness to leverage his math major background and programming skills made for the perfect research partner.”</blockquote><p>It should go without saying that any influencer pieces should have any and all financial relationships, including stake or investment, <em>prominently </em>disclosed. Given Rand’s industry cache, this is an important note. Irrespective of financial ties, Patrick isn’t a disinterested third party, just “helping with research”. He has direct financial incentive and stands to gain from the positioning highlighted in the article — namely that existing tools are inaccurate and expensive.</p><p>Rand claims he “hadn’t seen Gumshoe’s product in action before completing 95% of this research work.” Even if true, this is irrelevant. Patrick, who did see the product, was the research partner handling data, programming, and methodology.</p><h3>Methodology</h3><p>Rand pre-empts criticism of his methodology by stating he’s not a professional researcher and inviting others to repeat the experiments. Fair enough. This isn’t a peer-reviewed paper and he’s not pretending it is. He is the head of a data company though, has published dozens of studies, and is extremely adept at research.</p><p>He does borrow specific techniques:<a href="https://medium.com/@lewis.fantom9291/beginners-guide-to-pairwise-correlation-pearson-correlation-coefficient-8a7a832c2bfd"> pairwise correlation</a> and<a href="https://en.wikipedia.org/wiki/Semantic_similarity"> semantic similarity</a> scoring, from a<a href="https://arxiv.org/html/2505.23799v1#:~:text=Large%20language%20models%20(LLMs)%20are,input%20be%20more%20broadly%20used"> Carnegie Mellon paper (Wu et al., EMNLP 2025)</a> that he describes as “a model for this work.” That paper had 2,976 human raters evaluating 14,880 response pairs with formal inter-rater reliability testing, cross-validation, and ethics board approval. Borrowing the methodology implies a degree of rigour. The reader should know the source material was built to a different standard.</p><p>Worth noting: the CMU paper’s central finding was that automated consistency metrics don’t align well with human perceptions of consistency and that this gap is <em>worse on real-world prompts than on synthetic ones</em>. That’s directly relevant to every tool in the space, including Patrick’s, since synthetic prompts are the backbone of scaled AI visibility tracking.</p><p>Also worth noting: Rand acknowledges brand sentiment briefly but dismisses it. From a GEO perspective, this is a serious omission. A brand appearing in 80% of responses means very little if an LLM consistently frames it with qualifiers like “though some users report quality issues” or “considered a budget option”. Visibility and quality/accuracy are distinct metrics and the study measures only one.</p><p>Now, here’s where I’ll probably surprise you — the methodology is largely fine for what it is.</p><p>600 volunteers with different devices, settings, chat histories, temperature configurations, and locations. Messy, chaotic, completely uncontrolled. And honestly? That’s what the real world looks like. People don’t standardize their ChatGPT settings before searching for headphones. Rand’s methodology captures a snapshot of the ecosystem. It’s a legitimate choice for a preliminary study.</p><p>Where the methodology does fall apart is in the inclusion of Google’s AI Overviews alongside ChatGPT and Claude. AI Overviews are not conversational AI. They’re a search feature embedded in Google’s existing ecosystem, shaped by traditional ranking signals and filtered through Google’s user personalization.<a href="https://developers.google.com/search/docs/appearance/ai-features"> They don’t always appear</a>. The same query can trigger an AIO one time and not the next, based on factors that have nothing to do with generative AI consistency. That’s Google’s triggering logic, not LLM behaviour. ChatGPT and Claude always return a response when you ask them a question.</p><p>By lumping AIO results alongside ChatGPT and Claude, the study conflates variance caused by generative engine inconsistency (which is what Rand claims to be measuring) with variance caused by Google’s personalization and display logic, which is a different problem entirely.</p><p>Most practitioners working in this space already draw a line between conversational AI optimization and AI Overview optimization. They’re related, and the skills overlap, but the measurement challenges are different enough that combining them in a single study muddies the findings. The research would have been stronger, and the conclusions more credible, if AIO had been treated separately.</p><h3>Core Arguments</h3><h3>Argument 1: AI visibility tools are inaccurate.</h3><p>By and large, I agree with this argument. My experience conducting competitive intelligence for Gander tells me the same. If you’ve heard me speak or been part of my <a href="https://gander.li/6x0ZXlu">ongoing industry research</a>, you’ll know I use this example when describing accuracy:</p><p>“If five of us ask ChatGPT the same five questions, five times, we’ll get 25 different answers.”</p><p>Variability is an understandable part of an optimization practice. Gumshoe’s own blog states:</p><p>“We found that individual AI responses are highly variable (you’ll rarely see the same answers twice), but the <em>frequency</em> with which a brand appears across many queries converges to a stable signal.”</p><p>The purpose of the article is to showcase that variability is so high that it doesn’t make sense to spend money on a visibility tool. Yet, their own research actually contradicts that statement. Rand opens stating that an estimated $100M+ per year is being spent on AI tracking, with zero published research validating the metrics, and a comparison to “sketchy SEO salespeople” who could weaponize the inconsistency.</p><p>His framing is clear: this is an industry selling snake oil to gullible executives.</p><p>Before presenting a single number, Rand primes the reader to view every tool, every metric, and every provider through the lens of bad faith. This is a textbook example of <a href="https://en.wikipedia.org/wiki/Poisoning_the_well">poisoning the well</a>. By the time the<a href="https://images.sparktoro.com/blog/wp-content/uploads/2026/01/Slide6.png"> beautiful graphs</a> appear, the audience has already been told how to feel about them.</p><p>Effective rhetoric? Absolutely. A valid, empirical argument? No.</p><p>Rand also states: “I could find absolutely no research showing whether AI tools are consistent enough, when prompted for lists of brand/product recommendations, to produce valid visibility metrics.” That may well be true. But the absence of published research proving the metrics work is not the same as evidence that they don’t. There’s no published research disproving them either. <em>The sword cuts both ways.</em></p><p>This is barely a two-year-old industry. SEO didn’t have peer-reviewed validation of domain authority, page rank, or any of its core metrics within its first two years. Even foundational technical SEO elements, like structured data, were widely contested. The gap Rand identifies is real, but it’s an indictment of the research community’s pace, not proof that all practitioners are operating in bad faith.</p><p>His research data tells a different story.</p><p>When prompting for recommendations about digital marketing consultants with expertise in e-commerce: Smartsites appears in 85 of 95 responses. Prompting for West Coast Cancer care: City of Hope appears in 69 of 71 responses. Noise cancelling headphones? Bose, Sony, Sennheiser, and Apple appear 55–77% across 994 responses from 142 wildly different human-crafted prompts. His conclusion, in his own words: visibility percentage is “(probably) a reasonable metric.”</p><p>And if there were any doubt about how the research was received by his own collaborators, Gumshoe published a blog post two weeks after the article dropped, titled “How Much Data Do You Need to Measure AI Visibility with Confidence?” It opens with “Recent research from SparkToro and Gumshoe” and proceeds to build the statistical foundation for Gumshoe’s product configuration directly on top of these findings.</p><p>The research set out to prove AI visibility tracking is a boondoggle. It ended up becoming a product validation exercise for the co-researcher’s company, and each of its “expensive” competitors.</p><h3>Argument 2: Individual AI responses are effectively random</h3><p>Rand states that there is a less than a 1% chance that ChatGPT or Google AI will give you the same list of brands in any two responses out of 100 runs. Less than 1 in 1,000 for the same list in the same order. Every response is unique in composition, sequence, and length. His framing: “These tools are probability engines: they’re designed to generate unique answers every time. Thinking of them as sources of truth or consistency is provably nonsensical.”</p><p>This is correct. And it’s not new information. Anyone working seriously in this space already knows that individual AI responses are non-deterministic. LLMs operate on token probability distributions. Temperature, sampling methods, and context windows are designed to produce varied outputs. Pointing out that a probability engine produces probabilistic outputs is a description of the architecture.</p><p>The implicit comparison here is to traditional search, and it’s doing some heavy lifting. Rand’s audience is SEOs, and the unspoken frame is: imagine if Google gave you completely different results every time you searched. But Google does vary results by personalization, location, device, and search history. The SEO industry didn’t collapse when we discovered that two people Googling the same query see different results. It adapted its metrics. This is the same challenge in a different system.</p><p>In the last section, we discussed that there was enormous overlap between these supposedly unique lists. So how can there be less than a 1% chance of getting the same list twice? My hypothesis is that Rand is measuring exact composition matches. If one response returns five brands and the next returns four of the same five, that’s a “different list.” Add one brand, drop another, change the count, it’s now a different list. The 1 in 1,000 stat he presents for order matching is an even stricter condition applied to an already rare subset. Further, we’re seeing a data set where brand presence is consistent but the order in which they’re mentioned is variable.</p><p>If the number of brands mentioned is what’s driving classification as unique, and Rand confirms that responses ranged from “as few as 2–3 recommendations, and equally often 10 or more”, then the “chaos” isn’t in which brands the LLM recommends. It’s in how many it decides to mention in any given response. That’s a very different claim than “AI recommendations are random”. Individual responses recommending varying numbers of brands doesn’t invalidate aggregate measurement. Nobody calls polling a boondoggle because each individual response is unpredictable. The signal is in the aggregate. Rand’s own frequency data demonstrates this.</p><p>I’d love to verify their findings. The raw data is technically public. Patrick built a mini-site where you can browse responses. But the data isn’t in a format that allows independent reproduction of the 1 in 100 or 1 in 1,000 calculations. The article doesn’t define what constitutes “the same list.” And on their face, these numbers are difficult to reconcile with the frequency data presented in the same article. I’m sure other nerds like me would appreciate seeing your raw data.</p><p>EDIT: Our lead AI Engineer, Mehrad, called me out on this argument. His addition:</p><blockquote><em>Your defense of visibility % is built on the observation that the same core brands reliably surface across varied prompts, which is compelling and well-supported. But that consistency actually makes the ordering problem more pointed, not less. If the consideration set is stable, then the rank position isn’t shifting because of noise or small sample sizes. It’s shifting within an otherwise predictable system, which is a stronger argument </em>against <em>rank tracking than Rand’s original framing.</em></blockquote><p>You’re pushing back on the 1 in 1,000 stat as too strict, but you don’t fully reconcile the fact that consistent brand presence and unreliable rank order coexist in the same data.</p><p>If certain brands appear consistently across hundreds of responses regardless of how the list is ordered, that consistency is telling you something meaningful about how those brands are represented in the model’s training data or retrieved content. They’ve earned a place in an LLM’s consideration set in a way that others haven’t.</p><p>That’s essentially the operational definition of GEO. The goal isn’t to rank 1 in an AI response, the way you’d target position 1 in Google. The goal is to become a brand the model reliably associates with a given intent or topic space, to be in the consideration set at all, and to stay there consistently.</p><p>The brands that appear in 80–90% of responses, despite variable ordering, are the case studies that organic search practitioners should study. They’re not gaming a ranking algorithm. They’re genuinely embedded in the model’s understanding of a category.</p><p>He’s right. I completely missed that in my original draft. Contrary to traditional SERPs, we have zero evidence that the order in which a brand is mentioned in any given response influences click through. None. Gander does track this as part of its core visibility metrics and it’s largely because our hypothesis is that it will matter in the long term.</p><h3>The system isn’t random, it’s measurable</h3><p>Before we get to visibility as a metric, it’s worth pausing on a question Rand never asks: if AI recommendations are essentially random, “<a href="https://c.tenor.com/cxkyTik70aYAAAAd/tenor.gif">spicy autocomplete</a>”, as he puts it, then why do measurable on site quality signals predict citation rates?</p><p><a href="https://arxiv.org/abs/2509.10762">Kumar &amp; Palkhouski</a> (2025) audited 1,702 citations across Brave, Google AIO, and Perplexity using 70 B2B SaaS prompts and 1,100 unique URLs. They found that overall page quality predicted citation with an odds ratio of 4.2x. Pages cited by multiple engines scored 71% higher on quality metrics than single-engine citations. The signals that mattered most, metadata freshness, semantic HTML, structured data, were consistent across engines. While I’d like to see ChatGPT, CoPilot, Gemini, Perplexity, and Claude measured, the study’s findings are interesting.</p><p>The findings from this study, and others like it, matter because it reframes the question. We’re not asking whether AI outputs are variable, they are. We’re asking whether the system underlying those outputs responds to measurable, optimizable signals. The answer, so far, is yes. Which means the question isn’t “should we track visibility?” It’s “how do we track it well?”</p><h3>Argument 3: Visibility % is imprecise and requires impractical sample sizes</h3><p>As I had mentioned earlier on, Rand concedes that visibility percentage is “(probably) a reasonable metric.” He describes it as a “dartboard-pattern-like answer.” When looking at City of Hope’s 97% visibility rate in ChatGPT, he says “I’m not sure it means anything.”</p><p>I think it may. When comparing a localized search, such as “best West Coast cancer hospital”, we need to consider that AI search is a more accurate reflection of reality. As early as 2023, Michael King posited that third-party placement is disproportionately weighted in LLM results. This means that LLMs will mine directories with reviews and social media platforms to better understand people’s experience with an organization. Contrast with a more broader topic like best headphones, it may explain why City of Hope has a strong visibility rate in the study.</p><p>Strip the rhetorical qualifiers and what remains is straightforward: visibility reported as a percentage works. The study’s data demonstrates convergence across wildly uncontrolled conditions. Gumshoe’s blog post two weeks later didn’t treat it as “probably reasonable”. Instead, they built Bernoulli trial math on top of it and used it to justify their product’s default configuration. You don’t run statistical power calculations on a metric you think is meaningless.</p><p>The “dartboard-pattern-like answer” framing sounds damning until you think about what a dartboard pattern actually is. It’s a distribution around a central tendency. That’s how statistical sampling works. Every poll, every clinical trial, every quality control process produces exactly this: variance in individual observations, stability in the aggregate. Calling the output a dartboard pattern describes the metric. That’s what statistics look like.</p><p>Now, the sample size question. Rand argues you need 60–100+ runs per prompt to get meaningful visibility data. I agree — it’s why we designed Gander to run more prompts than any other platform. It’s the best argument for automated tooling. No search professional is manually running prompts 100 times, across the most popular four or five engines, and pasting results into a spreadsheet. If visibility as a metric requires scale to be statistically valid, then the solution is a platform that operates at scale.</p><p>Now, not every platform is created equal. One of the most popular limits web searches to two (default is between four and six, but can be up to ten), saving them money. Running queries is expensive, and VCs expect a return. What this means is that the results are not indicative of what would be shown to the average person, and query sourcing is limited by more than 50%, conservatively. That is legitimately shady because unless you look under the hood, you’d never know that you’re getting markedly different results.</p><p>And let’s be honest about the timeframe here. Rand’s study ran over November and December 2025, which is roughly two months of data collection. Is SEO any different? Has anyone ever looked at two months of organic ranking data and concluded that SEO tracking is a waste of money because positions fluctuate? Of course not. We accept that positions move. We track the trend. We look at visibility over time. This is the same discipline applied to a different system, and the same patience is required.</p><p>Where I’ll give Rand credit: his point about accuracy matters. AI tools recommended defunct businesses, inactive social influencers, and non-existent TikTok accounts during this research. That’s a real problem, particularly for branded search where precision matters. Hallucinations are less of an issue than they were a year ago, but they haven’t been eliminated. Any tool tracking AI visibility needs to account for accuracy in responses, not just frequency of mentions and citations or share of voice. That’s a legitimate problem that platforms, and practitioners, need to account for.</p><h3>Argument 4: Prompt diversity is an unsolved problem that compounds the tracking challenge</h3><p>Rand asked 142 people to write prompts about the same thing and found the average semantic similarity between their prompts was 0.081. On a scale where 1.0 is identical, that means there’s negligible overlap. He calls synthetic prompts as a proxy for real user queries “a huge problem” and “a giant hole in AI tracking.”</p><p><em>This is genuinely useful data</em>. The industry should pay attention to it. People don’t prompt an LLM the way they search Google. They’re conversational, specific, idiosyncratic, tangential. Many AI visibility platforms organize tracking around keyword clusters, an SEO paradigm that Rand’s own data suggests is not a 1:1 comparable. Search professionals cannot rely on 2–5 keywords as a proxy when writing prompts for their clients.</p><p>Despite near-zero semantic similarity in the queries, responses were largely similar when presenting companies, products, and brands. My hypothesis is that intent matters more than the language used in a prompt. Whether a user asks an LLM “What are the best noise cancelling headphones for a flight?” or directs it to “Tell me the best noise cancelling headphones on the market”, I think is less relevant than the purpose of the product — to block out noise while listening to something else. In instances where factors like weight or physical size matter, <a href="https://gander.li/Wsf52MF">users will correct and clarify their follow up prompts</a>. Practitioners should be prepared to track follow-up prompts.</p><p>If that hypothesis holds, it has <a href="https://gander.li/A1kymHj">significant implications for how we think about tracking</a>. It means organizing measurement around topics and intents rather than specific query strings isn’t just practical, it’s how the models themselves operate. And it means the prompt diversity “problem” isn’t a hole in AI tracking. It’s evidence that generative engines are more robust than the surface-level variation suggests.</p><p>The parallel to traditional search is worth noting. Long-tail keyword diversity has always been a challenge in SEO. Millions of unique queries, infinite variation. Nobody concluded that because users search in wildly different ways, tracking organic visibility was pointless. The industry built tools that clustered queries by intent and measured visibility at the topic level. This is the same problem in a new system, and it calls for the same solution.</p><h3>Argument 5: The burden of proof is on the tools, and they aren’t meeting it</h3><p>Rand closes with a call to action: stop paying for AI tracking tools that don’t publish transparent, publicly reviewable research. He lists open questions: minimum sample sizes for statistical significance, API vs. web interface scraping, prompt diversity requirements, and frames them as preconditions the industry hasn’t cleared. His message to buyers: demand proof or walk away.</p><p>He’s right. The industry should be more transparent. Published methodologies, public data, reviewable research. If you’re charging marketers thousands per month to track something, you should be able to show your math. Fighting this point would look defensive, and it would be wrong. The call for transparency is the<a href="https://c.tenor.com/RgVpEEPY5bcAAAAd/tenor.gif"> strongest and most constructive part of the entire article</a>.</p><p>There’s an irony here. Rand just demonstrated that the first person to actually do the research validated the practice. The burden of proof argument works if the evidence, once gathered, shows the tools are broken. His own evidence showed the opposite. He set out to prove AI visibility tracking is an expensive waste of time, gathered data, and concluded that visibility percentage is meaningful, if done well. The burden was met. By him. The answer wasn’t the one his headline promised.</p><p>The unanswered questions he lists are real. But they’re operational, not existential.</p><ul><li>How many runs do you need for statistical significance?</li><li>How closely do API calls mirror web interface results?</li><li>What diversity of prompts is required for a given sector?</li></ul><p>These are calibration questions, not evidence that all tools are fundamentally flawed. Every measurement discipline has open methodological questions. SEO has been debating the validity of domain authority for a decade. That didn’t stop anyone from tracking organic visibility.</p><p>One piece of rhetoric worth noting: “If Patrick and I can do it in our spare time while running multiple companies, you know who else can do it?” This simultaneously positions the research as a casual side project, lowering the bar for methodological scrutiny, and shames venture-backed companies for not publishing their own work. Both framings can’t hold at the same time. Either this kind of research requires rigorous, well-funded effort, or it’s something two busy guys can knock out between D&amp;D sessions.</p><p>The industry deserves the former, and the standard should apply evenly. Hell, I’m pretty busy building a bootstrapped<a href="https://takeagander.ai/"> AI analytics platform</a> and found the time to read his study, corresponding research, and write a rebuttal.</p><h3>Parting Thoughts</h3><p>What’s surprising is the question that Rand doesn’t ask: does increased visibility predict business outcomes?</p><p>His research validates that visibility % is measurable. It shows the metric is statistically consistent across varied conditions. It never asks whether that metric correlates with anything that matters: traffic, brand awareness, pipeline, revenue. Does a 97% visibility rate translate into more patients calling City of Hope? Does Smartsites appearing in 85/95 of responses drive inbound leads? We don’t know, because the research stops at “the number is real” and never gets to “the number is useful.”</p><p><em>That’s the actual open question for this industry, and it’s</em><a href="https://media3.giphy.com/media/v1.Y2lkPTc5MGI3NjExZXN0bmo3YmVneTFoMnJicjJ0eXJleW80azQ5djVmZzJrbXA1NTN2bSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/15BuyagtKucHm/giphy.gif"><em> bigger than anything in this article</em></a><em>.</em></p><p>Did you appreciate the depth and rigour in this research breakdown? We built Gander with the same discipline. Daily prompt runs across ChatGPT, Gemini, Perplexity, and AI Overviews — plus a suite of technical tools that show you exactly if, and how often your clients are cited across the most popular LLMs. We help you understand why they appear, and what to do about it if they don’t.</p><p>We are built for agencies who <em>stake their reputation on data</em>. Don’t guess. Take a Gander.</p><p><a href="https://gander.li/MHjWBxP">BOOK A DEMO</a> or <br><a href="https://gander.li/uMiIhug">TRY RISK FREE FOR 14 DAYS</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c6cfd0dc468c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Students (Really) Use ChatGPT]]></title>
            <link>https://medium.com/@AdamMalamis/how-students-really-use-chatgpt-7b68a4b3a1fc?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b68a4b3a1fc</guid>
            <category><![CDATA[generative-engine]]></category>
            <category><![CDATA[ai-search]]></category>
            <category><![CDATA[geo]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Tue, 30 Sep 2025 12:27:52 GMT</pubDate>
            <atom:updated>2025-09-30T12:27:52.255Z</atom:updated>
            <content:encoded><![CDATA[<h3>Summary</h3><p>Below we provide a lay summary of Ammari et al’s (<a href="https://sites.comminfo.rutgers.edu/tammari/">Rutgers</a>) <a href="https://arxiv.org/html/2505.24126v1">longitudinal research</a> on how students interact with ChatGPT. The study was conducted from December 2022 to January 2024 with results published in May 2025.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3gM9Lbn0XX_xTLgU" /></figure><p>A recent study analyzing how undergraduate students use ChatGPT provides a detailed picture of user behavior that moves beyond self-reported surveys to examine naturalistic interaction logs. The research, titled “<em>How Students (Really) Use ChatGPT: Uncovering Experiences Among Undergraduate Students</em>“, sought to understand the day-to-day ways students integrate the AI tool into their academic routines by observing their actual, self-directed usage patterns.</p><p>The findings reveal a complex relationship where students treat the AI as both a tool and a learning partner, actively working to manage its limitations. The analysis identified five dominant categories of use that frame the student-AI interaction.</p><h3>What are the Five Dominant Categories of use?</h3><p>The study systematically categorized thousands of interactions, identifying five primary ways students engage with the platform:</p><ul><li><strong>Information Seeking:</strong> Students used the tool to ask questions on a wide range of subjects, including academic, medical, and cultural topics. This behavior extends beyond simple fact-checking into more complex information discovery.</li><li><strong>Content Generation:</strong> A significant use case was the generation of original content. This included drafting academic and professional documents, using the AI as a co-creator in the writing process.</li><li><strong>Language Refinement:</strong> Users frequently employed ChatGPT as an advanced editing tool to refine their own writing, improving clarity, style, and grammar.</li><li><strong>Meta-Cognitive Engagement</strong>: The research observed students using the AI to reflect on their own learning processes. This “meta-cognitive” interaction involved using the chatbot to co-regulate their progress and manage their learning journey. &lt;define meta cognitive&gt;</li><li><strong>Conversational Repair:</strong> Users seem to persevere when an answer engine produces an error or misunderstands a prompt. Instead, they actively engage in “conversational repair,” rephrasing questions, providing clarification and context, and adapting their prompts to guide the AI toward the desired outcome.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*O7YVs5STuoazQ7Nu" /></figure><h3>Are there predictors of long-term engagement?</h3><p>The study also conducted behavioral modeling to determine which usage patterns predicted sustained, long-term engagement with the platform. The analysis found that users were most likely to continue using ChatGPT when they were engaged in structured, goal-driven tasks. Specific examples of these “sticky” tasks include writing code, solving multiple-choice questions, and preparing job applications. This suggests that the tool’s utility in accomplishing specific, tangible goals is a stronger driver of retention than casual, exploratory use.</p><p><em>Note: The authors grounded their research with two separate frameworks: </em><a href="https://uwaterloo.ca/centre-for-teaching-excellence/catalogs/tip-sheets/self-directed-learning-four-step-process"><em>Self-Directed Learning</em></a><em> and </em><a href="https://en.wikipedia.org/wiki/Uses_and_gratifications_theory"><em>Uses and Gratifications Theory</em></a><em>. The latter was particularly interesting to me because the initial theory was established nearly 85 years ago with radio listeners! It was then broken down into five main assumptions in the 70’s that I believe are still relevant over five decades later.</em></p><h3>The AI as a “Socio-Technical Actor”</h3><p>A central conclusion of the research is that students interact with ChatGPT as more than just a passive cognitive tool. The paper frames ChatGPT as a “socio-technical actor that co-constructs the learning experience with the student”. The observed behaviors, particularly the persistence shown in conversational repair and the use of meta-cognitive reflection, indicate that students are forming a dynamic, collaborative relationship with the technology. They actively manage its affordances and limitations, adapting their own strategies to work with the system effectively.</p><p>The study reveals that users are not passive recipients of information but are instead active, adaptive partners in a dialogue, engaging in a range of sophisticated behaviors to achieve their goals.</p><h3>What are the implications for AI search?</h3><p>The behaviors identified in the study signal a significant departure from the interaction model that has defined the digital age: the search engine. This shift has profound implications for how users develop loyalty and trust with online platforms, explaining the unprecedented speed of generative AI adoption and a disproportionate conversion rate for purchasing products and services (4.4x, relative to traditional search).</p><p>Traditional search is an inherently one-sided, “flat” interaction. The user submits a query, and the platform returns a ranked list of potential answers. The dialogue begins and ends with this single transaction. Loyalty in this paradigm is built on efficiency and the perceived authority of the algorithm. Some would argue it was simply built from a lack of options. Trust is placed in Google’s ability to sort the web’s information correctly and deliver the most relevant link. The relationship is with a tool, not a partner.</p><p>Ammari’s findings illustrate a starkly different, more collaborative model. The prevalence of “conversational repair” is the most telling indicator of this change. Unlike a failed search query, which often leads to abandonment or a completely new query, users of generative AI invest their own effort to correct the system’s course. They rephrase, clarify, and guide the AI, treating the interaction as a dialogue to be improved rather than a transaction that has failed. This willingness to co-create the outcome signifies a deeper level of engagement and a new form of trust — not in the AI’s infallibility, but in its capacity to be guided toward a “correct” answer.</p><p>This collaborative dynamic is the bedrock of a new, more resilient form of platform loyalty. The research shows that long-term engagement is anchored by “goal-driven tasks” like writing code or preparing a job application. When a platform becomes integral to achieving a tangible outcome, it moves from being a utility to being a partner. This task-oriented loyalty is far stickier than the transactional loyalty of traditional search. The platform is no longer just a source of information; it is a collaborator in the user’s personal and professional workflows.</p><p>This sense of partnership or collaboration could be an explanation for the immense and rapid adoption of generative AI. The relationship between user and platform is becoming… well, relational. Trust is built on the platform’s ability to provide incrementally better responses based on the user-provided context.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b68a4b3a1fc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is GEO? An In-Depth Look at Generative Engine Optimization]]></title>
            <link>https://medium.com/@AdamMalamis/what-is-geo-an-in-depth-look-at-generative-engine-optimization-3b9cf55fc55b?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/3b9cf55fc55b</guid>
            <category><![CDATA[seo]]></category>
            <category><![CDATA[ai-search]]></category>
            <category><![CDATA[digital-marketing]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Sat, 27 Sep 2025 22:22:12 GMT</pubDate>
            <atom:updated>2025-09-27T22:22:12.840Z</atom:updated>
            <content:encoded><![CDATA[<p>The way people search has changed.</p><p>Not long ago, almost every online journey began with Google. Today, millions of users use tools like ChatGPT, Gemini, or Perplexity and get full answers on the spot. Gartner warns this change could see organic traffic <a href="https://www.gartner.com/en/webinar/630172/1392785?">fall by 50% or more by 2028</a> — signalling that traditional SEO is no longer the reliable path to visibility it once was.</p><p>Even a well-ranked page might never appear in an AI-generated response. <a href="https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/the-economic-potential-of-generative-ai-the-next-productivity-frontier">Generative Engine Optimization (GEO)</a> is emerging as the answer. Instead of chasing rankings alone, GEO focuses on shaping your content so it can be understood, trusted, and cited by AI systems.</p><p>The goal isn’t just to attract a click; it’s to become part of the actual response your potential customers read. That makes GEO both a challenge and an opportunity. For companies willing to adapt, it opens a new path to visibility at the exact moment decisions are made.</p><p>For those who ignore it, the risk is invisibility in the very conversations that are now shaping purchasing behavior.</p><h3>What is Generative Engine Optimization?</h3><p>Generative Engine Optimization, or GEO, is the process of preparing your content so it can appear in AI-driven search results. GEO goes beyond Google’s algorithms. It looks at how tools like ChatGPT and Perplexity pull in and present information.</p><p>Generative models prefer short, direct explanations they can drop straight into an answer. Break content into clear sections, frame it around real questions, and back it with reliable sources. That’s what makes it more likely to appear.</p><p>Think of it as the next step in SEO. Where SEO earned you a place on the search results page, GEO earns you a place inside the answer itself. For businesses, that’s a defining shift, because if you’re not part of the AI’s response, you may never be seen at all when people look for what you offer.</p><h3>How is GEO different from SEO?</h3><p>It’s easy to confuse GEO with SEO, but they target different goals. SEO is designed to win visibility on search engines like Google. GEO is designed to do the same, but this time, inside AI-generated answers.</p><p>Traditional SEO relies on signals such as backlinks, keyword density, and metadata. These elements help a page climb rankings, but they don’t guarantee a mention when someone asks a generative engine a question.</p><p>GEO works differently. Rather than rewarding keyword stuffing, GEO favors content that reads naturally — clear, credible, and easy to follow.</p><p><strong>Key differences include:</strong></p><ul><li><strong>Focus:</strong> SEO helps a page climb the rankings, but GEO is about having your knowledge show up directly inside the answer. In logistics, it’s the difference between showing up on a search results page about freight rates and having your shipping data quoted directly when someone asks an AI why costs are rising.</li><li><strong>Format:</strong> SEO rewards long guides, but GEO leans on bite-sized clarity. A hospital blog might publish a deep dive into diabetes care for SEO — but GEO favors short Q&amp;As that an AI can lift and use to answer a worried patient’s late-night query.</li><li><strong>Signals:</strong> SEO still leans on backlinks and site health. GEO pays closer attention to how clearly you explain things. A conservation charity doesn’t need a dozen links to prove value — it needs a well-structured piece that spells out, in plain language, how reforestation changes local ecosystems.</li><li><strong>Outcome:</strong> SEO may bring a click to your finance page. GEO can put your firm’s take on retirement saving right inside the AI’s advice, meaning the customer sees you before they even think about visiting a site.</li></ul><p>In short, what we’re saying is that SEO positions you near the search, while GEO positions you inside the answer.</p><h3>Why does GEO matter in 2025?</h3><p>Search habits are shifting fast. Younger audiences already turn to <a href="https://searchengineland.com/google-leads-gen-z-ai-search-behavior-survey-459809">TikTok, Reddit, and AI chat tools before Google</a>. As generative search becomes built into browsers and devices, being “AI-visible” won’t be optional. It will decide whether a business is part of the conversation or left out completely.</p><p>The stakes are high. GEO directly affects brand awareness, lead generation, and authority. If an AI highlights your competitor’s product and skips yours, the choice may already be made before a customer reaches your site.</p><p><strong>Key reasons GEO matters now:</strong></p><ul><li><strong>Reduced friction:</strong> Generative tools collapse the research journey into a single answer.</li><li><strong>First-mover advantage</strong>: Very few companies are optimizing, so the first to act will stand out. Evidence shows that efforts compound over time.</li><li><strong>Authority</strong>: Being cited in an AI response builds <a href="https://arxiv.org/abs/2501.01303?">immediate trust</a>, as research shows users rate AI content as more credible when citations are present, even without checking them.</li></ul><p>At its core, GEO is about future-proofing. Companies that adapt early shape how AI engines present information in their industry and benefit from returns that build over time. Those who hold back risk watching competitors set the story.</p><h3>What are the key elements of GEO?</h3><p>In Generative Engine Optimization, there are repeatable practices that give your content a much stronger chance of being pulled into AI-driven answers. These methods don’t replace good writing; rather, they sharpen it so that both people and machines can trust and reuse your expertise.</p><p>Here are the key elements to look for:</p><h3>Structured content</h3><p>AI engines favor clarity. That means organizing information under clear headers, using short paragraphs, and focusing on one idea at a time.</p><p>Instead of burying key points in long blocks, GEO requires content that feels easy to skim and simple to extract. Think of each section as a self-contained block that could be lifted out and still make sense.</p><h3>Schema markup</h3><p><a href="https://schema.org/">Schema markup</a> tells machines exactly what your content represents. Adding structured data for FAQs, products, or reviews makes it easier for AI to categorize and reuse your work confidently.</p><p>Without schema, engines may misinterpret context or miss it altogether. With it, you’re signaling: here’s what this is, here’s who it’s for, and here’s why it matters.</p><h3>E-E-A-T and authority</h3><p>Google’s framework of Experience, Expertise, Authoritativeness, and Trustworthiness (<a href="https://developers.google.com/search/blog/2022/12/e-e-a-t">E-E-A-T</a>) translates directly into GEO. Generative engines seek voices they can trust. That means using bylines, showing credentials, citing sources, and keeping claims specific.</p><p>As such, content that demonstrates lived experience or expert knowledge is far more likely to be surfaced than generic marketing copy. That authority is your entry ticket.</p><h3>Conversational style</h3><p>Engines are trained on conversational data, so they respond well to writing that mirrors natural speech. Overly stiff or technical wording risks being overlooked. GEO content should feel professional but accessible, i.e. the tone you’d use explaining a subject to an informed colleague.</p><p>In plain English, varied sentence length, and natural rhythm help ensure your answers can be quoted directly.</p><h3>Question-led content</h3><p>One of the easiest GEO wins is building content around the questions people actually ask. Headings framed as questions (“Why does GEO matter?”) mirror the way users type or speak into AI.</p><p>Each answer then becomes a ready-made snippet that can be pulled into responses. By structuring content around queries, you reduce friction for both readers and machines.</p><h3>Technical foundations</h3><p>Even the best writers struggle without the right technical foundations beneath them. Engines still favor pages that load quickly, are WCAG accessible, work on mobile, and use secure connections.</p><p>Internal linking also helps build a knowledge network around your brand, while removing intrusive pop-ups ensures AI can crawl your content smoothly. Strong technical health may not get you visibility on its own, but without it, your GEO efforts can stall.</p><h3>What challenges does GEO bring?</h3><p>GEO offers real advantages, but it isn’t without obstacles. The first is uncertainty. Because generative search is new, there are no definitive rulebooks or long-established best practices. Companies are experimenting, and results can vary widely depending on the engine and the niche.</p><p>The second challenge is resource demand. GEO requires content that’s well-written, structured, and technically supported. That means investment in both strategy and production. Smaller businesses may find it difficult to compete with brands able to generate high volumes of optimized material.</p><p><strong>Other hurdles include:</strong></p><ul><li>Lack of clear reporting or analytics for AI visibility</li><li>Risk of misinformation when engines paraphrase poorly</li><li>Constantly shifting technical standards as models evolve</li></ul><p>GEO is promising, but it demands adaptability. Those who treat it as a one-off campaign will struggle, and traditional SEOs are often poorly equipped for the shift. Marketers who approach it as ongoing work will be better positioned to capture its full benefits.</p><h3>How can companies start implementing GEO?</h3><p>The good news is you don’t need to overhaul your entire content strategy to start. But the sooner you adapt SEO best practices for GEO, the sooner you gain an edge while others lag behind.</p><p>Start by reviewing your existing content. Does it answer real questions in clear, concise blocks? Are headings written in a way that mirrors how people speak to AI?</p><p>Also, where gaps exist in your library of content, perhaps try creating new pieces or repurposing old non-GEO-focused content so that it focuses on direct, practical explanations.</p><p><strong>Practical starting points:</strong></p><ul><li>Rework top-performing SEO pages into question-led formats</li><li>Add schema markup for FAQs, products, or reviews</li><li>Audit author bios and citations to build visible authority</li><li>Remove technical barriers such as pop-ups or slow load times</li><li>Link related articles together to create a clear knowledge map</li><li>Ensure your website is accessible</li></ul><p>From there, track how your content is referenced by AI tools and adjust over time. Remember, GEO isn’t a quick win, as it’s a discipline that grows stronger with steady, consistent effort.</p><h3>Getting On Board With GEO</h3><p>Generative Engine Optimization will soon become a requirement for companies wanting visibility. As search habits shift from links to answers, businesses that adapt will be the ones customers actually see.</p><p>GEO gives you the chance to appear inside those answers, not just alongside them. That means the path forward is no longer about ‘tricking’ an algorithm into ranking your content — as SEO practitioners have done for years.</p><p>It’s now about producing credible, structured, and human content that machines can easily understand and reuse. Companies that commit early will shape how their industry is represented in AI-driven search.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3b9cf55fc55b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How do I get cited in ChatGPT? A Writer’s Guide to the GEO-16 Framework]]></title>
            <link>https://medium.com/@AdamMalamis/how-do-i-get-cited-in-chatgpt-a-writers-guide-to-the-geo-16-framework-5660a102294a?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/5660a102294a</guid>
            <category><![CDATA[generative-engine-seo]]></category>
            <category><![CDATA[content-marketing]]></category>
            <category><![CDATA[ai-search]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Sat, 27 Sep 2025 22:01:18 GMT</pubDate>
            <atom:updated>2025-09-27T22:15:10.625Z</atom:updated>
            <content:encoded><![CDATA[<h3>Summary</h3><p>How might we structure our writing so that it’s cited by AI answer engines? We explore Kumar and Palkhouski’s <a href="https://arxiv.org/html/2509.10762">GEO-16 framework</a>, a data-driven model suggesting a correlation between 16 on-page signals and inclusion in answer engine responses. We’ll discuss how adapting our approach to elements like <a href="https://www.ibm.com/think/topics/structured-vs-unstructured-data">structured data</a> and <a href="https://www.a11yproject.com/posts/what-is-semantic-html/">semantic HTML</a> can potentially improve content discoverability and increase the likelihood of becoming a trusted source for generative engines.</p><figure><img alt="Author writing on an antique typewriter at an old fashioned desk." src="https://cdn-images-1.medium.com/max/1024/1*sIF1JruS4zwcQJQMZngzhg.png" /></figure><h3>What Page Signals Improve AI Citation Likelihood?</h3><p>Here are practical tips from research to help answer engines reference your articles:</p><ul><li>Give prominence to a well-organized structure: Arrange your content logically by incorporating distinct headings and subheadings to streamline information flow.</li><li>Concentrate on reader-centric content: Strive for clarity and brevity in your writing, catering to the audience’s needs. Research indicates that articles with high readability and a straightforward layout are more easily navigated and utilized by AI systems.</li><li>Utilize structured data: Employ structured data formats like Article, TechArticle, or FAQPage schema to annotate your articles. This facilitates AI comprehension of your content’s context and substance.</li><li>Maintain content freshness: Regularly update your articles and visibly display the date of the last update to indicate relevance to AI algorithms.</li><li>Substantiate arguments with evidence and citations: Bolster your assertions with proof and proper referencing to establish credibility and expertise, crucial factors for AI evaluations.</li><li>Employ semantic HTML: Ensure your articles are composed in neat, semantic HTML code by utilizing suitable tags for different types of content, enhancing machine interpretability.</li><li>Optimize for snippet extraction: Craft content that can be readily extracted as a concise snippet by using clear, succinct language and furnishing direct responses to common queries.</li></ul><h3>Prioritize a Well-Organized Structure</h3><p>Arrange your content logically by incorporating distinct headings and subheadings to segment your content. Think of your headings (H1, H2, H3) as a table of contents for users (and AI). A clear, logical hierarchy allows both people and machines to instantly grasp the main topics of your article and understand the relationship between different concepts.</p><h3>Concentrate on Reader-Centric Content</h3><p>Strive for clarity and brevity in your writing, catering to your audience’s needs. Research indicates that articles with high readability and a straightforward layout are more easily navigated and integrated by answer engines. This aligns with the “people-first” principle; when content is genuinely useful and easy for a person to understand, it’s also well-structured for an AI to interpret.</p><p>Depending on the audience, we tend to err on the side of more simple, writing at a Grade 8 level (based on <a href="https://readable.com/readability/flesch-reading-ease-flesch-kincaid-grade-level/">Flesch-Kincaid grade score</a>).</p><h3>Implement Structured Data</h3><p>Employ <a href="https://schema.org/">structured data formats</a> like Article, TechArticle, or FAQPage schema to annotate your articles. This facilitates AI comprehension of your content’s context and substance. Think of structured data as a “translator for machines.” By adding structured metadata, you are explicitly telling answer engines details like the publication date, author, or the questions and answers on the page, removing any ambiguity.</p><h3>Maintain Content Freshness</h3><p>Regularly update your articles and visibly display the date of the last update to indicate relevance to answer engines. In a rapidly changing field, content freshness is a significant trust signal. It shows that the information is not stale and that the author is actively maintaining its accuracy, making it a more reliable source for citation.</p><h3>Link Appropriately</h3><p>Bolster your articles with the appropriate references (i.e. links) to establish credibility and expertise. Linking out to authoritative studies, established sources, or primary research helps answer engines better understand your content in context. It demonstrates that your claims are well-researched and supported by a wider community.</p><h3>Employ Semantic HTML</h3><p>Ensure your articles are composed in neat, semantic HTML code by utilizing suitable tags for different types of content, enhancing machine interpretability. Instead of using a generic &lt;div&gt; tag, for example, using &lt;article&gt;, &lt;blockquote&gt;, or &lt;aside&gt; gives the AI precise information about what that block of content represents.</p><h3>Optimize for Snippet Extraction</h3><p>Craft content that can be readily extracted as a concise snippet by using clear, succinct language and furnishing direct answers to common queries. This often involves an “answer-first” writing style, where a key question is answered directly in the first paragraph. By providing a clear, self-contained answer upfront, you make it incredibly easy for an answer engine to select your content as the most useful response.</p><p>The GEO-16 research study gives us an empirical framework for thinking about online writing and <a href="https://gandergeo.com/resources/gander-blog/what-is-geo-an-in-depth-look-at-generative-engine-optimization/">generative engine optimization</a> (GEO). Our focus should be shifting toward creating content that is clear, organized, and trustworthy. The good news is that the very things that help an answer engine understand your article, like a logical structure and proper referencing, are the same things that help a human reader. By concentrating on these building blocks of quality, we give our work the best possible chance to be seen as a helpful source worth citing.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5660a102294a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using AI Tools to Augment User Experience Research Workflows]]></title>
            <link>https://medium.com/researchops-community/using-ai-tools-to-augment-user-experience-research-workflows-b10e9bddec9a?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/b10e9bddec9a</guid>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Mon, 21 Apr 2025 23:35:40 GMT</pubDate>
            <atom:updated>2025-08-09T05:32:01.757Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="A woman in front of a white board with multi-coloured sticky notes, leading a workshop." src="https://cdn-images-1.medium.com/max/1024/1*Xd8TK1ORlFGPTvK_NuweRQ.png" /><figcaption>Image generated by midjourney.</figcaption></figure><p>The integration of AI into UX research is no longer a futuristic concept — it’s happening now, and it’s reshaping how teams gather, analyze, and act on user insights. While AI shouldn’t replace human intuition or the nuanced understanding that experienced UX professionals bring, it can dramatically enhance efficiency, reduce manual effort, and uncover patterns that might otherwise go unnoticed.</p><p>This article explores six key ways AI can improve your UX workflows, from research planning to data analysis and sentiment tracking. Each section provides examples of how AI can optimize research efforts while also outlining potential pitfalls and the importance of human oversight.</p><h3>1. AI as a Research Planning Assistant</h3><p>User research requires meticulous planning, from defining objectives to selecting the right participants. Traditionally, UX professionals develop research plans manually, drafting detailed study outlines, interview questions, and recruitment strategies. This process demands significant time and effort, particularly for large-scale studies. AI can streamline research planning by analyzing past projects, generating structured research plans, and suggesting key questions based on existing data. With AI-powered tools, we can create research outlines that ensure consistency and thoroughness across studies. Additionally, AI can assist in prioritizing research objectives by analyzing existing user behavior data (e.g. purchasing trends, visitor behaviour) to determine which areas require the most focus.</p><p>While AI-generated research plans can be a helpful starting point, they often lack the nuanced judgment that experienced researchers bring. AI suggestions might miss industry-specific context or emerging user concerns that a human researcher would naturally account for.</p><p><strong>Example Use-Case: Planning a Usability Study</strong></p><p>A UX team needs to conduct a usability study for a new dashboard feature but struggles with structuring an efficient research plan. Traditionally, they manually draft objectives, questions, and participant criteria. Here, you can use AI to analyze past usability studies and generate a preliminary research plan, suggesting relevant interview questions and target demographics. While this saves time, the team must review the plan to ensure the activities make sense given the budget, timeline, and desired outcomes. For example, your research plan probably doesn’t need to include card sorting if you’re A/B testing a new call to action.</p><blockquote>However, you should always review questions to identify and document bias, ensure overall relevance, and avoid leading participants.</blockquote><h3>2. Efficient Data Analysis</h3><p>Speaking from experience, sometimes the sheer volume of data can be overwhelming, especially if you have a tight turnaround. Prior to incorporating AI in our workflow, we would manually sift through surveys, usability test results, and interview transcripts to identify common themes. This process is slow and prone to bias, as researchers may inadvertently focus on expected trends rather than uncovering new insights.</p><blockquote>AI can dramatically enhance data analysis by automatically detecting patterns, clustering responses, and ranking insights based on their relevance.</blockquote><p>For example, AI can efficiently scan an Excel file containing survey responses to identify frequently mentioned pain points, making it easier for UX teams to prioritize issues. It can also cross-reference findings from multiple research sources, providing a more holistic view of user behavior. Moreover, AI reduces human error by surfacing unexpected patterns that might be overlooked in manual analysis.</p><p>The challenge here is twofold: “living in” the data (really going through and understanding transcripts and outputs) is extremely valuable. Time, space, and repetition often provide insights that, as an experienced professional, is valuable to the project. Also, we must validate any automated findings, ensuring that the patterns identified genuinely reflect user needs.</p><p><strong>Example Use-Case: Analyzing Survey Responses</strong></p><p>You’ve been tasked with synthesizing a customer satisfaction survey with thousands of responses. Without AI, we would typically spend hours sifting through the responses in multiple passes, coding responses to find common complaints. AI accelerates this process by clustering similar responses, identifying recurring themes, and ranking issues based on their frequency. It gives us the ability to quickly identify obvious pain points — like if our pricing table is unclear, or if response times have lead to customer attrition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xRqTJISR6SmQ00WT" /><figcaption>Photo by <a href="https://unsplash.com/@uxindo?utm_source=medium&amp;utm_medium=referral">UX Indonesia</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>3. Automated Transcription Services</h3><p>User interviews and usability tests generate vast amounts of qualitative data, often in the form of recorded conversations. Before we’d be forced to manually transcribe sessions or pay a premium for a dedicated transcription service. The task was either time-consuming and tedious or expensive — often both. AI-driven transcription services can convert audio to text within minutes, allowing UX teams to focus on analysis rather than transcription.</p><p>Beyond speed, AI transcription tools also improve organization. Many AI-powered services can differentiate between speakers, generate timestamps, and even highlight key themes within transcripts. This makes it easier to navigate large amounts of qualitative data and retrieve relevant insights quickly. However, AI transcription is not perfect — accents, background noise, and technical jargon can reduce accuracy.</p><h3>4. Sentiment Analysis for User Feedback</h3><p>Gathering user feedback can be extremely valuable for refining digital experiences, but manually analyzing open-ended responses is laborious. AI-powered tools can evaluate hundreds of entries in minutes, classifying them as positive, negative, or neutral.</p><blockquote>AI can detect emotional tones within user reviews, social media comments, and survey responses, helping UX teams identify frustration points and areas of delight.</blockquote><p>Beyond simple sentiment detection, AI can highlight specific emotional triggers, such as repeated complaints about confusing navigation or praise for an existing feature. AI can also rank feedback by urgency (provided you give it direction on how to do so, e.g. by frequency), helping UX teams prioritize the most critical pain points. However, sentiment analysis tools can misinterpret context, especially sarcasm or industry-specific terminology. They also hallucinate and often have trouble evaluating larger datasets. As tools, like ChatGPT, become more powerful, we should be able to upload thousands of entries in a single CSV. As of March 2025, I haven’t found a tool that will <em>reliably</em> review thousands of entries.</p><p>Instead, I conduct sentiment analysis in batches and commit time for an abridged manual review.</p><p><strong>Example Use-Case: Monitoring User Complaints in App Reviews</strong></p><p>You are one of three UX professionals on the product team, responsible for regularly analyzing app store reviews to identify the top three usability issues. Instead of spending weeks manually reviewing feedback — especially when incorporating external sources like TrustPilot — you streamline the process by exporting the past quarter’s reviews into a CSV file.</p><p>To make the data more manageable, you divide the reviews into smaller, digestible chunks. You then break the task into sub-tasks, using separate prompts to flag negative sentiments, analyze their frequency, and categorize complaints into themes such as navigation issues or slow load times. Once the AI processing is complete, you review the dataset, ensuring sentiment classifications are accurate and refining any misclassified feedback as needed.</p><h3>5. Generating Initial Insights from Qualitative Research</h3><p>Turning raw data into actionable insights is a core function of UX research, but the process is often slow. AI can speed up this step by summarizing qualitative research, identifying patterns, and even suggesting action items. For instance, you can use an AI tool to analyze usability testing sessions to highlight common usability friction points. It can also link research findings to your strategic plan or quarterly revenue goals, making insights more applicable to product development.</p><p>These insights help UX teams move faster, especially in fast-paced product development cycles. However, AI-generated conclusions should be considered a starting point rather than a final decision. AI lacks the deep contextual understanding that human researchers bring, which means insights must be carefully reviewed before implementation.</p><p><strong>Example Use-Case: Reviewing Open-Ended Survey Responses</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eur5bVuYrapexVAl" /><figcaption>Photo by <a href="https://unsplash.com/@firmbee?utm_source=medium&amp;utm_medium=referral">Firmbee.com</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>You’re part of the design team at a fintech company and have recently conducted a financial literacy survey, asking users about their budgeting and spending habits. The survey receives thousands of open-ended responses, making manual analysis overwhelming.</p><blockquote>Traditionally, your team would manually sift through the responses, categorizing feedback and identifying common themes.</blockquote><blockquote>Instead, you automate the first stage of analysis by processing the responses through a natural language tool.</blockquote><p>Your prompt instructs it to scan responses and detect recurring themes — such as difficulty tracking expenses, confusion around credit scores, or a preference for automated savings features. Additionally, you tell it to quantify the frequency of each theme, ranking them in order of importance based on user feedback.</p><p>In this example, you could also instruct the tool to generate preliminary summaries that link research findings to business goals. For example, if a significant number of users express frustration with tracking expenses, this insight could directly support the case for developing a new budgeting feature. AI can also highlight demographic patterns, showing how different age groups or income levels experience financial challenges differently, helping you tailor solutions for various user segments.</p><p>Once the tool has completed its analysis, your team conducts a human review to ensure accuracy and extract deeper meaning from the findings. You refine the insights by adding qualitative nuance that AI might have missed and identifying opportunities for further exploration. These validated insights then feed into content strategies, in-app messaging, and UX improvements designed to educate users and convert them into engaged subscribers.</p><h3>6. Complex Data Analysis: Theming, Coding, and Tagging</h3><p>For large research projects, coding qualitative data and tagging themes is one of the most time-intensive tasks. Traditionally, UX researchers manually code user responses, categorizing feedback into meaningful themes. AI can automate this process by recognizing patterns in qualitative data and applying predefined codes to responses. It can also generate suggested metadata tags, making it easier to organize and retrieve insights.</p><p>AI-driven analysis enables UX teams to scale research efforts without increasing manual workload. It also enhances collaboration, as structured AI-generated insights allow teams to share findings more efficiently. AI-generated themes may not always align with defined research objectives or with specified business goals.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3w4ckY6bAH9mlZir" /><figcaption>Photo by <a href="https://unsplash.com/@patrickperkins?utm_source=medium&amp;utm_medium=referral">Patrick Perkins</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>This presents an opportunity for your team to add strategic value. Instead of simply reporting your findings, your team can bridge the gap between research insights and organizational priorities — tying qualitative themes back to department goals such as revenue growth, customer retention, or subscriber engagement. For example, if your initial analysis reveals that users struggle with onboarding, your team can highlight how improving this experience could directly impact subscriber conversion rates. If sentiment analysis surfaces frustration with billing processes, these insights could support business cases for UX-driven policy changes that reduce churn.</p><p>By going beyond raw data and positioning insights within the larger business context, your team demonstrates its ability to drive meaningful impact. This shift elevates UX research from a support function to a strategic asset, making research findings more actionable and reinforcing the role of UX in shaping key business decisions.</p><h3>Operationalizing AI in Your Team</h3><p><a href="https://media1.giphy.com/media/v1.Y2lkPTc5MGI3NjExOHRjYzZ3anFjcDhic2d0NTM2NWl2aGV3YXJseTA5NGl6NnkwY3hoaCZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/tTc43DeTm2kkJTrI2G/giphy.gif">I love a good SOP. Really.</a></p><p>Standard operating procedures SOPs) are where we share what we think are best practices with our team. They translate operational goals into a concise list (<a href="https://media2.giphy.com/media/v1.Y2lkPTc5MGI3NjExZ3hiZmNxOTdkMmZ3a241bHVic2pmbG92cDF5NDFyMzl6bXNlMmw4eSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/Cz6TlrRVVyv9S/giphy.gif">which I also love</a>). My philosophy on what makes a good SOP is straightforward — write one, and keep rewriting it until new hires can accomplish the task without asking questions. Simple, yeah?</p><p>As your team experiments with AI-assisted synthesis, your SOP should include usage thresholds — how, when, and why to use AI. For example: “AI-generated insights must be reviewed by your research lead before being disseminated to marketing.” They should also define approval flows and audit mechanisms that flag what’s been touched by AI and what hasn’t. These SOPs create structure and to explain your well-documented process when challenged.</p><p>Integrating AI into your team’s workflow also means preparing them to think critically about its output. ReOps can lead this by developing training materials focused on prompt design, evaluating AI-generated synthesis, and knowing when to step in with human judgment.</p><p>Remember your first time doing sentiment analysis? I bet you do. Do you remember the fifth, sixth, twentieth? Probably not. You developed a reflex — an intuition for what was important and what wasn’t. A major concern for senior research leadership is how to help build this reflex in an age of automation. That could include one-on-ones, solid decision-making rubrics (I also love a good rubric, which are frameworks that help you evaluate and compare different alternatives based on a set of predefined criteria and weights ), or peer review.</p><p>Researchers should know how to spot hallucinations, conduct sample verifications, and recognize when their LLM is confidently wrong. Embedding review checkpoints into SOPs, before insights are published or presented, helps solidify that habit. As these tools evolve, so should your playbook. Versioning your SOPs is part of operational rigor.</p><p>What does ethical transparency look like when you’re summarizing interviews with LLMs or using chatbots for screening? When is it appropriate — or required — to tell participants their responses are being analyzed by AI? Consider working with legal to define the extent of your team’s transparency and build standard language into consent forms, research reports, and stakeholder documentation. It’s also worth having a top-level conversation with senior leadership about what internal materials are acceptable to input into LLMs. A publicly available strategic plan or annual report is one thing — proprietary data about an unreleased product is quite another.</p><h3>Final Thoughts</h3><p>AI is not a replacement for UX professionals. For now, we definitely need an “adult in the room”. There are numerous AI tools that, when used effectively and by an experienced UX professional, can elevate research capabilities and drive more informed decision-making.</p><p>By incorporating AI into various stages of the UX workflow — planning, data analysis, transcription, sentiment tracking, and qualitative coding — teams can decrease costs and increase efficiency without sacrificing the depth of human insight. However, AI-generated findings must always be critically evaluated. As AI continues to evolve, the most successful UX professionals will be those who maintain a strong foundation in user-centered research principles while expanding the tools in their ever-expanding toolbox.</p><p><a href="https://www.linkedin.com/in/adammalamis/"><em>Adam</em></a><em> is a UX professional with over 20 years experience. He leads a team of senior designers and developers at </em><a href="https://simalam.com/"><em>Simalam</em></a><em>, specializing in building accessible, user-tested digital tools, platforms and sites. NNG &amp; CPACC certified.</em></p><p><strong><em>If you have an idea for an article you can </em></strong><a href="https://medium.com/researchops-community/submitting-articles-to-the-researchops-community-9b097397f5f3"><strong><em>submit it on our form</em></strong></a><strong><em>. The team at ResearchOps can help arrange support with structuring, writing and editing.</em></strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b10e9bddec9a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/researchops-community/using-ai-tools-to-augment-user-experience-research-workflows-b10e9bddec9a">Using AI Tools to Augment User Experience Research Workflows</a> was originally published in <a href="https://medium.com/researchops-community">ResearchOps Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Top 10 Accessibility Basics for New Web Designers]]></title>
            <link>https://medium.com/@AdamMalamis/top-10-accessibility-basics-for-new-web-designers-ffdfacd42a50?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/ffdfacd42a50</guid>
            <category><![CDATA[usability]]></category>
            <category><![CDATA[web-design]]></category>
            <category><![CDATA[design-process]]></category>
            <category><![CDATA[accessibility]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Sat, 22 Mar 2025 16:57:09 GMT</pubDate>
            <atom:updated>2025-03-22T16:57:09.363Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5XtGLiwHOiwCkktYPC7Y5Q.jpeg" /></figure><p>Accessibility in design is a foundational responsibility of designers aiming to create inclusive digital environments. By understanding and integrating principles such as inclusive design, adherence to accessibility guidelines, empathy for diverse user needs, and thoughtful considerations in typography, color, and navigation, designers elevate their work. Prioritizing accessibility enriches user experiences universally and ensures digital products are welcoming and usable for everyone.</p><h3>1. Understanding Accessibility as Inclusive Design</h3><p>Inclusive design considers diverse user abilities and needs from the outset, rather than addressing accessibility retrospectively. The Nielsen Norman Group (NNG) highlights inclusive design as a critical practice, emphasizing that designers who anticipate a broad spectrum of user requirements significantly enhance user satisfaction and product adoption. By designing with inclusivity, products become more adaptable, reducing barriers for individuals with varying levels of ability. Inclusive design not only addresses permanent disabilities but also accommodates temporary limitations, such as injuries, situational impairments, and varying technological contexts.</p><h3>2. Legal Frameworks and Guidelines</h3><p>Legal frameworks such as the Americans with Disabilities Act (ADA) and the globally recognized Web Content Accessibility Guidelines (WCAG) guide designers in establishing baseline accessibility standards. These guidelines, particularly WCAG, provide detailed criteria for accessible digital content, covering aspects like perceivability, operability, understandability, and robustness. Compliance with these standards ensures products avoid legal pitfalls and reach wider audiences effectively. Organizations such as the International Association of Accessibility Professionals (IAAP) provide resources and training to ensure designers and teams maintain up-to-date knowledge of these critical guidelines. Staying informed of evolving accessibility regulations is vital, as failing to comply can result in significant legal and reputational repercussions for organizations.</p><h3>3. How Disabilities Impact Digital Interaction</h3><p>Understanding diverse user needs begins with empathy and awareness of different types of disabilities — visual, auditory, cognitive, and motor impairments, among others. For example, users with visual impairments often rely on screen readers, necessitating that designers ensure textual descriptions of visual content are clear and informative. Similarly, users with motor impairments depend heavily on keyboard navigation, underscoring the importance of keyboard-accessible interactions. Additionally, recognizing cognitive disabilities involves simplifying interfaces and providing clear instructions, reducing the cognitive load, and enhancing user comfort and efficiency.</p><h3>4. Accessible Typography</h3><p>Typography significantly impacts digital readability. Designers should select typefaces that enhance clarity and ease of reading, especially for users with visual impairments or dyslexia. NNG recommends avoiding overly decorative fonts, maintaining sufficient contrast between text and background, and ensuring font sizes are large enough to be readable across devices. Proper spacing, line height, and alignment further support improved readability and reduced cognitive strain. Designers should also consider the context in which typography will be viewed, optimizing text layout for different screen sizes and resolutions to ensure universal readability.</p><h3>5. Color Accessibility and Contrast</h3><p>Color usage directly influences accessibility. Sufficient contrast between foreground text and background colors is essential to ensure content remains distinguishable and readable. According to WCAG guidelines, contrast ratios of at least 4.5:1 for normal text and 3:1 for large text significantly enhance readability for visually impaired users. Moreover, designers should avoid using color alone to convey critical information, employing additional visual cues or text labels to ensure clarity for users with color vision deficiencies. Tools such as color contrast checkers can help designers validate their designs, ensuring color choices are inclusive and effective.</p><h3>6. Navigational Accessibility</h3><p>Navigational accessibility is integral to creating intuitive user experiences. Clear and predictable navigation structures empower users, including those utilizing assistive technologies, to find information and complete tasks effortlessly. NNG emphasizes consistency in navigation patterns, descriptive link texts, and logical focus order, facilitating smoother interactions for all users. Designing intuitive navigation aids users with cognitive impairments by reducing complexity and decision fatigue. Designers must also consider dynamic content, such as expandable menus or interactive forms, ensuring these remain accessible to all users through appropriate labeling, ARIA roles, and keyboard navigability.</p><h3>7. Testing Accessibility Effectively During the Design Process</h3><p>Testing for accessibility early and consistently throughout the design process ensures issues are identified and resolved efficiently. Tools such as WAVE, Axe, and Google’s Lighthouse are essential resources, providing automated checks for common accessibility issues like color contrast, missing alt texts, and ARIA roles. However, automated testing alone isn’t sufficient. Combining automated tools with manual testing, including screen reader testing (e.g., JAWS, NVDA, VoiceOver), offers deeper insights into actual user experiences. Regular usability testing sessions involving users with disabilities further validate design choices and expose issues not caught by automated methods.</p><h3>8. Incorporating Keyboard Accessibility into Your Design Workflow</h3><p>Keyboard accessibility is critical, enabling users with motor disabilities or screen reader users to interact efficiently. Designers should verify keyboard navigability by ensuring logical focus order, clearly visible focus indicators, and fully accessible interactive components. Regularly conducting keyboard-only testing as part of the design cycle ensures smoother transitions from design to development, significantly enhancing overall accessibility.</p><h3>9. Responsive and Mobile-First Design as Foundations for Accessibility</h3><p>Responsive and mobile-first approaches inherently support accessibility by prioritizing simplicity and clarity across devices. Mobile-first design emphasizes essential content and functionality, which aligns closely with accessibility principles by reducing clutter and focusing on user needs. As NNG notes, mobile-first designs also inherently support simpler navigation structures, larger touch targets, and clearer content hierarchies, significantly enhancing accessibility for all users, particularly those with motor or visual impairments.</p><h3>10. Common Accessibility Pitfalls in Popular Design Trends</h3><p>Current design trends such as animations, sliders, pop-ups, and auto-playing videos often introduce accessibility barriers. Fast or looping animations can cause discomfort or seizures in sensitive users and should include pause controls or be minimal and subtle. Sliders and carousels often fail keyboard navigation tests or confuse screen readers due to unclear semantics and navigation flow. Similarly, pop-ups and modals can trap keyboard users or screen readers if not properly implemented with clear dismiss functionality and appropriate focus management.</p><h3>Conclusion</h3><p>Integrating accessibility deeply into the design process enhances the usability and inclusivity of digital products, enriching experiences for a broader audience. Rather than viewing accessibility as simply compliance with legal requirements, designers should embrace it as an essential component of quality design and innovation. As technology evolves, ongoing education, staying informed about emerging standards, and actively engaging with diverse user groups become even more critical. Ultimately, accessibility represents a commitment to empathy, inclusivity, and quality, driving designers toward more thoughtful, universally accessible solutions and making digital spaces welcoming and empowering for everyone.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ffdfacd42a50" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using AI Tools to Augment User Experience Research Workflows]]></title>
            <link>https://medium.com/@AdamMalamis/using-ai-tools-to-augment-user-experience-research-workflows-0e41140f7b1b?source=rss-66797233241b------2</link>
            <guid isPermaLink="false">https://medium.com/p/0e41140f7b1b</guid>
            <category><![CDATA[workflow]]></category>
            <category><![CDATA[user-experience-design]]></category>
            <category><![CDATA[ai-tools]]></category>
            <category><![CDATA[user-experience-research]]></category>
            <category><![CDATA[user-experience]]></category>
            <dc:creator><![CDATA[Adam Malamis]]></dc:creator>
            <pubDate>Sat, 22 Mar 2025 16:54:52 GMT</pubDate>
            <atom:updated>2025-04-21T23:31:50.911Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Xd8TK1ORlFGPTvK_NuweRQ.png" /><figcaption>Image generated by midjourney.</figcaption></figure><p>The integration of AI into UX research is no longer a futuristic concept — it’s happening now, and it’s reshaping how teams gather, analyze, and act on user insights. While AI shouldn’t replace human intuition or the nuanced understanding that experienced UX professionals bring, it can dramatically enhance efficiency, reduce manual effort, and uncover patterns that might otherwise go unnoticed.</p><p>This article explores six key ways AI can improve your UX workflows, from research planning to data analysis and sentiment tracking. Each section provides examples of how AI can optimize research efforts while also outlining potential pitfalls and the importance of human oversight. By the end of this article, you’ll have a clearer understanding of how to leverage AI tools effectively within your existing workflows.</p><h3>AI as a Research Planning Assistant</h3><p>User research requires meticulous planning, from defining objectives to selecting the right participants. Traditionally, UX professionals develop research plans manually, drafting detailed study outlines, interview questions, and recruitment strategies. This process demands significant time and effort, particularly for large-scale studies. AI can streamline research planning by analyzing past projects, generating structured research plans, and suggesting key questions based on existing data. With AI-powered tools, we can create research outlines that ensure consistency and thoroughness across studies. Additionally, AI can assist in prioritizing research objectives by analyzing existing user behavior data (e.g. purchasing trends, visitor behaviour) to determine which areas require the most focus.</p><p>While AI-generated research plans can be a helpful starting point, they often lack the nuanced judgment that experienced researchers bring. AI suggestions might miss industry-specific context or emerging user concerns that a human researcher would naturally account for.</p><p><strong>Example Use-Case: Planning a Usability Study</strong></p><p>A UX team needs to conduct a usability study for a new dashboard feature but struggles with structuring an efficient research plan. Traditionally, they manually draft objectives, questions, and participant criteria. Here, you can use AI to analyze past usability studies and generate a preliminary research plan, suggesting relevant interview questions and target demographics. While this saves time, the team must review the plan to ensure the activities make sense given the budget, timeline, and desired outcomes. For example, your research plan probably doesn’t need to include card sorting if you’re AB testing a new call to action.</p><p>You should always review questions to identify and document bias, ensure overall relevance, and avoid leading participants.</p><h3>Efficient Data Analysis</h3><p>Speaking from experience, sometimes the sheer volume of data can be overwhelming, especially if you have a tight turnaround. Prior to incorporating AI in our workflow, we would manually sift through surveys, usability test results, and interview transcripts to identify common themes. This process is slow and prone to bias, as researchers may inadvertently focus on expected trends rather than uncovering new insights. AI can dramatically enhance data analysis by automatically detecting patterns, clustering responses, and ranking insights based on their relevance.</p><p>For example, AI can efficiently scan an Excel file containing survey responses to identify frequently mentioned pain points, making it easier for UX teams to prioritize issues. It can also cross-reference findings from multiple research sources, providing a more holistic view of user behavior. Moreover, AI reduces human error by surfacing unexpected patterns that might be overlooked in manual analysis.</p><p>The challenge here is twofold: “living in” the data is extremely valuable. Time, space, and repetition often provide insights that, as an experienced professional, is valuable to the project. Also, we must validate any automated findings, ensuring that the patterns identified genuinely reflect user needs.</p><p><strong>Example Use-Case: Analyzing Survey Responses</strong></p><p>You’ve been tasked with synthesizing a customer satisfaction survey with thousands of responses. Without AI, we would typically spend hours sifting through the responses in multiple passes, coding responses to find common complaints. AI accelerates this process by clustering similar responses, identifying recurring themes, and ranking issues based on their frequency. It gives us the ability to quickly identify obvious pain points — like if our pricing table is unclear, or if response times have lead to customer attrition.</p><h3>Automated Transcription Services</h3><p>User interviews and usability tests generate vast amounts of qualitative data, often in the form of recorded conversations. Before we’d be forced to manually transcribe sessions or pay a premium for a dedicated transcription service. The task was either time-consuming and tedious or expensive — often both. AI-driven transcription services can convert audio to text within minutes, allowing UX teams to focus on analysis rather than transcription.</p><p>Beyond speed, AI transcription tools also improve organization. Many AI-powered services can differentiate between speakers, generate timestamps, and even highlight key themes within transcripts. This makes it easier to navigate large amounts of qualitative data and retrieve relevant insights quickly. However, AI transcription is not perfect — accents, background noise, and technical jargon can reduce accuracy.</p><h3>Sentiment Analysis for User Feedback</h3><p>Gathering user feedback can be extremely valuable for refining digital experiences, but manually analyzing open-ended responses is laborious. AI-powered tools can evaluate hundreds of entries in minutes, classifying them as positive, negative, or neutral. AI can detect emotional tones within user reviews, social media comments, and survey responses, helping UX teams identify frustration points and areas of delight.</p><p>Beyond simple sentiment detection, AI can highlight specific emotional triggers, such as repeated complaints about confusing navigation or praise for an existing feature. AI can also rank feedback by urgency (provided you give it direction on how to do so, e.g. by frequency), helping UX teams prioritize the most critical pain points. However, sentiment analysis tools can misinterpret context, especially sarcasm or industry-specific terminology. They also hallucinate and often have trouble evaluating larger datasets. As tools, like ChatGPT, become more powerful, we should be able to upload thousands of entries in a single CSV. As of March 2025, I haven’t found a tool that will <em>reliably</em> review thousands of entries.</p><p>Instead, I conduct sentiment analysis in batches and commit time for an abridged manual review.</p><p><strong>Example Use-Case: Monitoring User Complaints in App Reviews</strong></p><p>You are one of three UX professionals on the product team, responsible for regularly analyzing app store reviews to identify the top three usability issues. Instead of spending weeks manually reviewing feedback — especially when incorporating external sources like TrustPilot — you streamline the process by exporting the past quarter’s reviews into a CSV file.</p><p>To make the data more manageable, you divide the reviews into smaller, digestible chunks. You then break the task into sub-tasks, using separate prompts to flag negative sentiments, analyze their frequency, and categorize complaints into themes such as navigation issues or slow load times. Once the AI processing is complete, you review the dataset, ensuring sentiment classifications are accurate and refining any misclassified feedback as needed.</p><h3>Generating Initial Insights from Qualitative Research</h3><p>Turning raw data into actionable insights is a core function of UX research, but the process is often slow. AI can speed up this step by summarizing qualitative research, identifying patterns, and even suggesting action items. For instance, you can use an AI tool to analyze usability testing sessions to highlight common usability friction points. It can also link research findings to your strategic plan or quarterly revenue goals, making insights more applicable to product development.</p><p>These insights help UX teams move faster, especially in fast-paced product development cycles. However, AI-generated conclusions should be considered a starting point rather than a final decision. AI lacks the deep contextual understanding that human researchers bring, which means insights must be carefully reviewed before implementation.</p><p><strong>Example Use-Case: Reviewing Open-Ended Survey Responses</strong></p><p>You’re part of the design team at a fintech company and have recently conducted a financial literacy survey, asking users about their budgeting and spending habits. The survey receives thousands of open-ended responses, making manual analysis overwhelming.</p><p>Traditionally, your team would manually sift through the responses, categorizing feedback and identifying common themes.</p><p>Instead, you automate the first stage of analysis by processing the responses through a natural language tool. Your prompt instructs it to scan responses and detect recurring themes — such as difficulty tracking expenses, confusion around credit scores, or a preference for automated savings features. Additionally, you tell it to quantify the frequency of each theme, ranking them in order of importance based on user feedback.</p><p>In this example, you could also instruct the tool to generate preliminary summaries that link research findings to business goals. For example, if a significant number of users express frustration with tracking expenses, this insight could directly support the case for developing a new budgeting feature. AI can also highlight demographic patterns, showing how different age groups or income levels experience financial challenges differently, helping you tailor solutions for various user segments.</p><p>Once the tool has completed its analysis, your team conducts a human review to ensure accuracy and extract deeper meaning from the findings. You refine the insights by adding qualitative nuance that AI might have missed and identifying opportunities for further exploration. These validated insights then feed into content strategies, in-app messaging, and UX improvements designed to educate users and convert them into engaged subscribers.</p><h3>Complex Data Analysis: Theming, Coding, and Tagging</h3><p>For large research projects, coding qualitative data and tagging themes is one of the most time-intensive tasks. Traditionally, UX researchers manually code user responses, categorizing feedback into meaningful themes. AI can automate this process by recognizing patterns in qualitative data and applying predefined codes to responses. It can also generate suggested metadata tags, making it easier to organize and retrieve insights.</p><p>AI-driven analysis enables UX teams to scale research efforts without increasing manual workload. It also enhances collaboration, as structured AI-generated insights allow teams to share findings more efficiently. AI-generated themes may not always align with defined research objectives or with specified business goals.</p><p>This presents an opportunity for your team to add strategic value. Instead of simply reporting your findings, your team can bridge the gap between research insights and organizational priorities — tying qualitative themes back to department goals such as revenue growth, customer retention, or subscriber engagement. For example, if your initial analysis reveals that users struggle with onboarding, your team can highlight how improving this experience could directly impact subscriber conversion rates. If sentiment analysis surfaces frustration with billing processes, these insights could support business cases for UX-driven policy changes that reduce churn.</p><p>By going beyond raw data and positioning insights within the larger business context, your team demonstrates its ability to drive meaningful impact. This shift elevates UX research from a support function to a strategic asset, making research findings more actionable and reinforcing the role of UX in shaping key business decisions.</p><h3>Final Thoughts</h3><p>AI is not a replacement for UX professionals. For now, we definitely need an “adult in the room”. There are numerous AI tools that, when used effectively and by an experienced UX professional, can elevate research capabilities and drive more informed decision-making.</p><p>By incorporating AI into various stages of the UX workflow — planning, data analysis, transcription, sentiment tracking, and qualitative coding — teams can decrease costs and increase efficiency without sacrificing the depth of human insight. However, AI-generated findings must always be critically evaluated. As AI continues to evolve, the most successful UX professionals will be those who maintain a strong foundation in user-centered research principles while expanding the tools in their ever-expanding toolbox.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0e41140f7b1b" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>