<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jia-Ru (Edith) Chung on Medium]]></title>
        <description><![CDATA[Stories by Jia-Ru (Edith) Chung on Medium]]></description>
        <link>https://medium.com/@janajiaruchung?source=rss-a1ff058aedd5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*gkEZcFhYuUlVXG8XKZmW_w.jpeg</url>
            <title>Stories by Jia-Ru (Edith) Chung on Medium</title>
            <link>https://medium.com/@janajiaruchung?source=rss-a1ff058aedd5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 16:43:23 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@janajiaruchung/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Treating Research Programs as Products: From Manual Prototypes to Scalable Evaluation Systems]]></title>
            <link>https://janajiaruchung.medium.com/treating-research-programs-as-products-from-manual-prototypes-to-scalable-evaluation-systems-c5cb14041186?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/c5cb14041186</guid>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[surveys]]></category>
            <category><![CDATA[research]]></category>
            <category><![CDATA[evaluation]]></category>
            <category><![CDATA[program]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Mon, 16 Nov 2026 18:41:15 GMT</pubDate>
            <atom:updated>2026-11-16T18:41:15.885Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How research quality, product thinking, iterative development, and stakeholder collaboration transform individual research workflows into sustainable organizational capabilities.</em></p><p>Research programs are often designed as processes: define objectives, collect data, analyze results, and deliver findings.</p><p>But what happens when the goal is not simply to complete a study, but to build an evaluation system that supports multiple workstreams, adapts to changing requirements, and produces reliable insights over time?</p><p>Through my experience building scalable evaluation programs at Microsoft and Google, I learned an important lesson: <strong>An evaluation program can be developed and managed like a product.</strong></p><p>Rather than designing a complete system upfront, I approached program development incrementally. Starting with a single workstream and manual processes, I established research quality standards, validated the measurement approach, incorporated stakeholder feedback, improved reporting, and gradually expanded the framework through automation and integration.</p><p>This approach combines technical rigor, product thinking, strategic prioritization, cross-functional collaboration, and continuous improvement.</p><h3>1. Establish Research Quality Standards From the Beginning</h3><p>When building a scalable evaluation program, one of the most important lessons I’ve learned is that <strong>research quality must be established from the beginning, not added after the system has been built.</strong></p><p>Starting with a minimum viable framework does not mean compromising methodological rigor. Even a manual prototype should follow established research best practices to ensure that the data collected is valid, reliable, consistent, and suitable for decision-making.</p><p>Before launching the first evaluation, I focus on several foundational elements.</p><p><strong>Measurement validity:</strong> Define clear research objectives, constructs, and success metrics to ensure we’re measuring the right concepts. Survey questions and evaluation criteria should accurately reflect the experiences or outcomes we’re trying to understand.</p><p><strong>Measurement reliability:</strong> Standardize questions, rating scales, evaluation guidelines, and data collection procedures so results remain consistent across participants, evaluators, workstreams, and measurement periods.</p><p><strong>Sampling and representativeness:</strong> Establish appropriate sampling strategies, eligibility criteria, and recruitment procedures. Evaluate potential selection and nonresponse bias to understand whether findings can be generalized to the intended population.</p><p><strong>Data collection quality:</strong> Apply survey design best practices, including neutral wording, clear response options, appropriate survey length, and consistent administration. Pilot-test instruments and collection workflows before full deployment.</p><p><strong>Data integrity:</strong> Define data schemas, required fields, unique identifiers, validation rules, and procedures for handling missing, duplicate, or inconsistent records. These standards become especially important when integrating multiple data sources.</p><p><strong>Analytical and reporting standards:</strong> Establish appropriate statistical methods, quality checks, documentation practices, and reporting conventions to ensure findings are reproducible, interpretable, and communicated without overstating conclusions.</p><p>In my experience, establishing these standards early creates a stronger foundation for subsequent automation and expansion.</p><p>For example, when moving from manual surveys to a platform such as Qualtrics, standardized questions, consistent metric definitions, and validated data structures make it easier to maintain comparability across survey waves and workstreams.</p><p>Without these foundations, automation can simply reproduce existing measurement problems at a larger scale.</p><p><strong>Key takeaway: Start small in functionality, but establish high standards for research validity, reliability, and data quality from day one.</strong></p><h3>2. Start Small: Build a Manual Prototype Before Automating</h3><p>When establishing a research program from scratch, it’s tempting to begin with sophisticated dashboards, automated pipelines, and comprehensive integrations.</p><p>However, automating an unvalidated process can introduce complexity before the team fully understands its requirements.</p><p>A more effective approach is to start with a minimum viable evaluation framework.</p><p>In my Microsoft experience, the program began with manual survey collection and analysis for a focused workstream before evolving toward a more standardized, scalable process supported by Qualtrics.</p><p>This initial phase helped establish the essential components: research objectives, survey questions, measurement definitions, data-quality checks, analytical methods, and reporting expectations.</p><p>Working manually also created opportunities to identify practical problems early, such as inconsistent data formats, unclear metrics, repetitive analytical tasks, and difficulties interpreting results.</p><p>These observations informed what needed to be standardized or automated.</p><p>Starting small also allowed the framework to be tested against actual stakeholder needs before committing to a more complex technical implementation.</p><p>The objective was not to create a perfect system immediately, but to establish a reliable foundation that could evolve as requirements became clearer.</p><p><strong>Key takeaway: Validate the measurement process before investing in automation. A simple, reliable prototype is more valuable than a sophisticated system built on untested assumptions.</strong></p><h3>3. Treat Stakeholders as Users of the Research System</h3><p>An evaluation program is only valuable if stakeholders can understand its outputs and use them to make decisions.</p><p>Researchers may prioritize methodological rigor, while product managers, engineers, operations teams, and business leaders often have different analytical needs.</p><p>For example, product teams may want to identify usability problems, operational teams may focus on workflow efficiency, and leadership may need to understand performance trends across multiple applications.</p><p>These needs should influence how the evaluation system evolves.</p><p>Just as product teams gather customer feedback, research program builders can collect stakeholder feedback on early prototypes, reports, and dashboards.</p><p>In practice, this means reviewing outputs with stakeholders, understanding which findings are difficult to interpret, identifying missing information, and refining the presentation accordingly.</p><p>An early report may contain technically accurate results but require substantial explanation. Over time, clearer visualizations, standardized metric definitions, and more intuitive information hierarchies can make the same analysis easier to review and act upon.</p><p>Stakeholder collaboration also helps ensure that new features address actual decision-making needs rather than simply adding analytical complexity.</p><p><strong>Key takeaway: Stakeholder feedback should shape the research system itself, not just the product being evaluated.</strong></p><h3>4. Iterate on Technical Quality and Reporting Usability</h3><p>Treating an evaluation program as a product means continuously improving two dimensions: analytical reliability and stakeholder usability.</p><h3>Technical iteration</h3><p>As a program evolves, data structures, survey requirements, and analytical methods may need to change.</p><p>Technical improvements can include standardizing survey fields, refining data-cleaning procedures, strengthening validation checks, improving statistical analysis, and reducing repetitive manual work.</p><p>In the early stages, these tasks may be performed manually to understand where errors occur and which steps require consistent rules.</p><p>Once the process is stable, repeatable components can be automated.</p><p>For example, data-cleaning scripts can standardize response formats, apply quality checks, calculate performance metrics, and generate consistent analytical outputs.</p><h3>Reporting iteration</h3><p>Technical accuracy does not automatically make findings useful.</p><p>Reports should also evolve based on how stakeholders interpret and use them.</p><p>Improvements might include clearer metric definitions, more intuitive charts, longitudinal comparisons, simplified summaries, and better visibility into recurring problems.</p><p>Rather than presenting every available metric with equal emphasis, reports can prioritize overall performance, significant changes, recurring issues, and recommended actions.</p><p>This iterative process helps ensure that technical improvements and stakeholder usability evolve together.</p><p><strong>Key takeaway: A scalable evaluation program must be both analytically reliable and easy for stakeholders to understand and use.</strong></p><h3>5. Expand From One Workstream to Multiple Workstreams</h3><p>One of the most important transitions in program development is moving from a successful individual workflow to a reusable framework.</p><p>A single workstream provides an opportunity to test the process under manageable conditions.</p><p>Once the measurement approach is validated, the next challenge is determining which components can be reused across additional workstreams.</p><p>For example, core satisfaction metrics, survey-quality checks, data-processing procedures, and reporting structures may be standardized.</p><p>At the same time, individual workstreams may require different questions, operational measures, or analytical approaches.</p><p>The objective is not to force every team into an identical framework. It is to establish a common foundation that supports consistency while accommodating meaningful differences.</p><p>In my Microsoft experience, the evaluation effort expanded into a longitudinal measurement program covering approximately 33 internal applications.</p><p>As the scope increased, standardization became increasingly important for comparing performance, maintaining report quality, and identifying recurring experience problems across applications.</p><p>This transition also changed the nature of the research work.</p><p>Instead of focusing only on individual findings, the program needed to support ongoing measurement, cross-application comparisons, and stakeholder prioritization.</p><p><strong>Key takeaway: Scalability comes from standardizing reusable components while preserving flexibility for different workstreams.</strong></p><h3>6. Introduce Platform Integration and Automation When the Process Is Ready</h3><p>Once the manual workflow is validated and requirements become clearer, platform integration and automation can help improve efficiency and consistency.</p><p>In my Microsoft experience, this included evolving from manual survey processes toward Qualtrics-supported collection and more standardized analytical workflows.</p><p>Survey platforms can help manage distribution, response collection, structured metadata, and recurring measurement activities.</p><p>Complementary SQL or Python workflows can support data cleaning, segmentation, statistical analysis, quality validation, and reporting.</p><p>However, automation should be introduced selectively.</p><p>A useful way to prioritize automation is to ask whether a task is repeated frequently, follows consistent rules, creates substantial manual effort, or introduces avoidable errors.</p><p>For example, automating routine data validation may create greater value than introducing an advanced visualization that stakeholders rarely use.</p><p>Automation also requires ongoing maintenance.</p><p>Changes to survey questions, data structures, or reporting requirements can affect existing pipelines. Documentation, validation tests, and version control help maintain reliability as the system evolves.</p><p><strong>Key takeaway: Automation should strengthen a validated process, not substitute for understanding how the process should work.</strong></p><h3>7. Prioritize Improvements Based on Stakeholder and Program Value</h3><p>As an evaluation program matures, stakeholders may request additional metrics, integrations, dashboards, and analytical features.</p><p>This creates a product management challenge: deciding what to improve next.</p><p>I approach these decisions by considering analytical quality, stakeholder value, frequency of use, scalability, and implementation effort.</p><p>For example, a data-validation issue affecting multiple workstreams should generally take priority over a cosmetic reporting improvement.</p><p>Once the core framework is reliable, improving visualization clarity, reducing manual analysis time, or expanding functionality to additional workstreams may offer greater organizational value.</p><p>This also requires balancing immediate needs with long-term maintainability.</p><p>A highly customized feature may benefit one stakeholder group but create complexity for the broader program.</p><p>A reusable improvement, by contrast, may generate value across multiple teams and evaluation cycles.</p><p><strong>Key takeaway: Prioritize improvements that strengthen reliability, usability, and scalability rather than expanding functionality for its own sake.</strong></p><h3>8. Build Continuous Quality Assurance Into the Program</h3><p>In a traditional research project, quality assurance is often concentrated around data collection, analysis, and final reporting.</p><p>In a scalable evaluation program, quality assurance must become an ongoing capability.</p><p>This includes monitoring response completeness, validating analytical outputs, reviewing metric definitions, documenting methodological changes, and ensuring consistency across measurement periods.</p><p>As new workstreams are introduced, the program must also account for differences in data availability, sample sizes, user populations, and operational contexts.</p><p>For example, not every application may have sufficient responses for reliable comparisons.</p><p>A rigorous framework should make these limitations visible rather than forcing incomplete data into misleading rankings.</p><p>Standardized quality checks help ensure that scaling the program does not compromise the credibility of its findings.</p><p><strong>Key takeaway: Research quality should be embedded in the workflow and maintained as the program expands.</strong></p><h3>9. Measure the Success of the Evaluation Program Itself</h3><p>If an evaluation program is treated as a product, it should have its own success criteria.</p><p>Traditional research measures often focus on the quality and quantity of studies delivered.</p><p>A product-oriented approach also considers whether the evaluation system improves efficiency, reliability, stakeholder understanding, and organizational decision-making.</p><p><strong>Operational efficiency:</strong> Has automation reduced repetitive data preparation, analysis, or reporting work?</p><p><strong>Analytical reliability:</strong> Are metrics consistently defined, validated, and reproducible?</p><p><strong>Stakeholder usability:</strong> Can teams interpret findings and identify priorities without extensive explanation?</p><p><strong>Adoption:</strong> Are multiple workstreams consistently using the framework?</p><p><strong>Decision impact:</strong> Are findings informing product improvements, operational decisions, or strategic priorities?</p><p><strong>Scalability:</strong> Can additional workstreams adopt the framework without rebuilding the process from scratch?</p><p>These measures help distinguish a technically completed system from a genuinely useful organizational capability.</p><p>A dashboard may be accurate but rarely used. An automated pipeline may save time but fail to address stakeholder needs.</p><p>Continuous evaluation of the program itself helps identify these gaps and guide future iterations.</p><p><strong>Key takeaway: The success of an evaluation program depends on both the quality of its outputs and the value it creates for its users.</strong></p><h3>A Practical Framework: From Manual Prototype to Scalable Program</h3><p>The evolution of a research program can be organized into seven stages.</p><p><strong>1. Establish Quality Standards:</strong> Define research objectives, valid and reliable metrics, sampling procedures, data schemas, quality checks, and reporting principles before collecting data.</p><p><strong>2. Prototype:</strong> Start with one workstream, manual survey collection, core metrics, and basic analysis to establish a minimum viable measurement framework.</p><p><strong>3. Validate:</strong> Evaluate data quality, methodological reliability, stakeholder needs, and reporting usability before expanding.</p><p><strong>4. Iterate:</strong> Refine survey questions, analytical methods, data structures, visualizations, and reporting based on feedback and observed limitations.</p><p><strong>5. Standardize:</strong> Establish reusable metric definitions, data-processing procedures, quality checks, and reporting templates.</p><p><strong>6. Integrate and Scale:</strong> Introduce survey platforms such as Qualtrics, automate repeatable workflows, and expand the framework across additional workstreams.</p><p><strong>7. Monitor and Improve:</strong> Continuously evaluate data quality, stakeholder adoption, analytical efficiency, and business value.</p><p>These stages are iterative rather than strictly linear. New requirements may reveal the need to revisit earlier design decisions, while ongoing monitoring may identify opportunities for further improvement.</p><h3>Final Thoughts: Research Infrastructure Is Also a Product</h3><p>One of my biggest lessons from building evaluation programs at Microsoft and Google is that <strong>scalable research requires more than strong analytical methods. It requires product thinking.</strong></p><p>Starting with a single workstream and manual prototype allowed the measurement approach to be tested before introducing additional complexity.</p><p>Establishing research quality standards from the beginning ensured that the framework could expand without compromising the validity, reliability, or consistency of its findings.</p><p>As the program evolved, stakeholder feedback helped shape reporting usability, technical improvements strengthened analytical reliability, and standardization made it possible to expand across multiple workstreams.</p><p>Platform integration and automation then supported more efficient, repeatable evaluation.</p><p>The objective was not simply to automate research or produce more reports.</p><p>It was to establish a reliable, accessible, and reusable framework that could support continuous performance monitoring and evidence-based decisions.</p><p>This perspective changes how we define research impact.</p><p>Beyond delivering individual studies, researchers can contribute by building the infrastructure, standards, and workflows that allow organizations to generate and use evidence consistently.</p><p><strong>The ultimate value of a scalable evaluation program is not just the research it produces today, but the organization’s ability to continue producing reliable insights and making better decisions tomorrow.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c5cb14041186" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Strategic Thinking Is a Skill You Can Systematize: A Six-Step Framework for Better Decisions]]></title>
            <link>https://janajiaruchung.medium.com/strategic-thinking-is-a-skill-you-can-systematize-a-six-step-framework-for-better-decisions-eed616f35534?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/eed616f35534</guid>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[business]]></category>
            <category><![CDATA[strategy]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Sun, 15 Nov 2026 01:15:21 GMT</pubDate>
            <atom:updated>2026-11-15T01:15:21.068Z</atom:updated>
            <content:encoded><![CDATA[<p><em>How to move beyond analysis, align stakeholders, make smarter tradeoffs, and turn insights into measurable impact.</em></p><p>Technical expertise helps us determine whether our analysis is accurate. Strategic thinking helps us determine whether we’re solving the right problem in the first place.</p><p>In research, analytics, product development, and business operations, professionals often have established frameworks for evaluating technical quality. We examine validity, reliability, bias, statistical significance, and the limitations of our methods.</p><p>But when it comes to strategic thinking, the process is often less explicit.</p><p>What makes a recommendation strategic? How do we decide which problems deserve attention? How do we move from identifying an opportunity to creating measurable business value?</p><p>Strategic thinking is sometimes treated as an abstract leadership quality, but much of it can be developed through a systematic approach.</p><p>I use a six-step framework to organize strategic problem-solving:</p><p><strong>Frame → Align → Prioritize → Execute → Measure → Scale</strong></p><p>This framework can be applied across user research, product management, business analytics, operations, and organizational transformation.</p><h3>1. Frame: Are We Solving the Right Problem?</h3><p>Strategic thinking begins before selecting a research method, building a dashboard, or proposing a solution.</p><p>It starts with understanding the business problem and the decision that needs to be made.</p><p>Imagine a company reports declining customer satisfaction.</p><p>An analytical response might be to investigate satisfaction scores, segment customers, and identify negative feedback themes.</p><p>A strategic response begins with broader questions.</p><p>What business outcome are we trying to improve? Is declining satisfaction the underlying problem or merely a symptom? Which customer journeys or business processes are affected? What decisions will the findings inform? What would success look like?</p><p>For example, declining satisfaction might reflect delayed deliveries, poor customer support, confusing product experiences, or unmet expectations.</p><p>Each explanation could require a different intervention.</p><p>A useful problem statement connects the observed issue, affected population, desired outcome, and business context.</p><p>Instead of saying, “Customer satisfaction has declined,” we might frame the problem as:</p><p>“We need to identify which service failures contribute most to customer dissatisfaction and determine which improvements could deliver the greatest customer and operational value.”</p><p>The second statement provides a clearer foundation for decision-making.</p><p><strong>Strategic principle: Define the decision before defining the analysis.</strong></p><h3>2. Align: Are Stakeholders Working Toward the Same Outcome?</h3><p>Even a well-defined problem can be difficult to solve when stakeholders have different objectives.</p><p>Product teams may prioritize usability and adoption. Operations teams may focus on efficiency and cost. Finance may emphasize return on investment. Leadership may be concerned with long-term growth and organizational risk.</p><p>These perspectives are not necessarily conflicting. They often reflect different responsibilities.</p><p>Strategic alignment involves understanding these priorities and establishing shared success criteria.</p><p>A practical approach begins by identifying the decision-makers, stakeholders, execution owners, organizational constraints, and desired outcomes.</p><p>Stakeholder workshops, discovery interviews, journey mapping, and collaborative prioritization exercises can help establish a common understanding.</p><p>For example, when improving customer onboarding, Product may want to simplify the experience, Operations may want to reduce manual processing, and Compliance may need to preserve verification requirements.</p><p>A strategic approach would identify a shared objective, such as increasing successful onboarding while maintaining operational efficiency and compliance.</p><p>However, alignment does not require everyone to agree on every detail.</p><p>The objective is to establish a shared understanding of the problem, the evidence, and the criteria that will guide decisions.</p><p><strong>Strategic principle: Alignment is not consensus on every solution; it is agreement on the problem, objectives, and decision-making process.</strong></p><h3>3. Prioritize: What Should We Solve First?</h3><p>Organizations rarely lack problems to solve. More often, they lack sufficient resources to address every opportunity simultaneously.</p><p>This makes prioritization one of the most important strategic capabilities.</p><p>Imagine a company identifies two customer experience problems.</p><p>Problem A has a customer satisfaction score of 2.5 out of 5, affects approximately 500 customers per month, and requires relatively little implementation effort.</p><p>Problem B has a satisfaction score of 3.2 out of 5, affects approximately 15,000 customers per month, and requires moderate implementation effort.</p><p>If we prioritize solely by satisfaction score, Problem A appears more urgent.</p><p>However, Problem B affects substantially more customers and may create greater overall business impact.</p><p>A strategic assessment considers several dimensions.</p><p><strong>Customer impact:</strong> How many customers are affected, and how severely?</p><p><strong>Business value:</strong> What is the potential effect on revenue, retention, cost, efficiency, or other organizational outcomes?</p><p><strong>Feasibility:</strong> What resources, dependencies, and implementation effort are required?</p><p><strong>Risk:</strong> Are there compliance, security, reputational, or operational concerns?</p><p><strong>Confidence:</strong> How strong is the evidence supporting the proposed intervention?</p><p>Frameworks such as RICE (Reach, Impact, Confidence, Effort) can help structure prioritization, but no scoring model should replace professional judgment.</p><p>For example, a low-frequency issue involving significant customer harm or regulatory risk may deserve immediate attention regardless of its numerical score.</p><p>Prioritization also requires acknowledging opportunity costs. Choosing one initiative means delaying or declining another.</p><p>Strong strategic judgment makes those tradeoffs explicit.</p><p><strong>Strategic principle: Prioritization is about making defensible tradeoffs, not simply ranking metrics.</strong></p><h3>4. Execute: How Do Insights Become Action?</h3><p>Identifying an important problem does not guarantee that it will be solved.</p><p>Many organizations produce valuable research findings, analytical reports, and strategic recommendations that never translate into meaningful change.</p><p>The gap between insight and execution often involves unclear ownership, competing priorities, insufficient resources, or organizational resistance.</p><p>A strategic recommendation should therefore include more than what needs to change.</p><p>It should explain the proposed action, the expected benefit, the implementation owner, relevant dependencies, the timeline, and the criteria for success.</p><p>For complex or uncertain initiatives, a pilot can be an effective starting point.</p><p>Suppose an analysis identifies a confusing onboarding process as a major contributor to customer abandonment.</p><p>Rather than immediately redesigning the entire experience, a team might test clearer instructions or a simplified verification step.</p><p>A focused experiment reduces implementation risk and generates evidence before broader investment.</p><p>Execution also requires influence.</p><p>When stakeholders disagree, the goal should not be to defend a recommendation at all costs. It should be to understand concerns, clarify tradeoffs, and identify a feasible path forward.</p><p>For example, a product team may support a redesign, while an engineering team raises concerns about technical complexity.</p><p>A strategic response might involve identifying a smaller intervention that addresses the most important customer friction without requiring extensive architectural changes.</p><p>This preserves the business objective while adapting the implementation approach.</p><p><strong>Strategic principle: An insight creates value only when it informs a decision or action.</strong></p><h3>5. Measure: Did the Decision Actually Create Value?</h3><p>Implementation is not the same as success.</p><p>A feature can launch on time, a process can be redesigned, and a dashboard can be delivered without producing meaningful improvements.</p><p>Strategic measurement connects actions to outcomes.</p><p>This requires defining success before implementation.</p><p>For example, an organization improving customer onboarding might establish several levels of measurement.</p><p>At the customer experience level, it might track satisfaction, perceived ease, and clarity.</p><p>At the behavioral level, it might measure completion rates, abandonment, and repeated attempts.</p><p>At the operational level, it might examine processing time, support contacts, and rework.</p><p>At the business level, it might evaluate activation, retention, and cost per successful onboarding.</p><p>These measures help distinguish whether an intervention improved the experience, changed customer behavior, increased operational efficiency, or delivered broader business value.</p><p>Whenever feasible, organizations should use appropriate experimental or quasi-experimental designs to evaluate impact.</p><p>A randomized A/B test can help estimate the causal effect of a change. When randomization is impractical, methods such as difference-in-differences or interrupted time-series analysis may be useful, provided their assumptions are credible.</p><p>It is also important to distinguish correlation from causation.</p><p>For example, if satisfaction improves after a product redesign, the redesign may have contributed to the improvement. However, seasonal changes, shifts in customer composition, or other simultaneous initiatives could also explain the result.</p><p>Strong strategic measurement acknowledges uncertainty rather than overstating impact.</p><p>Finally, measurement should inform the next decision. If the intervention does not produce the expected result, the team should investigate why and determine whether to iterate, expand, or discontinue the initiative.</p><p><strong>Strategic principle: Measure outcomes, not just outputs, and distinguish observed changes from demonstrated causal effects.</strong></p><h3>6. Scale: Can Success Become a Repeatable Capability?</h3><p>A successful initiative creates immediate value.</p><p>A scalable initiative creates the ability to deliver value repeatedly.</p><p>This is where strategic thinking moves beyond individual projects toward organizational capability building.</p><p>Consider a team that develops an effective approach to identifying customer journey friction.</p><p>If every subsequent project requires rebuilding the survey, data pipeline, analytical process, and reporting framework, the organization remains dependent on one-time efforts.</p><p>A more scalable approach might include standardized metrics, reusable data models, automated analytical workflows, documented methods, governance practices, and ongoing performance monitoring.</p><p>For example, a team that successfully evaluates one onboarding journey could develop a reusable measurement framework for other customer journeys.</p><p>The framework might standardize core metrics, data quality checks, reporting structures, and evaluation procedures while allowing teams to adapt to different business contexts.</p><p>Scaling also requires recognizing that a solution effective in one context may not work identically elsewhere.</p><p>Before expanding a successful pilot, organizations should evaluate differences in customer populations, operational environments, technical dependencies, and implementation capacity.</p><p>The objective is not to standardize everything.</p><p>It is to standardize what should remain consistent while preserving flexibility where context matters.</p><p><strong>Strategic principle: Sustainable impact comes from building capabilities, not just completing projects.</strong></p><h3>Bringing the Six Steps Together</h3><p>The six steps form a continuous strategic decision-making cycle.</p><p><strong>Frame:</strong> Define the business problem and the decision that needs to be made.</p><p><strong>Align:</strong> Establish shared objectives, stakeholder responsibilities, and success criteria.</p><p><strong>Prioritize:</strong> Evaluate opportunities based on impact, feasibility, risk, and available evidence.</p><p><strong>Execute:</strong> Translate recommendations into actionable initiatives with clear ownership.</p><p><strong>Measure:</strong> Evaluate outcomes, quantify business impact, and identify opportunities for improvement.</p><p><strong>Scale:</strong> Standardize successful practices and build sustainable organizational capabilities.</p><p>The process is not strictly linear.</p><p>New evidence may require reframing the problem. Implementation constraints may change priorities. Measurement may reveal unexpected outcomes. Scaling may expose new organizational challenges.</p><p>Strategic thinking requires revisiting earlier decisions as conditions evolve.</p><h3>Ten Questions to Strengthen Strategic Judgment</h3><p>The six-step process can be supplemented with ten questions that serve as a practical strategic quality checklist.</p><p><strong>1. Business problem framing:</strong> Are we addressing the underlying problem or merely a symptom?</p><p><strong>2. Customer-centric thinking:</strong> Which customer needs, pain points, and outcomes matter most?</p><p><strong>3. Strategic alignment:</strong> How does this initiative support broader organizational objectives?</p><p><strong>4. Stakeholder alignment:</strong> Who needs to participate in the decision and implementation?</p><p><strong>5. Prioritization:</strong> Why is this opportunity more important than the alternatives?</p><p><strong>6. Tradeoffs:</strong> What are we choosing not to do, and what risks are we accepting?</p><p><strong>7. Execution:</strong> Who owns the action, and what could prevent successful implementation?</p><p><strong>8. Risk and uncertainty:</strong> Which assumptions remain unvalidated, and how can we reduce uncertainty?</p><p><strong>9. Business impact:</strong> How will we determine whether the decision created measurable value?</p><p><strong>10. Scalability:</strong> Can the approach be sustained, reused, or improved over time?</p><p>These questions can be applied during project planning, stakeholder discussions, strategic reviews, and leadership presentations.</p><p>They also provide a useful way to evaluate the quality of a recommendation before presenting it.</p><h3>Technical Rigor and Strategic Judgment Are Complementary</h3><p>Technical and strategic expertise are sometimes treated as separate capabilities.</p><p>In practice, they reinforce one another.</p><p>Technical rigor ensures that evidence is credible. It addresses questions about data quality, validity, reliability, statistical uncertainty, and analytical limitations.</p><p>Strategic judgment ensures that the evidence is relevant to important decisions. It addresses business priorities, organizational constraints, tradeoffs, execution, and measurable value.</p><p>Neither is sufficient on its own.</p><p>A technically sophisticated analysis may have little impact if it addresses a low-priority problem.</p><p>A compelling strategy may fail if it rests on unreliable evidence or unrealistic assumptions.</p><p>The strongest decision-making processes combine both.</p><p><strong>Technical rigor asks: Can we trust the evidence?</strong></p><p><strong>Strategic judgment asks: What should we do with it, and why?</strong></p><p>This distinction is especially important for researchers, analysts, and technical professionals who want to increase their strategic influence.</p><p>Developing strategic expertise does not mean abandoning technical depth. It means using that expertise to support higher-quality decisions.</p><h3>Final Thoughts: Strategy Is About Better Decisions</h3><p>Strategic thinking does not require having the most ambitious vision, the most complicated framework, or the highest-ranking title.</p><p>It requires the ability to connect problems, evidence, stakeholders, decisions, and outcomes.</p><p>The six-step framework — <strong>Frame, Align, Prioritize, Execute, Measure, Scale</strong> — offers a practical way to develop that capability.</p><p>It encourages us to move beyond asking whether our analysis is correct and toward asking whether our work is helping the organization make better decisions.</p><p>Ultimately, the value of strategy is not measured by the number of recommendations produced or initiatives launched.</p><p><strong>It is measured by the quality of decisions made, the outcomes achieved, and the organization’s ability to sustain and improve those outcomes over time.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eed616f35534" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Survey Research Validity: Common Pitfalls From Questionnaire Design to Reporting]]></title>
            <link>https://janajiaruchung.medium.com/survey-research-validity-common-pitfalls-from-questionnaire-design-to-reporting-26f3442a9fb5?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/26f3442a9fb5</guid>
            <category><![CDATA[market-research]]></category>
            <category><![CDATA[surveys]]></category>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[science]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Sun, 15 Nov 2026 01:11:57 GMT</pubDate>
            <atom:updated>2026-11-15T01:11:57.125Z</atom:updated>
            <content:encoded><![CDATA[<p>Good UX research isn’t just about collecting data quickly. It’s about collecting <strong>trustworthy evidence that supports better product decisions.</strong></p><p>But even a well-designed survey can produce misleading findings. Problems can emerge at any stage, from defining what to measure and writing questions to recruiting participants, analyzing responses, and communicating results.</p><p><strong>The key question is: Can we trust these insights to make decisions?</strong></p><p>This article walks through six stages of survey research, highlighting common validity and reliability issues, why they matter, and how to prevent them.</p><h3>Stage 1: Define Research Objectives and Measurement Constructs</h3><p><strong>Key question: Are we measuring the right things?</strong></p><p>Before writing survey questions, researchers need to clarify their research objectives, define the concepts they want to measure, and determine which metrics best represent those concepts.</p><h3>Issue 1: Construct Validity — Measuring the Wrong Concept</h3><p>Construct validity concerns whether a measurement accurately represents the concept it is intended to measure.</p><p><strong>Common pitfalls:</strong></p><ul><li>Using satisfaction as a proxy for usability.</li><li>Assuming click-through rate (CTR) measures user understanding.</li><li>Treating task completion as evidence of an effortless experience.</li></ul><p>For example, a customer may successfully complete a banking transaction but still experience confusion, frustration, or unnecessary effort. Task completion alone does not capture the full experience.</p><p><strong>Best practices:</strong></p><ul><li>Clearly define each construct before selecting metrics.</li><li>Use validated instruments, such as SUS for perceived usability or NASA-TLX for perceived workload, when appropriate.</li><li>Combine self-reported measures with behavioral and operational data.</li></ul><h3>Issue 2: Content Validity — Missing Important Dimensions</h3><p>Content validity examines whether the questions adequately cover the relevant dimensions of a concept.</p><p><strong>Common pitfalls:</strong></p><ul><li>Measuring customer experience using only overall satisfaction.</li><li>Evaluating usability without asking about navigation, efficiency, or errors.</li><li>Focusing only on successful users while overlooking users who abandoned a task.</li></ul><p><strong>Best practices:</strong></p><ul><li>Break broad concepts into measurable dimensions.</li><li>Map survey questions to research objectives, user journeys, or Jobs to Be Done (JTBD).</li><li>Review whether important aspects of the experience are missing.</li></ul><p><strong>Example:</strong> To evaluate an online banking experience, researchers might measure task completion, ease of use, trust, perceived security, and overall satisfaction rather than relying on a single satisfaction question.</p><h3>Stage 2: Design and Validate the Questionnaire</h3><p><strong>Key question: Will respondents understand and answer the questions as intended?</strong></p><p>Even when research objectives are clear, poorly designed questions can introduce measurement errors.</p><h3>Issue 3: Face Validity — Questions That Don’t Appear to Measure the Intended Concept</h3><p>Face validity concerns whether a measure appears relevant and appropriate to respondents or reviewers. It is a useful initial check, but it does not establish that a measure is scientifically valid.</p><p><strong>Common pitfalls:</strong></p><ul><li>Using technical terminology unfamiliar to respondents.</li><li>Asking questions that appear unrelated to the research objective.</li><li>Using ambiguous wording.</li></ul><p><strong>Best practices:</strong></p><ul><li>Use clear, plain language.</li><li>Conduct cognitive interviews to understand how respondents interpret questions.</li><li>Pilot test questionnaires before launch.</li></ul><h3>Issue 4: Response Bias — Question Wording Influences Answers</h3><p>Poor questionnaire design can systematically influence how participants respond.</p><p><strong>Common pitfalls:</strong></p><ul><li>Leading questions: “How helpful was our improved dashboard?”</li><li>Double-barreled questions: “Was the application fast and easy to use?”</li><li>Social desirability bias: Respondents provide answers they believe are expected.</li><li>Acquiescence bias: Respondents tend to agree with statements regardless of content.</li></ul><p><strong>Best practices:</strong></p><ul><li>Use neutral, single-concept questions.</li><li>Provide balanced response options.</li><li>Randomize question or option order when appropriate.</li><li>Include “Not applicable” or “Don’t know” options where meaningful.</li></ul><p><strong>Example:</strong></p><p>Instead of asking, “How satisfied are you with our easy-to-use application?”</p><p>Ask, “Overall, how satisfied or dissatisfied are you with your experience using the application?”</p><h3>Stage 3: Sampling and Data Collection</h3><p><strong>Key question: Are we collecting data from the right people in realistic conditions?</strong></p><p>A perfectly written questionnaire can still produce misleading insights if the sample does not represent the intended population or the survey is administered in an inappropriate context.</p><h3>Issue 5: External Validity — Results Don’t Generalize</h3><p>External validity concerns whether findings apply beyond the participants and conditions studied.</p><p><strong>Common pitfalls:</strong></p><ul><li>Surveying only highly engaged customers.</li><li>Recruiting only internal employees to represent external customers.</li><li>Generalizing findings from a small or narrowly defined sample.</li><li>Ignoring nonresponse bias.</li></ul><p><strong>Best practices:</strong></p><ul><li>Define the target population and sampling frame.</li><li>Recruit across relevant user segments.</li><li>Compare respondent characteristics with the target population.</li><li>Apply weighting when justified and report its limitations.</li><li>Clearly communicate the population to which findings can reasonably generalize.</li></ul><p><strong>Example:</strong> If only 5% of invited customers respond, the results may still provide valuable insights. However, researchers should investigate whether respondents differ systematically from nonrespondents before generalizing the findings.</p><h3>Issue 6: Ecological Validity — Survey Conditions Don’t Reflect Real Experiences</h3><p>Ecological validity concerns how closely research conditions reflect the context of actual product use.</p><p><strong>Common pitfalls:</strong></p><ul><li>Asking users to recall experiences from several months earlier.</li><li>Surveying participants about features they have never used.</li><li>Evaluating realistic workflows through hypothetical scenarios alone.</li></ul><p><strong>Best practices:</strong></p><ul><li>Trigger surveys shortly after relevant interactions.</li><li>Use contextual or in-product surveys when appropriate.</li><li>Screen participants for relevant experience.</li><li>Complement survey findings with product telemetry or observational research.</li></ul><p><strong>Example:</strong> Asking customers about their experience immediately after completing a withdrawal may provide more contextually relevant feedback than asking them to recall the transaction three months later.</p><h3>Stage 4: Data Cleaning and Measurement Reliability</h3><p><strong>Key question: Are responses consistent, complete, and suitable for analysis?</strong></p><p>Before analyzing survey results, researchers should assess data quality and determine whether their measures are sufficiently reliable.</p><h3>Issue 7: Reliability — Inconsistent Measurement</h3><p>Reliability refers to the consistency of a measurement. A measure may be reliable without being valid, but unreliable measurement undermines confidence in its findings.</p><p><strong>Common pitfalls:</strong></p><ul><li>Changing question wording between survey waves.</li><li>Using inconsistent response scales.</li><li>Applying different coding rules across researchers.</li><li>Combining items that do not consistently measure the same construct.</li></ul><p><strong>Best practices:</strong></p><ul><li>Standardize questionnaires and coding procedures.</li><li>Maintain consistent wording and scales for longitudinal comparisons.</li><li>Evaluate internal consistency for appropriate multi-item scales.</li><li>Assess inter-rater agreement when human coding is involved.</li></ul><h3>Issue 8: Data Quality — Missing or Unusable Responses</h3><p>Poor-quality responses can distort estimates and introduce bias.</p><p><strong>Common pitfalls:</strong></p><ul><li>Duplicate submissions.</li><li>Straight-lining or inattentive responses.</li><li>High item nonresponse.</li><li>Excluding incomplete responses without examining potential bias.</li></ul><p><strong>Best practices:</strong></p><ul><li>Establish data-quality rules before analysis.</li><li>Examine missing-data patterns.</li><li>Investigate suspicious responses rather than automatically deleting them.</li><li>Document exclusion criteria and sample sizes.</li></ul><p><strong>Example:</strong> If dissatisfied customers are more likely to abandon a lengthy questionnaire, analyzing only completed surveys may overestimate satisfaction.</p><h3>Stage 5: Statistical Analysis and Interpretation</h3><p><strong>Key question: Do the data support the conclusions we’re making?</strong></p><p>Even high-quality survey data can lead to incorrect decisions when researchers apply inappropriate statistical methods or interpret associations as causal effects.</p><h3>Issue 9: Statistical Conclusion Validity — Misleading Statistical Results</h3><p>Statistical conclusion validity concerns whether the statistical evidence supports the conclusions drawn about relationships or differences.</p><p><strong>Common pitfalls:</strong></p><ul><li>Conducting underpowered studies.</li><li>Reporting only p-values without effect sizes.</li><li>Testing many hypotheses without accounting for multiple comparisons.</li><li>Using statistical tests that do not fit the data or research design.</li><li>Ignoring uncertainty in small samples.</li></ul><p><strong>Best practices:</strong></p><ul><li>Plan sample sizes and minimum detectable effects when hypothesis testing is intended.</li><li>Select statistical methods based on measurement scales and study design.</li><li>Report effect sizes and confidence intervals.</li><li>Account for multiple comparisons when appropriate.</li><li>Distinguish statistical significance from practical importance.</li></ul><p><strong>Example:</strong> A satisfaction increase from 4.0 to 4.1 may be statistically significant with a large sample, but researchers still need to determine whether the difference is meaningful for customers and the business.</p><h3>Issue 10: Internal Validity — Confusing Correlation With Causation</h3><p>Internal validity is particularly important when researchers claim that a product change caused an observed outcome.</p><p><strong>Common pitfalls:</strong></p><ul><li>Attributing higher satisfaction to a redesign without considering other changes.</li><li>Comparing pre- and post-launch results from different customer populations.</li><li>Ignoring seasonality, marketing campaigns, or concurrent releases.</li></ul><p><strong>Best practices:</strong></p><ul><li>Use randomized experiments or credible comparison groups when feasible.</li><li>Track relevant confounding variables.</li><li>Maintain comparable measurement procedures across periods.</li><li>Avoid causal claims when the study design supports only associations.</li></ul><p><strong>Example:</strong> Satisfaction increases by 12% after a product redesign. However, a support policy change and a seasonal shift in customer activity occurred during the same period.</p><p>Without an appropriate comparison, researchers cannot confidently attribute the improvement to the redesign alone.</p><h3>Stage 6: Reporting Findings and Influencing Decisions</h3><p><strong>Key question: Are we communicating findings accurately, including their limitations?</strong></p><p>Research validity does not end after statistical analysis. How findings are summarized, visualized, and interpreted can influence whether stakeholders make appropriate decisions.</p><h3>Issue 11: Overgeneralization — Making Claims Beyond the Evidence</h3><p><strong>Common pitfalls:</strong></p><ul><li>Claiming “customers prefer Feature A” when only a small subgroup was surveyed.</li><li>Presenting directional findings as definitive conclusions.</li><li>Ignoring differences across user segments.</li><li>Treating statistically nonsignificant results as proof that no difference exists.</li></ul><p><strong>Best practices:</strong></p><ul><li>Specify the population, sample size, and research context.</li><li>Distinguish observed findings from hypotheses.</li><li>Communicate uncertainty and important limitations.</li><li>Identify which findings require further validation.</li></ul><h3>Issue 12: Misleading Reporting — Presenting Numbers Without Context</h3><p><strong>Common pitfalls:</strong></p><ul><li>Reporting percentage changes without baseline values.</li><li>Presenting averages without sample sizes.</li><li>Highlighting favorable metrics while omitting contradictory evidence.</li><li>Using charts that exaggerate small differences.</li></ul><p><strong>Best practices:</strong></p><ul><li>Report baseline and follow-up values.</li><li>Include sample sizes, denominators, and uncertainty estimates where appropriate.</li><li>Explain whether changes are statistically and practically meaningful.</li><li>Connect findings to specific, evidence-supported recommendations.</li></ul><p><strong>Example:</strong></p><p>Instead of reporting:</p><p>“Customer satisfaction improved by 15%.”</p><p>Report:</p><p>“Average customer satisfaction increased from 3.40 to 3.91 on a five-point scale, a 15% relative increase. Further analysis is needed to determine whether the improvement can be attributed to the redesign.”</p><p>This provides stakeholders with a clearer understanding of the magnitude and meaning of the result.</p><h3>Quick Reference: Validity and Reliability Across the Survey Lifecycle</h3><p><strong>1. Research Objectives — Construct and Content Validity</strong></p><ul><li><strong>Main Issues:</strong> Construct and content validity</li><li><strong>Key Question:</strong> Are we measuring the right concepts and dimensions?</li></ul><p><strong>2. Questionnaire Design — Face Validity and Response Bias</strong></p><ul><li><strong>Main Issues:</strong> Face validity and response bias</li><li><strong>Key Question:</strong> Are questions clear, understandable, and unbiased?</li></ul><p><strong>3. Sampling &amp; Collection — External and Ecological Validity</strong></p><ul><li><strong>Main Issues:</strong> External and ecological validity</li><li><strong>Key Question:</strong> Are respondents representative, and is the research context realistic?</li></ul><p><strong>4. Data Quality — Reliability and Missing-Data Bias</strong></p><ul><li><strong>Main Issues:</strong> Reliability and missing-data bias</li><li><strong>Key Question:</strong> Are measurements consistent and responses usable?</li></ul><p><strong>5. Analysis — Statistical Conclusion and Internal Validity</strong></p><ul><li><strong>Main Issues:</strong> Statistical conclusion and internal validity</li><li><strong>Key Question:</strong> Are statistical conclusions justified, and can we make causal claims?</li></ul><p><strong>6. Reporting — Overgeneralization and Misleading Interpretation</strong></p><ul><li><strong>Main Issues:</strong> Overgeneralization and misleading interpretation</li><li><strong>Key Question:</strong> Are findings and limitations communicated accurately?</li></ul><p>These issues are not exclusive to individual stages. For example, construct validity begins with measurement planning but can also be affected by questionnaire wording, while external validity depends on sampling decisions and how findings are reported.</p><h3>Final Takeaway: Validity Is a Continuous Research Responsibility</h3><p>Strong survey research requires more than selecting the right statistical test or collecting a large number of responses.</p><p>Validity and reliability need to be considered throughout the research lifecycle, from defining objectives and designing questionnaires to collecting data, analyzing results, and communicating recommendations.</p><p><strong>The goal is not to eliminate every possible source of bias. It is to identify potential threats, reduce them where possible, and communicate remaining uncertainty transparently.</strong></p><p>When leadership asks, “How confident are we in these findings?” researchers should be able to explain not only what the data show, but also how the study was designed, what limitations remain, and which decisions the evidence can reasonably support.</p><p>That’s how survey data becomes trustworthy evidence for product decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26f3442a9fb5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Influence Change When People Resist: 7 Practical Strategies for Driving Adoption]]></title>
            <link>https://janajiaruchung.medium.com/how-to-influence-change-when-people-resist-7-practical-strategies-for-driving-adoption-ce375fd5ebd2?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce375fd5ebd2</guid>
            <category><![CDATA[change-management]]></category>
            <category><![CDATA[ux-research]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Mon, 09 Nov 2026 22:43:29 GMT</pubDate>
            <atom:updated>2026-11-09T22:43:29.636Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Change doesn’t fail simply because people resist it. It often fails because we underestimate what it takes for people to adopt something new.</em></p><p>Organizations constantly introduce new technologies, workflows, policies, and processes to improve efficiency and performance. Yet even when a proposed change is supported by research and data, implementation can be surprisingly difficult.</p><p>Stakeholders may disagree with the findings. Employees may prefer existing workflows. Leadership may support an initiative but hesitate to allocate resources. And even when a new solution launches successfully, people may continue using the old one.</p><p><strong>The challenge isn’t just convincing people that change is necessary. It’s helping them understand, participate in, and sustain that change.</strong></p><p>Here are seven practical strategies for influencing stakeholders, overcoming resistance, and driving meaningful adoption.</p><h3>1. Understand the Resistance Before Trying to Overcome It</h3><p>Resistance is often treated as an obstacle, but it can be valuable information.</p><p>People rarely resist change without a reason. They may worry about losing control, increasing their workload, disrupting established routines, or being held accountable for unfamiliar outcomes.</p><p>Before proposing solutions, ask:</p><ul><li>What are stakeholders concerned about losing?</li><li>What incentives encourage them to maintain the status quo?</li><li>What risks do they perceive that we haven’t addressed?</li><li>What would make the proposed change easier to accept?</li></ul><p>For example, imagine introducing a new analytics dashboard to replace manual reporting.</p><p>Leadership may welcome faster reporting, while analysts resist because they distrust automated calculations or fear losing flexibility.</p><p>Rather than insisting on adoption, involve analysts in validating the dashboard’s accuracy and identifying exceptions.</p><p><strong>Practical tip:</strong> Treat resistance as a discovery opportunity. Use stakeholder interviews, workflow observations, and open discussions to uncover the real barriers.</p><h3>2. Connect the Change to Outcomes People Care About</h3><p>One of the biggest mistakes in change management is explaining what needs to change without explaining why it matters.</p><p>Different stakeholders care about different outcomes.</p><p>Executives may prioritize business growth, cost reduction, and risk management. Product managers may focus on adoption and customer retention. Employees may care about workload, efficiency, and job performance.</p><p>The same initiative should be communicated differently depending on the audience.</p><p>Instead of saying:</p><p>“We need to implement a new customer feedback system.”</p><p>Try:</p><p>“Our current feedback process makes it difficult to identify recurring customer issues. A centralized system could help us detect problems earlier, prioritize improvements, and reduce repeated support contacts.”</p><p>The second statement connects the proposed solution to a recognizable problem and a measurable outcome.</p><p><strong>Practical tip:</strong> Frame your recommendation using three elements: the existing problem, the cost of maintaining the status quo, and the expected value of change.</p><h3>3. Involve Stakeholders Early to Build Ownership</h3><p>People are more likely to support changes they helped shape.</p><p>When stakeholders are involved only after decisions have been finalized, they may feel that the change is being imposed on them.</p><p>Instead, engage them during problem definition, solution development, and implementation planning.</p><p>For example, when redesigning an internal enterprise workflow, invite employees to identify pain points, evaluate prototypes, and help prioritize improvements.</p><p>This creates two benefits: better solutions informed by real operational constraints and stronger stakeholder ownership.</p><p>However, participation should be meaningful. Asking for feedback without being willing to act on it can damage trust.</p><p><strong>Practical tip:</strong> Establish a small group of representatives from affected teams. Give them a clear role in evaluating alternatives, identifying risks, and shaping the rollout.</p><h3>4. Use Evidence to Influence, Not Just to Prove You’re Right</h3><p>Data is powerful, but presenting more data does not automatically create agreement.</p><p>A research report showing low task completion or poor usability may demonstrate that a problem exists. It does not necessarily convince stakeholders that a proposed solution is worth prioritizing.</p><p>To influence decisions, connect evidence to business impact.</p><p>For example:</p><p><strong>Research finding:</strong> Users struggle to locate the correct policy settings.</p><p><strong>Operational implication:</strong> Users spend more time completing tasks and may make configuration errors.</p><p><strong>Recommended action:</strong> Improve navigation and test whether the redesign increases task success and reduces completion time.</p><p>Evidence becomes more persuasive when stakeholders understand its implications for their goals.</p><p>Also, be transparent about uncertainty. Distinguish observed findings from hypotheses and projected benefits.</p><p><strong>Practical tip:</strong> Present evidence as a decision-making tool, not simply a collection of research findings.</p><h3>5. Start Small and Demonstrate Quick Wins</h3><p>Large-scale transformations can create uncertainty because they require substantial investment before results become visible.</p><p>A pilot program reduces that uncertainty.</p><p>Rather than rolling out a new process across an entire organization, begin with one team, one workflow, or one clearly defined use case.</p><p>For example, if you want to establish a continuous customer experience measurement program, start with a high-volume customer journey.</p><p>Measure baseline satisfaction and task completion, introduce the improvement, and evaluate the results.</p><p>If the pilot demonstrates value, use the findings to build support for expansion.</p><p>Quick wins are especially useful when stakeholders are skeptical about implementation costs or operational disruption.</p><p><strong>Practical tip:</strong> Choose pilots that are small enough to manage but important enough to demonstrate meaningful business value.</p><h3>6. Reduce the Effort Required to Adopt the Change</h3><p>Even when people agree that a new process is better, they may not adopt it.</p><p>Why?</p><p>Because agreement and behavior change are different things.</p><p>Adoption requires people to learn new skills, adjust routines, and sometimes temporarily become less efficient.</p><p>Successful change management therefore focuses on reducing friction.</p><p>This might include integrating new tools into existing workflows, providing practical training, offering clear documentation, or identifying internal champions who can support colleagues.</p><p>For instance, introducing a new research insights repository will not automatically encourage product teams to use it.</p><p>Adoption becomes easier when insights are searchable, linked to product decisions, and incorporated into existing planning meetings.</p><p><strong>Practical tip:</strong> Make the desired behavior easier than the old behavior whenever possible. Don’t rely exclusively on communication or training.</p><h3>7. Measure Adoption, Not Just Implementation</h3><p>A project can be delivered on time and still fail to create meaningful change.</p><p>Launching a new tool, completing training, or publishing a policy doesn’t necessarily mean that employees have adopted it.</p><p>To evaluate change, measure three levels:</p><ul><li><strong>Implementation:</strong> Was the change delivered? Track rollout completion and training coverage.</li><li><strong>Adoption:</strong> Are people actually using it? Track active usage, workflow completion, and repeat usage.</li><li><strong>Impact:</strong> Is the change making a difference? Track time saved, error reduction, satisfaction, and business outcomes.</li></ul><p>For example, a new internal workflow might achieve 95% training completion but only 40% active adoption.</p><p>That gap suggests the problem is no longer awareness. It may involve usability, incentives, technical barriers, or workflow compatibility.</p><p>Track these metrics over time, segment results by relevant user groups, and combine quantitative trends with qualitative feedback to understand why adoption succeeds or stalls.</p><p><strong>Practical tip:</strong> Define adoption and success metrics before implementation, then establish a regular feedback loop to identify barriers and adjust the rollout.</p><h3>A Practical Framework: Listen, Align, Pilot, Enable, Measure</h3><p>When implementing change, I find it useful to think about five connected activities:</p><p><strong>Listen:</strong> Understand stakeholder concerns, motivations, and workflow constraints.</p><p><strong>Align:</strong> Connect the proposed change to shared priorities and measurable outcomes.</p><p><strong>Pilot:</strong> Test the change on a manageable scale and demonstrate its value.</p><p><strong>Enable:</strong> Remove adoption barriers through training, workflow integration, and ongoing support.</p><p><strong>Measure:</strong> Monitor adoption and impact, gather feedback, and continuously improve.</p><p>This framework is particularly useful when you need to influence decisions without having direct authority over the teams responsible for implementation.</p><h3>Final Thoughts: Influence Is Built Through Trust, Not Authority</h3><p>Effective change management is not about winning arguments or convincing everyone to agree immediately.</p><p>It is about understanding different perspectives, creating shared goals, reducing uncertainty, and demonstrating measurable value.</p><p>Resistance does not always mean people are unwilling to change. Sometimes it means the proposed change has not adequately addressed their needs, risks, or operational realities.</p><p>The strongest change leaders don’t simply deliver recommendations. They build the relationships, evidence, processes, and feedback mechanisms that make adoption possible.</p><p><strong>Because meaningful change isn’t achieved when a recommendation is approved. It’s achieved when people adopt a better way of working — and the improvement lasts.</strong></p><p><strong>Medium tags:</strong> Change Management, Leadership, Stakeholder Management, Organizational Change, Product Management</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce375fd5ebd2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What You Must Care About When Doing B2B Enterprise User Research]]></title>
            <link>https://janajiaruchung.medium.com/what-you-must-care-about-when-doing-b2b-enterprise-user-research-6507b5a07936?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/6507b5a07936</guid>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[enterprise]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Thu, 05 Feb 2026 19:30:07 GMT</pubDate>
            <atom:updated>2026-02-05T19:30:07.019Z</atom:updated>
            <content:encoded><![CDATA[<p><em>(And Why Consumer UX Thinking Often Fails Here)</em></p><p>B2B enterprise user research is not “consumer UX, but bigger.”</p><p>It’s a fundamentally different problem space — one where <strong>users aren’t buyers</strong>, <strong>success isn’t delight</strong>, and <strong>a single friction point can cost millions</strong>.</p><p>After working across complex systems — AI tools, internal platforms, security, cloud, healthcare, and enterprise productivity — I’ve learned that great enterprise research requires a different mindset, different methods, and different measures of success.</p><p>Here’s what truly matters.</p><h3>1. Users ≠ Buyers ≠ Decision-Makers</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NnGGqAbzgYw3WvY1.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*NBESAzzSuN8P45Fp" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Rj-QP96KiuJtVBg3" /></figure><p>One of the fastest ways enterprise research fails is by focusing on “the user” as a single persona.</p><p>In reality, enterprise products live inside <strong>a system of roles</strong>:</p><ul><li><strong>End users</strong> who operate the tool daily</li><li><strong>Economic buyers</strong> who control budgets</li><li><strong>Technical buyers</strong> who care about security, scalability, and integration</li><li><strong>Compliance, legal, and finance</strong> who can veto decisions</li><li><strong>Champions</strong> who advocate internally</li></ul><p>Each role evaluates the product differently — and friction for <em>any</em> of them can stall adoption.</p><p><strong>Enterprise research must ask:</strong></p><ul><li>Who lives with this product vs who approves it?</li><li>Whose metrics define success?</li><li>Whose pain can quietly kill adoption?</li></ul><blockquote><em>In enterprise UX, insight often lives in the tension </em>between<em> stakeholders.</em></blockquote><h3>2. The Real Job-to-Be-Done Is Organizational, Not Just Individual</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*C41aghEbl2lqsv5w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*349_kd1ggP6djlq7.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Rr7EnwbptEE1iqYB.jpg" /></figure><p>Enterprise users don’t use tools for fun or convenience — they use them to <strong>reduce risk, make decisions, and unblock work</strong>.</p><p>That means:</p><ul><li>Edge cases matter more than happy paths</li><li>Exception handling matters more than speed</li><li>Hand-offs between teams matter more than micro-interactions</li></ul><p>Key questions enterprise research should surface:</p><ul><li>What decisions does this tool support?</li><li>What happens if it fails?</li><li>What workarounds existed before?</li><li>Where do humans override the system — and why?</li></ul><blockquote><em>In enterprise UX, supporting a defensible decision often matters more than completing a task.</em></blockquote><h3>3. Trust, Reliability, and Interpretability Are Core UX Requirements</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*3ShESqNML_uVav_B" /></figure><p>Enterprise users don’t ask, <em>“Is this delightful?”</em><br> They ask, <em>“Can I trust this — and defend it?”</em></p><p>High-impact enterprise UX supports:</p><ul><li>Explainability (“Why did the system do this?”)</li><li>Traceability (logs, history, audit trails)</li><li>Confidence under scrutiny (managers, auditors, customers)</li></ul><p>This is especially critical in:</p><ul><li>AI-powered systems</li><li>Security, finance, healthcare, and compliance products</li></ul><p>If a user can’t explain the system’s output to leadership, <strong>they won’t use it — no matter how usable it is</strong>.</p><h3>4. Context Matters More Than Screens</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-aBnNfIMAVRmBqb2.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DI3WD-Ebj5mP29Dx.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*c_b1dFAi6208L_nq.jpg" /></figure><p>Enterprise work rarely happens in a clean, single-screen environment.</p><p>It happens:</p><ul><li>Across multiple monitors</li><li>While juggling email, chat, tickets, dashboards</li><li>Under time pressure</li><li>With partial attention</li></ul><p>This is why enterprise research favors:</p><ul><li>Contextual inquiry (remote or in-person)</li><li>Shadowing and workflow walkthroughs</li><li>Artifact reviews (spreadsheets, tickets, dashboards)</li><li>Diary studies for long or episodic work</li><li>Telemetry and logs paired with interviews</li></ul><blockquote><em>Screen-only usability testing often misses the real work.</em></blockquote><h3>5. Measure What Leadership Actually Cares About</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PFnvX2U4espS9MEk.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*1EYZBaEaMcqA6tgd.png" /></figure><p>Enterprise success is not DAUs or engagement streaks.</p><p>Leadership cares about:</p><ul><li>Time saved or errors reduced</li><li>Risk exposure lowered</li><li>Productivity per role</li><li>Training time</li><li>Support ticket deflection</li><li>Retention and expansion readiness</li></ul><p>Strong enterprise research connects:</p><ul><li><strong>UX metrics</strong> (task success, confidence, SUS)</li><li><strong>Business outcomes</strong> (cost, efficiency, risk)</li></ul><blockquote><em>The most powerful enterprise research ties usability directly to money or risk.</em></blockquote><h3>6. Constraints Are Real — Design Around Them</h3><p>Enterprise products come with immovable constraints:</p><ul><li>Legacy systems</li><li>Security and compliance rules</li><li>Long procurement cycles</li><li>Role-based permissions</li><li>Limited iteration velocity</li></ul><p>Instead of fighting these, effective researchers:</p><ul><li>Identify <strong>high-leverage improvements</strong></li><li>Document <strong>trade-offs clearly</strong></li><li>Provide <strong>decision frameworks</strong>, not just findings</li></ul><blockquote><em>Enterprise leaders don’t expect perfection — they expect clarity under constraints.</em></blockquote><h3>7. Research Is Influence, Not Just Insight</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kSsdoowYklvkzvJz.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/0*XcZeM5afWb0R6E-i.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*gHosfczaxkqSkJbP" /></figure><p>Enterprise research must land differently for different audiences:</p><ul><li>Designers → actionable design implications</li><li>PMs → prioritization and roadmap alignment</li><li>Executives → risk, cost, growth, and confidence</li><li>Sales &amp; CS → enablement and positioning</li></ul><p>Strong enterprise deliverables:</p><ul><li>Are role-aware</li><li>Highlight trade-offs</li><li>Explain consequences of inaction</li><li>Align recommendations to near-term roadmaps</li></ul><blockquote><em>In enterprise environments, research only matters if it moves decisions.</em></blockquote><h3>Final Takeaway</h3><p>B2B enterprise UX research isn’t about polishing interfaces.</p><p>It’s about:</p><ul><li>Supporting real decisions</li><li>Reducing organizational risk</li><li>Navigating constraints</li><li>Influencing complex systems</li><li>Making work defensible, reliable, and scalable</li></ul><p>If consumer UX is about <strong>delight</strong>, enterprise UX is about <strong>confidence</strong>.</p><p>And great enterprise researchers know the difference.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6507b5a07936" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Common SQL & Python Snippets Every UX Researcher Should Know (When & Why to Use Them)]]></title>
            <link>https://janajiaruchung.medium.com/common-sql-python-snippets-every-ux-researcher-should-know-when-why-to-use-them-f3241c8eed19?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/f3241c8eed19</guid>
            <category><![CDATA[quant-ux]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[analytics]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Thu, 29 Jan 2026 03:33:05 GMT</pubDate>
            <atom:updated>2026-01-29T03:33:05.888Z</atom:updated>
            <content:encoded><![CDATA[<p>As UX Research evolves, the role increasingly sits at the intersection of <strong>user behavior, data, and decision-making</strong>. Today’s UX Researchers — especially Quant and Mixed-Methods UXR — are expected not just to <em>collect</em> data, but to <strong>analyze it, interpret it, and translate it into actionable insights</strong> for product, design, and leadership teams.</p><p>You don’t need to be a data scientist — but you <em>do</em> need to be fluent in a small, high-impact set of SQL and Python patterns.</p><p>This article covers the <strong>most common SQL and Python snippets a UXR needs to know</strong>, along with <strong>when and why</strong> you’d use them in real research scenarios.</p><h3>Why SQL and Python Matter for UXR</h3><p>SQL and Python enable UXRs to:</p><ul><li>Analyze <strong>behavioral logs</strong> alongside surveys and interviews</li><li>Validate qualitative insights with <strong>quantitative evidence</strong></li><li>Answer ad-hoc stakeholder questions quickly and confidently</li><li>Own the full research workflow, from raw data to insight</li></ul><p>In interviews, these skills signal that you can move from <strong>insight → impact</strong>, not just reporting.</p><h3>SQL Essentials for UX Research</h3><p>SQL is your tool for <strong>extracting and structuring behavioral data</strong>.</p><h3>1. Basic Aggregations (Engagement &amp; Frequency)</h3><pre>SELECT user_id,<br>       COUNT(*) AS event_count<br>FROM user_events<br>GROUP BY user_id;</pre><p><strong>When to use</strong></p><ul><li>Measuring engagement</li><li>Identifying heavy vs light users</li><li>Understanding task frequency</li></ul><p><strong>Why it matters</strong><br> This is often the first step in answering questions like:</p><blockquote><em>“How often are users actually doing this?”</em></blockquote><h3>2. CASE WHEN for UX Metrics</h3><pre>AVG(CASE WHEN completed = 1 THEN 1 ELSE 0 END) AS completion_rate</pre><p><strong>When to use</strong></p><ul><li>Task completion rate</li><li>Conversion rate</li><li>Error rate</li></ul><p><strong>Why it matters</strong><br> This converts raw logs into <strong>UX metrics leadership understands</strong>.</p><h3>3. Window Functions (Critical for UXR)</h3><p>Window functions let you analyze behavior <strong>without collapsing rows</strong>, which is essential for user journeys.</p><h3>Example: Previous User Action</h3><pre>LAG(event_name)<br>OVER (<br>  PARTITION BY user_id<br>  ORDER BY event_time<br>) AS previous_event</pre><p><strong>When to use</strong></p><ul><li>Funnels</li><li>Feature adoption</li><li>Career or lifecycle transition</li></ul><p><strong>Why it matters</strong><br> UX is temporal. Window functions preserve <strong>sequence and context</strong>.</p><h3>4. Top-N per Group (Prioritization)</h3><pre>WITH ranked AS (<br>  SELECT pain_point,<br>         COUNT(*) AS mentions,<br>         ROW_NUMBER() OVER (<br>           PARTITION BY persona<br>           ORDER BY COUNT(*) DESC<br>         ) AS rn<br>  FROM feedback<br>  GROUP BY persona, pain_point<br>)<br>SELECT *<br>FROM ranked<br>WHERE rn &lt;= 3;</pre><p><strong>When to use</strong></p><ul><li>Top pain points per persona</li><li>Top tasks per segment</li></ul><p><strong>Why it matters</strong><br> This directly supports <strong>roadmap and prioritization decisions</strong>.</p><h3>5. Deduplication / Latest Record</h3><pre>SELECT *<br>FROM events e<br>JOIN (<br>  SELECT user_id, MAX(event_time) AS latest<br>  FROM events<br>  GROUP BY user_id<br>) m<br>ON e.user_id = m.user_id<br>AND e.event_time = m.latest;</pre><p><strong>When to use</strong></p><ul><li>Latest survey response</li><li>Most recent session</li><li>Current user state</li></ul><p><strong>Why it matters</strong><br> Dirty or duplicated data can invalidate research conclusions.</p><h3>Python Essentials for UX Research</h3><p>Python is where you <strong>clean, analyze, and explain</strong> data.</p><h3>1. Inspecting Survey Data</h3><pre>df.head()<br>df.info()<br>df.describe()</pre><p><strong>When to use</strong></p><ul><li>First step with any dataset</li></ul><p><strong>Why it matters</strong><br> Interviewers love hearing:</p><blockquote><em>“I always inspect missingness, ranges, and types first.”</em></blockquote><h3>2. Data Cleaning (High-Signal Skill)</h3><pre>df = df.dropna(subset=[&quot;user_id&quot;])<br>df[&quot;csat&quot;] = df[&quot;csat&quot;].clip(1, 5)</pre><p><strong>When to use</strong></p><ul><li>Surveys</li><li>Behavioral exports</li><li>Human evaluations</li></ul><p><strong>Why it matters</strong><br> UX insights are only as good as the <strong>data quality</strong>.</p><h3>3. Computing UX Metrics</h3><pre>df.groupby(&quot;variant&quot;)[&quot;csat&quot;].mean()<br>df[&quot;task_completed&quot;].mean()</pre><p><strong>When to use</strong></p><ul><li>A/B tests</li><li>Feature comparisons</li><li>Experience benchmarking</li></ul><p><strong>Why it matters</strong><br> This bridges research and experimentation.</p><h3>4. Statistical Tests (Explain the Why, Not Just the Code)</h3><h3>T-Test (A/B Testing)</h3><pre>from scipy.stats import ttest_ind</pre><pre>ttest_ind(<br>    df[df.variant==&quot;A&quot;].csat,<br>    df[df.variant==&quot;B&quot;].csat<br>)</pre><p><strong>When to use</strong></p><ul><li>Comparing means between variants</li></ul><p><strong>Why it matters</strong><br> UXRs use stats to <strong>support decisions</strong>, not chase p-values.</p><h3>Chi-Square (Categorical Outcomes)</h3><pre>from scipy.stats import chi2_contingency</pre><pre>chi2_contingency(pd.crosstab(df.variant, df.completed))</pre><p><strong>When to use</strong></p><ul><li>Completion vs non-completion</li><li>Error vs no error</li></ul><h3>5. Regression for Driver Analysis</h3><pre>import statsmodels.api as sm</pre><pre>X = df[[&quot;ease_of_use&quot;, &quot;speed&quot;]]<br>y = df[&quot;csat&quot;]<br>X = sm.add_constant(X)<br>model = sm.OLS(y, X).fit()</pre><p><strong>When to use</strong></p><ul><li>Identifying what drives satisfaction</li><li>Prioritizing design improvements</li></ul><p><strong>Why it matters</strong><br> Regression helps answer:</p><blockquote><em>“What should we fix first?”</em></blockquote><h3>6. Qual + Quant Triangulation</h3><pre>df.groupby(&quot;theme&quot;)[&quot;csat&quot;].mean()</pre><p><strong>When to use</strong></p><ul><li>Linking interview themes to metrics</li></ul><p><strong>Why it matters</strong><br> This is where UX Research truly differentiates itself.</p><h3>What UXRs Don’t Need to Over-Index On</h3><p>You <em>don’t</em> need:</p><ul><li>Deep machine learning</li><li>LeetCode-style SQL puzzles</li><li>Complex optimization algorithms</li></ul><p>You <em>do</em> need:</p><ul><li>Clarity</li><li>Rigor</li><li>Business relevance</li></ul><h3>Final Takeaway</h3><p>The strongest UX Researchers use SQL and Python not to show technical prowess, but to <strong>tell clearer, more convincing stories about users</strong>.</p><p>If you can:</p><ul><li>Extract behavioral data</li><li>Clean and analyze it responsibly</li><li>Explain what it means — and why it matters —</li></ul><p>You are operating at a <strong>senior UXR level</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f3241c8eed19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Idea to Beta: What UX Researchers Should Really Know About Product Testing Stages]]></title>
            <link>https://janajiaruchung.medium.com/from-idea-to-beta-what-ux-researchers-should-really-know-about-product-testing-stages-9d9e6db61f0c?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/9d9e6db61f0c</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[ux-research]]></category>
            <category><![CDATA[ux]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Mon, 19 Jan 2026 17:42:42 GMT</pubDate>
            <atom:updated>2026-01-19T17:43:20.894Z</atom:updated>
            <content:encoded><![CDATA[<p>In product teams, the terms <em>Alpha</em> and <em>Beta</em> testing get thrown around constantly. But many UX researchers — especially those moving between startups, AI products, and enterprise environments — see these stages misunderstood or misapplied.</p><p>Even worse, teams often think testing stage is defined by <strong>sample size</strong>, when in reality it’s defined by <strong>product maturity, risk level, and who is testing</strong>.</p><p>This article breaks down:</p><ul><li>What Alpha and Beta testing actually mean</li><li>Why stage ≠ number of users</li><li>What realistically happens between conception and Beta</li><li>How long it usually takes to get there</li></ul><h3>🧪 Alpha Testing: Internal Reality Check</h3><p>Alpha testing is the first time a product that is <em>almost</em> complete is tested in realistic conditions — internally.</p><p><strong>Who participates</strong></p><ul><li>Employees</li><li>Product + QA teams</li><li>Friendly internal testers</li></ul><p><strong>Primary goals</strong></p><ul><li>Catch critical bugs</li><li>Validate core flows work</li><li>Identify blockers that would break external use</li><li>Stress unfinished features</li></ul><p><strong>Environment</strong></p><ul><li>Controlled</li><li>Pre-production</li><li>Often unstable</li></ul><p><strong>Key mindset</strong></p><blockquote><em>Can this product survive real usage without breaking?</em></blockquote><p>Alpha is not about scale, metrics, or satisfaction.<br> It’s about <strong>stability and survivability</strong>.</p><p>Typical size: <strong>10–50 testers</strong> (occasionally up to 100)</p><h3>🌍 Beta Testing: Real-World Validation</h3><p>Beta begins once the product is stable enough to leave internal walls.</p><p>This is where UX researchers become essential.</p><p><strong>Who participates</strong></p><ul><li>Real users</li><li>Target segments</li><li>Invite-only customers or public users</li></ul><p><strong>Primary goals</strong></p><ul><li>Validate product-market fit</li><li>Measure usability &amp; satisfaction</li><li>Discover edge cases</li><li>Observe natural behavior</li><li>Identify missing value</li></ul><p><strong>Environment</strong></p><ul><li>Real-world devices &amp; networks</li><li>Uncontrolled usage contexts</li></ul><p><strong>Key mindset</strong></p><blockquote><em>Does this product work for real people in their daily lives?</em></blockquote><p>Typical size: <strong>200–500 users</strong> for meaningful signal</p><h3>🚫 The Common Myth: “400 Users Means Alpha”</h3><p>Wrong.</p><p>Testing stage is NOT defined by the number of users.<br> It’s defined by:</p><ul><li>Product maturity</li><li>Risk tolerance</li><li>Tester type (internal vs external)</li></ul><p>Here’s the real rule:</p><p>UsersStage&lt;50Alpha50–150Late Alpha / Pilot Beta200–500Beta ✅1000+Public Beta / Soft Launch</p><p>If you’re running a study with 400 external users, you are in <strong>Beta</strong>, even if the product feels early.</p><h3>⏳ How Long Does It Take to Get From Idea → Beta?</h3><p>This is where many researchers underestimate product reality.</p><p>Beta doesn’t happen right after prototyping.<br> There’s a structured maturity path before a product is safe for real users.</p><h3>Phase 1 — Ideation &amp; Validation (2–4 weeks)</h3><ul><li>Define the problem</li><li>Run early user interviews</li><li>Identify JTBD</li><li>Narrow MVP scope</li></ul><h3>Phase 2 — MVP Development (6–16 weeks)</h3><ul><li>Build core flows</li><li>Backend + data structure</li><li>Feature wiring</li><li>AI integration (if applicable)</li><li>Internal usability checks</li></ul><h3>Phase 3 — Alpha Testing (2–4 weeks)</h3><ul><li>Internal dogfooding</li><li>Crash fixes</li><li>Flow validation</li><li>Remove blockers</li></ul><h3>Phase 4 — Beta Preparation (2–4 weeks)</h3><ul><li>Analytics instrumentation</li><li>Scaling infrastructure</li><li>Recruit external users</li><li>Feedback systems</li></ul><h3>📊 Typical Time to Reach Beta</h3><p>Product TypeTime to BetaSimple web/mobile app2–4 monthsAI-powered product4–8 monthsMarketplace product6–12 monthsHardware + software12–24 months</p><h3>💡 What Actually Slows Teams Down (Not Engineering)</h3><p>From a UX research lens, most delays before Beta come from:</p><ul><li>Poor problem definition</li><li>Scope creep</li><li>Weak onboarding</li><li>Missing analytics</li><li>Recruiting the wrong users</li><li>Rebuilding after usability failures</li></ul><p>In other words:</p><blockquote><em>Products rarely fail because they’re not built.<br> They fail because they’re built without validated user reality.</em></blockquote><h3>🎯 Why This Matters for UX Researchers</h3><p>Understanding Alpha vs Beta helps you:</p><ul><li>Push back when teams want large-scale testing too early</li><li>Frame realistic timelines in roadmaps</li><li>Design studies aligned to product maturity</li><li>Avoid mislabeling evaluative research as “Alpha”</li><li>Set stakeholder expectations on risk and data quality</li></ul><p>When researchers align testing stage with product maturity instead of arbitrary numbers, we move from being <em>test executors</em> to true <strong>product risk strategists</strong>.</p><h3>Simple Takeaway</h3><ul><li>Alpha = Internal stability check</li><li>Beta = External real-world validation</li><li>400 users = Beta</li><li>Expect ~4–6 months minimum from idea to Beta for AI/mobile products</li></ul><p>Knowing this isn’t just terminology — it’s product literacy.</p><p>Also add that besides alpha beta what testing stages are</p><p>Love that addition — here’s an expanded Medium-ready section appended naturally to the article 👇</p><h3>🔁 Beyond Alpha and Beta: Other Product Testing Stages UX Researchers Should Know</h3><p>Alpha and Beta are only part of a much bigger product maturity pipeline.<br> Strong UX researchers understand the <em>full lifecycle</em> because each stage requires different methods, risk levels, and research goals.</p><p>Here’s the broader testing spectrum used in real product development:</p><h3>1. 💡 Concept Testing (Pre-Product)</h3><p><strong>When</strong>: Idea stage, before anything is built<br> <strong>Goal</strong>: Validate problem relevance &amp; value<br> <strong>Methods</strong>:</p><ul><li>Interviews</li><li>JTBD exploration</li><li>Fake door tests</li><li>Value proposition testing</li></ul><blockquote><em>Are we solving a real problem users care about?</em></blockquote><h3>2. 🎨 Prototype Testing</h3><p><strong>When</strong>: Early design phase<br> <strong>Goal</strong>: Validate flows, IA, mental models<br> <strong>Methods</strong>:</p><ul><li>Usability testing (low/high fidelity)</li><li>Card sorting</li><li>Task walkthroughs</li></ul><blockquote><em>Can users understand and navigate the idea?</em></blockquote><h3>3. 🧪 Alpha Testing (Internal)</h3><p>(covered earlier)</p><blockquote><em>Does the product function without breaking?</em></blockquote><h3>4. 🌍 Beta Testing (External)</h3><p>(covered earlier)</p><blockquote><em>Does it work in the real world for real users?</em></blockquote><h3>5. 🚀 Soft Launch / Pilot Release</h3><p><strong>When</strong>: Limited real market exposure<br> <strong>Goal</strong>:</p><ul><li>Validate performance at scale</li><li>Monitor retention</li><li>Catch operational risks<br> <strong>Methods</strong>:</li><li>Behavioral analytics</li><li>Funnel analysis</li><li>Early VoC</li></ul><blockquote><em>Does the product sustain real usage under live conditions?</em></blockquote><h3>6. 📈 General Availability (GA)</h3><p><strong>When</strong>: Official public release<br> <strong>Goal</strong>:</p><ul><li>Monitor satisfaction</li><li>Optimize features</li><li>Improve engagement<br> <strong>Methods</strong>:</li><li>Surveys</li><li>Continuous usability</li><li>Experimentation</li><li>Segmentation</li></ul><blockquote><em>How do we optimize and grow from here?</em></blockquote><h3>7. 🔄 Post-Launch Optimization / Iterative Testing</h3><p><strong>When</strong>: Ongoing<br> <strong>Goal</strong>:</p><ul><li>Improve adoption</li><li>Reduce friction</li><li>Expand features<br> <strong>Methods</strong>:</li><li>A/B testing</li><li>Longitudinal research</li><li>Advanced analytics</li><li>Behavioral segmentation</li></ul><blockquote><em>How do we continuously improve and scale?</em></blockquote><h3>🧭 Why This Expanded View Matters</h3><p>UX researchers often get pulled in too late (Beta) or misused too early (Alpha with external users). Knowing all stages helps you:</p><ul><li>Match method to maturity</li><li>Protect users from unstable builds</li><li>Frame research strategically</li><li>Influence product timing</li><li>Speak the language of PM &amp; Engineering</li></ul><p>The best researchers don’t just test usability — <br> they guide <strong>product readiness across every stage of maturity</strong>.</p><p>incorporate all these into an article</p><p>Here’s the fully integrated Medium-style article with all sections combined and polished 👇</p><h3>From Idea to Beta: A Practical Guide to Product Testing Stages for UX Researchers</h3><p>In product teams, the terms <em>Alpha</em> and <em>Beta</em> testing are everywhere.<br> Yet many UX researchers see them misunderstood, misused, or incorrectly tied to sample size instead of product maturity.</p><p>Even experienced teams often assume:</p><blockquote><em>“If we test with hundreds of users, it must still be Alpha.”</em></blockquote><p>Not true.</p><p>Testing stages are defined by <strong>product readiness, risk level, and tester type</strong> — not the number of participants.</p><p>This guide breaks down:</p><ul><li>What Alpha and Beta really mean</li><li>Why stage ≠ sample size</li><li>The full lifecycle of product testing</li><li>A realistic timeline from idea → Beta</li><li>How UX researchers should operate at each phase</li></ul><h3>🧭 The Full Product Testing Lifecycle</h3><p>Before Alpha and Beta even begin, products move through multiple validation stages. Each stage serves a different research purpose.</p><h3>💡 1. Concept Testing (Pre-Build)</h3><p><strong>When</strong>: Idea stage<br> <strong>Goal</strong>: Validate problem relevance &amp; desirability<br> <strong>UX Methods</strong>:</p><ul><li>User interviews</li><li>JTBD exploration</li><li>Fake door tests</li><li>Value proposition validation</li></ul><blockquote><em>Are we solving something users actually need?</em></blockquote><h3>🎨 2. Prototype Testing</h3><p><strong>When</strong>: Early design<br> <strong>Goal</strong>: Validate structure &amp; flows<br> <strong>UX Methods</strong>:</p><ul><li>Usability testing</li><li>Card sorting</li><li>Click tests</li></ul><blockquote><em>Can users understand and navigate the solution?</em></blockquote><h3>🧪 3. Alpha Testing — Internal Reality Check</h3><p>Alpha is the first realistic test of a near-complete product inside the company.</p><p><strong>Who tests</strong></p><ul><li>Employees</li><li>QA</li><li>Internal dogfooding users</li></ul><p><strong>Goal</strong></p><ul><li>Catch crashes</li><li>Fix broken flows</li><li>Remove blockers</li><li>Stabilize core features</li></ul><p><strong>Size</strong></p><ul><li>Usually 10–50 testers</li></ul><p><strong>Key question</strong></p><blockquote><em>Does it technically work without breaking?</em></blockquote><p>Alpha is about <em>functionality</em>, not scale or satisfaction.</p><h3>🌍 4. Beta Testing — Real-World Validation</h3><p>Beta begins once the product is stable enough for external users.</p><p><strong>Who tests</strong></p><ul><li>Real target users</li><li>Early adopters</li><li>Selected customers</li></ul><p><strong>Goal</strong></p><ul><li>Validate usability</li><li>Identify edge cases</li><li>Measure satisfaction</li><li>Observe real behaviors</li></ul><p><strong>Typical size</strong></p><ul><li>200–500 users</li></ul><p><strong>Key question</strong></p><blockquote><em>Does it work naturally for real people?</em></blockquote><h3>🚫 Sample Size Does NOT Define Stage</h3><p>UsersTesting Stage&lt;50Alpha50–150Late Alpha / Pilot200–500Beta ✅1000+Soft Launch / Public Beta</p><p>If you’re working with 400 external users, you are firmly in Beta — even if stakeholders mislabel it.</p><h3>🚀 5. Soft Launch / Pilot Release</h3><p><strong>When</strong>: Limited live rollout<br> <strong>Goal</strong></p><ul><li>Test scalability</li><li>Monitor performance</li><li>Early retention &amp; behavior tracking</li></ul><h3>📈 6. General Availability (GA)</h3><p><strong>When</strong>: Public release<br> <strong>UX Focus</strong></p><ul><li>Continuous feedback</li><li>Surveys &amp; CSAT</li><li>Experimentation</li><li>Feature optimization</li></ul><h3>🔄 7. Post-Launch Optimization</h3><p><strong>Goal</strong></p><ul><li>Improve adoption</li><li>Reduce friction</li><li>Expand features<br> <strong>Methods</strong></li><li>A/B testing</li><li>Behavioral analytics</li><li>Segmentation</li></ul><h3>⏳ How Long Does It Take to Reach Beta?</h3><p>UX researchers often underestimate the time needed to move from idea → Beta.</p><p>Here’s the realistic product maturity path:</p><h3>Phase 1 — Ideation &amp; Validation (2–4 weeks)</h3><ul><li>Define problem</li><li>Conduct discovery interviews</li><li>Identify JTBD</li><li>Scope MVP</li></ul><h3>Phase 2 — MVP Development (6–16 weeks)</h3><ul><li>Core features built</li><li>Backend &amp; infrastructure</li><li>Initial usability validation</li></ul><h3>Phase 3 — Alpha Testing (2–4 weeks)</h3><ul><li>Internal dogfooding</li><li>Bug fixing</li><li>Flow stabilization</li></ul><h3>Phase 4 — Beta Prep (2–4 weeks)</h3><ul><li>Instrument analytics</li><li>Recruit users</li><li>Prepare feedback loops</li></ul><h3>📊 Typical Time to Beta by Product Type</h3><p>Product TypeTime to BetaSimple mobile/web2–4 monthsAI-powered4–8 monthsMarketplace6–12 monthsHardware + software12–24 months</p><h3>💡 What Actually Delays Beta (UX-Related)</h3><p>From a research perspective, delays usually come from:</p><ul><li>Poor onboarding</li><li>Undefined core use case</li><li>Scope creep</li><li>Weak IA</li><li>Recruiting the wrong testers</li><li>Missing analytics</li><li>Rebuilding after usability failure</li></ul><h3>🎯 Why UX Researchers Must Understand Testing Stages</h3><p>Knowing the lifecycle allows researchers to:</p><ul><li>Align method with product maturity</li><li>Push back on premature scale testing</li><li>Set realistic roadmap expectations</li><li>Protect external users from unstable builds</li><li>Provide better product strategy guidance</li></ul><p>Strong researchers don’t just run usability.<br> They manage <strong>product risk across stages</strong>.</p><h3>Final Takeaways</h3><ul><li>Alpha = Internal stability testing</li><li>Beta = External real-world validation</li><li>Stage is defined by product readiness, not sample size</li><li>400 users = Beta</li><li>Expect ~4–6 months minimum to reach Beta for AI/mobile products</li></ul><p>Product literacy is now a core UX research skill.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9d9e6db61f0c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Choose the Right Statistical Test: A Practical Guide for UX & Product Research]]></title>
            <link>https://janajiaruchung.medium.com/how-to-choose-the-right-statistical-test-a-practical-guide-for-ux-product-research-635cb91dd608?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/635cb91dd608</guid>
            <category><![CDATA[surveys]]></category>
            <category><![CDATA[ux-research]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Fri, 16 Jan 2026 18:45:57 GMT</pubDate>
            <atom:updated>2026-01-16T18:45:57.940Z</atom:updated>
            <content:encoded><![CDATA[<p>When people ask, <em>“Which statistical test should I use?”</em>, the honest answer is:<br> <strong>it depends on your study design, your data type, and your sample size</strong>.</p><p>In applied UX and product research, choosing the right test isn’t about memorizing formulas. It’s about making <strong>defensible decisions under real-world constraints</strong> — small samples, imperfect data, and high-stakes product decisions.</p><p>In this article, I’ll walk through <strong>the key factors you should consider when selecting a statistical test</strong>, and I’ll ground everything in a real scenario:</p><blockquote><strong><em>Example context used throughout:</em></strong><em><br> A </em><strong><em>within-subject (longitudinal) study</em></strong><em>, comparing </em>post-migration<em> vs </em>3 months after migration<em>, with a </em><strong><em>small sample size (n = 23 users)</em></strong><em>.</em></blockquote><h3>The First Thing to Decide: Study Design</h3><p>Before touching any statistics, I always answer this question:</p><h3>Are you comparing the same users over time, or different users?</h3><ul><li><strong>Same users measured multiple times</strong><br> → <em>Within-subject / paired / longitudinal</em></li><li><strong>Different users at each time point</strong><br> → <em>Between-subject / independent samples</em></li></ul><p>This single decision dramatically changes which tests are appropriate.</p><p>In my example, the design is <strong>within-subject</strong>:</p><ul><li>The <strong>same 23 users</strong> are measured post-migration and again 3 months later.</li><li>Each participant acts as their own control, which reduces noise and variance.</li></ul><h3>The Second Thing to Decide: Outcome Type</h3><p>Next, I clarify <strong>what kind of data I’m analyzing</strong>:</p><ul><li><strong>Binary</strong><br> Yes / No outcomes (e.g., task completion)</li><li><strong>Ordinal</strong><br> Ordered categories (e.g., Likert scales like 1–5 CSAT)</li><li><strong>Continuous</strong><br> Time, counts, or composite scores</li></ul><p>This matters because different tests are designed for different outcome structures.</p><h3>The Third Thing to Decide: Sample Size Reality</h3><p>Sample size determines how cautious you need to be.</p><p>With <strong>small samples (like n = 23)</strong>:</p><ul><li>Assumptions matter more</li><li>P-values are unstable</li><li>Effect size and direction become more important than “significance”</li></ul><p>In my example:</p><ul><li><strong>n = 23 total users</strong></li><li>This pushes me toward <strong>paired designs</strong>, <strong>exact or non-parametric tests</strong>, and <strong>conservative interpretation</strong></li></ul><h3>Putting It Together: A Practical Decision Table</h3><p>Here’s the table I effectively carry in my head when choosing tests:</p><p>Outcome TypeStudy DesignSample SizeRecommended Test<strong>Binary (Yes/No)</strong>Within-subjectSmall (n ≈ 20–30)<strong>McNemar’s test</strong> (exact if needed)<strong>Binary (Yes/No)</strong>Between-subjectSmall<strong>Fisher’s Exact TestOrdinal (Likert)</strong>Within-subjectSmall<strong>Wilcoxon signed-rank testOrdinal (Likert)</strong>Within-subjectModerate<strong>Paired t-test</strong> (pragmatic)<strong>Ordinal (Likert)</strong>Between-subjectSmall<strong>Mann–Whitney U testOrdinal (Likert)</strong>Between-subjectModerate<strong>Welch’s t-test</strong></p><p>This table isn’t about “right vs wrong.”<br> It’s about <strong>choosing the most defensible option given your constraints</strong>.</p><h3>Applying This to My Example (Within-Subject, n = 23)</h3><p>Let’s make this concrete.</p><h3>Binary outcome: Task completion (Yes / No)</h3><ul><li><strong>Design:</strong> Within-subject (same users over time)</li><li><strong>Sample size:</strong> Small (n = 23)</li></ul><p><strong>What I use:</strong><br> 👉 <strong>McNemar’s test</strong></p><p><strong>Why:</strong><br> McNemar’s test is designed specifically for paired binary data. It focuses on <strong>who changed</strong> (failed → succeeded or vice versa), rather than comparing group averages.</p><p>This is exactly what matters in a post-migration comparison.</p><h3>Ordinal outcome: Likert-scale CSAT (1–5)</h3><ul><li><strong>Design:</strong> Within-subject</li><li><strong>Sample size:</strong> Small</li></ul><p><strong>What I use:</strong></p><ul><li><strong>Paired t-test</strong> as a pragmatic default <em>if difference scores are reasonably symmetric</em></li><li><strong>Wilcoxon signed-rank test</strong> as a conservative fallback</li></ul><p><strong>Why:</strong><br> Because this is a within-user comparison, variance between users is not the main concern. The question becomes whether the <strong>distribution of change</strong> is stable enough to justify a parametric test.</p><p>When in doubt, Wilcoxon keeps conclusions robust.</p><h3>Why Student’s vs Welch’s t-test Is Not the Question Here</h3><p>An important clarification:</p><ul><li><strong>Student’s vs Welch’s t-test applies to independent samples</strong></li><li>In <strong>within-subject designs</strong>, that distinction is irrelevant</li></ul><p>For paired data:</p><ul><li>Variance equality between groups doesn’t matter</li><li>Each participant is their own control</li></ul><p>That’s why, in my example, the real choice is:</p><ul><li><strong>Paired t-test vs Wilcoxon</strong>, not Student’s vs Welch’s</li></ul><h3>What I Emphasize More Than the Test Itself</h3><p>Especially with <strong>small samples</strong>, the test is not the headline.</p><p>What matters more:</p><ul><li><strong>Direction of change</strong></li><li><strong>Magnitude of effect</strong></li><li><strong>Consistency across metrics</strong></li><li><strong>Alignment with qualitative feedback or telemetry</strong></li></ul><p>I treat p-values as <strong>supporting evidence</strong>, not proof.</p><h3>The Principle I Follow</h3><blockquote><strong><em>The goal of statistical testing in UX research is not to “win” significance — it’s to reduce risk and inform decisions responsibly.</em></strong></blockquote><h3>One-Sentence Summary</h3><blockquote><strong><em>When choosing a statistical test, I start with design (within vs between), then outcome type, then sample size — using paired and non-parametric tests for small longitudinal samples like my n=23 example, and focusing on effect size and direction over p-values.</em></strong></blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=635cb91dd608" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Create Great Surveys: From Sampling to Decision-Ready Insights]]></title>
            <link>https://janajiaruchung.medium.com/how-to-create-great-surveys-that-people-actually-answer-cb1116ae312f?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/cb1116ae312f</guid>
            <category><![CDATA[surveys]]></category>
            <category><![CDATA[ux-research]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Thu, 15 Jan 2026 22:51:20 GMT</pubDate>
            <atom:updated>2026-01-26T01:53:36.463Z</atom:updated>
            <content:encoded><![CDATA[<p>Surveys are one of the most misused tools in product and UX research.</p><p>Teams run them constantly — yet still walk away with vague insights, conflicting results, or dashboards no one fully trusts. The issue is rarely sample size or advanced statistics.<br> It’s survey design.</p><p>A great survey doesn’t ask more questions.<br> It asks the <em>right</em> ones — clearly, respectfully, and with intent.</p><p>This article walks through the end-to-end survey lifecycle, showing how to design surveys that produce decision-ready insights instead of noise.</p><h3>Start With the Right People</h3><p>A perfectly written survey fails if it reaches the wrong audience.</p><p>Before drafting questions, clarify who just experienced the behavior you care about — and who should be excluded. Identify meaningful subgroups like tenure, role, or usage frequency, and be mindful of how often you’re surveying the same users.</p><p>The best surveys are triggered as close to the experience as possible. Convenience samples and over-surveying power users quietly distort results.</p><p>Sampling isn’t a logistics step. It defines what your data means.</p><h3>Screener Creation: Protect Data Quality Up Front</h3><p>Before a respondent ever sees your survey, a screener decides whether their answers will be trustworthy.</p><p>A good screener ensures:</p><ul><li>The respondent actually experienced the product or flow</li><li>The experience is recent enough to recall accurately</li><li>The respondent fits the population the decision is about</li></ul><p>Effective screener questions are factual, neutral, and unambiguous:</p><ul><li>“Did you book a meeting room in the past 30 days?”</li><li>“Which of the following tools did you use today?”</li><li>“Were you able to complete the task yourself?”</li></ul><p>Avoid screeners that test opinions, leak survey intent, or exclude edge cases unintentionally. Over-screening reduces representativeness; under-screening contaminates your data.</p><p>A strong screener is invisible when done well — and painful when skipped.</p><h3>Design Surveys Around Decisions, Not Curiosity</h3><p>Surveys don’t exist to “learn in general.” They exist to reduce uncertainty around a specific decision.</p><p>Before writing a single question, ask:</p><ul><li>What decision will this survey inform?</li><li>Who will act on the results?</li><li>What would change if the numbers move?</li></ul><p>If you can’t explain the decision in one sentence, the survey isn’t ready. Most effective surveys tie to one to three decisions, take under three minutes, and prioritize questions by importance — not narrative flow.</p><h3>Measure Clear Constructs</h3><p>Vague goals produce vague data.</p><p>Instead of measuring “experience” or “satisfaction,” define the construct you care about: ease, trust, confidence, reliability, friction, or task success. Each question should map cleanly to one construct. If it doesn’t, it’s either doing too much — or nothing useful.</p><p>Clarity here enables clean analysis later.</p><h3>Choose Question Types Intentionally</h3><p>Different question types serve different purposes.</p><p>Likert scales capture attitudes and perceptions. Binary questions verify facts or completion. Multiple-choice questions surface drivers and categories. Open-ended questions explain <em>why</em> — they shouldn’t be used for discovery at scale.</p><p>If you don’t know how you’ll analyze the text, don’t ask for it.</p><h3>Why Likert Scales Work</h3><p>People don’t experience products in binaries. They think in gradients.</p><p>Likert scales capture both direction and intensity, aligning with how humans evaluate experiences. Five-point scales reduce fatigue and noise; seven-point scales add sensitivity when nuance matters. Both outperform yes/no questions for subjective measurement and enable richer downstream analysis.</p><h3>Write Questions That Don’t Bias Answers</h3><p>Strong survey questions are simple and neutral.</p><p>They cover one idea, avoid assumptions, specify a timeframe, and use plain language. Every adjective you add — “fast,” “intuitive,” “seamless” — pushes respondents toward an answer.</p><p>Ask what happened, not what you hope happened.</p><h3>Pilot Before You Scale</h3><p>Pilot testing is a non-negotiable quality gate.</p><p>Test with a small group, ask participants to paraphrase questions, measure completion time, and note hesitation or confusion. If respondents interpret a question differently than you intended, the data is already compromised.</p><h3>Collect Data Consistently</h3><p>Execution matters as much as design.</p><p>Trigger surveys immediately after relevant actions, keep timing consistent across waves, and monitor response quality — not just response rate. Watch for straight-lining, drop-offs, and missing data.</p><p>Consistency is what makes trends meaningful.</p><h3>Plan Analysis Before Seeing Results</h3><p>Decide how you’ll analyze the data before it arrives.</p><p>Are you tracking trends, comparing groups, or diagnosing drivers? What’s the primary metric — and what supports it? Choosing an analysis plan after seeing results invites bias.</p><h3>Segment With Purpose — or Don’t</h3><p>Segmentation is powerful when differences are expected and actionable — such as new versus experienced users or heavy versus light usage. It’s counterproductive when sample sizes are small or decisions won’t change.</p><p>Clarity beats complexity.</p><h3>Look Beyond Averages</h3><p>Means rarely tell the full story.</p><p>Examine distributions, top- and bottom-box scores, trends over time, and drivers of outcomes like trust or satisfaction. The value of a survey isn’t the number — it’s the insight behind it.</p><h3>An End-to-End Example: Microsoft Digital Employee Experience</h3><p>At Microsoft Digital, continuous employee surveys support internal tools for booking meeting rooms and shared spaces, in partnership with Global Workspace Services.</p><p>Surveys are event-triggered after bookings, stratified by building and region. A short monthly tracker measures ease of booking, reliability, and meeting success, while rotating modules explore issues like AV quality or room accuracy.</p><p>Results are analyzed alongside telemetry to identify primary drivers of dissatisfaction. This approach consistently surfaces the small number of buildings responsible for most negative experiences — giving engineering teams clear, prioritized fixes.</p><p>The lesson: trust, trend integrity, and action matter more than survey length.</p><h3>Close the Loop</h3><p>A survey is only successful if it leads to action.</p><p>Share results quickly, connect insights to decisions, communicate what changed, and track metrics after fixes ship. When people see their feedback matter, response quality improves over time.The Survey Design Loop</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/0*5KYVOmM9De2yhG52" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uHZjODOplbYOfy4S.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/698/0*XqpIv7NPJPwQ1CYf.jpg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cb1116ae312f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Agentic AI in the Real World: One Pattern, Many Systems]]></title>
            <link>https://janajiaruchung.medium.com/agentic-ai-in-the-real-world-one-pattern-many-systems-5e840e3002f0?source=rss-a1ff058aedd5------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e840e3002f0</guid>
            <category><![CDATA[ai-agent]]></category>
            <dc:creator><![CDATA[Jia-Ru (Edith) Chung]]></dc:creator>
            <pubDate>Fri, 09 Jan 2026 20:36:47 GMT</pubDate>
            <atom:updated>2026-01-09T20:36:47.173Z</atom:updated>
            <content:encoded><![CDATA[<p>Agentic AI shows up in very different places today — consumer health apps, enterprise supply chains, support sites, and healthcare operations. At first glance, these domains seem unrelated. But if you look closely, they share the same operational shape.</p><p>This article explains <strong>why agentic AI exists</strong>, <strong>where it breaks</strong>, <strong>why humans stay in the loop</strong>, and <strong>how confidence thresholds make these systems trustworthy</strong> — using examples from Fitbit, support sites, OCT/IFS supply-chain tools, and healthcare operations.</p><h3>Why Agentic AI Exists</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oxuH0TEVeHm8VqYl.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7QYfxAmIjb8J6huA.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/322/0*zUOiwDI4-9I_ZXk9" /></figure><p>Agentic AI exists to fill a gap that traditional automation and chatbots never solved.</p><p>Most real work is:</p><ul><li><strong>Multi-step</strong></li><li><strong>Cross-system</strong></li><li><strong>Ongoing over time</strong></li><li><strong>Ambiguous at the edges</strong></li></ul><p>Before agentic AI, organizations had two bad options:</p><ul><li><strong>Rules and RPA</strong> — fast but brittle, collapsing under edge cases</li><li><strong>Humans</strong> — flexible but slow, expensive, and hard to scale</li></ul><p>Agentic AI is designed to sit between these extremes. It can <strong>hold a goal</strong>, <strong>track state over time</strong>, <strong>coordinate actions across tools</strong>, and <strong>decide when to escalate</strong>. In other words, it doesn’t just answer questions — it <strong>owns coordination work</strong>.</p><h3>The One Agentic Pattern Behind Many Systems</h3><p>Across domains, agentic AI follows the same loop:</p><ol><li><strong>Sense</strong> signals continuously</li><li><strong>Reason</strong> toward a persistent goal</li><li><strong>Act</strong> within defined guardrails</li><li><strong>Escalate</strong> to humans when confidence or risk thresholds are crossed</li></ol><p>This loop is identical whether the system is helping a user sleep better, reconciling supply-chain forecasts, resolving a support issue, or coordinating healthcare operations.</p><h3>How This Shows Up in Practice</h3><h3>Consumer Health (Fitbit)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3XqFsBwHCBOtQJNs.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Y_QeYHJHaLjwaQYl.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/950/0*ChWm-jtg-rjVj7xg.png" /></figure><p>In consumer health, agentic AI coordinates <strong>continuous physiological signals</strong> — sleep, heart rate variability, activity, stress — toward goals like recovery or consistency. The system adapts guidance over time, knowing when to nudge, when to pause, and when to stay silent.</p><p>Crucially, it avoids diagnosis. The value is <strong>orchestration</strong>, not authority.</p><h3>Enterprise Supply Chain (OCT &amp; IFS)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sKJ3F0Lx1DdxfFcG.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hwLTDMhMc0O-L4dV.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9z8AX4pMUbtrBO_F.jpg" /></figure><p>In enterprise systems like OCT and IFS, agentic AI reconciles <strong>capacity, forecasts, and constraints</strong> across tools. Humans traditionally do this by hand — comparing dashboards, running scenarios, and escalating risks.</p><p>Agentic AI replaces that coordination work by continuously monitoring divergence, simulating outcomes, and escalating only when thresholds are crossed. Planners keep decision ownership; AI handles the glue.</p><h3>Support Sites</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*J8Fyog06k7R---C-.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*QuPME5mcJbQkJ0rj.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*eGgKJlWE7NsI39nG" /></figure><p>Support systems deal with <strong>real-time ambiguity</strong>. Users arrive with situations, not well-formed questions. Agentic AI discovers intent across turns, chooses the next step (article, workflow, clarification), tracks progress toward resolution, and escalates with context when needed.</p><p>The success metric isn’t automation rate — it’s <strong>resolution without trust loss</strong>.</p><h3>Healthcare Operations</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*V11oqmlTBlviN2B7" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/258/0*vDV6Nggym7CBnr8t" /></figure><p>In healthcare, agentic AI belongs in <strong>coordination</strong>, not diagnosis. It orchestrates intake, scheduling, prior authorization, follow-ups, and exception detection across fragmented systems.</p><p>Here, conservative behavior is essential: escalate early, stay within scope, and preserve human accountability.</p><h3>Where Agentic AI Breaks</h3><p>Agentic AI fails fastest when it’s used where autonomy increases risk faster than it reduces effort:</p><ul><li><strong>High-risk, irreversible actions</strong></li><li><strong>Emotionally charged situations</strong></li><li><strong>Rare or novel issues</strong></li><li><strong>Unclear success criteria</strong></li><li><strong>Immature systems without observability</strong></li></ul><p>Treating agentic AI as “set and forget” is the shortest path to losing Ops trust.</p><h3>Why Humans Stay in the Loop</h3><p>Humans are not a fallback — they’re a <strong>designed component</strong>.</p><p>Across all domains:</p><ul><li>AI provides speed, consistency, and scale</li><li>Humans provide judgment, accountability, and trust</li></ul><p>Edge cases — not averages — define success. Humans handle those edges.</p><h3>What “Conservative” Really Means</h3><p>Conservative agentic AI doesn’t mean slow or timid. It means:</p><blockquote><strong><em>The system is biased toward avoiding overreach and irreversible harm, even if that means acting less autonomously.</em></strong></blockquote><p>Conservative systems:</p><ul><li>Act only when confidence and reversibility are high</li><li>Escalate early when uncertainty rises</li><li>Use careful language</li><li>Expand autonomy gradually</li></ul><p>Trust compounds. Overconfidence destroys it.</p><h3>The Confidence-Threshold Framework</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5Sy0cxkp9crtDJgJ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bEZEr1vux0VgXdxG.jpg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*8G9g3TZQk_-sUzRo" /></figure><p>A confidence-threshold framework replaces binary automation with graduated autonomy:</p><p><strong>Level 1 — Observe</strong><br> Low confidence → monitor silently</p><p><strong>Level 2 — Recommend</strong><br> Medium confidence → suggest actions, require human approval</p><p><strong>Level 3 — Act with Guardrails</strong><br> High confidence + low risk → execute predefined actions</p><p><strong>Level 4 — Escalate</strong><br> High risk → stop and hand off immediately</p><p>Confidence is composite — signal clarity, data completeness, pattern familiarity, reversibility, and risk severity — not just model certainty.</p><p>This framework makes autonomy <strong>explicit</strong>, predictable, and governable.</p><h3>The Unifying Insight</h3><p>Agentic AI succeeds across Fitbit, OCT/IFS, support sites, and healthcare for the same reason:</p><blockquote><strong><em>They all involve continuous signals, evolving goals, high coordination cost, and unavoidable human accountability.</em></strong></blockquote><p>What changes is risk tolerance and time horizon — not the architecture.</p><h3>Final Takeaway</h3><p>Agentic AI is not about replacing people.<br> It’s about <strong>replacing coordination work</strong> — the tracking, sequencing, and context-passing that exhausts humans and slows systems.</p><p>When designed conservatively, with confidence thresholds and human-in-the-loop escalation, agentic AI becomes a durable orchestration layer — one that scales without breaking trust.</p><p>That’s what makes it real.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e840e3002f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>