<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Xiaoxiao on Medium]]></title>
        <description><![CDATA[Stories by Xiaoxiao on Medium]]></description>
        <link>https://medium.com/@xiaoxiaox?source=rss-165394cecf64------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*o4v0dlZyYJrJwe-_J8voyg.jpeg</url>
            <title>Stories by Xiaoxiao on Medium</title>
            <link>https://medium.com/@xiaoxiaox?source=rss-165394cecf64------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 11:58:16 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@xiaoxiaox/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Describing What Changed: A Descriptive Offline Evaluation Framework for Recommendation Models]]></title>
            <link>https://medium.com/@xiaoxiaox/describing-what-changed-a-descriptive-offline-evaluation-framework-for-recommendation-models-9f4f7c11ea49?source=rss-165394cecf64------2</link>
            <guid isPermaLink="false">https://medium.com/p/9f4f7c11ea49</guid>
            <category><![CDATA[recommendation-system]]></category>
            <category><![CDATA[model-evaluation]]></category>
            <category><![CDATA[recommender-systems]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Xiaoxiao]]></dc:creator>
            <pubDate>Sat, 12 Jul 2025 20:58:35 GMT</pubDate>
            <atom:updated>2025-07-12T20:58:35.431Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2GtaYxZCXNt_uvoP18_FuQ.png" /></figure><p>When we evaluate recommendation models, especially in large systems like social networks, media or e-commerce, much of our insight work comes down to one simple but difficult question:</p><p><strong>What actually changed?</strong></p><p>Sure, we have our top-line metrics — click-through rate, dwell time, skip rate — but to truly understand <em>why</em> one model performs better than another (or not), we need to go deeper. We need to describe the recommendations themselves.</p><p>In this post, I’ll share an early structure for a <strong>descriptive (offline) evaluation framework</strong>: a way to systematically surface and interpret changes in recommendation outputs. The goal is to make these evaluations more explainable, repeatable, and ultimately more actionable — especially as we move toward more generative and personalized systems.</p><h3>🧱 1. Start With What You Have: Tools and Vocabularies</h3><p>Most mature recommendation systems already have infrastructure in place that we can build on:</p><ul><li>Dashboards that summarize model outputs</li><li>Metrics that help track behavioral shifts</li><li>Logging systems with metadata about content, users and models</li></ul><p>These tools give us a foothold. The real value comes when we layer on a more structured way to <em>describe</em> what these outputs mean.</p><h3>🎯 2. Tailor the Granularity: What’s the Right Level of Description?</h3><p>Not every evaluation needs the same lens. In some cases, looking at the distribution by some simple dimensions is enough. In others, you might need:</p><ul><li>Segment-level breakdowns (e.g. by user cohort or intent)</li><li>Topic clusters or genre taxonomies</li><li>Interactions between recent behavior and recommended content</li></ul><p>The key here is to adapt the granularity to the question at hand. Are you debugging personalization? Then maybe you need user-level entropy. Are you evaluating diversity? Think distributional concentration or evenness.</p><h3>📊 3. Extend the Metrics: Go Beyond CTR</h3><p>Traditional KPIs often obscure more than they reveal. We need richer, more descriptive metrics that illuminate how recommendations behave:</p><ul><li><strong>Personalization</strong>: Measure how tailored the outputs are to different users. Techniques include item-level variance, user-item cosine distance, or clustering overlap.</li><li><strong>Diversity</strong>: Consider inequality metrics like the Gini index to quantify skew in content exposure. Are we over-recommending the same creators or genres?</li><li><strong>Reactivity</strong>: Track how quickly recommendations adapt to new user signals. Lag metrics or signal-weighting ratios can be useful here.</li></ul><p>These aren’t replacements for performance metrics — they’re complements that help explain them.</p><h3>🧠 4. Add Context: What Should (and Shouldn’t) Happen?</h3><p>Every model has intent baked into it. A good descriptive framework reflects that by answering:</p><ul><li>What kinds of improvement do we expect? (e.g. higher diversity, better topical alignment)</li><li>What behaviors are we cautious about? (e.g. hallucinated items, incoherent transitions)</li><li>Where might trade-offs exist? (e.g. personalization vs. serendipity)</li></ul><p>This is where collaboration with modeling and product teams becomes essential. Your evaluation is only as good as your hypothess.</p><h3>🎲 5. Sample Smartly: Especially When Using Simulated Data</h3><p>If your evaluations rely on simulated or offline data:</p><ul><li>Ensure your sample represents key user/content distributions</li><li>Identify when to oversample edge cases for stress testing</li><li>Align your sampling strategy with your evaluation goals (debugging vs. benchmarking)</li></ul><p>Sampling isn’t just a preprocessing step — it’s part of the evaluation logic itself.</p><h3>🔁 6. Make It Repeatable: From One-Offs to Pipelines</h3><p>Instead of reinventing the wheel every time a new model lands, we can:</p><ul><li>Build modular diagnostic pipelines with reusable templates</li><li>Define a consistent vocabulary for describing outputs across model types</li><li>Share evaluation patterns across teams to reduce cognitive load and increase speed</li></ul><p>The long game here is a system of evaluation — not just a set of tools, but a shared way of thinking.</p><h3>🚀 What’s Next?</h3><p>If you’re working on recommendation models, especially those involving generative or dynamic elements, this framework is something you can start applying today. You don’t need to overhaul your stack — just begin by asking:</p><blockquote><em>“What changed, and how do I describe it in a way that someone else can understand?”</em></blockquote><p>That’s the foundation for deeper insight, better model selection, and ultimately a more trustworthy system.</p><p><em>Disclaimer: The views expressed in this post are my own and do not represent those of any employer, past or present. This framework is based on general industry practices and personal experience, and does not reflect or disclose any proprietary systems, data, or strategy.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9f4f7c11ea49" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Zodiac Signs, MBTI, and Machine Learning: A Tale of Two Cognitive Styles]]></title>
            <link>https://medium.com/@xiaoxiaox/zodiac-signs-mbti-and-machine-learning-a-tale-of-two-cognitive-styles-6a28666d23fd?source=rss-165394cecf64------2</link>
            <guid isPermaLink="false">https://medium.com/p/6a28666d23fd</guid>
            <category><![CDATA[generative-ai-use-cases]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Xiaoxiao]]></dc:creator>
            <pubDate>Tue, 01 Jul 2025 04:37:41 GMT</pubDate>
            <atom:updated>2025-07-01T04:52:10.282Z</atom:updated>
            <content:encoded><![CDATA[<h4><strong>Why thinking in generative vs. discriminative terms might explain more than you think</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*47CwrVUWumZtkCqyV8wecw.png" /></figure><p>We often think of machine learning as a purely technical domain — all code, math, and model tuning. But at its core, it reflects something deeply human: the way we interpret and structure the world. And sometimes, these structures echo our cognitive habits in surprisingly poetic ways.</p><p>Take, for example, how we explain personality.</p><p>On one end, there’s astrology: your birth time determines your zodiac sign, and from that, a cascade of personality traits unfolds. You might be a Pisces — intuitive, emotional, perhaps artistically inclined. Or a Capricorn — pragmatic, responsible, goal-oriented. The input is simple (a timestamp), but the output is rich, vivid, and sprawling.</p><p>Now contrast that with MBTI (Myers-Briggs Type Indicator). You answer a battery of questions, the system analyzes your preferences and behavior patterns, and then assigns you one of 16 types: INFP, ESTJ, and so on. Here, the input is noisy and complex (your real-life responses), and the output is a compressed summary: a label.</p><p>If you’re already sensing a machine learning analogy here — you’re absolutely right.</p><p><strong>Generative vs. Discriminative Thinking</strong></p><p>In machine learning, <strong>generative models</strong> aim to capture how data is generated. Given some underlying variable (often a latent code), they can sample or produce rich, detailed outputs. Think of GPT generating text, or a GAN producing faces.</p><p><strong>Discriminative models</strong>, on the other hand, are focused on boundaries. They don’t try to explain how data comes to be. Instead, they ask: given what I see, what label or decision should I make?</p><p>Now reread the descriptions of astrology and MBTI.</p><ul><li>Astrology is <strong>generative</strong>: from a low-dimensional input (birth data), it constructs a high-dimensional personality narrative. It “generates” identity.</li><li>MBTI is <strong>discriminative</strong>: from high-dimensional behavioral signals, it distills a compact label. It “classifies” identity.</li></ul><p>One expands. One compresses.</p><p>This isn’t a critique or endorsement of either system. Rather, it’s a reflection of two cognitive modes we often toggle between — even outside machine learning.</p><p><strong>Why This Matters</strong></p><p>Understanding this generative/discriminative duality helps us grasp not just ML model architecture, but human reasoning itself:</p><ul><li>When we try to understand <em>why</em> someone did something, we often default to generative reasoning: What underlying motives or traits produced this behavior?</li><li>When we make quick judgments — hire or not, trust or not — we use discriminative reasoning: What label best fits this pattern?</li></ul><p>Both styles have power. Both can mislead.</p><p><strong>A Final Thought</strong></p><p>So the next time you catch yourself justifying a friend’s stubbornness with “Well, she <em>is</em> a Taurus,” or deciding someone’s an INTJ after five minutes of talking — pause and ask:</p><p>Am I generating a story, or making a classification?</p><p>And maybe, somewhere between zodiac signs and acronyms, you’ll find a deeper understanding of both models and minds.</p><p><em>The views expressed here are my own and do not reflect those of my employer.</em></p><p><em>Special thanks to my invisible writing buddy for helping me clarify my thoughts.</em>🪄</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6a28666d23fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>