<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Dev Pal on Medium]]></title>
        <description><![CDATA[Stories by Dev Pal on Medium]]></description>
        <link>https://medium.com/@devpal?source=rss-33cf2565f954------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*MlknNcO3AribepyKPNVY4g.jpeg</url>
            <title>Stories by Dev Pal on Medium</title>
            <link>https://medium.com/@devpal?source=rss-33cf2565f954------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 10:00:42 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@devpal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Subtle Play Behind Generative AI’s Long Answers]]></title>
            <link>https://medium.com/@devpal/the-subtle-play-behind-generative-ais-long-answers-93472c55434f?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/93472c55434f</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[claude]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Sun, 13 Sep 2026 12:06:04 GMT</pubDate>
            <atom:updated>2026-09-13T12:06:32.133Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Why longer answers may be about more than just being helpful.</em></h4><p><em>Have you ever asked Claude or ChatGPT a question and found yourself reading long answers? What if that is intentional — to drive engagement — to keep you on the app.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*-yrSyxYo2ixc7IcX.png" /></figure><p>For technology companies, claiming and keeping customer attention — and turning it into revenue — is a subtle play. Social media companies like Meta and TikTok mastered it by bringing social interaction online and layering an advertising model on top. Users returned because their friends were there, notifications pulled them back, and eventually the feed itself became the habit. The more they used the platforms, the more data these companies accumulated, making advertising increasingly targeted.</p><p><strong>Generative AI answers are deliberately long prose and not bulleted.</strong></p><ol><li><strong><em>When you read long text, you invariably spend more time on the app (the source)</em></strong> which are key KPIs on investor decks for why these companies continue to get funding.</li><li><strong><em>Long text has more context that can go wrong (the trigger)</em> </strong>It gets interesting — long text brings in more context from saved memory or external sources and it may not serve the user need. As a result, it is <strong>triggers the user to respond.</strong></li><li><strong><em>User response is logged and acts as reinforcement loop for the AI to learn</em></strong> about the user and update its model on how it should respond when it faces a similar question.</li></ol><p>It’s useful context for when you find yourself skimming AI text and upping your reading speed.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=93472c55434f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[To A/B or not to A/B?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@devpal/to-a-b-or-not-to-a-b-bd663ed78c0c?source=rss-33cf2565f954------2"><img src="https://cdn-images-1.medium.com/max/1405/0*lV2uIWRuBERqwrWJ.png" width="1405"></a></p><p class="medium-feed-snippet">Practical guide on choosing the right A/B test</p><p class="medium-feed-link"><a href="https://medium.com/@devpal/to-a-b-or-not-to-a-b-bd663ed78c0c?source=rss-33cf2565f954------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@devpal/to-a-b-or-not-to-a-b-bd663ed78c0c?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd663ed78c0c</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[ab-testing]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[causal-inference]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Thu, 30 Apr 2026 21:05:10 GMT</pubDate>
            <atom:updated>2026-04-30T21:05:28.404Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Goals & North Star Metrics: A Content Recommendations Deep Dive]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@devpal/goals-north-star-metrics-a-content-recommendations-deep-dive-aca0a4d6ddc7?source=rss-33cf2565f954------2"><img src="https://cdn-images-1.medium.com/max/1376/1*om0PJxKXPPavGDupSQeYGA.png" width="1376"></a></p><p class="medium-feed-snippet">KPIs for Consumer Features: A Framework</p><p class="medium-feed-link"><a href="https://medium.com/@devpal/goals-north-star-metrics-a-content-recommendations-deep-dive-aca0a4d6ddc7?source=rss-33cf2565f954------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@devpal/goals-north-star-metrics-a-content-recommendations-deep-dive-aca0a4d6ddc7?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/aca0a4d6ddc7</guid>
            <category><![CDATA[metrics]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ab-testing]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Tue, 31 Mar 2026 20:30:07 GMT</pubDate>
            <atom:updated>2026-03-31T20:30:07.155Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[ChatGPT Plus vs Claude Pro: Why I switched?]]></title>
            <link>https://medium.com/@devpal/chatgpt-plus-vs-claude-pro-why-i-switched-ba3268ee9bf0?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/ba3268ee9bf0</guid>
            <category><![CDATA[anthropic-claude]]></category>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[claude]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Fri, 27 Feb 2026 19:44:10 GMT</pubDate>
            <atom:updated>2026-02-27T19:44:51.633Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Yj39wcINpuVPpLkB.png" /></figure><p>Two products. Same price. One is trying to keep you engaged. The other is trying to get you out of the chat faster. I start with facts, look at human criteria and end with an opinion.</p><h3>The Facts</h3><ol><li><strong>Context window:</strong> Claude 200K vs ChatGPT 128K-<strong>Claude wins</strong></li><li><strong>Persistent memory:</strong> Claude’s memory is user-visible ChatGPT’s memory is more deeply integrated, more contextual -<strong>ChatGPT wins</strong></li><li><strong>File uploads:</strong> ChatGPT handles docs and PDFs more reliably -<strong>ChatGPT wins</strong></li><li><strong>Response length:</strong> ChatGPT is padded and recap-heavy ; Claude is concise -<strong>Claude wins</strong></li><li><strong>Refusals:</strong> Claude engages on nuance, explains boundaries. ChatGPT has more corp-speak standardized safety responses given its mass-y audience -<strong>Draw</strong></li><li><strong>Web search:</strong> Both have it; ChatGPT’s is more mature and better cited -<strong>ChatGPT wins</strong></li><li><strong>Coding:</strong> ChatGPT executes code in-chat. Claude excels at reasoning -<strong>Draw</strong></li><li><strong>Multimodal ecosystem:</strong> ChatGPT has voice, image gen, data analysis; Claude has none-<strong>ChatGPT wins</strong></li><li><strong>Hallucination honesty:</strong> Claude admits uncertainty; ChatGPT fabricates more, keeps bad news away -<strong>Claude wins</strong></li><li><strong>Pricing:</strong> Both $20/month -<strong>Draw</strong></li></ol><h3>The “Human” Criteria</h3><ol><li><strong>The “feel” of talking to it</strong> — Claude pushes back, disagrees, has opinions. ChatGPT tends to agree with you and tell you what you want to hear. That’s a huge daily-use difference that doesn’t fit neatly into “quality.”</li><li><strong>Does it actually follow your instructions?</strong> — You said write it in 3 bullet points, it gave you 7 lengthy paragraphs. You said be concise, it wrote an essay. Instruction-following is its own paradigm.</li><li><strong>Consistency</strong> — Run the same prompt twice. If this were an experiment, variance matters as much as the mean.</li><li><strong>Memory</strong> -Does it remember what you told it last Tuesday?</li><li><strong>When it says no</strong> -Both models refuse things, but how and when they do it feels very different. One feels like a cautious lawyer, one feels like a curious colleague who occasionally hits a wall. Worth naming.</li><li><strong>Speed on a bad day</strong> -Not peak performance, but what’s the experience at 2pm on a Tuesday when everyone’s using it?</li></ol><h3>The Opinion</h3><p>After the criteria and facts, here’s what I keep coming back to: these two products feel built around different incentives.</p><p>ChatGPT behaves like a friendly consumer-scale platform. Answers are long, thorough and optimized for time spent which is important for retention. Tone skews agreeable to offer a much needed “kick” — the same dopamine loop that keeps you scrolling on Facebook and Instagram.. Treading on questions with nuance, safety is prioritized. With over 800 Million users, engagement is key.</p><p>Claude feels built for a different environment. The tone is tighter and precise. It signals uncertainty more easily and pushes back more often with shorter responses. It feels designed for people who <strong>care about output quality over conversational comfort.</strong></p><p>Consumer-scale products tend to <strong>optimize for time spent and retention.</strong><br>Enterprise-oriented tools tend to <strong>optimize for precision and correctness.</strong></p><p>At $20/month, they are playing distinctly different games.<br>You’re choosing the right collaborator.</p><p>I know my choice.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ba3268ee9bf0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You’re Prompting LLMs Wrong: The Role Design Playbook]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@devpal/youre-prompting-llms-wrong-the-role-design-playbook-db80d0e06a07?source=rss-33cf2565f954------2"><img src="https://cdn-images-1.medium.com/max/639/0*mAuS9g4ehERqJMjO.png" width="639"></a></p><p class="medium-feed-snippet">Role design &gt; Prompt engineering</p><p class="medium-feed-link"><a href="https://medium.com/@devpal/youre-prompting-llms-wrong-the-role-design-playbook-db80d0e06a07?source=rss-33cf2565f954------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@devpal/youre-prompting-llms-wrong-the-role-design-playbook-db80d0e06a07?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/db80d0e06a07</guid>
            <category><![CDATA[prompt-engineering]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[writing-prompts]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[sentiment-analysis]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Fri, 30 Jan 2026 22:17:59 GMT</pubDate>
            <atom:updated>2026-01-30T22:17:59.477Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[How I use ChatGPT?]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@devpal/how-i-use-chatgpt-067f6401f5f8?source=rss-33cf2565f954------2"><img src="https://cdn-images-1.medium.com/max/761/0*jIn_SLqOlp2ADhqG.jpeg" width="761"></a></p><p class="medium-feed-snippet">A practical guide to configuring ChatGPT</p><p class="medium-feed-link"><a href="https://medium.com/@devpal/how-i-use-chatgpt-067f6401f5f8?source=rss-33cf2565f954------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@devpal/how-i-use-chatgpt-067f6401f5f8?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/067f6401f5f8</guid>
            <category><![CDATA[chatgpt]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[writing-prompts]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Wed, 14 Jan 2026 01:28:03 GMT</pubDate>
            <atom:updated>2026-01-14T01:28:03.283Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Product Case Study 101: A 4 step approach]]></title>
            <description><![CDATA[<div class="medium-feed-item"><p class="medium-feed-image"><a href="https://medium.com/@devpal/product-case-study-101-a-4-step-approach-a7645c779772?source=rss-33cf2565f954------2"><img src="https://cdn-images-1.medium.com/max/1200/0*rLqEIbbqnGw0BtyF.png" width="1200"></a></p><p class="medium-feed-snippet">Data Scientists on consumer products are often asked &#x201C;Daily Active Users are down, can you investigate why?&#x201D; or &#x201C;Logins are down&#x201D;&#x200A;&#x2014;&#x200A;I am&#x2026;</p><p class="medium-feed-link"><a href="https://medium.com/@devpal/product-case-study-101-a-4-step-approach-a7645c779772?source=rss-33cf2565f954------2">Continue reading on Medium »</a></p></div>]]></description>
            <link>https://medium.com/@devpal/product-case-study-101-a-4-step-approach-a7645c779772?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7645c779772</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[case-study]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Tue, 23 Sep 2025 18:33:35 GMT</pubDate>
            <atom:updated>2025-09-23T18:33:35.947Z</atom:updated>
        </item>
        <item>
            <title><![CDATA[Data Science Playbook Part 4 — Rent Rewards]]></title>
            <link>https://medium.com/@devpal/data-science-playbook-part-4-rent-rewards-c9d557e88189?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/c9d557e88189</guid>
            <category><![CDATA[startup]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[fintech]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Mon, 25 Aug 2025 20:45:48 GMT</pubDate>
            <atom:updated>2025-08-26T20:29:29.616Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Science Playbook Part 4 — Rent Rewards</h3><h3>The Premise</h3><p>When I joined Rent Rewards as the fourth data hire, I stepped into a company trying to turn rent → into a rewards ecosystem locally in a neighborhood premised by the fact a large XX% share of consumer wallet took place within 5 miles of where a person lived.</p><p>My role quickly expanded to supporting data for two pods → <strong>Dining</strong> and <strong>Payments</strong>, while also supporting our Customer Success team. That meant one week I’d be debugging why a rent payment failed, and the next I’d be supporting analytics on dining partnerships that could drive incremental GMV.</p><h3>Scaling Customer Success with Data</h3><p>Early on, we had recurring issues across rent payments, loyalty points, and customer support escalations. Instead of chasing each problem, I built frameworks to:</p><ul><li>Identify the root cause of recurring issues.</li><li>Quantify their impact on loyalty and churn.</li><li>Prioritize engineering fixes with a data-backed case.</li></ul><p>This shifted CS from a reactive function into a <em>data-informed partner</em> to product.</p><h3>Reimagining Dining as a Growth Pod</h3><p>Dining was where Rent Rewards could move from “just a rent card” to an everyday lifestyle product grounded in the “neighborhood”. We experimented with multiple engagement levers:</p><ul><li><strong>Exclusive partnerships</strong> in neighborhoods where Rent Rewards was the “<em>only”</em> accepted payment method.</li><li><strong>Mobile checkout</strong> in restaurants through the Rent Rewards app.</li><li><strong>Split pay</strong> and <strong>guest pay</strong>, making it easy for members to bring friends into the ecosystem.</li><li><strong>House accounts</strong> where Rent Rewardsstored balances for members and automatically debited at partner restaurants.</li></ul><p>These weren’t just product features → they were experiments in <em>behavioral design</em>. Every transaction became an opportunity to <strong>close the loop between users, merchants, and rewards.</strong></p><h3>Merchant Dashboard</h3><p>We didn’t just think about users: we built data pipelines and dashboards for merchants. Running semantic level dbt models, we surfaced insights on volume, repeat visits, and lift from Rent Rewards-exclusive offers. Merchants could log in and “<em>see the value”</em> of the partnership, and use targeted offers to target users “hyperlocally” that lived in the neighborhood. That transparency was key to scaling partnerships.</p><h3>Ridesharing Promos: Cross selling partnerships</h3><p>One of my favorite projects was quantifying Ride-sharing promos run through Rent Rewards. We discovered that <strong>XX% of renters who got into </strong>Rent Rewards<strong> rental Properties did not have a specific Ride sharing app (</strong>we had partnered with) in their wallet. That insight helped us position Rent Rewards not just as a rent rewards company, but as part of the everyday mobility driving stronger mindshare.</p><h3>Lessons Learned</h3><p>My time at Rent Rewards reinforced a few truths about scaling data science in <strong>early consumer products:</strong></p><ol><li><strong>CX helps you learn what’s wrong at a startup</strong> — Fixing customer support tickets helped me understand the core product failures that customers were facing (e.g. Merchant names on transactions on the Rent Rewards app did not name the merchant as we were the “merchant of record” )</li><li><strong>Ecosystems win when both sides see value</strong> — Building merchant dashboards and showing merchants “on a map” the targeted customer opportunity from neighborhood Rent Rewards properties meant more value in targeted promos.</li><li><strong>Non-obvious data creates leverage</strong> — Rent, dining, and rides may look unrelated, but tying them together created the flywheel.</li></ol><h3>Why This Matters</h3><p>Rent Rewards proved that rewards aren’t just about points → its about neighborhood loyalty (building a local rewards ecosystem where the input is rent).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c9d557e88189" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Science Playbook Pt 3 — Learnings at Postmates]]></title>
            <link>https://medium.com/@devpal/data-science-playbook-pt-3-learnings-at-postmates-26427ba27694?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/26427ba27694</guid>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[experimentation]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[data-analytics]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Fri, 08 Aug 2025 21:20:50 GMT</pubDate>
            <atom:updated>2025-08-08T21:20:50.829Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Science Playbook Pt 3 — Learnings at Postmates</h3><h3>The Background</h3><p><a href="https://postmates.com/">Postmates</a> was the first to pioneer a <a href="https://www.investopedia.com/terms/g/gig-economy.asp">gig-economy</a> based food delivery model, and was eventually surpassed by Tony Xu’s Doordash in marketshare, to be acquired by Uber after Uber’s efforts to acquire GrubHub fell through, and to claim Postmates’ youthful brand, its’ merchant network and the Los Angeles market, where Postmates’ had more than 50% marketshare.</p><figure><img alt="Postmates’ delivery partner in San Francisco PC: https://sfbike.org/" src="https://cdn-images-1.medium.com/max/1000/0*2LCbZtJPDQYvsGLR.jpg" /></figure><p>I joined the Postmates’ Consumer Data Science team of three analysts, where we were given a free hand to figure out the key challenges to driving consumer retention, increase ordering behavior and grow basket size (also, known as average order value).</p><p>Postmates’ delivery robot seen in San Francisco Credit: Bloomberg</p><p><strong>The Problem</strong></p><p>We saw 3 hurdles for consumers as they navigated our app</p><ol><li><strong>Selection</strong> — Did users like the merchants we had?</li><li><strong>Availability</strong> — Was this merchant available and would the delivery come in time?</li><li><strong>Price</strong> — Did the price meet user’s expectations?</li></ol><p>Quickly, I figured that these 3 hurdles were the biggest hurdles to go over, with <strong>selection</strong> finding something you liked involved a <a href="https://en.wikipedia.org/wiki/Decision-making_paradox">decision paradox</a> with more and more merchants, and more and more items. If you got through selection, users faced increasing <strong>price sensitivities</strong> to order a pizza at $14.99, and if they got through <strong>selection and availability</strong>, the merchant may not delivery on time or worse be closed affecting <strong>availability.</strong></p><p><strong>The Solution</strong></p><p>The framing of the solution became</p><blockquote><em>How do we convert all users, who have </em><strong><em>selection</em></strong><em> (find something they like? → Add to Cart), who have </em><strong><em>availability</em></strong><em> (merchant open and delivery within 30 mins) and at a </em><strong><em>price they like?</em></strong></blockquote><p>These users look like users at the <strong>Checkout Page — just before the order is placed.</strong></p><p>I reckoned we should be <strong>converting 100% of users at checkout</strong> who have hurdled through selection, availability, price.</p><p>What were the <strong>key reasons Checkout users</strong> dropped off?</p><ol><li><strong>Pricing at checkout (e.g Merchant fee, Delivery fee, Service fee)</strong></li><li>Address issues</li><li>Miscellaneous (e.g. Changed their mind)</li></ol><p>What were the <strong>key reasons pricing led to dropoff</strong> for Checkout customers?</p><ol><li><strong>Built a classification model to predict the “relative” and “directional impact” of lowered merchant fees</strong>, lowered delivery fees and lowered platform service fees to figure out which fee played a role at price sensitivities at checkout</li><li>Concluded that <strong>reducing delivery fees</strong> <strong>made the most gain on </strong>user checkout</li><li><strong>Rolled out A/B tests to test free delivery for new users</strong> in the LA market, given demand was consistent</li><li><strong>Led to an X% uptick in new users placing their first order on Postmates’</strong> in the Los Angeles market</li><li>F<strong>irst orders led to 65% second orders and 35% third orders</strong>, that generated more revenue for the platform.</li></ol><p><strong>Retention = Growth</strong> and thereby by understanding the context of “why” users at checkout didn’t convert (e.g. Pricing) and solving for it based on the parameters that had the most sensitivity (e.g. Delivery fee over other fees), we were able to get the first orders in for <strong>X% more users</strong>, <strong>who we retained and generated more lifetime revenue offsetting for their acquisition costs.</strong></p><p>Checkout <a href="https://dpal.substack.com/p/data-science-playbook-pt-1-learnings">Part 1 — Instagram and Zynga</a> and <a href="https://dpal.substack.com/p/data-science-playbook-pt-2-learnings">Part 2 — TikTok</a> here</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=26427ba27694" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data Science Playbook Pt 2 — Learnings at TikTok]]></title>
            <link>https://medium.com/@devpal/data-science-playbook-pt-2-learnings-at-tiktok-d2aa077555af?source=rss-33cf2565f954------2</link>
            <guid isPermaLink="false">https://medium.com/p/d2aa077555af</guid>
            <category><![CDATA[data-analytics]]></category>
            <category><![CDATA[tiktok-app]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[product]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <dc:creator><![CDATA[Dev Pal]]></dc:creator>
            <pubDate>Thu, 24 Jul 2025 22:01:05 GMT</pubDate>
            <atom:updated>2025-11-12T18:51:40.596Z</atom:updated>
            <content:encoded><![CDATA[<h3>Data Science Playbook Pt 2 — Learnings at TikTok</h3><p>Recaps learnings at TikTok Shop and going from 0 to $XX GMV</p><p>China has an involution culture — which is competitive and asks you to “roll and turn inwards. Bytedance, TikTok’s parent company entered the US market with the acquisition of a startup called musical.ly, and then grew to replicate Douyin (the name of the Chinese TikTok app) to scale and find mass adoption in the US market. While the US market is makes roughly 50–80% of the revenue of any international consumer app that is ad-driven, China makes roughly 80% of overall TikTok revenue, in broadstrokes.</p><p><strong>Background</strong><br>After my stint on launching Notes at Instagram, I moved over to TikTok, 3 months prior to the E-commerce launch. While most tech companies in the US operate in a pod structure with design, data, product and engineering, the set up for TikTok Shop was different. It was an operations-driven company, looking to blueprint its’ success framework in Asia and bring it to the US market with minimal changes. The Operation leader (let’s called them X), was insistent on using the same rubric to find creators who would sell products with the same enthusiasm as live creators in China, sold products on Douyin. At Meta, bug identification to solution took around 1–2 days, when launching a feature, at TikTok, bugs were identified and neutralized within hours. There was a ton of velocity and eagerness on launching commerce and while TikTok’s revenue so far was ad-monetized revenue, there was increasing <strong>appetite for using a closed loop model (User sees a product → User buys a product fulfilled by TikTok Shop)</strong> vs open-loop (User sees a product → User clicks ad) and semi closed loop (User sees a product → User buys a product fulfilled by third party). Owning the entire experience meant that we had to build trust early as users tend to buy low-priced items (commodity items) first before trusting a platform to buy a TV, Microwave. The stage was set.</p><p><strong>My role</strong></p><p>I spent my initial few months on the Recommendation team where we assessed the number of commerce videos we could show on Short form video controlling for churn. Next, as someone who could access US user data at a granular level (which my Chinese colleagues couldn’t), we had to figure out the following</p><ol><li>Where do we place the Shop tab with minimal disruption to organic traffic?</li><li>What products do feature first to our users?</li><li>Do we feature products with higher quality / high cost vs lower quality / low cost?</li><li>How do we acquire the right products, incubate them and grow them to be “hero products” such that there is reliability of revenue?</li></ol><p>In the first few months, we hustled our way to the first question — understanding that we were displacing XX% of traffic that was flowing in from the iconic For You Page (FYP) to the Friends tab, while Friending was Meta’s core competency (Entertainment being TikTok’s) , it was something that TikTok was playing catchup to (as Meta did for Short Form video). Once we figured that out, there were 3 areas where commerce was served up — LIVE, Short Form Video and Shop Tab (the tab we replaced the Friends tab with, that looked exactly like Amazon’s home page).</p><p>For the next few months, I spent an inordinate amount of time working with a Senior Director (who was locally based, and spoke english → both being rarities in TikTok at the time — where I found myself lucky), to work on a strategy for the following.</p><ol><li>How do we acquire the right products across industry verticals like Fashion, Beauty, Healthcare, Furniture, etc?</li><li>How do we in Minimum Viable way, figure out which products drove the highest traction to “incubate” them?</li><li>How do we ensure a Pareto with 80% of our revenue insured by stable “hero” products that customers kept buying over time?</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Swb0dV1ya0jCHpDL.png" /></figure><p>Source: newsroom.tiktok.com</p><p><strong>Trading off between $ and Order</strong></p><p>For this we had two metrics — GPM and OPM — GPM referred to Gross Merchandise Volume that we generated for 1000 impressions of a product and OPM referred to as Orders we generated for 1000 impressions of a product. Initially, to drive transaction frequency (or, in other words to get the user to buy more products → we focused on driving purchases of low priced products that would drive more mindshare while the long tail of users would finally buy furniture → a model that TEMU and other marketplaces also do).</p><p><strong>Product exposure in exchange for price-match</strong></p><p>Then, we boosted products for sellers who would help reduce the prices for us to be competitive with Amazon, TEMU in exchange for traffic. We were in the business of trading off eyeballs in exchange for ad revenue, GMV sales through product, and in this case sellers helped match the lowest price on the market for more and more exposure. By now, we had scaled GMV ground up from $0 to $XXM, in less than 4 months. I continued to advise the various business teams on (1) Price ($0-$10 vs $20-$30) , (2) Category (e.g. Furniture vs Sport) and (3) Industry (e.g. Home vs Fashion). Although Collectibles (e.g. trading cards) generated high user repeat purchases, high average order value, we deprioritized it since we had to go for on-brand categories like Fashion. There were competitive landscape considerations, given startups like Whatnot primarily drove the Collectibles (trading cards, pokémon cards) marketshare through live (a perishable card reveal use case that generates engagement through principle of scarcity — a “shiny” card that drives intrigue and therefore, sales. With TikTok Shop, the idea was to be a trendy marketplace that would capitalize on the trends that emerged from organic engagement — Fashion was on point.</p><p><strong>Causal link between reviews driving sales</strong></p><p>Lastly, I moved to the User Product Reviews team where the objective was to (1) Grow user reviews (2) understand the causal link between user reviews → sales. My correlative studies so far showed that it was difficult to link sales to reviews on the app — products with reviews had to have sales and therefore the two were related but I quickly wanted to establish a causal link by running a test where hid the star rating altogether to see the impact on conversion. We had seen earlier that given the price was low, conversion rate would inflect even at moderate/ low avg rating — meaning that it was entirely possible for a product with 3.5 rating to have high sales, given the price was &lt;$10.</p><p>In addition to this, the advent of AI took over, and we had to fast follow with <strong>AI Insights</strong> (e.g. tags that summarized wisdom of crowds from reviews such as “T-shirt fits well”) and <strong>AI Summaries</strong> (long prose version of Insights), that were already mature in places like Amazon and were building perception for what a user might expect off reviews. However, the number of users scrolling and seeing detailed reviews were very low and we had to incentivize — (a) users adding more reviews after purchases (b) users seeing detailed reviews to create an appetite for these AI products. All this while, we hadn’t talked about whether the sentiment of a review would actually lead to a proclivity of user to purchase a product, on a marketplace highly weighted by <strong>price, promotion and discounting.</strong></p><p>While this is a brief overview, watch out for Part 3 where I will discuss more on product analytics / data science journeys at consumer tech.</p><p>Please <a href="https://dpal.substack.com/p/data-science-playbook-pt-1-learnings">read Part 1: Instagram and Zynga</a> for my learnings there</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d2aa077555af" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>