<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Product Science - Medium]]></title>
        <description><![CDATA[About the application of data science to User Experience problems and product development - Medium]]></description>
        <link>https://medium.com/productscience?source=rss----43a6e3ba3ec7---4</link>
        <image>
            <url>https://cdn-images-1.medium.com/proxy/1*TGH72Nnw24QL3iV9IOm4VA.png</url>
            <title>Product Science - Medium</title>
            <link>https://medium.com/productscience?source=rss----43a6e3ba3ec7---4</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 20:02:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/feed/productscience" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A/B Testing: Materials for In-Depth Study]]></title>
            <link>https://medium.com/productscience/awesome-ab-experiments-list-718bbb7bcd26?source=rss----43a6e3ba3ec7---4</link>
            <guid isPermaLink="false">https://medium.com/p/718bbb7bcd26</guid>
            <category><![CDATA[ab-testing]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Anton Martsen]]></dc:creator>
            <pubDate>Thu, 27 Sep 2018 11:00:19 GMT</pubDate>
            <atom:updated>2018-10-29T18:45:07.309Z</atom:updated>
            <content:encoded><![CDATA[<p>Recently, I got involved in the adventure of A/B testing. I decided to go beyond simple math and online calculators and dig deeper into this field. In this journey, I found many useful articles, papers, researches and so on. If you, like me, want to become professional in A/B testing, then check the following links in this post. You’ll find many exciting things.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rei8EES-LqLWKtBc" /><figcaption>“group of people with signages” by <a href="https://unsplash.com/@tchompalov?utm_source=medium&amp;utm_medium=referral">Vlad Tchompalov</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h4>Overview of the A/B testing field</h4><p>These links are great to refresh your knowledge, remember the basic formulas, and to learn common mistakes and solutions for them.</p><ol><li><a href="http://hookedondata.org/Guidelines-for-AB-Testing/">Guidelines For Ab Testing — Hooked on Data</a></li><li><a href="http://bit.ly/expSurvey">Controlled Experiments on the Web: Survey and Practical Guide</a></li><li><a href="http://bit.ly/expRulesOfThumb">Seven Rules of Thumb for Web Site Experimenters</a></li></ol><h4>Advanced Topics</h4><h4>Sequential Analysis</h4><ol><li><a href="http://www.evanmiller.org/sequential-ab-testing.html">Simple Sequential A/B Testing — Evan Miller</a></li><li><a href="https://www.auduno.com/2014/12/25/rapid-a-b-testing-with-sequential-analysis/">Rapid A/B-testing with Sequential Analysis | Audun M Øygard</a></li><li><a href="https://www.auduno.com/2015/01/04/estimation-in-sequential-analysis/">Estimation in Sequential Analysis | Audun M Øygard</a></li></ol><h4>Peeking Problem</h4><ol><li><a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">How Not To Run an A/B Test — Evan Miller</a> with wrong conclusions and great response to it <a href="http://elem.com/~btilly/ab-testing-multiple-looks/part1-rigorous.html">A/B Testing Rigorously (without losing your job)</a> (and extension <a href="http://elem.com/~btilly/ab-testing-multiple-looks/part2-limited-data.html">A/B Testing With Limited Data</a>)</li><li><a href="https://www.lucidchart.com/blog/the-fatal-flaw-of-ab-tests-peeking">The Fatal Flaw of A/B Tests: Peeking | Lucidchart Blog</a></li><li><a href="https://blog.acolyer.org/2017/09/28/peeking-at-ab-tests-continuous-monitoring-without-pain/">Peeking at A/B tests: continuous monitoring without pain | the morning paper</a></li></ol><h4>Bayesian</h4><ol><li><a href="https://conversionxl.com/blog/bayesian-frequentist-ab-testing/">Bayesian vs Frequentist A/B Testing (and Does it Even Matter?)</a></li><li>Discussion on Reddit — <a href="https://www.reddit.com/r/statistics/comments/68utz6/frequentist_or_bayesian_ab_testing_methodology/">Frequentist or Bayesian AB Testing Methodology? : statistics</a></li><li><a href="http://cdn2.hubspot.net/hubfs/310840/VWO_SmartStats_technical_whitepaper.pdf">Bayesian A/B Testing at VWO</a></li></ol><h4>Bayesian vs. Peeking Problem</h4><ol><li><a href="http://varianceexplained.org/r/bayesian-ab-testing/">Is Bayesian A/B Testing Immune to Peeking? Not Exactly — Variance Explained</a></li><li><a href="http://blog.analytics-toolkit.com/2017/bayesian-ab-testing-not-immune-to-optional-stopping-issues/">Bayesian AB Testing is Not Immune to Optional Stopping Issues | Analytics-Toolkit.com</a></li></ol><h4>A/A</h4><ol><li><a href="https://1drv.ms/w/s!AuRxCGEOCRKGkdA7TdRyXAOyY-XzRg">The A/A Test</a></li></ol><h4>More</h4><p>In addition to authors that wrote previous papers and articles, I recommend you to check out these resources:</p><ol><li>Ronny Kohavi is a Microsoft Technical Fellow and Vice President of Analysis &amp; Experimentation. <em>You need to explore his project </em><a href="https://exp-platform.com/"><em>ExP Platform</em></a>. Also look for his <a href="https://www.quora.com/What-are-some-good-books-to-read-on-the-practical-side-of-online-experimental-design/answer/Ronny-Kohavi?share=bffabcb1&amp;srid=IWro">recommendations on what to read</a>.</li><li><a href="https://eytan.github.io/">Eytan Bakshy’s blog</a>. Eytan is a senior scientist on the Facebook Core Data Science Team, who lead the Adaptive Experimentation group.</li><li>Evan Miller is a developer of statistical software. He has a series of A/B testing articles on <a href="https://www.evanmiller.org/">his website</a>.</li><li>Papers from the <a href="http://www.kdd.org/">SIGKDD</a> conferences. It’s a community for data mining, data science, and analytics.</li><li>The story “<a href="https://blog.sumall.com/journal/optimizely-got-me-fired.html">How Optimizely (Almost) Got Me Fired</a>” and a <a href="http://pages.optimizely.com/rs/optimizely/images/stats_engine_technical_paper.pdf">paper about The New Stats Engine</a>, where they fix the problems.</li><li>Companies Blogs:</li></ol><ul><li>Uber — <a href="https://eng.uber.com/xp/">Under the Hood of Uber’s Experimentation Platform</a></li><li>Netflix — <a href="https://medium.com/netflix-techblog/quasi-experimentation-at-netflix-566b57d2e362">Quasi Experimentation at Netflix — Netflix TechBlog — Medium</a></li><li>Airbnb — <a href="https://medium.com/airbnb-engineering/https-medium-com-jonathan-parks-scaling-erf-23fd17c91166">Scaling Airbnb’s Experimentation Platform — Airbnb Engineering &amp; Data Science — Medium</a></li><li>Booking — <a href="https://booking.ai/how-booking-com-increases-the-power-of-online-experiments-with-cuped-995d186fff1d">How Booking.com increases the power of online experiments with CUPED</a></li><li>Facebook — <a href="https://research.fb.com/efficient-tuning-of-online-systems-using-bayesian-optimization/">Efficient tuning of online systems using Bayesian optimization — Facebook Research</a></li><li>Quora – <a href="https://data.quora.com/">A Robust Statistical Test for Ratio Metrics</a></li><li>Yandex – <a href="https://research.yandex.com/tutorials/online-evaluation/">Online Evaluation for Effective Web Service Development</a></li></ul><p>P.S. This article will be updated periodically.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=718bbb7bcd26" width="1" height="1" alt=""><hr><p><a href="https://medium.com/productscience/awesome-ab-experiments-list-718bbb7bcd26">A/B Testing: Materials for In-Depth Study</a> was originally published in <a href="https://medium.com/productscience">Product Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>