<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Aleix Ruiz de Villa on Medium]]></title>
        <description><![CDATA[Stories by Aleix Ruiz de Villa on Medium]]></description>
        <link>https://medium.com/@aleixrvr?source=rss-5e972bddb181------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*w1D5KvY9YjuL26mj-LtGEg.jpeg</url>
            <title>Stories by Aleix Ruiz de Villa on Medium</title>
            <link>https://medium.com/@aleixrvr?source=rss-5e972bddb181------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 13:46:43 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@aleixrvr/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Propensity Scores and Inverse Probability Weighting in Causal Inference]]></title>
            <link>https://medium.com/data-science/propensity-scores-and-inverse-probability-weighting-in-causal-inference-97aa53f3b6ce?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/97aa53f3b6ce</guid>
            <category><![CDATA[hands-on-tutorials]]></category>
            <category><![CDATA[propensity-model]]></category>
            <category><![CDATA[bcn-causal-algo]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[editors-pick]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Mon, 01 Feb 2021 18:16:02 GMT</pubDate>
            <atom:updated>2021-03-31T18:32:36.294Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/bcn-causal-algo">Bcn Causal ALGO</a>, <a href="https://towardsdatascience.com/tagged/hands-on-tutorials">Hands-on Tutorials</a></h4><h4>A global overview</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/0*cy4B9dhVPUq2i73J.jpg" /><figcaption>Source: <a href="https://pixabay.com/es/photos/horizontales-edad-pesos-930716/">https://pixabay.com/es/photos/horizontales-edad-pesos-930716/</a></figcaption></figure><p>This is a joint work with <a href="https://medium.com/u/c05c0c26d946">Núria Correa Mañas</a>, <a href="https://medium.com/u/b3fed4bbda55">Jesus Cerquides</a>, <a href="https://medium.com/u/57efda3708cd">Joan Capdevila Pujol</a> and Borja <a href="https://medium.com/u/cd162631459a">Velasco</a> within the <a href="https://medium.com/u/5637a0e9712">Causal ALGO Bcn</a>. You can find a hands-on post by <a href="https://medium.com/u/c05c0c26d946">Núria Correa Mañas</a> <a href="https://nuricorrea.medium.com/a-hands-on-introduction-to-propensity-score-use-for-beginners-856302b632ac">here</a>!</p><p>In this post we are going to talk about two well known techniques used to calculate Average Treatment Effects (ATEs): propensity score analysis and inverse probability weighting. This post assumes you have the very basic notions of causal inference, that is you understand the problem of estimating effects under the presence of confounding.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zGdHYFtuqCnL1I9W" /><figcaption>Confounding variables are those that affect the distribution of the treatment and at the same time the outcome. Image by Author</figcaption></figure><p>The main strength of propensity score analysis is its ability to reduce a multidimensional problem into an unidimensional one. Once propensity scores have been calculated for each observation, we can ensure that we are properly comparing two possibly different populations, the treatment and control groups. Moreover, the effect of the treatment can be posteriorly calculated only based on such scores. Inverse probability weighting contributes with a different numerical formula with the same objective, calculating ATEs.</p><h3>ATEs reminder</h3><p>ATEs stand for average treatment effects. That is, when you have two groups, treated and untreated patients, you want to see which is the effect of the treatment into some outcome (probability to recover, for instance). Unless you run a Randomized Controlled Trial (RCT), you cannot just look at the difference of the recovery between the two, because treated and non treated groups potentially have very different set of attributes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*BqVDCDp73_aM8seF-MXcZw.png" /><figcaption>Treatment groups may have different attributes distribution. Image by Author</figcaption></figure><p>Let us remind first the basics of confounding adjustment through the very well know Kidney Stones problem. In this problem, doctors wanted to know which, among two treatments A and B, had a better recovery rate. When they had a look at the data they found this strange situation.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/968/1*2kBCqO0KstMWZwnKp0Gobg.png" /><figcaption>Simpson’s paradox — Kidney Stones. Image by Author</figcaption></figure><p>When data was stratified by the size of the stone, treatment A was better in both cases. However when analyzed altogether, treatment B was better. It turned out that doctors were assigning treatment previously guessing the size of the stone. Since treatment A was surgery, while B was some kind of pills, larger stones (which were more difficult to cure) tended to be mostly treated by treatment A, producing an imbalance of distributions among treatments. This dynamic can be represented through a Directed Acyclic Graph (DAG)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*W0A95j7MTcn3Dp-d7RHgRg.png" /><figcaption>Image by Author</figcaption></figure><p>In this case X is a variable, but the same formulation works when it is a vector, which will be the case in practice. To remove the effect of confounding (variables that affect both the treatment and outcome), provided no confounder is missing, we can calculate the ATE with the following <strong>adjustment formula.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*vfNCx1GweylqYtMCKcau8g.png" /></figure><p>This formula basically says what would happen if you have treatment <em>t</em>, to everyone, not just the particular selection of patients that happened in the past. Understanding properly this formula takes some time and we will not explain here. References on this topic can be found <a href="http://bayes.cs.ucla.edu/PRIMER/">here</a>.</p><p>Our main goal is to know what would happen if we gave treatment A or B to everyone.</p><h3>Inverse Probability Weighting</h3><p>As we have seen, the main problem with analyzing data is that groups are imbalanced. This can possible create a biased comparison between groups.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AJfER368fKECjseKWCKILA.png" /><figcaption>Motivational example with made up data. Image by Author</figcaption></figure><p>One idea that could come up to our mind is what if we re-weight each groups in a way that they <strong>reflect the global distribution of sizes </strong>instead that the ones of each treatment group. For instance, for the large stones subgroup</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JMLRQS-vM6uGnVn1MgIfkQ.png" /><figcaption>Image by Author</figcaption></figure><p>we can use the weights shown on the figure. These weights precisely are the inverses of the propensity score, the probability of being assigned to a particular treatment group, given patients attributes (we will talk in more detail about this in the next section).</p><p>This intuition can be formally reflected in the following formula, where, multiplying by the propensity score, we arrive at the Inverse Probability Weighting formula</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aAS-5gOI0flCMCn-9rcMzA.png" /><figcaption>Inverse probability weighting formula</figcaption></figure><p>This formula has a numerical problem. We have to divide by the propensity score. In some cases, for some <em>x</em>, it may happen that the probability of treatment may be very low, which can easily increase its variance. So it is not always advised.</p><h3>Propensity Score</h3><p>Let’s view the adjustment formula from another point of view. Recall</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/654/1*vfNCx1GweylqYtMCKcau8g.png" /></figure><p>where X is a vector of attribute with all potential confounders. Imagine you want to compare the effect of the treated and non treated patients to the outcome <em>y.</em> You would like to compare</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*69tHMPNA_Wpax7e2qgr5_Q.png" /></figure><p>So the previous formula can be read as take a patient with attributes <em>x</em>, and calculate the impact of the treatment and non treatment</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/558/1*9fZTAQvkqCsMgSDTVH0wkg.png" /></figure><p>and add these quantities weighting with the frequency you will see these attributes <em>x</em>, i.e. <em>P(x)</em>.</p><p>However there is a problem with this approach: each patient only has one version of the treatment (treated or not), so one of the two terms will not be available (unless of course you have another patient with the same exact attributes, which doesn’t happen often). This is why, in some cases, causal inference problems can be seen as a missing data problem.</p><p>One approximate solution is to find for each patient with attributes <em>x</em>, another with attributes <em>x’</em> as close as possible. This is known as <strong>matching</strong>. This intuitive idea comes to a practical problem, mainly to define what “close as possible” means. This is because different attributes may impact differently on our problem and it is not clear how to prioritize them accordingly.</p><p>Rubins and Rosenbaum in “The central role of the propensity score in observational studies for causal effects” (1983) came to a solution. To find a comparable patient, you don’t need to find another with the same attributes. It is enough to find one which has the same probability of being chosen! This quantity is known as <strong>propensity score</strong>.</p><p>That is, for each set of attributes <em>x</em>, you need to calculate</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/358/1*2IZF0-_Hgy5XagFYhB_8CQ.png" /></figure><p>This can be done with logistic regression (or, in fact, with any machine learning model that suits you).</p><p>Calculating propensity scores for the whole population helps us manage the <strong>common support assumption.</strong> Again, for each patient we need to find a similar within the other treatment group. What if there is some type of patients in one group too different from the rest of the oposite group? this means we cannot find a proper match, so we cannot say what would happen with this patient if we had done the alternative treatment.</p><p>Visualizing the distributions of both groups we can identify regions were there is no proper match.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/638/1*6A3Zn45B9ekEo6PcMDNseg.png" /><figcaption>source: <a href="https://www.slideshare.net/erf_latest/propensity-score-matching-methods">https://www.slideshare.net/erf_latest/propensity-score-matching-methods</a></figcaption></figure><p>Those patients outside the common support should be removed from the comparison. Rubin and Rosenbaum’s paper ensures that after this selection, the comparison is proper, meaning unbiased, so it can be formally done.</p><p>Propensity scores can also be used to calculate ATEs. With some calculations (omitted for simplicity), we can see that the difference of treated and non treated can be calculated in a similar fashion than before, but only using propensity scores</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DItWz3KZogFoMGmUEMfFxg.png" /></figure><p>This formula leads to different algorithms to calculate the ATE, some proposed in Rubin and Rosenbaum’s paper.</p><ul><li>Version 1: For each treated patient find another untreated with same propensity score and calculate the difference in outcomes. Average all the results.</li><li>Version 2: For each level of propensity p sample a treated and untreated patient with that level and calculate the difference. Average all the results.</li><li>Sub-classification: Bin propensity scores and for each level p, calculate the mean treatment difference. Average all the results.</li></ul><h3>Conclusions</h3><ul><li>Average Treatment Effects can be calculated using equivalent formulations: inverse probability weighting and propensity scores</li><li>Inverse Probability Weighting, since potentially dividing by small probabilities can suffer from large variations</li><li>Propensity Scores can be used to find a region of common support</li><li>Propensity Scores can be used in a variety of ways to calculate ATEs</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=97aa53f3b6ce" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/propensity-scores-and-inverse-probability-weighting-in-causal-inference-97aa53f3b6ce">Propensity Scores and Inverse Probability Weighting in Causal Inference</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Causality: AB Testing + Causal Inference]]></title>
            <link>https://aleixrvr.medium.com/causality-ab-testing-causal-inference-14d149933ff2?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/14d149933ff2</guid>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[ab-testing]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Thu, 12 Dec 2019 19:52:51 GMT</pubDate>
            <atom:updated>2019-12-12T19:52:51.193Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><p>Introductory talk at BcnAnalytics</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E4hNh5mReGzL24roRI7HGQ.jpeg" /></figure><p><a href="https://medium.com/u/fefe9b56e142">Bartek Skorulski</a> and myself we gave an introductory talk at one of the regular Barcelona’s events of <a href="http://bcnanalytics.com/">BcnAnalytics</a>. We focused on how combining both techniques, AB Testing and Causal Inference, can give a comprehensive solution to causality problems in businesses. You can see the event here!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FrQKPrry6g0g%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DrQKPrry6g0g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FrQKPrry6g0g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b100f95ff40bf1ccfd779f98393703fc/href">https://medium.com/media/b100f95ff40bf1ccfd779f98393703fc/href</a></iframe><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=14d149933ff2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What to expect from a causal inference business project: an executive’s guide III]]></title>
            <link>https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-iii-53ba7d34a6cc?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/53ba7d34a6cc</guid>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Thu, 12 Sep 2019 15:03:16 GMT</pubDate>
            <atom:updated>2019-09-13T10:49:43.175Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><h4>Part III: Where causal inference stands in the current AI, Big Data, Data Science, Statistics, and Machine Learning scene?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Kqva9RPXxLtM8QyIq13DAw.jpeg" /></figure><p>This is the third part of the post “What to expect from a causal inference business project: an executive’s guide”. You will find the second one <a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-ii-10e521115cb0">here</a>.</p><p>Most of these words have fuzzy meaning, at least at a popular level. Let me define first what will they mean some of them in this post.</p><p><strong>Big data</strong>: All the computing infrastructure devoted to providing access and calculations for querying, preprocessing data or training models with large data sets (they do not fit in your laptop).</p><p>One of the main ideas in big data technologies is that the more data you have, the better. A priori, that’s a fair assumption. However, in some cases, it mistakenly leads to thinking that</p><ul><li>If you have a very large dataset, data speaks by itself and you don’t even need modeling.</li><li>The more degree of detail or granularity the better. For instance, if you want to know where people have been and you track their position each second or millisecond, when actually only hourly data is relevant (depending on the application of course)</li></ul><p><strong>Machine learning: </strong>This is the technical area of expertise devoted to predictive systems. It is a combination of statistics, computing, and optimization. Most of the applications you see in media are in the area of supervised learning. In supervised learning, you have a historical database with many observations, wherein each observation you have a context description and a response variable, the one you want to predict. For example, if you want to automatically read a car plate’s number, you will have a database with images of car plates (context description) and the actual number of each car plate (response variable). This data set is typically labeled by humans (you need a human to read each image and annotate its number). The objective is to build an algorithm or system that for each new car plate image (not in your historical database), it is capable of automatically telling you which is the number in it. The essence of supervised learning is learning by trying to copy the past.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*i70NeQtKrcCVLABGSwvY5Q.png" /><figcaption>source: <a href="https://www.smbc-comics.com/">smbc-comics</a></figcaption></figure><p>Currently, there is an increasing interest in reinforcement learning. Reinforcement learning’s concern is not about recreating the past, but to learn how to take actions optimally. It is widely used in training machines to play games. A very popular case was DeepMind training a machine with reinforcement learning to <a href="https://deepmind.com/research/case-studies/alphago-the-story-so-far">win at Go</a>. There is even a <a href="https://www.alphagomovie.com/">documentary</a> about that!</p><p>Current reinforcement learning relies on machine learning techniques. Moreover, it is mostly used in tasks where you have a simulator of the task you want to learn to do. For example, in learning to play games, you have a computer that can simulate a game.</p><p><strong>AI: </strong>This is probably the fuzziest word among those. We will consider AI as a combination of machine learning, robotics and similar techniques that try to replicate human behavior. You can see a lot of companies talking about AI. They are in fact applying machine learning in tasks related to image, sound or text.</p><p><strong>Data Science: </strong>This is an activity focused on applications, mostly in business, related to predictive analytics and getting insight from data. The core knowledge is a combination of statistics and machine learning, with lots of programming (and close relation to big data).</p><h4>What’s the difference between machine learning and traditional statistics?</h4><p>Machine learning and statistics differ in aim and usage. Let me show it through an example. Again imagine you work in the CausalNews newspaper. You have users paying a monthly subscription and want to increase your retention. Each customer chooses a content type (topic) and a price. You also know from her, her age and recent activity (how often reads your news on her mobile phone, whether she contacted you complaining about problems accessing your website, …). You are interested in knowing for each customer which is the probability of staying the next period.</p><p>Both a machine learning practitioner and statistician will build a model (formula, algorithm, …) relating customer information and probability of staying</p><blockquote>probability of staying = <em>f</em>(age, activity, content type, price)</blockquote><p>Although machine learning and statistics have different types of validating their models, it is even possible that both build the same model! How is the machine learning practitioner and statistician going to use the model?</p><p><em>Machine learning practitioner</em>: Give me all your customers. I will evaluate the probability of staying in the next period. You can start contacting the ones with more risk of stopping their subscription.</p><p><em>Statistician</em>: Younger people have more activity but are willing to pay less. Maybe you want to change your content a bit to get older people. Even more, price is not affecting much to your retention, so it seems you can slightly increase it.</p><p>From this example you can see that <em>machine learning is usually more operational, while statistics is more strategic</em>. The focus of machine learning is devoted to the next period (short term). Statistics is devoted to understanding the main factors affecting your customer’s retention so that you can build medium-large term policies to improve them.</p><blockquote>Machine learning is more operational while causal inference is more strategic</blockquote><h4>What’s the role of causality in all this game?</h4><p>In this example, causality has the same interest as statistics. Help you understand what are the key factors affecting your business. That’s what we call getting insights from data.</p><p>Statistics have been doing this job successfully for many decades. However, it also has some limitations. There is no explicit formalism for causality: causality has an intrinsic directionality that is not reflected in correlations, there is no definition of interventions, … one of the main concerns in statistics is addressing uncertainty (a highly difficult problem). Causality uses knowledge from statistics and goes one step further focusing entirely on cause-effect relationships.</p><h4>Machine learning vs causality cultures</h4><p>There are some crucial key points that are different in both disciplines. While in machine learning, as in big data, the more data the better. In causality, this is far than enough. Having more data only helps to get more precise estimates, but it is not helpful to have correct (unbiased) estimates! You can have infinite data, and yet missed major confounders producing totally mistaken results.</p><p>Either causal inference and machine learning rely on models, so both consider that data hardly speaks by itself. However, in machine learning generally you don’t need any domain knowledge, while in causal inference domain knowledge is <em>essential.</em> Human interaction is a must. That probably will make automate machine learning tasks much faster than causal inference ones.</p><p>Model validation is essentially different in each one. The oracle for most machine learning applications is <a href="https://en.wikipedia.org/wiki/Cross-validation_(statistics)">cross-validation</a> (evaluating your model in an unseen subset of data to estimate the predicting accuracy of the model). In causality, cross-validation is not enough. You need to test your assumptions with and A/B test.</p><p>Besides all this, causality is starting to help to improve machine learning. For example in areas as recommender systems or reinforcement learning. However, this is much more technical and a completely different story, so we will not talk about it here (yet).</p><h3>Why now?</h3><p>Well, that’s a difficult question. There has been a lot of effort from Judea Pearl and colleagues making causal inference more understandable by writing books more accessible for a wider audience. In parallel, there have been many advances in research in epidemiology using the potential outcomes framework (and alternative but equivalent way of dealing with causal inference).</p><p>On the other side, besides the impressive results of deep learning (currently the most popular machine learning technique), we are finding some limitations in the typical supervised learning approach. <a href="https://en.wikipedia.org/wiki/Yoshua_Bengio">Yoshua Bengio</a>, one of the pioneers in deep learning, who received a Turing award for his work in this area, warned, in <a href="https://www.technologyreview.com/s/612434/one-of-the-fathers-of-ai-is-worried-about-its-future/">this</a> interview, about the necessity of including causality in the current machine learning point of view: “I’m not saying I want to forget deep learning… But we need to be able to extend it to do things like reasoning, learning causality, and exploring the world in order to learn and acquire information.”</p><p>Gary Marcus recently wrote an article in the <a href="https://www.nytimes.com/2019/09/06/opinion/ai-explainability.html">New York Times</a> also talking about similar necessities.</p><p>It is true that the process from research to application takes some time. The most popular techniques used nowadays in machine learning started in the 80’s and 90’s. All the research plus the huge advances in computers made them popular today. In that sense, there has been research in causal inference for decades and it seems it is flourishing now.</p><blockquote>We are already taking most of our decisions based on observational data. Let’s do it with a stronger base!</blockquote><h3>Keep reading about causal inference</h3><p>If you want to know more you can read other posts from this blog and the references therein (<a href="https://towardsdatascience.com/why-do-we-need-causality-in-data-science-aec710da021e">https://towardsdatascience.com/why-do-we-need-causality-in-data-science-aec710da021e</a>). There is also a very nice blog by Uber that I would recommend “<a href="https://eng.uber.com/causal-inference-at-uber/">Using Causal Inference to Improve the Uber User Experience</a>” and the books and courses:</p><ul><li>Judea Pearl and Dana Mackenzie’s “<a href="http://bayes.cs.ucla.edu/WHY/">The Book of Why</a>”</li><li>Miguel Hernán course “<a href="https://www.edx.org/course/causal-diagrams-draw-your-assumptions-before-your-conclusions">Causal Diagrams: Draw Your Assumptions Before Your Conclusions</a>”</li><li>Michael D. Ryall, Aaron L. Bramson “<a href="https://www.crcpress.com/Inference-and-Intervention-Causal-Models-for-Business-Analysis/Ryall-Bramson/p/book/9780415657600">Inference and Intervention: Causal Models for Business Analysis</a>”</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53ba7d34a6cc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-iii-53ba7d34a6cc">What to expect from a causal inference business project: an executive’s guide III</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What to expect from a causal inference business project: an executive’s guide II]]></title>
            <link>https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-ii-10e521115cb0?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/10e521115cb0</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <category><![CDATA[causality]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Thu, 12 Sep 2019 15:01:39 GMT</pubDate>
            <atom:updated>2019-09-13T10:49:57.245Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><h4>Part II: Which are the project key points you need to know</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GBakzeYhaokWDwn8FNCzvw.jpeg" /></figure><p>This is the second part of the post “<a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-i-7c5a35c03147">What to expect from a causal inference business project: an executive’s guide</a>”. You will find the third part <a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-iii-53ba7d34a6cc">here</a>.</p><h3>Causal Modeling</h3><p>Casual inference models how variables affect each other. Based on this information, uses some calculation tools to answer questions like what would have happened if instead of doing this I had done that? can I have an estimate of the effect of a variable to another?</p><p>Causal inference provides a broad-brush approach to get preliminary estimates of causal effects. If you want more definitive conclusions, you should go, whenever is possible, for more precise and clear measurements with A/B tests. These do not suffer from confounding and you don’t need any modeling, beyond statistical calculations.</p><p>For modeling relationships with variables, you use a graph (see “<a href="https://towardsdatascience.com/use-causal-graphs-4e3af630cf64">Use causal graphs!</a>”). This will be the foundation of your analysis. Such graph will be based on your human domain knowledge. You are modeling the process that creates your data and how variables affect each other. In the book “<a href="https://www.crcpress.com/Inference-and-Intervention-Causal-Models-for-Business-Analysis/Ryall-Bramson/p/book/9780415657600">Inference and Intervention: Causal Models for Business Analysis</a>” you can find in detail how to build this graph with many business examples.</p><h4>Adding more variables</h4><p>The advertising example from the previous post is a very simplified example. We should expect to have a larger set of relevant variables. For instance, in most cases, we will include seasonality. Culture and sports news usually have different months of activity. And it may be well possible that you used different platforms at different moments in time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GWwIJz_9dsI03tLtomRYTg.png" /></figure><p>It’s important that you include <strong>all</strong><em> </em>the variables that you think may be relevant to your analysis<em> </em>and argue why is it so<em>. </em>This is an exercise that takes some time. Here is a list of types of variables you may want to consider:</p><ul><li>Affecting your causal variable</li><li>Affecting your effect variable</li><li>Contextual</li><li>Ones you can intervene or influence</li><li>Ones you don’t have data but are important, unobserved, missing or similar.</li></ul><h4>Main risk</h4><p>What would happen if we knew that the topic is relevant, but for some reason, we didn’t have this information? In this case, the topic is an unobserved variable, as shown in the following graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MTijh2qKQlUwTNfDDD5RJg.png" /><figcaption>Unobserved topic</figcaption></figure><p>Causal inference tells us that in this case, it is <em>impossible</em> to give a precise estimate of the effectiveness of the media platform. In reality, the lower is the effect of our confounder, the better estimate we have. So we shouldn’t stop here. We should look for other variables closely related to topic that would be enough to run the analysis. We are in fact approximating reality, as a physicist would do. The better the modeling process, the more realistic the results we will get.</p><blockquote>The main risk in causal inference is missing relevant confounders in our analysis.</blockquote><h4>Causal Estimates</h4><p>The last part is taking into account all the information represented in the graph to calculate causal estimates of each arrow we are interested in. We are not going to explain this part because we would need to enter into technical considerations. If you are interested in knowing more, you can read the previous posts from this blog starting from “<a href="https://towardsdatascience.com/why-do-we-need-causality-in-data-science-aec710da021e">Why do we need causality in data science</a>”.</p><h3>What can you expect from a causal inference analysis?</h3><p>From this example, we can have an idea of what is the focus of causal inference.</p><ul><li>Assessing confounding.</li><li>Separating and estimating causal effects, assigning causal credit to each of the causes.</li><li>Deciding whether we can give an answer with our data, we need to include more variables in our analysis or saying we cannot give a proper estimate.</li><li>Using graphs as a communication tool to explicit your objectives, risks and assumptions.</li><li>Quantitative estimates of causal effects.</li></ul><p>There is also a collateral benefit. The exercise of creating a graph describing your business process makes you pose many interesting questions about your business and clarify some concepts you didn’t know before.</p><p>What are the risks of not using causal inference? Causal inference decreases the chances that your analysis is totally wrong. How much wrong? well as we saw in Simpson’s paradox you can easily arrive to the totally opposed conclusion from what reality is.</p><p>Does causal inference guarantee that my analysis is right? No, as we saw, you can still be missing confounders or modeling your causal relations wrong and getting false conclusions! In case you can perform an A/B test, you should take definitive conclusions with it. If you want to perform many tests, you can still use causal inference to prioritize your tests! However, if you cannot perform such tests, causal inference is currently the best tool to make estimates about causal effects.</p><p>You can continue to the third part <a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-iii-53ba7d34a6cc">here</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=10e521115cb0" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-ii-10e521115cb0">What to expect from a causal inference business project: an executive’s guide II</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What to expect from a causal inference business project: an executive’s guide I]]></title>
            <link>https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-i-7c5a35c03147?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/7c5a35c03147</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[causal-inference]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Thu, 12 Sep 2019 15:00:15 GMT</pubDate>
            <atom:updated>2019-09-25T08:44:08.546Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><h4>Part I: When do you need casual inference?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DlVwhasn0P8HBOrV82S8qQ.jpeg" /></figure><p>This is the fifth post on a series about causal inference and data science. The previous one was <a href="https://towardsdatascience.com/solving-simpsons-paradox-e85433c68d03">“Solving Simpson’s Paradox”</a>. You will find the second part of this post <a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-ii-10e521115cb0">here</a>.</p><p>Causal inference is a new language to model causality to help understand better causes and impacts so that we can make better decisions. Here we will explain how it can help a company or organization to gain insights from their data. This post is written for those in a data-driven company, not necessarily technical staff, who want to understand which are the key points in a causal inference project.</p><p>Note: there are two frameworks for causal inference, the potential outcomes, and the DAG’s (Directed Acyclic Graph) approach. Although both are mathematically equivalent, we will focus on the one based on graphs (DAGs).</p><h3>When do I need causal inference?</h3><p>Imagine your company has developed a new website. It is deployed and you wonder whether it has a positive impact or not. To do that, you measure the number of users and compare the old vs the new webpages, getting this plot:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AHd37RYEUiq3cOSkUbwsbw.png" /><figcaption>Number of web users</figcaption></figure><p>Now you ask, is the new webpage a success? Apparently, you have good results, but if you think carefully, you will see it is not that clear. There are some possible explanations: this month traditionally has more activity, your competition has done something wrong, some conditions in your context have changed, … In fact, there is an infinite number of possible reasons.</p><p>How do we address this? Running an A/B test. <em>Before deploying your website</em>, you split randomly your users into two groups assigning them to one of the two versions. Some will enter the new one and some the old one. The point here is that they are all affected by the same context, ensuring that comparison between them is comparable.</p><p>However, A/B tests are not always possible. There are situations where it’s too expensive, it is unethical (imagine you wanted to see if smoking produced cancer on children) or it is unfeasible (your competition has launched a new product — you cannot ask them to run an A/B test for you to measure its effect on your product!). Still, you may probably have some historical data that you think may help you in some way. Well… <strong><em>what you need then is causal inference! </em></strong>You might think that it is most of the cases?! The answer is yes, it is most of the cases.</p><h3>I have data and interpreting it seems pretty simple, I don’t see the problem!</h3><p>There is a huge difference between data that has been obtained <em>experimentally</em> (with an A/B test) and <em>observational</em> data (without an A/B test). Check out the following example from “<a href="https://economix.blogs.nytimes.com/2013/05/01/can-every-group-be-worse-than-average-yes/">Can Every Group Be Worse Than Average? Yes</a>”, where analyzing the evolution of wages from 2000–2013 you have this result</p><h4>Median change in real weekly wages, 2000–13</h4><p><em>Total: +0.9%<br>High School Dropouts: -7.9%<br>High School Graduates, No College: -4.7%<br>Some College: -7.6%<br>Bachelor’s or Higher: -1.2%</em></p><p><em>You have a global increase </em>(+0.9%)<em>, while in each subgroup wages decrease</em>. What?! Imagine you have to make decisions in this scenario: are things going better or worse? It’s not a trivial answer. It may seem a calculation error, but in fact, it is not. When the total and its subgroups have opposite directions, we call it Simpson’s paradox. If you want are a more detailed example, take a look at the kidney stones case in “<a href="https://towardsdatascience.com/solving-simpsons-paradox-e85433c68d03">Solving Simpson’s Paradox</a>”. Again, the same raw data sustains two totally opposed conclusions at the same time!</p><p>Simpson’s paradox only happens in observational data and it is just a particular problem you may find analyzing observational data. Although it seems a rarity, it happens more often than you think. In general, you must be aware of many different biases you may be introducing in your analysis: there is a huge <a href="https://catalogofbias.org/biases/">variety of them</a>!</p><blockquote>The conclusion is that dealing with observational data can be really tricky and can lead easily to wrong conclusions.</blockquote><h3>Let’s see an example</h3><p>Imagine you work in a newspaper called CausalNews. To reach a wider audience, you use two different social media platforms to post your news, Media4You, and MediaForU. Since you are spending a lot of money on them, you want to know which one is more efficient for your company.</p><p>Your data may look like this</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5p3eyPwzO-eke4jnjhEFMg.png" /><figcaption>Example data set</figcaption></figure><p>Now, MediaForU is giving you (10+5+1)/3 = 5.3k visitors per news, while Media4You is only giving you 3k. So that’s it, MediaForU is better!</p><p>Well… as you may have imagined, it’s not that simple. You have two different types of news: culture and sports.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L8_72CcDiMmnyU8x-nVIXg.png" /><figcaption>Introducing news topics</figcaption></figure><p>Typically sports have more readers than culture. Media4You only got cultural news, which is the less popular topic, so the comparison was not made under the same circumstances. In fact, Media4You worked better for the culture section. We can describe the situation with the following graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u_1fCw8TmnzfTLyJbsQSFA.png" /><figcaption>Advertising example</figcaption></figure><p>For many different reasons, you tried both platforms in an uneven way (<em>distribution policy</em>). Thus, which topic the news belongs to affect which platform is used. At the same time, the topic itself has a different type of readers and both platforms may have different levels of performance.</p><p>In our direct calculation (5.3k vs 3k), we were mixing two effects: the effectiveness of the topic and the effectiveness of the platform. If we want to measure the effectiveness of the platform, the topic is acting as a <strong>confounder </strong>because the topic is affecting both at the same time. We are not sure whether we are seeing the effectiveness of the platform or the effectiveness of the topic.</p><p>At this moment, we naturally would like to estimate:</p><ul><li><em>Platform Effectiveness</em>: if both platforms received the same distribution of topics, which one would create more visits?</li><li><em>Topic Effectiveness</em>: What is the intrinsic interest of readers in each topic, regardless of our marketing policy?</li></ul><p>This sums up to estimate the causal effects represented in the diagram by their relative arrows. In the following post, we will describe in more detail how causal inference becomes a methodology to estimate those effects step by step, making explicit the assumptions and risks we are taking in our analysis.</p><p>You can continue reading <a href="https://medium.com/@aleixrvr/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-ii-10e521115cb0">here</a>.</p><h4>Acknowledgments:</h4><p>I would like to thank Nicole Thompson for all the advice she gave me while working together about how to communicate difficult ideas more easily. Thanks, Nicole!</p><h4>References:</h4><ul><li>Judea Pearl and Dana Mackenzie’s “<a href="http://bayes.cs.ucla.edu/WHY/">The Book of Why</a>”</li><li>Miguel Hernán course “<a href="https://www.edx.org/course/causal-diagrams-draw-your-assumptions-before-your-conclusions">Causal Diagrams: Draw Your Assumptions Before Your Conclusions</a>”</li><li>Michael D. Ryall, Aaron L. Bramson “<a href="https://www.crcpress.com/Inference-and-Intervention-Causal-Models-for-Business-Analysis/Ryall-Bramson/p/book/9780415657600">Inference and Intervention: Causal Models for Business Analysis</a>”</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7c5a35c03147" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/what-to-expect-from-a-causal-inference-business-project-an-executives-guide-i-7c5a35c03147">What to expect from a causal inference business project: an executive’s guide I</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Solving Simpson’s Paradox]]></title>
            <link>https://medium.com/data-science/solving-simpsons-paradox-e85433c68d03?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/e85433c68d03</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Wed, 20 Feb 2019 17:53:02 GMT</pubDate>
            <atom:updated>2019-09-13T10:50:34.585Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><h4>Understand a key toy example in causal inference</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Ka-jepZki_H-1DPOpo7xA.jpeg" /></figure><p>This is the fourth post on a series about causal inference and data science. The previous one was <a href="https://towardsdatascience.com/observing-is-not-intervening-9fbd22679f44">“Observing is not intervening”</a>.</p><p>Simpson’s paradox is a great example. At first, it challenges our intuition, but then, if we are able to dissect it properly, gives a lot of ideas about how to handle analysis of observational data (data that hadn’t been obtained through a well-designed experiment). It appears in many data analysis. We will walk through it using the well-known case of kidney stones. The techniques explained here can be found in detail in Pearl’s et al “Causal Inference in Statistics: A Primer”.</p><h3>Kidney stones example</h3><p>In a hospital, a doctor was dealing with kidney stones. She had two treatments, say A and B. She distributed both among her patients in half (350 for A and 350 for B) and measured each treatment’s success. The results can be found in the following table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/486/1*Ez5h_TIVzNkNOy8YsPJrHQ.png" /></figure><p>Clearly, treatment B was the best. The job was done and she could go home. But…wait! She knew she had more information and wanted to have a look at it, just in case…She also had the size of the stones and maybe this information was relevant. Then she came up with this table:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*vNzA-CN2KLm0boD_ShPxSg.png" /></figure><p>Hmmm… For small stones A is better, for large stones A is better but aggregated B is better?! What the f@#*!!</p><h3>What’s going on?</h3><p>Then she recalled how patients had been selected. Once a new patient arrived, she had a guess on their stone. Treatment A was performing some kind of surgery while treatment B was basically giving them pills. She knew that giving pills to large stones was less effective, so the hard cases were assigned to treatment A. This process makes treatment comparison more difficult!</p><p>Using a graph we can show the data process as follows</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*S9rE_q0cQHZY7zzxYlZ3zA.png" /><figcaption>Data generation process</figcaption></figure><p>The size affects both the treatment assignment and the chances of recovery. This is called <strong>confounding</strong>, because you cannot distinguish the effect of treatment from the size on the recovery.</p><h3>Extreme group assignments</h3><p>To understand how assignments can affect our conclusions, imagine these two <em>made up </em>extreme cases:</p><ol><li>A receives only large stones while B receives only small ones</li><li>The other way round</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/734/1*xcmOghUWhPPZZ0LEZUtoYA.png" /><figcaption>Case 1</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/732/1*MUCDf8iuZDNi7gt4jIePLw.png" /><figcaption>Case 2</figcaption></figure><p>As you can see, assigning all hard cases to A, makes A seem to be the least efficient while assigning all hard cases to B makes B seem to be the least efficient. Makes sense!</p><h3>Interventions</h3><p>The question we want to answer is the following: if the hospital had to choose <em>only one treatment</em>, which one would it be?</p><p>As we saw in the previous post <a href="https://towardsdatascience.com/observing-is-not-intervening-9fbd22679f44">“Observing is not intervening”</a>, this can be expressed in a graph that represents the distribution we want to know.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/844/1*O7LPJ8cCaiZdmJV9sWevOQ.png" /><figcaption>Intervening on treatment</figcaption></figure><p>This distribution would respond to the case we gave treatment A to everyone. Intervention on a variable is defined by removing its dependencies with respect to its antecessors (those variables it depends on). The main question in causal inference is whether we can make inferences about this new intervened distribution only having information from the data generating distribution.</p><h3>Adjustment formula</h3><p>The trick is the following: if we focus on small stones, then all the effect of recovery can be explained from the treatment, so we can measure its effectivity</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_AHYiCWZ8ukoSLfpokd7nQ.png" /><figcaption>Conditioning on small size stones</figcaption></figure><p>We can do the same for large stones. But then how do we combine these two quantities? Well, in the way they had been assigned if size would have not affected the assignment of treatment. That means we use the global distribution of stone sizes instead of using the distribution from treatment assignment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*j86iio-4n8FgiO74uDAALw.png" /><figcaption>Distribution of sizes</figcaption></figure><p>As you can see, there is fairly the same number of large and small stones, while each treatment has an uneven distribution of sizes.</p><p>Writing this process down we get the so-called <strong>adjustment formula</strong>.</p><p>P(R|A) = P(R|Small, A) * P(Small) + P(R|Large, A) * P(Large)</p><p>It can be seen (it is not straightforward, though) that this formula is precisely what we were looking for: calculating the probability of recovery in the interventional graph, but only using data obtained from the data generation graph.</p><p>The same can be done for treatment B and then compare the results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/730/1*gCiHUX4qPc3Bovd-FLOqxQ.png" /></figure><p>We go from the initial ‘better B’ 3% up to a ‘better A’ 7%!!</p><h3>Well then, let’s always adjust!</h3><p>Not so fast…There are situations where adjustment will bring to wrong conclusions. In the example above, for measuring the effect of treatment, size becomes a “noisy thing” and we want to remove its effect. Think now in a different situation. Imagine you have treatments for some illness. Moreover, you know (because you measure it) that affects patients blood pressure. At the same time, you know (because you have seen it many times in your career) that blood pressure also affects your chances of recovery. In this case, you would have data generated in this way</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/890/1*xfLSFkgqi2Ft9lxrydlIkQ.png" /></figure><p>Now treatment has a direct way and an indirect way to affect your recovery. But you are interested in both! You don’t want to remove any effect from it. The idea is that in this case, you don’t need to apply any adjustment because the direct measurements will give you the right quantity. Or equivalently, if you intervene on treatment (recall that means removing the dependencies on the causes of the treatments) you are getting the same graph, so you have to do nothing!</p><h3>Conclusions</h3><p>This simple example gives us strong conclusions on how to analyze causal effects in observational data.</p><ol><li>Data does not speak on itself: the direct calculation may bring wrong conclusions</li><li>More data does not solve the problem: you could increase the number of patients and still get the same paradox!</li><li>Correlation is not enough: correlation is a symmetric function while causality is not. The latter has a clear directionality.</li><li>Different models lead to different conclusions: depending on the situation, we have to apply the adjustment formula or not, so having different conclusions. And this has been argued only using the graph!</li><li>Graphs are a great communication tool: we already saw it in “<a href="https://towardsdatascience.com/use-causal-graphs-4e3af630cf64">Use graphs!</a>” and we confirm it once more!</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e85433c68d03" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/solving-simpsons-paradox-e85433c68d03">Solving Simpson’s Paradox</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Observing is not intervening]]></title>
            <link>https://medium.com/data-science/observing-is-not-intervening-9fbd22679f44?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/9fbd22679f44</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Sun, 09 Dec 2018 18:39:16 GMT</pubDate>
            <atom:updated>2019-09-13T10:50:45.825Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><h4>And why conditional probabilities are not enough</h4><p>This is the third post on a series about causal inference and data science. The previous one was <a href="https://towardsdatascience.com/use-causal-graphs-4e3af630cf64">“Use causal graphs!”</a> and the next one is “<a href="https://towardsdatascience.com/solving-simpsons-paradox-e85433c68d03">Solving Simpson’s Paradox</a>”.</p><p>In causal inference we are interested in measuring the effect that a variable A , say a treatment for some particular disease, has on some other variable B, say the probability of recovery, often from observational data. This means that we are interested in measuring the differences of the probability of recovery between the cases A=treated vs A=untreated.</p><p>In data science and machine learning we are used to work with <strong>conditional probabilities</strong>, which may seem useful for this purpose. However, we will see with a simple example that conditioning is not enough.</p><h3>What is conditioning</h3><p>Conditioning on a value <em>X=x</em> means that, among all the possible values that <em>X</em> takes, we select the particular sample of value <em>x. </em>For example, If we had data</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*FDrMFPgbmRbBW94IWCHDog.png" /></figure><p>selecting on X=1 would lead to</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*OfEU83GTzwq6UFclq5GabQ.png" /></figure><h3>Drugstore example</h3><p>Imagine the following situation. We go to a drugstore because we’ve got a cold. They have two different drugs: Drug 1 and Drug 2. Drug 1 is more popular, so we ask for it first: 50% of the time they have it, so we take it and we recover. On the other 50% of the cases, 90% of the time they have a more expensive Drug 2 and we buy it and recover too. But in the remaining 10%, they’ve run out of both, so we keep our cold.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*jLg5ypXcaaO_nvtOBnPECw.png" /><figcaption>Drugstore selling diagram</figcaption></figure><p><em>What if the laboratory stops manufacturing Drug 2? What is our probability of recovery? </em>The answer seems clear: 50% of cases we will not recover.</p><p>Let’s see what happens if we try to answer this question with direct conditional probabilities. If we gather the previous information, we get the <em>Drugstore selling table</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*HDY7yaJQCwCFV_PsFKHIjg.png" /><figcaption>Drugstore selling table</figcaption></figure><p>Now, what is the probability of being cured when there is no Drug 2? For this, we need to select the sample corresponding to Drug 2 = No</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*c0pFbWZyg5pYPiVa8Bh1Cw.png" /><figcaption>No Drug 2 table</figcaption></figure><p>and calculate the probability of recovering</p><p>P(Recovering | Drug 2 = No) = 50 /( 50 + 5) ~ 91%</p><p>Which is a totally different result!</p><h3>What has happened?!</h3><p>We can model the data generation process using a causal graph.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*A20MOtaUFK9A__yecA6ZnQ.png" /><figcaption>Data generation graph</figcaption></figure><p>This graph reflects our observational data. Buying Drug 2 depends on whether Drug 1 was previously bought, and Recovery depends on which drugs have been bought. The data generated by this process is obtained in the <em>Drugstore selling table</em> above.</p><p>However, our question regarding the lack of Drug 2, regards to a different data generation process. One in which Drug 2 does not depend on Drug 1 anymore because Drug 2 is always missing.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/580/1*yxDX2O2i4uCbAY5Cuoa9Wg.png" /><figcaption>Interventional graph</figcaption></figure><p>This graph is called <strong>interventional graph </strong>which models what would happen if there never was Drug 2. If we could sample data from this graph, we would obtain the <em>Interventional data table</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/982/1*gjnp2uhAkLMnGQTJ7ufx2g.png" /><figcaption>Interventional data table</figcaption></figure><p>and we could directly calculate under the new probabilistic distribution P_intervened</p><p>P_intervened(Recovering | Drug 2 = No) = 50 /( 50 + 50)= 50%</p><p>which is the expected result. Why direct conditional probabilities didn’t do the job? Because they answer questions about data generated by the Data generation graph. In some cases is ok, but in this case, it is not enough.</p><p>This toy exercise is a simple example to understand the problems attached to this type of analysis in a general situation. Here we know how data is generated (because we did it). However, obtaining the interventional data table would not be so straightforward. The simple solution would be, go to reality, remove Drug 2 for some time and measure how many people recover. Of course, in many situations, this would have large unwanted consequences.</p><p><em>One of the main questions of causal inference is whether we can derive data from the interventional graph only from gathered observational data without having to go to reality and making an experiment</em>. Meaning, can we get the probability of Recovery in the interventional graph with historical data only? In the following post, we will explain how to obtain such result with the classical Simpson’s Paradox.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9fbd22679f44" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/observing-is-not-intervening-9fbd22679f44">Observing is not intervening</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Use causal graphs!]]></title>
            <link>https://medium.com/data-science/use-causal-graphs-4e3af630cf64?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/4e3af630cf64</guid>
            <category><![CDATA[casual-causal-inference]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Mon, 19 Nov 2018 13:34:05 GMT</pubDate>
            <atom:updated>2019-09-13T10:50:55.220Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><p>This is the second post of a series about causality in data science. You can check the first one “<a href="https://medium.com/@aleixrvr/why-do-we-need-causality-in-data-science-aec710da021e">Why do we need causality in data science?</a>” and the next one “<a href="https://towardsdatascience.com/observing-is-not-intervening-9fbd22679f44">Observing is not intervening</a>”. As we said, there are currently two principal frameworks for working with causality: potential outcomes and with graphs. Here we will continue explaining why is causal inference necessary and how graphs help with it.</p><h3>Graphical models</h3><p>Graphs are an awesome tool. Modeling causality through graphs brings an appropriate language to describe the dynamics of causality. Whenever we think an event A is a cause of B we draw an arrow in that direction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*TETnr1maS79Cn3yNveXfFw.png" /></figure><p>This means that your model is considering a possible causal relation from A to B. This ranges from “A is the main source of causation” to “A hardly explains anything about B”. In particular it comprises the case where A is <em>actually not</em> a cause of B. You would be overmodeling the reality, but the model would not be incorrect. <strong>That’s why, the key assumptions come when you do not put an arrow</strong>. In this case, you are claiming independence between variables, and you should be able to argue and defend why is it so. In summary, the more assumptions, the less arrows.</p><h3>Talking about correlations is not enough</h3><p>Graphs extend statistical language. The simplest example is that correlation is symmetric, while directed graphs are not. I can simply write</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*JfNksTiFsw8A3biHDhs4fg.png" /></figure><p>a causal relation between raining and having a wet floor as a graph. On the other hand, <em>correlation(Rain, Wet Floor) = correlation(Wet Floor, Rain). S</em>o correlation is directionality-blind. If we want to talk about causality, we need a language that can<em> </em>deal with causal behavior.</p><p>In terms of formulas for causal models, we would write</p><p><em>Wet Floor := f(Rain, Randomness)</em></p><p>which means that having a wet floor depends on the rain, and some other random variables (independent from the rain). The key here is the “:=” symbol, which means that rain affects wet floor, but not the other way round. This is not a mathematical equation where both sides are equivalent, but a programatic equation where the value of wet floor is calculated from rain, but the value of rain is not set from wet floor.</p><h3>Causal linear models</h3><p>Linear models are one of the key ingredients of statistics. Through the causal lens, they easily bring out interpretations of direct and indirect effects. For starters, consider the following example. An airplane company who wants to expand its business to a new airport. They choose an airport, bring their fleet, set up a price and sell flights tickets. For some months, they experiment with prices until they reach a profitable and sustainable price. Customers, when buying tickets, take into account the distance from their homes to the airport and the price of the flights. Since they are a growing company, they have been doing this for the past 5 years. These events could be described through the following graph, where <em>Distance </em>is the distance from the city bringing most of their clients, <em>Price</em> is the monthly mean price and <em>Flights</em> is the monthly mean number of flights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/528/1*He82WUfs0g2TcD50TBvnqw.png" /></figure><p>Now, they wonder, for the next airports they want to work with, <strong>how distance affects the number of flights</strong>? Looking at the data, they arrive at the conclusion that linear models are enough in this case. And then they perform a linear regression trying to express <em>Flights</em> from <em>Distance</em> and <em>Price.</em></p><p>The get the resulting linear model</p><p><em>Flights = -10 Distance + -5 Price</em></p><p>which can be read as the following: fixed a particular flight price (say 100 $), if you increase the distance, the number of flights will decrease in 10 units. However, this is not what you want! When you increase the distance, it will also affect the price (it cannot be fixed)! You are interested in how much units of <em>Flights</em> change when you change a unit of <em>Distance</em>,<em> </em>also called <strong>total effect</strong><em>.</em></p><p>From the graphical models perspective, we can put a linear model on each edge, and have the following set of equations:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*PAAOzlM5fgZ8lH6IU6vf1w.png" /></figure><p><em>Price := -3 Distance + Random_1</em></p><p><em>Flights := -10 Distance + -5 Price + Random_2</em></p><p>Now we see two effects of <em>Distance</em> to <em>Flights:</em></p><ul><li><strong>Direct effect</strong> (<em>Distance</em> -&gt; <em>Flights)</em>: Negative perception of customer to distance.</li><li><strong>Indirect effect</strong> (<em>Distance</em> -&gt; <em>Price</em> -&gt; <em>Flights): </em>Increase in distance, forces companies to reduce the price and that produces an increment of flights.</li></ul><p>Working with formulas above, the total effect can be obtained by substitution of variable <em>Price </em>by its relation with <em>Distance</em>:</p><p><em>Flights := -10 Distance + -5 Price + Error_2 = (-10 + (-5)(-3) ) Distance + Random_3 = 5 Distance + Random_3</em></p><p>The total effect in this case is 5 (= -10 + (-5)(-3)). But this has been a toy example. How could we calculate the total effect directly from data? In this case, you can aggregate all direct and indirect effects expressing <em>Flights</em> only from <em>Distance</em>. So the answer is performing a linear regression of <em>Flights</em> only from <em>Distance</em>.</p><p>Observe how conclusions differ from the first and last regression analysis. The first one concluded that distance had a negative effect of -10 in the number of flights, while the last one concluded a positive effect of 5! If management of prices compensates the negative perception of customers on distance, you can still get a positive effect on the number of flights while seeking for less accessible airports.</p><p>Conclusions:</p><ol><li>If you want to calculate the total effect, you should perform the regression of <em>Flights</em> from <em>Distance</em> (unlike the first regression putting all variables in the calculation).</li><li>The use of graphs helps a lot to understand, reason and debate about the whole analysis.</li></ol><h3>Data doesn’t speak by itself</h3><p>We have seen that to calculate the effect of distance to the number of flights, we have relied on the graph. Now imagine a different situation. The airline company, instead of the process described above, they have proceeded as following: they set up a price at which they want to set their flights. After, they look for an airport they think it will satisfy their needs. In this case, we would have the following graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*DOCOjcdaiG1QPPbNm1bGpA.png" /></figure><p>Note that the arrow from <em>Distance</em> to <em>Price</em> has reversed. In this case, changing a unit of distance does not change the price at all! So if we want to know how changing a unit of distance changes a unit in flights, we can directly perform the linear regression</p><p><em>Flights = -10 Distance + -5 Price</em></p><p>and the answer would be -10! This means that the way data is generated and how we model this process is really important. Moreover, this knowledge does not come from data, but from the particular domain itself. We could eventually have two identical data sets, but if the way they have been generated differs, we would probably have to analyze them in a different way.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4e3af630cf64" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/use-causal-graphs-4e3af630cf64">Use causal graphs!</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why do we need causality in data science?]]></title>
            <link>https://medium.com/data-science/why-do-we-need-causality-in-data-science-aec710da021e?source=rss-5e972bddb181------2</link>
            <guid isPermaLink="false">https://medium.com/p/aec710da021e</guid>
            <category><![CDATA[causality]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[casual-causal-inference]]></category>
            <category><![CDATA[causal-inference]]></category>
            <dc:creator><![CDATA[Aleix Ruiz de Villa]]></dc:creator>
            <pubDate>Sat, 10 Nov 2018 11:53:28 GMT</pubDate>
            <atom:updated>2019-09-13T10:51:02.740Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/casual-causal-inference">Casual Causal Inference</a></h4><p>This is a series of posts explaining why we need causal inference in data science and machine learning (next one is <a href="https://towardsdatascience.com/use-causal-graphs-4e3af630cf64">‘Use Graphs!’</a>). Causal inference brings a new fresh set of tools and perspectives that let us deal with old problems.</p><h3>When experimenting is not available</h3><p>First off, designing and running experiments (typically with A/B testing) is always better than using causal inference techniques: you don’t need to model how data is generated. If you can do that, go for it!</p><p>However, there are many situations where this is not entirely possible:</p><ul><li>If your experiments are unethical (you cannot make a child smoke to test whether it causes cancer or not)</li><li>The cause does not depend on you (the competence launches a new product and you want to measure its effect on your sales)</li><li>You have historical data and want to take the most of it.</li><li>Performing experiments is too costly in terms of money or impact, or they are too cumbersome to bring to practice</li></ul><h3>A little bit of history</h3><p>There are three main sources of influence in causal inference: computer science, statistics and epidemiology and econometrics. Active research on causality started in the 80’s.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/517/1*dHYqZ6K36bm6jxvWDkazVQ.png" /></figure><p>The computer science branch has been led by Judea Pearl. Its first influences go back to Sewell Wright in 1920’s, when he wrote about graphical models with linear functions. These techniques evolved and now are currently known as the Directed Acyclic Graphs (DAG) approach.</p><p>The framework most popular in statistics and epidemiology is known as Potential Outcomes framework and was proposed by Jerzy Neyman in 1923. This was the starting point for developing causal inference from a more statistical point of view. Donald Rubin is the most well known in this approach.</p><p>Both frameworks are equivalent, meaning that a theorem in one is a theorem in the other, and every assumption in one can be translated into an equivalent assumption in the other. Differences are a matter of usage. Some problems are easier to formulate in one framework, and some in the other. Jamie Robin and Thomas S. Richardson worked on a framework called Single World Intervention Graphs (SWIG) which acts as a mediator between the two frameworks.</p><p>There are some facts about causality that have been known for a while in econometrics. The techniques more popular among econometricians are:</p><ol><li>Controlled Regression</li><li>Regression Discontinuity Design</li><li>Difference-in-Differences</li><li>Fixed-Effects Regression</li><li>Instrumental Variables</li></ol><p>Of course, they can be formulated in terms of the previous frameworks.</p><h3>Philosophical questions (always in debate)</h3><p>The nature of causality is really hard. The paradox is that we use it continuously on a daily basis, it is common sense in a lot of cases, but finding a definition assessing in which cases A causes B in the real world is very difficult! We would all agree that raining causes the floor wet (I hope… we may find rain deniers, too). But how do we give a clear definition that distinguishes between the rain or the air being the cause of the wetness? How do we know that the rain is the main cause, and not another unobserved variable that causes rain and the floor wet at the same time? Ponder that as much as you’d like…In the eighteenth century, David Hume had already started to think about the nature of causality, and many philosophers have written about it.</p><p>The general agreement in the statistics community is that you cannot prove a causal effect at least without performing an experiment. When you deal with observational data (data obtained passively, without you experimenting), the most you can expect is to talk about correlation (probabilistic dependency). This has created a scenario where talking explicitly about causality in observational data is taboo.</p><p>Even though proving causality in such cases is not possible, there are benefits of talking about causality explicitly. The first one is easy to understand: most of our knowledge as humans about how the world works is observational. You don’t make experiments about all things you know. In some cases, it is not even possible: does the sun cause daylight? how do you make an experiment about it, switching the sun on and off? in this case you could try some type of surrogate experiment and argue that this validates the original hypothesis, but it is not straightforward either. Meanwhile, we all agree that the sun causes daylight. The second argument is to avoid misleadingness. When you analyze data is because you want arrive to some conclusions to take further actions. If you think in that way, is because you think those actions affect (and thus are a cause of) some quantity of interest. So, even you talk about correlations for technical correctness, you are going to use those insights in a causal way. So, if your objective is a causal one, you’d better talk explicitly.</p><h3>Where to start reading</h3><h4>Tutorials</h4><ul><li><a href="https://medium.com/u/1cdc1cca2f85">adam kelleher</a>’s <a href="https://medium.com/causal-data-science/causal-data-science-721ed63a4027">introductory series</a></li><li>Interpreting linear models from a causal perspective: “ <a href="https://ftp.cs.ucla.edu/pub/stat_ser/r432.pdf">Graphical Tools for Linear Structural Equation Modeling</a>”, Bryant Chen and Judea Pearl</li></ul><h4>Books</h4><ul><li>“<a href="https://www.wiley.com/en-us/Causal+Inference+in+Statistics%3A+A+Primer-p-9781119186847">Causal Inference in Statistics: A Primer</a>” , (DAGs framework) Pearl J., Glymour M. and Jewell N.P.</li><li>“<a href="https://www.hsph.harvard.edu/miguel-hernan/causal-inference-book/">Causal Inference</a>”, (Potential Outcomes framework) Hernán M.A. and Robins J.M.</li><li><a href="http://www.mostlyharmlesseconometrics.com/">“Mostly Harmless Econometrics: An Empiricists Companion”</a>, (Econometrics point of view) Angrist J.D. and Pishcke J.</li></ul><p>Who am I? I’m Aleix Ruiz de Villa (<a href="http://aleixruizdevilla.net/">http://aleixruizdevilla.net</a>), Ph.D. in mathematical analysis and ex-head of data science of some companies. I live in Barcelona, Catalonia, and I’m a cofounder of badass.cat and bacaina.cat.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aec710da021e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/why-do-we-need-causality-in-data-science-aec710da021e">Why do we need causality in data science?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>