<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tushar Bhatkal on Medium]]></title>
        <description><![CDATA[Stories by Tushar Bhatkal on Medium]]></description>
        <link>https://medium.com/@tushar.bhatkal?source=rss-8f80fd09c2c1------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*O1K_S7bBeT8axhceT9yVkQ.jpeg</url>
            <title>Stories by Tushar Bhatkal on Medium</title>
            <link>https://medium.com/@tushar.bhatkal?source=rss-8f80fd09c2c1------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 03:07:45 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tushar.bhatkal/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Kohli or Smith? Statistically Comparing the Two Best Batsmen in the World]]></title>
            <link>https://medium.com/@tushar.bhatkal/kohli-vs-smith-testing-an-hypothesis-in-5-steps-8a6b82ce4f44?source=rss-8f80fd09c2c1------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a6b82ce4f44</guid>
            <category><![CDATA[steve-smith]]></category>
            <category><![CDATA[cricket]]></category>
            <category><![CDATA[hypothesis-testing]]></category>
            <category><![CDATA[statistical-analysis]]></category>
            <category><![CDATA[virat-kohli]]></category>
            <dc:creator><![CDATA[Tushar Bhatkal]]></dc:creator>
            <pubDate>Thu, 26 Mar 2020 06:42:03 GMT</pubDate>
            <atom:updated>2020-03-27T01:41:34.038Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/715/1*AVMvLpRRrRVxDU1bZDHJxA.jpeg" /></figure><p>The Amazon documentary “<strong>The Test — A new era for Australia’s Team”, </strong>has shown <strong>Steve Smith’s</strong> craft in Test cricket. He has been consistently rated as one of the top-ranked Test batsmen in the world, according to the <a href="https://www.icc-cricket.com/rankings/mens/player-rankings/test">ICC Player Rankings</a>. Once he settles down in his innings he reaches that realm where you stop everything and watch the man do his thing.</p><p>Right behind him at number two is the Indian captain <strong>Virat Kohli</strong>, who has shown his dominance in all formats of the game. The two batsmen have almost similar records in Test cricket which got me thinking, does Kohli have a better statistical average per innings than Smith.</p><blockquote>Note : We are using average score over all the innings rather than the Batting Average which is the total number of runs scored by the batsmen divided by the number of times they have been out</blockquote><p>Steve Smith has an average score of 54 per innings. This helps us set our hypothesis statement — <strong>“Kohli has an average score greater than 54 per innings”</strong></p><p>To continue with this test of hypothesis we first need to decide the sample size (n) for the data. To get this we need to answer two questions -</p><ol><li>What is the confidence level?</li><li>How close to the mean do we want the value (or the error we are willing to accept)</li></ol><p>Kohli has played a total of <a href="http://www.howstat.com/cricket/Statistics/Players/PlayerBatGraph2.asp?PlayerID=3600">145 innings</a> over 9 years, with a <strong>standard deviation</strong> of <strong>56.9</strong>. Keeping in mind form, injuries and conditions, we decide to take a loose confidence level of <strong>80%</strong> and allows ourselves an error of <strong>8</strong> runs</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/265/1*YE_AakaagzT6Gia_zMxmdQ.png" /></figure><p>where, <strong>Z</strong> for 80% is <strong>1.28 </strong>(from <a href="http://www.growingknowing.com/GKStatsBookNormalTable1.html">tailed normal standard curve</a>), and <strong>E</strong> is <strong>8</strong>. Substituting the values we get <strong>n</strong> which is approximately equal to <strong>83</strong>.</p><p>Remember the following notations :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/572/1*xfTbJ1kBRMEtIwQPBjfYgA.png" /></figure><p>To decide which our Null and Alternate Hypothesis are and to test this Hypothesis we follow the five steps below.</p><blockquote><strong>STEP 1</strong> — Null Hypothesis has equality</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/515/1*2gCvMG1x6Zv_y7aDIAaHEQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*YmEjtmCEq75N8Ux0QDtnpA.png" /></figure><p>Our initial statement falls in the <strong>Alternate Hypothesis </strong>(greater than 54).</p><blockquote><strong>STEP 2</strong> — Alternate Hypothesis tells which side to reject</blockquote><p>For this step we need to calculate the <strong>Type I error</strong> (alpha) which is the <strong><em>Probability of Accepting the Alternate Hypothesis when the Null Hypothesis is True</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/899/1*k0d-K-yk1TlH4tPEj3jxPA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/649/1*9CiCvKmNHotTcG85Sg1CPQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/448/1*zGll-NBcYkcBPFG3gBBGFQ.png" /></figure><blockquote><strong>STEP 3</strong> — Get <strong>Z</strong> value based on <strong>alpha</strong> (Type I error)</blockquote><p>Using the Normal Standard Curves (Z-table) we find the Z for (0.5–0.2=0.3)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/316/1*gRvKdiT9DRzTedZTw-fqug.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/678/1*iCFcooDXcYiXlHuJglPJFw.png" /></figure><p>This Z value provides us with a decision rule to accept or reject the Null Hypothesis. Therefore, we accept the Null Hypothesis if Z from sample data is less than 0.84 and reject other wise.</p><blockquote><strong>STEP 4</strong> — Using sample data get <strong>Z</strong></blockquote><p>Our sample data consists of the scores from Kohli’s last <a href="http://www.howstat.com/cricket/Statistics/Players/PlayerBatGraph2.asp?PlayerID=3600">83 innings</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/245/1*IqMZDKe5ZR6ZkPo347svbA.png" /></figure><p>The required values are :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/547/1*Xdf7rdUDfUb7AxaRblj0YA.png" /></figure><p>Substituting these in the above formula, we get :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/233/1*IcApIhocCcz_LoKJ_6zUuw.png" /></figure><blockquote><strong>STEP 5</strong> — Compare the Z values from <strong>Step 3 and 4 </strong>and make the conclusion</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*pfsepNt0n_35AeyocnanzA.png" /></figure><p>Therefore, we <strong>Accept</strong> the <strong>Null Hypothesis</strong>, which implies that we <strong>disagree</strong> that <strong>Kohli has an average score greater than 54 per innings. </strong>However, we never agree or disagree directly while testing a Hypothesis. The right way to make this conclusion would be by saying -</p><blockquote>We do not have enough evidence to agree that Kohli has an average score greater than 54 per innings</blockquote><p>Watching these two batsmen over the last few years, one thing is for certain, if anyone has a chance of overtaking Smith as Number 1, it is King Kohli. After all he carries the hope of 1.3 billion people on his shoulders! No pressure :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a6b82ce4f44" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Propensity Score Matching (PSM) vs Bayesian Additive Regression Trees (BART) for Causal Inference]]></title>
            <link>https://medium.com/@tushar.bhatkal/propensity-score-matching-psm-vs-bayesian-additive-regression-trees-bart-for-causal-inference-360ddb8b86f2?source=rss-8f80fd09c2c1------2</link>
            <guid isPermaLink="false">https://medium.com/p/360ddb8b86f2</guid>
            <category><![CDATA[psm]]></category>
            <category><![CDATA[bart]]></category>
            <category><![CDATA[causal-inference]]></category>
            <category><![CDATA[r]]></category>
            <dc:creator><![CDATA[Tushar Bhatkal]]></dc:creator>
            <pubDate>Sun, 09 Feb 2020 15:54:30 GMT</pubDate>
            <atom:updated>2020-02-13T16:56:20.581Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/700/0*Qo59r3htul7OUVbu.png" /></figure><h3>What is Causal Inference?</h3><p>Causal Inference is the process of drawing a conclusion about a causal connection based on the response of the outcome when the cause (treated or not treated) is changed.</p><p>Misunderstanding the causal evidence presented by data can lead to lost time, money and lives.</p><p>Consider, <strong><em>X </em></strong>be a vector of observed covariates, <strong><em>Z</em></strong> be the binary treatment variable, <strong><em>Y(0)</em></strong> and <strong><em>Y(1) </em></strong>are potential outcome (Y(1) when treated [Z=1], Y(0) when not treated [Z=0]) and ‘<strong><em>Y’ </em></strong>is the observed outcome. The Simulation of these variables can viewed in the next section</p><p>Individual level causal effects compare potential outcomes.</p><blockquote>Individual Treatment Effect = Y ᵢ(1) — Y ᵢ(0)</blockquote><p>The typical goal in causal inference is to estimate the Average Treatment Effect (ATE). Depending on the estimand required for the analysis we either calculate <strong>Average Treatment Effect (ATE) </strong>which estimates the effect for the entire sample (treated and not treated individuals), <strong>Effect of Treatment on Treated (ATT)</strong> which provides estimates of the treatment effect for those who are in the treatement group, and <strong>Effect of Treatment on Control (ATC)</strong>, which provides an estimate of the treatment effect for the control group only.</p><blockquote>ATE = E[Y(1) — Y(0)]</blockquote><blockquote>ATT = E[Y(1)-Y(0)|Z=1]</blockquote><blockquote>ATC = E[Y(1)-Y(0)|Z=0]</blockquote><h3>Why Causal Inference is hard?</h3><p>The fundamental problem here is that we only observe <strong>one</strong> potential outcome for each individual (Either Y(1) or Y(0)) or in other words the counterfactual is missing. Statistical inference for causation can be made for randomized experiments, but not for observational studies.</p><p>For most causal questions we typically cannot perform experiments as the appropriate natural experiments are hard to find. In observational studies, one cannot rule out the possibility that <strong>confounding variables </strong>are responsible for group differences in the observed outcome.</p><blockquote><strong>Confounders</strong> are covariates that effect the outcome (<strong>Y</strong>) as well as the treatment (<strong>Z</strong>). In most policy studies covariates like <strong>Income</strong> tend to be confounders as the policies are made available depending on income bracket of the individual and the income also plays a role in the outcome of the policy.</blockquote><p>To overcome this, we can use constructed observational studies, however they require strong assumptions to be satisfied:</p><ul><li>Structural Assumptions : <strong>Ignorability</strong> (All confounders are known to the model), Stable Unit Treatment Assumption Value or <strong>SUTVA</strong> (It means that there is no interaction between the individuals and the treatment of one individual doesn&#39;t affect the other)</li><li>Parametric Assumptions : The parametric form of the model should be similar to the response (eg: for linear relations use linear models). This assumption gets stronger when try to adjust for many covariates (more complex models).</li></ul><h3>Propensity Score Matching</h3><p>Matching refers to the procedure of restructuring the original sample in preparation of statistical analysis.</p><p>In an ideal world, we want to control all the confounding covariates, in the vector <strong>X</strong>. However, not all of these are observable in most cases.</p><blockquote>The propensity score theory says that rather than controlling for all the variables in <strong>X</strong>, it is sufficient to control for just the propensity score <strong>e(X)</strong>. This <strong>e(X) </strong>can be considered as a one number summary of all the confounding covariates of <strong>X</strong>.</blockquote><blockquote>e(<strong>X</strong>) = P(Z=1|<strong>X</strong>) = E[Z|<strong>X</strong>]</blockquote><p>We can estimate e(X) with standard software using logistic or Inverse Logit functions.</p><blockquote>Therefore, <strong>if ignorability holds</strong> (there are no other confounding variables to be considered) and if we <strong>properly control</strong> for our estimate of e(X) we should be able to get unbiased treatment effects estimates.</blockquote><h3>Using BART for Causal Inference</h3><p><a href="https://projecteuclid.org/euclid.aoas/1273584455">Bayesian Additive Regression Trees</a> (BART) is the Bayesian form of boosted regression trees. It can be used to estimate causal effects as:</p><blockquote>E[Y(1)|X=x] — E[Y(0)|X=x] = f(1,x) — f(0,x)</blockquote><p>Each iteration of the BART Markov chain generates a new draw of <strong><em>f</em></strong> from the posterior distribution.Then average over this to get the posterior distribution for the sub-population under interest.</p><p>To estimate the average causal effect:</p><ul><li>Fit BART to the entire sample.</li><li>Get posterior prediction for each treated unit at <strong>the observed treatment condition </strong>and <strong>the counterfactual condition</strong>.</li><li>The differences between these predictions create posterior distributions for individual level treatments.</li><li>Averaging over this will give the posterior distribution for sub-populations of interest (full sample (ATE), treated (ATT), control (ATC))</li></ul><h3>Simulation Analysis</h3><p>Let’s simulate hypothetical data to understand the benefits and drawbacks of using PSM and BART.</p><p><a href="https://github.com/itushar/PSM-vs-BART-Analysis-and-Simulation/blob/master/PSMvsBART.md">The R Notebook for this simulation can be found on github link</a></p><h3>Data Description</h3><p>This analysis simulates hypothetical data for children with their family details. Here the treatment group consists of children who attend private schools. The question of interest here is the effect of attending private schools on their overall performance scores.</p><p>The data includes overall income of the family (<em>income</em>), Mother’s age (<em>momage</em>), father’s age (<em>dadage</em>), number of events attended by parents (events), number of siblings (<em>siblings</em>).</p><p>Since we are trying to estimate the <strong>effect of attending private schools</strong> on their overall performance scores. The estimand to be observed is the <strong>effect of the treatment on the treated</strong> or ATT. The <strong>ATT</strong> here would give us the effect of attending private schools on overall performance <strong>for those who indeed attended private school</strong>.</p><p>We create <strong>three</strong> different worlds with varying response and assumptions:</p><blockquote>The response surface for <strong>World A</strong> is linear and all the assumptions hold.</blockquote><figure><img alt="Potential Outcomes for World A" src="https://cdn-images-1.medium.com/max/1024/1*_zK4yIKR4Lop0G2PnKI8Ug.png" /></figure><blockquote>In <strong>World B</strong> one of the response surface should be <strong>non linear</strong> and other assumptions hold.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wDXjqIS-N4VkORwA3Rsl4g.png" /></figure><blockquote><strong>World C </strong>has the same response surface as <strong>World B</strong> but violates one of the assumptions.</blockquote><h3>Data Generation Process</h3><p>We generate the covariates (confounders) by using <a href="https://www.rdocumentation.org/packages/rockchalk/versions/1.8.144/topics/mvrnorm">mvrnorm</a> function which creates multi-variate normal distribution. Then calculate the ATE/ATT for all three worlds along with a plot of Confounder (here income) vs Outcome (Y or performance score). (<a href="https://github.com/itushar/PSM-vs-BART-Analysis-and-Simulation/blob/master/PSMvsBART.md">Click to view R notebook</a>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/299/1*EdrVgnvVbyIGIwaa1c5rKw.png" /></figure><p>Lets plot for Income vs Outcome</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T0xt0SqYPoPD3uWpaYCvTA.png" /></figure><p>For World A, Linearity holds and there is enough overlaps from the plot. Lets observe the same for World B and World C :</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/311/1*ObND0rJUqTUBKst0GcSMcg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DfawGARbAYglrvyxPk52FA.png" /></figure><p>For World B, Linearity dosen’t hold, Ignorability holds as all confounders are considered (for the Propensity score).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/291/1*cfvZkfRqh9UW8qD4gPaxUg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DDzA9OtFaZxfa8cJVhba5g.png" /></figure><p>For World C, Linearity dosen’t hold. Ignorability is also not satisfied as all the confounders are not considered (We use another confounder which isn’t considered in the model).</p><h3>Bias Analysis</h3><p>We use a linear model to understand the Bias for the estimates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/570/1*w0w5k5DdQEiBaehEMOyF0w.png" /></figure><p>In World A, all the assumptions hold and hence we see very low Bias in the estimated ATT. This estimate is also close to the SATT (Actual ATT measure for the Sample)</p><p>In World B, The response surface for the potential outcome when treated (Y1) is non-linear (polynomial function). Hence the Parametric Assumption are weak and the overlap reduces as well leading to an increase in the bias. This causes the estimated ATT for world B to move away from the SATT.</p><p>In World C, Ignorability dosen’t hold (or weak) as all the confounders aren’t considered. This causes further increase in the bias.</p><h3>Results</h3><h4>For Propensity Score Matching :</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YVMGx74V5G916rNEpuv3AA.png" /><figcaption><strong>SATT</strong> - Actual ATT of the Sample, <strong>ATT</strong> - Estimated ATT of the Sample using PSM</figcaption></figure><p>Using Linear Regression model, even though the model is not a good predictor it gives a good estimate assuming ignorability. A lot of times it ends up excluding key interactions and assumes linearity when the relation is actually non-linear. Using Propensity Score Matching we can compare estimates for constructed observational studies to original Randomized experiments. After PSM (Caliper Matching) we get the ATT as <strong>20.02 ± 0.13</strong> for World A.</p><blockquote>We can say that the children who attended private schools had overall performance scores that were 20 units higher than if they didn’t attend private schools.</blockquote><h4>For BART :</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/971/1*OL2pujdBiE2F_bak4nessA.png" /><figcaption><strong>SATT</strong> — Actual ATT of the Sample, <strong>ATT</strong> — Estimated ATT of the Sample using BART</figcaption></figure><p>We observe that BART gives estimate close to the SATT even when there is Bias present. It is able to find a flexible fit for the response surface even when it is non-linear.</p><p>We get the ATT as <strong>28.3 ± 0.73</strong> for World A. We can say that the children who attended private schools had overall performance scores that were 28 units higher than if they didn’t attend private schools.</p><h3>Conclusion</h3><p>Propensity Score Matching is useful to estimate causal effect when the confounding covariates are know (through research). Here the ignorability assumption must be strong as the inference is drawn using scores (e(X) or propensity score) derived from what we think the confounders are (forcing overlap)</p><p>Even if ignorability holds, imbalance and overlap could still be issues which let’s us evaluate the adequacy of the propensity scores and the model used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DJ9pKOiYkigK6ZhU7LSiOQ.png" /></figure><p>BART constructs a counterfactual for each data point with a certainty interval, so that for those points which have empirical counterfactual the certainty interval is small and those without the empirical counterfactual the interval gets bigger. Using the common support rule we can discard points which have high uncertainty. Hence even if there is lack of overlap it performs well.</p><p>Hence, BART is useful to estimate treatment effects when the confounders have lack of overlap. It incorporates the outcome for understanding what covariates are important common support. It also provides a flexible fit to varying response surface and can work on weak ignorability assumptions which is common in real world data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=360ddb8b86f2" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>