<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Elvyna Tunggawan on Medium]]></title>
        <description><![CDATA[Stories by Elvyna Tunggawan on Medium]]></description>
        <link>https://medium.com/@elvyna?source=rss-2ebdd87c1f57------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*m34gjBtHK2bPo5IJV3AbPQ.jpeg</url>
            <title>Stories by Elvyna Tunggawan on Medium</title>
            <link>https://medium.com/@elvyna?source=rss-2ebdd87c1f57------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 08:40:39 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@elvyna/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Evaluating Effect of Property Primary Photo Change]]></title>
            <link>https://medium.com/airy-science/evaluating-effect-of-property-primary-photo-change-901ba19760cf?source=rss-2ebdd87c1f57------2</link>
            <guid isPermaLink="false">https://medium.com/p/901ba19760cf</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[causal-impact]]></category>
            <category><![CDATA[airyrooms]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Elvyna Tunggawan]]></dc:creator>
            <pubDate>Fri, 05 Apr 2019 07:12:59 GMT</pubDate>
            <atom:updated>2019-04-05T10:06:17.750Z</atom:updated>
            <content:encoded><![CDATA[<h3>Evaluating the Effect of Property Primary Photo Changes</h3><p>As a technology company innovating in hospitality industry, Airy adopts agile development and lean startup method. Through build-measure-learn loop, we focus on being agile during build step and keep validating our product by measuring its value and learning from the results. Although learning is the last step in the loop, it is actually the first thing we do before starting the build step: we initially set business hypothesis and generate ideas. If we do not measure and learn from each iteration, we might end up building things which are not aligned with our business value [1].</p><p>Prior to measurement, we should already have a hypothesis. On one of our previous studies, we hypothesized that changing the angle of Airyrooms primary photo on search engine results page (SERP) to diagonal view will increase the performance of our conversion funnel (from SERP to property detail page). By displaying photo with diagonal view, we assumed visitors could get a broader view of the room and find their preferred rooms easier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/696/0*u6xitfcIhW7TLLd0" /><figcaption>Figure 1. Bed-front view</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/686/0*m2w2E5to3W9opHXz" /><figcaption>Figure 2. Diagonal view</figcaption></figure><p>This hypothesis could be validated by conducting a research. There are two common research designs [2]:</p><ol><li>Experimental study<br>In this study, researchers attempt to analyze the effect of an intervention by splitting the subject of interest into control and treatment groups.</li></ol><ul><li>Randomized controlled trial (RCT)<br>It is commonly conducted in online platform (known as split test or A/B test). On this experiment, users are assigned randomly to different treatment groups, then the researchers observe whether treatment A is better than B [3]. While RCT has low bias due to randomization, it is expensive and time consuming.</li><li>Quasi-experimental research<br>It shares similar characteristics with RCT, except on the randomization. Quasi-experimental research allows researchers to assign subjects to treatment or control group using other criterion. Although it is subject to confounding variable, it reduces the time and resources needed to conduct the experiment [4].</li></ul><p>2. Observational study<br>Observational study focuses on observing the effect of a treatment, without trying to change which subject is exposed to the treatment. This study is usually conducted when there are any ethical concerns or logistical constraints if the researchers proceed with experiment.</p><p>Considering that the impact of changing the angle of primary photo is lower than the required time and man-hour needed to set up randomized experiment, we decided to conduct quasi-experimental research. Using property as the subject of analysis, we assigned some properties from top-searched locations as treatment group (with diagonal view as primary photo) and kept other properties as control group. We sampled properties from top-searched locations to ensure that the properties are discovered on SERP during the experimentation period.</p><p>Our hypothesis statement is formalized below.<br>H0: properties with diagonal view photo have lower or equal conversion rate compared to properties with bed-front view photo.<br>H1: properties with diagonal view photo have higher conversion rate than properties with bed-front view photo.</p><p>There were two batches of experiment, with different properties listed in the treatment group. How did we measure the results?</p><h3>Causal impact analysis</h3><p>To evaluate the impact, we compare observation from the treatment group and the counterfactual. Since we could not know the counterfactual (i.e. observation that would have occurred without conducting the experiment), we construct a synthetic control group, which resembles the potential outcome given there is no experiment. The effect equals to the differences between actual observation (post intervention) and the synthetic control group.</p><p>There are three sources of information to construct a synthetic control group [5]:</p><ul><li>Time series data of the observation <strong>prior to intervention</strong>. <br>We used historical conversion rate of the properties before the experiment was conducted.</li><li><strong>Similar time series data</strong> which are predictive to the observation.<br>To determine the similarity, we could pick other data which have <strong>positive correlation</strong> to the observation. We picked other properties, i.e. those without photo changes, as the control group because its conversion rate has strong Pearson correlation with treatment groups from both batches (0.75 and 0.58 respectively). Those properties (red line) also have similar conversion trend to the treatment groups (cyan and yellow lines, see Figure 3).</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/876/0*6_zrlqv-Vl1AWQ3q" /><figcaption>Figure 3. Conversion rate of each group</figcaption></figure><ul><li>Available <strong>prior knowledge</strong> about the model parameters.<br>The synthetic control group is based on state-space time series model, which allows us to incorporate prior knowledge about volatility of the data. On this analysis, we do not set prior parameters and let the model determines the parameters itself.</li></ul><p>We used CausalImpact package on R, which resulted in 3 plots (Figure 4 and 5).</p><ul><li><strong>Original</strong>: shows the actual observation (the solid line) and the predicted value (dashed line).</li><li><strong>Pointwise</strong>: shows the difference between the actual observation and the predicted value per each data point.</li><li><strong>Cumulative</strong>: shows the cumulative difference between the actual observation and the predicted value.</li></ul><p><strong>First batch</strong></p><p>On most days after the experiment, the conversion rate is higher than the control group (pointwise plot on Figure 4). We observe 1.27% increase in the conversion rate after the intervention (with a 95% confidence interval [0.90%, 1.67%]), and the result is considered statistically significant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/861/0*sQv-pS7D9y9IrtQm" /><figcaption>Figure 4. CausalImpact results on the first batch</figcaption></figure><p><strong>Second batch</strong></p><p>On this batch, we observe 0.72% increase in the conversion rate after the intervention, with a 95% confidence interval [0.45%, 0.99%]. In spite of having lower increase compared to April 4 batch’s result, it still shows statistically significant result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/854/0*sU8W3zreuoVfL-b8" /><figcaption>Figure 5. CausalImpact results on the second batch</figcaption></figure><h3>Conclusion</h3><p>Although both analysis shows statistically significant results, we should get back to the stakeholders and discuss whether the observed 1.27% and 0.72% increase are considered significant or not in business view. We expect the experiment will increase conversion rate at least by 2%, then it doesn’t prove the hypothesis even though we have statistically significant experiment results. However, we still applied the changes because it is still positively impacting business.</p><p>Changing the angle of a photo sounds simple, yet it actually could improve conversion rate. This idea was proposed by one of our employees. At Airy, we encourage everyone to be bold and innovative. As an employee, everyone could propose an improvement idea (with justification, of course!) along with the hypothesis. A proposal might be implemented if it is aligned to the current company’s OKRs and is feasible to do. If you have lots of ideas and are eager to improving hospitality industry, no worries, <a href="https://www.airyrooms.com/careers">we’re hiring</a>!</p><h3>Reference</h3><p>[1] Burns, B. (2018, April 24). Lean Startup vs Agile. Retrieved from <a href="https://medium.com/giglabs/lean-startup-vs-agile-dab65069392e">https://medium.com/giglabs/lean-startup-vs-agile-dab65069392e</a></p><p>[2] Cohort studies, case control studies, RCTs*. (2016). Institute for Work and Health. Retrieved from <a href="https://www.iwh.on.ca/sites/iwh/files/iwh/at-work/at_work_83.pdf">https://www.iwh.on.ca/sites/iwh/files/iwh/at-work/at_work_83.pdf</a></p><p>[3] Tamburelli, G., &amp; Margara, A. (2014). Towards Automated A/B Testing. <em>International Symposium on Search Based Software Engineering</em>, pp. 184–198. Retrieved from <a href="https://home.deib.polimi.it/margara/papers/ssbse14.pdf">https://home.deib.polimi.it/margara/papers/ssbse14.pdf</a></p><p>[4] Shuttleworth, M. (2008, Aug 13). Quasi-Experimental Design. Retrieved from Explorable.com: <a href="https://explorable.com/quasi-experimental-design">https://explorable.com/quasi-experimental-design</a></p><p>[5] Brodersen, K., et. al. (2015). Inferring causal impact using Bayesian structural time-series models. <em>Annals of Applied Statistics</em>, <em>vol. 9 (2015)</em>, pp. 247–274. Retrieved from <a href="https://ai.google/research/pubs/pub41854">https://ai.google/research/pubs/pub41854</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=901ba19760cf" width="1" height="1" alt=""><hr><p><a href="https://medium.com/airy-science/evaluating-effect-of-property-primary-photo-change-901ba19760cf">Evaluating Effect of Property Primary Photo Change</a> was originally published in <a href="https://medium.com/airy-science">Airy ♥ Science</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[FIFA World Cup 2018: Do the top teams have significantly different skill?]]></title>
            <link>https://medium.com/@elvyna/fifa-world-cup-2018-do-the-top-teams-have-significantly-different-skill-b0a8df919403?source=rss-2ebdd87c1f57------2</link>
            <guid isPermaLink="false">https://medium.com/p/b0a8df919403</guid>
            <category><![CDATA[football]]></category>
            <category><![CDATA[hypothesis-testing]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Elvyna Tunggawan]]></dc:creator>
            <pubDate>Thu, 14 Jun 2018 17:43:01 GMT</pubDate>
            <atom:updated>2018-06-24T11:33:34.050Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*m0M1JpnefV7DPVA0FIMpQA.jpeg" /><figcaption>FIFA World Cup 2018 (Source: Forbes)</figcaption></figure><p>It’s already June, the 21st world cup has already begun! Everyone has his own favorite team; most of my relatives support popular national teams, such as Germany, Spain, Brazil, Argentina, and Portugal. On this post, I’ll use one-way ANOVA and a post-hoc test to analyze whether <a href="https://us.soccerway.com/teams/rankings/fifa/?ICID=TN_03_05_01">FIFA’s top 5 teams</a> have significantly different skill. At the time of writing (June 10, 2018), the top teams are: 1) Germany, 2) Brazil, 3) Belgium, 4) Portugal and 5) Argentina. Before I continue with the analysis, you might be wondering…</p><h3>What is ANOVA?</h3><p>Analysis of variance (ANOVA) is a statistical test to know whether there is any significant difference between groups. If you are familiar with hypothesis testing, you might consider using multiple <a href="http://www.statisticshowto.com/probability-and-statistics/t-test/">t-tests</a> for each pair.. but you’ll have more probability of getting the wrong conclusions.</p><p>For example, you can compare Germany-Brazil, Germany-Belgium, Germany-Portugal and so on using t-test, then conclude which teams are significantly different. Let’s say you have 5% probability of rejecting null hypothesis when it is true (i.e. conclude that the teams are different while they are actually similar — called as <a href="https://en.wikipedia.org/wiki/Type_I_and_type_II_errors">type I error</a>) for each test, and you do the test 10 times.. then the probability of <em>correctly reject </em>null hypothesis is 0.95¹⁰ = 59.87%, which means you’ll have 100% − 59.87% = 40.13% probability of having the type I error. You might not want to use this conclusion, right? That’s where ANOVA comes to the rescue! Here are the advantages of ANOVA:</p><ul><li>it can test more than 3 groups (cool, there are 5 teams to be analyzed here!)</li><li>it can be used when there are more than 1 independent variables (e.g. skill, age — to simplify, I’ll only analyze based on skill)</li><li>it analyzes variance instead of comparing means; which compares variance between teams and variance within teams (you can find more detailed explanation <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3916511/">here</a>)</li></ul><p>There are <a href="http://www.statisticssolutions.com/the-various-forms-of-anova/">several types of ANOVA</a>. Here, I’ll use one-way ANOVA to compare overall skill (dependent variable) between teams (independent variable).</p><p>Although ANOVA can tell us <em>whether </em>the groups are different, we need to do another post-hoc test to check <em>which</em> groups are the different ones. Here comes Tukey Honest Significant Different (HSD) test! Basically, Tukey test conducts pairwise comparison between groups and checks which pair has significant difference.</p><p>Alright, let’s jump into the analysis! <em>Note: code implementation can be accessed </em><a href="https://github.com/vexenta/data-analysis/blob/master/jupyter-notebook/2018-06-08%20FIFA%20World%20Cup%202018%20-%20Tukey%20Test.ipynb"><em>here</em></a><em>.</em></p><h3>Exploratory Analysis</h3><p>The dataset is retrieved from <a href="https://www.kaggle.com/thec03u5/fifa-18-demo-player-dataset/data">Kaggle</a>, which contains 75 columns related to player’s identity, skills, and also preferred positions. Each player could have more than one preferred positions — I’ll take the first preferred position as his main position.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/840/1*68VdUzhHREPk42xPpr-iKw.png" /><figcaption>Figure 1 Dataset preview</figcaption></figure><p>There are 17,981 players from 165 nations, in which each nation has various amount of players. England, Germany, France, Spain, and Argentina have the most players on the dataset. Meanwhile, there are less players as Wing compared to Goalkeeper.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/754/1*CuRtEVpkuhzCRqLxrSx8qw.png" /><figcaption>Figure 2 Player counts by nation and position</figcaption></figure><p>From now on, I’ll focus on FIFA’s top 5 teams as mentioned at the beginning. Please keep in mind that each team has different number of players — and those players aren’t only the ones included as world cup squad.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/472/1*u6o0PefxLvucMRyI6y6hrw.png" /><figcaption>Figure 3 Overall skill distribution across teams</figcaption></figure><p>By observing the distribution, Brazil seems to have the highest overall skill, followed by Portugal, Belgium, Argentina, and Germany. While Brazil, Argentina and Germany have more than 800 players on the dataset, Belgium and Portugal only have 272 and 367 players respectively — thus perhaps the values haven’t converged to the population mean (refer to <a href="https://en.wikipedia.org/wiki/Law_of_large_numbers">law of large numbers</a>). Okay, now let’s see whether those groups are significantly different or not.</p><h3>Hypothesis Test</h3><blockquote>Null hypothesis: the top 5 teams have similar overall skill.<br>Alternative hypothesis: the top 5 teams have different overall skill.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*6lTjXcGJYwTzzGBgDU6Luw.png" /><figcaption>Figure 4 One-way ANOVA using scipy.stats package</figcaption></figure><p>The p-value lies far below our significant level (5%), so it can be concluded that the overall skill across the top 5 teams are significantly different.. but which are the different ones?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*nYsqlFbdvHMIgasx8HY11Q.png" /><figcaption>Figure 5 Tukey HSD test</figcaption></figure><p>Most of the teams have significantly different overall skill, except Argentina-Belgium and Brazil-Portugal. These two pairs have similar range of overall skill. Belgium does have higher average of overall skill, but it has more varied skill compared to Argentina. Meanwhile, Brazil has higher range of overall skill than Portugal (yes, Brazil has won the world cup 5 times, while Portugal’s current best achievement is 1966&#39;s semi final).</p><p>Set aside other factors, if I am asked to select one of these top 5 teams as the most potential champion, of course I’ll count on Brazil. However, it would be nice to see more surprises; should we have a new world champion?</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FTJC1Gjl46HA0E%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FTJC1Gjl46HA0E%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FTJC1Gjl46HA0E%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="244" frameborder="0" scrolling="no"><a href="https://medium.com/media/38b3a90f573f29c7576f115065d184a5/href">https://medium.com/media/38b3a90f573f29c7576f115065d184a5/href</a></iframe><p><strong>UPDATE<br></strong>Thanks to <a href="https://medium.com/u/b3808bf8dd5a">Ali Akbar</a> for noticing the miscalculation of type I error example.</p><h3>References</h3><ul><li>Python for Data Analysis Part 26: Analysis of Variance (ANOVA). <a href="http://hamelg.blogspot.com/2015/11/python-for-data-analysis-part-16_23.html">http://hamelg.blogspot.com/2015/11/python-for-data-analysis-part-16_23.html</a>. Accessed at June 9, 2018.</li><li>Four ways to conduct one-way ANOVAs with Python. <a href="https://www.marsja.se/four-ways-to-conduct-one-way-anovas-using-python/">https://www.marsja.se/four-ways-to-conduct-one-way-anovas-using-python/</a>. Accessed at June 9, 2018.</li><li>Difference Between One Way and Two Way ANOVA. <a href="https://keydifferences.com/difference-between-one-way-and-two-way-anova.html">https://keydifferences.com/difference-between-one-way-and-two-way-anova.html</a>. Accessed at June 9, 2018.</li><li>The T-Test. <a href="https://socialresearchmethods.net/kb/stat_t.php">https://socialresearchmethods.net/kb/stat_t.php</a>. Accessed at June 14, 2018.</li><li>Tukey Test / Tukey Procedure / Honest Significant Difference. <a href="http://www.statisticshowto.com/tukey-test-honest-significant-difference/">http://www.statisticshowto.com/tukey-test-honest-significant-difference/</a>. Accessed at June 9, 2018.</li><li>Hae-Young, Kim. (2014). Analysis of variance (ANOVA) comparing means of more than two groups. <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3916511/">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3916511/</a>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b0a8df919403" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>