<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sarah Nabelsi on Medium]]></title>
        <description><![CDATA[Stories by Sarah Nabelsi on Medium]]></description>
        <link>https://medium.com/@s.nooravi17?source=rss-9745e35c4a41------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*aBeTq3nCmAMZzelb</url>
            <title>Stories by Sarah Nabelsi on Medium</title>
            <link>https://medium.com/@s.nooravi17?source=rss-9745e35c4a41------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 05:47:05 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@s.nooravi17/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Should I Become a Data Scientist]]></title>
            <link>https://medium.com/glad-community/should-i-become-a-data-scientist-d3938941f395?source=rss-9745e35c4a41------2</link>
            <guid isPermaLink="false">https://medium.com/p/d3938941f395</guid>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sarah Nabelsi]]></dc:creator>
            <pubDate>Fri, 14 Dec 2018 20:18:36 GMT</pubDate>
            <atom:updated>2018-12-14T20:18:36.020Z</atom:updated>
            <content:encoded><![CDATA[<p>There is no shortage of articles attempting to lay out a step-by-step process of how to become a data scientist. “It’s easy! Are you a recent graduate? Do this… Are you changing careers? Do that… And make sure you’re focusing on the top skills: coding, statistics, machine learning, storytelling, databases, big data… Need resources? Check out Andrew Ng’s Coursera ML course, …”. Although these are important things to consider once you have made up your mind to pursue a career in data science, I hope to answer the question that should come <strong>before</strong> all of this. It’s the question that should be on every aspiring data scientist’s mind: “should I become a data scientist?” This question addresses the <em>why</em> before you try to answer the <em>how</em>. What is it about the field that <em>draws</em> you in and will <em>keep</em> you in it and excited for years to come?</p><p>In order to answer this question, it’s important to understand how we got here and where we are headed. Because by having a full picture of the data science landscape, you can determine whether data science makes sense for you.</p><p><strong>Where it all started…­­</strong></p><p>Before the convergence of computer science, data technology, visualization, mathematics, and statistics into what we call data science today, these fields existed in siloes — independently laying the groundwork for the tools and products we are now able to develop, things like: Oculus, Google Home, Amazon Alexa, self-driving cars, recommendation engines, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xmRqMCYw8ps-XMYTz-RR8g.png" /></figure><p>The foundational ideas have been around for decades… early scientists dating back to the pre-1800s, coming from wide range of backgrounds, worked on developing our first computers, calculus, probability theory, and algorithms like: CNNs, reinforcement learning, least squares regression. With the explosion in data and computational power, we are able to resurrect these decade old ideas and apply them to real-world problems.</p><p>In 2009 and 2012, articles were published by McKinsey and the Harvard Business Review, hyping up the role of the data scientist, showing how they were revolutionizing the way businesses are operating and how they would be critical to future business success. They not only saw the advantage of a data-driven approach, but also the importance of utilizing predictive analytics into the future in order to remain competitive and relevant. Around the same time in 2011, Andrew Ng came out with a free online course on machine learning, and the curse of AI FOMO (fear of missing out) kicked in.</p><p><strong>Where we are now…</strong></p><p>Companies began the search for highly skilled individuals to help them collect, store, visualize and make sense of all their data. “You want the title and the high pay? You got it! Just please come and come quick.” With very little knowledge of what they were looking for, job postings went up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*E4ll8CO9sw_mj53AX-e_IA.png" /></figure><p>If you searched ZipRecuiter today, you’d find over 190k open data science positions currently open, each one looking for their own data unicorn. Thus, in an effort to get talent in the door, the definition of what it meant to be a data scientist soon widened with definitions varying from company to company and person to person.</p><p>On the other hand, candidates saw a great opportunity: a career with high pay, high demand, and the promise of job security and glory. Everyone rushed to develop all the right skills with one goal in mind: to hold the “sexist job of the 21st century”.</p><p>We have the demand and we have the supply, so what’s the problem? Well, the problem isn’t a shortage of programs to support that demand and capitalize on the hype. It feels like every day there are new courses being developed to satisfy the cravings from aspiring data scientist to break into the field: master’s programs, boot camps and online courses. It’s an arms race to make the right courses with the promise of a Machine Learning job at the end of it. “No PhD? No problem. Just three to six months and a small investment of ~10–15k and you’ll be guaranteed a well-paid job upon graduation.” (wink)</p><p>These programs are designed to be a one-stop-shop for everything data science: you learn the programming, the visualization, the modeling — it’s all there. What you soon discover is that many (surely, not all) of the business problems being faced can be solved using similar approaches, so if you’re looking to apply some algorithm, chances are there’s a library that already exists to help you do just that. Simple right?</p><p><strong>Hold up…</strong></p><p>If you’ve been paying attention so far, you will have picked up on a few important things so far:</p><p>1. By getting ahead of themselves, companies are hiring data scientist before they have even started collecting the right data (i.e. they are suffering from the <a href="https://towardsdatascience.com/the-cold-start-problem-with-artificial-intelligence-49938ed3f612"><strong>Cold Start Problem of AI</strong></a>), meaning you will need to be involved in <em>every</em> step of the data pipeline including data collection, storage, and visualization before you can get to the modeling.</p><p>2. Rushing to get a job in data science (going through one of the above-mentioned methods) means you will be competing against hundreds of thousands of others in the same exact position. Expect that they will have similar projects to yours and similar experience. To get yourself noticed, you will need to find a way to differentiate yourself: <strong>showing your creativity and grittiness.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/225/1*MoG6Vx3oIDWSJzZBxcsH1g.png" /></figure><p>3. <strong>Chances are you won’t be developing algorithms from scratch</strong>. Unless you have a lot of extra time on your hands, you’ll most likely on the existing and well-trusted libraries. Why compete against a group of PhDs that helped develop the library and risk putting something less than optimal into production unless you had to develop something specific to your use-case.</p><p><strong>What’s to come…</strong></p><p>I believe, a few major things will change from here.</p><p>First, the data scientist role will become <em>well-defined</em> in terms of the skills needed to do the job and the expected impact on businesses. Second, automation of AI models will come. Don’t believe me? It’s already happening with products like Google AutoML and software packages like TPot. What seemed like the most technically challenging and appealing part of the job will be automated. As they say now: “No ML expertise? No problem!”</p><p>The implications of the trajectory on which we are headed is that:</p><ol><li>No longer will data engineers or data analysts be mistakenly given the title or the pay of a “real” data scientist, which will likely dwindle down the number of data scientists in the market.</li><li>Human insight and oversight will become increasingly more important. Not to say that technical expertise will become less important, but the importance of connecting the analytics to the business and knowing how software interacts with people will increase (i.e the human side of data science).</li><li>Lastly, since the Machine Learning part of the job behaves most consistently, it will be the first part to be automated (and we see that happening already). This means, that most of the value that data scientists will bring will involve the unification of messy data, preparation for modeling, and evaluation.</li></ol><p>Therefore, when trying to answer that question for yourself “should you become a data scientist?”, imagine yourself through the entire hype cycle. Could you see yourself growing in fast growing and evolving field that will challenge you in every way? Can you embrace the change and adapt yourself just as fast? If so, then this is the career for you…welcome aboard.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d3938941f395" width="1" height="1" alt=""><hr><p><a href="https://medium.com/glad-community/should-i-become-a-data-scientist-d3938941f395">Should I Become a Data Scientist</a> was originally published in <a href="https://medium.com/glad-community">GLAD Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creativity in Data Science]]></title>
            <link>https://medium.com/glad-community/creativity-in-data-science-1faf259ca643?source=rss-9745e35c4a41------2</link>
            <guid isPermaLink="false">https://medium.com/p/1faf259ca643</guid>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sarah Nabelsi]]></dc:creator>
            <pubDate>Sun, 18 Nov 2018 20:21:18 GMT</pubDate>
            <atom:updated>2018-11-18T20:21:18.124Z</atom:updated>
            <content:encoded><![CDATA[<p>Most people in the data science space might find it hard to say what they do is creative: writing SQL queries, building models, designing A/B tests, and creating dashboards. That all sounds pretty technical — where does the creativity come in? I wondered the same thing until I took the time to research creativity and understand what it is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_2VSnWIQuJvYCWVh" /></figure><p>Turns out, our biggest mistake is confounding creativity with the arts (i.e. only a painter, writer, musician, or actor can be creative). Either that, or we think that only the most brilliant minds can be truly creative — those that win Nobel prizes or make breakthroughs in their fields. Well, I am here to tell you this is wrong…</p><p>Businesses all over the world rely on creative people in all fields to move their business forward — to continue to innovate and win against the competition. So, what is creativity then?</p><p>In the words of Ken Robinson “creativity is the process of having original ideas that have value”. So, it boils down to three things:</p><ol><li>First and foremost, it’s a process. It’s not a moment in time, but something that happens over time. As John Doerr said “Ideas are easy. Execution is everything. It takes a team to win”. Coming up with the idea is only part of the process — proper execution on that idea will take time.</li><li>Next, the idea needs to be original. Original ideas don’t have to be new to the whole world, they just have to be new to you or to your community. Take Henry Ford for example: his inspiration for the car assembly line came when he saw the disassembly lines used to process meat! Not a new idea, but new to his industry!</li><li>Lastly, it has to be something of value. Is this worth-while? Judge the ideas on whether they are useful.</li></ol><p>Now when it comes to creativity, there is one main prerequisite. It’s the necessity of domain knowledge. For example, you can’t be creative on the piano if you don’t have some familiarity with piano. Sure, you can find a way to play some keys, but it won’t be the same as if you had been playing for several years. The same thing applies to other domains like mathematics — imagine being creative in your approach to solving a problem if you didn’t understand its mathematical underpinning. So, domain knowledge here is key.</p><p>Now, what is creativity in data science? We say it’s important and seek creative candidates… but how do we define it and cultivate it in ourselves and in our teams? Before saying what creativity in data science is, let’s talk about what it isn’t…</p><p>Well, it isn’t brute forcing your favorite algorithms to a dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*nvXGb3OO1D4Mt7l_JVOVNQ.jpeg" /></figure><p>This approach will lead you to miss much of what is critical to that domain (be it script writing, gaming, finance, etc.). You might end up neglecting to account for an important feature while modeling or limit your analysis resulting in uncovering only the obvious. New and creative ideas will come when you take the time to understand the nuances of the domain and ask the hard questions of your data.</p><p>That leads us to our definition of creativity in data science, which is: the marrying of statistical knowledge with domain knowledge and a little bit of mind wandering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/274/0*4rNCyQ010A1I_ZOU" /></figure><p>This touches on the importance of both domains (analytics and industry), but also highlights this idea of allowing your mind to wander. This means keeping an open mind while experiencing or learning something new and imagining how you could use that new knowledge and apply it to a problem you are trying to solve.</p><p>More tangibly, creativity in data science can be anything from innovative features for modeling, development of new tools, cool new ways to visualize data, or even the types of data that is pulled for analysis. What’s interesting in data is that everyone will do things differently, depending on how they think about the problem. When put that way, almost everything we do in data science can be creative if we think outside the box a little bit…</p><p>The best way I could think to describe creativity in a candidate or in an approach is when they give you this moment of “wow, I never thought about it that way.” Ideally as a company or team, you want to maximize the number of moments like this — keep good ideas flowing, prioritize, and execute.</p><p>How are you being creative in your work today?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1faf259ca643" width="1" height="1" alt=""><hr><p><a href="https://medium.com/glad-community/creativity-in-data-science-1faf259ca643">Creativity in Data Science</a> was originally published in <a href="https://medium.com/glad-community">GLAD Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You say you want statistical significance?]]></title>
            <link>https://medium.com/glad-community/you-say-you-want-statistical-significance-76de0046df88?source=rss-9745e35c4a41------2</link>
            <guid isPermaLink="false">https://medium.com/p/76de0046df88</guid>
            <category><![CDATA[marketing]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sarah Nabelsi]]></dc:creator>
            <pubDate>Sun, 18 Nov 2018 20:18:26 GMT</pubDate>
            <atom:updated>2018-11-18T20:18:26.916Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*0v9nJTPg_CJC-Xddx47keg.png" /></figure><blockquote>“What is the minimum sample needed to determine statistical significance?”</blockquote><p>This is possibly the most common question I have received working as a data scientist for companies that are looking to test new features in their product, searching for new creative direction, or aiming to optimize user experience.</p><p>Let’s first start with all the pitfalls I have personally witnessed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*HOSiJa1Uf9jbP1rEkp3qPg.jpeg" /></figure><h3>Pitfall Number 1:</h3><p>Since I work in marketing and we are mainly looking at conversion rates, the main argument I hear is:</p><blockquote>“We don’t have enough installs/clicks to determine statistical significance”</blockquote><p>However, if you are trying to measure something like click through rate (CTR = clicks/impressions) then you will actually care about your entire sample, not just those who clicked on your ad. That is, you care about all the (unique) impressions you served. Likewise, if you were measuring conversion rate (CVR = installs/clicks) then you would care about all clicks to the site/app.</p><p>This is similar to how you would think about a clinical trial. Your sample is determined as the total number of people in each treatment group, not just those who reacted to the treatment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_V_ClhYJNM1uvtOQ59E6zw.jpeg" /></figure><p>Now that we have that straight, we can move into the other pitfalls that come up..</p><h3>Pitfall Number 2:</h3><p>Let’s assume the test objectives have been set, budget has been allocated, and the test is in progress. Of course, everyone is eager to how the test is performing because, well, decisions need to be made and time is money. So someone is dedicated to constantly monitoring the test, checking significance every step of the way.</p><p>If this is what you are doing, STOP! This will, many times more than not, lead you to ending a test early the moment significance is found and lead to false positives.</p><p>To gain a better understanding, assume you have an experiment set up and you evaluate results after 200 observations and 500 observations. Below are the different scenarios that can result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*s78qNvFDnIAmA3MSrsX7MQ.png" /><figcaption>From: <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">http://www.evanmiller.org/how-not-to-run-an-ab-test.html</a></figcaption></figure><p>However, if you ended the test the moment you saw significance, it would result in something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*OGdFdHrQh5mYUdkgyoO4LQ.png" /><figcaption>From: <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">http://www.evanmiller.org/how-not-to-run-an-ab-test.html</a></figcaption></figure><p>Notice that this method has led you to increasing the ratio of significant to insignificant results. “<em>Therefore, the reported significance level — the “percent of the time the observed difference is due to chance” — will be wrong.”</em></p><p>For more about the implications of this, read <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">here</a>.</p><h3>Pitfall Number 3:</h3><p>That moment when <strong>statistical</strong> significance is determined and there is no conversation around <strong>practical </strong>significance. Sometimes it could be as small as a 0.5% difference between control and treatment group!</p><p>I mean great, it’s statistically significant, but is it actionable? This will depend on the context…</p><p>Now that we’ve covered all that, let’s discuss specific actions we can take to overcome these issues.</p><ol><li>Determine sample size <strong>before</strong> starting the test (aka Power Analysis).</li><li>Refrain from checking the results of your test until you have reached the specified number of samples!</li></ol><p>This will help immensely with getting clean and accurate results of your test, which could have costed a LOT of money.</p><p>So let’s get started…</p><h3>Minimum Sample Size</h3><p>Let’s say you got tasked with this job — to find the minimum sample size for an upcoming experiment. What would you do? Google a sample size calculator, right? THAT’S RIGHT! Which will lead you to sites like:</p><ul><li><a href="http://www.evanmiller.org/ab-testing/sample-size.html">Sample Size Calculator</a></li><li><a href="https://www.optimizely.com/sample-size-calculator/?conversion=4.75&amp;effect=30&amp;significance=90">A/B Test Sample Size Calculator</a></li></ul><p>Easy peasy. You start playing with the numbers, but wait..</p><blockquote>What’s an effect size?? Why does a smaller effect size lead to a much larger sample size?? Power, what’s power? BAH!</blockquote><p>You start looking up effect size, and dig deep into your memory try to remember the statistics you learned years ago! Not happening..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*N4oZ4ezHq7ZuyvF-OQ6M3Q.jpeg" /></figure><p>Okay, okay..breathe! Let’s start from the beginning.</p><p>To determine the minimum sample size needed for proper evaluation, we need to understand the intimate relationship between all the factors that play into the calculation, which are:</p><ul><li>Effect Size</li><li>Significance level (Probability of a Type 1 Error)</li><li>Power (Probability of a Type 2 Error)</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FEkvB0eVUM9pja%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FEkvB0eVUM9pja%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FEkvB0eVUM9pja%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="234" frameborder="0" scrolling="no"><a href="https://medium.com/media/911af3313b541c8813c2c6fd4b45e8fe/href">https://medium.com/media/911af3313b541c8813c2c6fd4b45e8fe/href</a></iframe><p>For the last two inputs, we can use recommended values that are used in academia (the rules can be more flexible for industry applications) which is a significance (alpha) value of 0.05 and a power of 0.8. So, that only leaves us with one value that we need to determine — effect size. Great, right? Maybe…</p><p>This particular input requires a little bit of thought (and some light calculations) since effect size will change depending on what you are trying to measure.</p><p>By definition, effect size represents the difference in means between treatment groups you want to detect. Keep in mind: the smaller the effect you want to measure, the more samples you will need. So, you’ll want to think about the minimum practical difference you care to measure, given the constraint of your budget.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*qE5-aIn9xpNNwgFxK664Og.png" /></figure><p>Cohen’s d represents the <em>standardized</em> version of effect size where values of 0.1, 0.3, and 0.5 correspond to small, medium, and large effects. But depending on your use-case, the standardized<em> </em>effect size you want to measure may be very small (even smaller than 0.1!).</p><p>For example, say you have a control group and a treatment group and you want to measure an absolute difference of 1%. Let’s also assume you know the baseline conversion rate to be 4% from historical knowledge or past experiments. Then the relative effect will be (5–4)/4 or 25% — and since it’s a binomial distribution, you can also calculate the standard deviation as sqrt(p*q) = 0.195 giving a standardized effect size of 0.01/0.195 or 0.05. Then the <em>absolute</em> effect size is 1%, the <em>relative</em> effect size is 25% and the <em>standardized</em> effect size will be about 0.05.</p><p>Armed with this information, you can now take the effect size, alpha, and power and plug into your favorite sample size calculator. Or you can use R by leveraging the pwr library and using the following or similar <a href="https://www.statmethods.net/stats/power.html">code</a>:</p><p><strong>pwr.anova.test(k = , n = , f = , sig.level = , power = )</strong></p><p>By filling in k (number of treatment groups), f (standardized effect size), sig.level (0.05) and power (0.8), it will spit back out to you the recommended sample size needed per variant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*gTzWis839Cc_nVpW7UPlgQ.gif" /></figure><p>Congrats! Now it’s on you to educate your company about why they shouldn’t constantly check the results of the test until you’ve reached at least that many in your sample.</p><p>Queue the end of the test. Now it’s up to you to evaluate the test, but I’ll save that for another day..</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=76de0046df88" width="1" height="1" alt=""><hr><p><a href="https://medium.com/glad-community/you-say-you-want-statistical-significance-76de0046df88">You say you want statistical significance?</a> was originally published in <a href="https://medium.com/glad-community">GLAD Community</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creativity in Data Science]]></title>
            <link>https://medium.datadriveninvestor.com/creativity-in-data-science-7aa4d1f07964?source=rss-9745e35c4a41------2</link>
            <guid isPermaLink="false">https://medium.com/p/7aa4d1f07964</guid>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Sarah Nabelsi]]></dc:creator>
            <pubDate>Mon, 05 Nov 2018 18:27:47 GMT</pubDate>
            <atom:updated>2018-11-05T18:27:47.430Z</atom:updated>
            <content:encoded><![CDATA[<p>Most people in the data science space might find it hard to say what they do is creative: writing SQL queries, building models, designing A/B tests, and creating dashboards. That all sounds pretty technical — where does the creativity come in? I wondered the same thing until I took the time to research creativity and understand what it is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_2VSnWIQuJvYCWVh" /></figure><p>Turns out, our biggest mistake is confounding creativity with the arts (i.e. only a painter, writer, musician, or actor can be creative). Either that, or we think that only the most brilliant minds can be truly creative — those that win Nobel prizes or make breakthroughs in their fields. Well, I am here to tell you this is wrong…</p><p>Businesses all over the world rely on creative people in all fields to move their business forward — to continue to innovate and win against the competition. So, what is creativity then?</p><p>In the words of Ken Robinson “creativity is the process of having original ideas that have value”. So, it boils down to three things:</p><ol><li>First and foremost, it’s a process. It’s not a moment in time, but something that happens over time. As John Doerr said “Ideas are easy. Execution is everything. It takes a team to win”. Coming up with the idea is only part of the process — proper execution on that idea will take time.</li><li>Next, the idea needs to be original. Original ideas don’t have to be new to the whole world, they just have to be new to you or to your community. Take Henry Ford for example: his inspiration for the car assembly line came when he saw the disassembly lines used to process meat! Not a new idea, but new to his industry!</li><li>Lastly, it has to be something of value. Is this worth-while? Judge the ideas on whether they are useful.</li></ol><p>Now when it comes to creativity, there is one main prerequisite. It’s the necessity of domain knowledge. For example, you can’t be creative on the piano if you don’t have some familiarity with piano. Sure, you can find a way to play some keys, but it won’t be the same as if you had been playing for several years. The same thing applies to other domains like mathematics — imagine being creative in your approach to solving a problem if you didn’t understand its mathematical underpinning. So, domain knowledge here is key.</p><p>Now, what is creativity in data science? We say it’s important and seek creative candidates… but how do we define it and cultivate it in ourselves and in our teams? Before saying what creativity in data science is, let’s talk about what it isn’t…</p><p>Well, it isn’t brute forcing your favorite algorithms to a dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/275/1*nvXGb3OO1D4Mt7l_JVOVNQ.jpeg" /></figure><p>This approach will lead you to miss much of what is critical to that domain (be it script writing, gaming, finance, etc.). You might end up neglecting to account for an important feature while modeling or limit your analysis resulting in uncovering only the obvious. New and creative ideas will come when you take the time to understand the nuances of the domain and ask the hard questions of your data.</p><p>That leads us to our definition of creativity in data science, which is: the marrying of statistical knowledge with domain knowledge and a little bit of mind wandering.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/274/0*4rNCyQ010A1I_ZOU" /></figure><p>This touches on the importance of both domains (analytics and industry), but also highlights this idea of allowing your mind to wander. This means keeping an open mind while experiencing or learning something new and imagining how you could use that new knowledge and apply it to a problem you are trying to solve.</p><p>More tangibly, creativity in data science can be anything from innovative features for modeling, development of new tools, cool new ways to visualize data, or even the types of data that is pulled for analysis. What’s interesting in data is that everyone will do things differently, depending on how they think about the problem. When put that way, almost everything we do in data science can be creative if we think outside the box a little bit…</p><p>The best way I could think to describe creativity in a candidate or in an approach is when they give you this moment of “wow, I never thought about it that way.” Ideally as a company or team, you want to maximize the number of moments like this — keep good ideas flowing, prioritize, and execute.</p><p>How are you being creative in your work today?</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7aa4d1f07964" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/creativity-in-data-science-7aa4d1f07964">Creativity in Data Science</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[You say you want statistical significance?]]></title>
            <link>https://medium.datadriveninvestor.com/you-say-you-want-statistical-significance-aefc35aa377f?source=rss-9745e35c4a41------2</link>
            <guid isPermaLink="false">https://medium.com/p/aefc35aa377f</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[marketing]]></category>
            <dc:creator><![CDATA[Sarah Nabelsi]]></dc:creator>
            <pubDate>Sat, 22 Sep 2018 02:05:10 GMT</pubDate>
            <atom:updated>2018-10-05T17:16:57.662Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/539/1*0v9nJTPg_CJC-Xddx47keg.png" /></figure><blockquote>“What is the minimum sample needed to determine statistical significance?”</blockquote><p>This is possibly the most common question I have received working as a data scientist for companies that are looking to test new features in their product, searching for new creative direction, or aiming to optimize user experience.</p><p>Let’s first start with all the pitfalls I have personally witnessed:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/770/1*HOSiJa1Uf9jbP1rEkp3qPg.jpeg" /></figure><h3>Pitfall Number 1:</h3><p>Since I work in marketing and we are mainly looking at conversion rates, the main argument I hear is:</p><blockquote>“We don’t have enough installs/clicks to determine statistical significance”</blockquote><p>However, if you are trying to measure something like click through rate (CTR = clicks/impressions) then you will actually care about your entire sample, not just those who clicked on your ad. That is, you care about all the (unique) impressions you served. Likewise, if you were measuring conversion rate (CVR = installs/clicks) then you would care about all clicks to the site/app.</p><p>This is similar to how you would think about a clinical trial. Your sample is determined as the total number of people in each treatment group, not just those who reacted to the treatment.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_V_ClhYJNM1uvtOQ59E6zw.jpeg" /></figure><p>Now that we have that straight, we can move into the other pitfalls that come up..</p><h3>Pitfall Number 2:</h3><p>Let’s assume the test objectives have been set, budget has been allocated, and the test is in progress. Of course, everyone is eager to how the test is performing because, well, decisions need to be made and time is money. So someone is dedicated to constantly monitoring the test, checking significance every step of the way.</p><p>If this is what you are doing, STOP! This will, many times more than not, lead you to ending a test early the moment significance is found and lead to false positives.</p><p>To gain a better understanding, assume you have an experiment set up and you evaluate results after 200 observations and 500 observations. Below are the different scenarios that can result:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/605/1*s78qNvFDnIAmA3MSrsX7MQ.png" /><figcaption>From: <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">http://www.evanmiller.org/how-not-to-run-an-ab-test.html</a></figcaption></figure><p>However, if you ended the test the moment you saw significance, it would result in something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*OGdFdHrQh5mYUdkgyoO4LQ.png" /><figcaption>From: <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">http://www.evanmiller.org/how-not-to-run-an-ab-test.html</a></figcaption></figure><p>Notice that this method has led you to increasing the ratio of significant to insignificant results. “<em>Therefore, the reported significance level — the “percent of the time the observed difference is due to chance” — will be wrong.”</em></p><p>For more about the implications of this, read <a href="http://www.evanmiller.org/how-not-to-run-an-ab-test.html">here</a>.</p><h3>Pitfall Number 3:</h3><p>That moment when <strong>statistical</strong> significance is determined and there is no conversation around <strong>practical </strong>significance. Sometimes it could be as small as a 0.5% difference between control and treatment group!</p><p>I mean great, it’s statistically significant, but is it actionable? This will depend on the context…</p><p>Now that we’ve covered all that, let’s discuss specific actions we can take to overcome these issues.</p><ol><li>Determine sample size <strong>before</strong> starting the test (aka Power Analysis).</li><li>Refrain from checking the results of your test until you have reached the specified number of samples!</li></ol><p>This will help immensely with getting clean and accurate results of your test, which could have costed a LOT of money.</p><p>So let’s get started…</p><h3>Minimum Sample Size</h3><p>Let’s say you got tasked with this job — to find the minimum sample size for an upcoming experiment. What would you do? Google a sample size calculator, right? THAT’S RIGHT! Which will lead you to sites like:</p><ul><li><a href="http://www.evanmiller.org/ab-testing/sample-size.html">Sample Size Calculator</a></li><li><a href="https://www.optimizely.com/sample-size-calculator/?conversion=4.75&amp;effect=30&amp;significance=90">A/B Test Sample Size Calculator</a></li></ul><p>Easy peasy. You start playing with the numbers, but wait..</p><blockquote>What’s an effect size?? Why does a smaller effect size lead to a much larger sample size?? Power, what’s power? BAH!</blockquote><p>You start looking up effect size, and dig deep into your memory try to remember the statistics you learned years ago! Not happening..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/300/1*N4oZ4ezHq7ZuyvF-OQ6M3Q.jpeg" /></figure><p>Okay, okay..breathe! Let’s start from the beginning.</p><p>To determine the minimum sample size needed for proper evaluation, we need to understand the intimate relationship between all the factors that play into the calculation, which are:</p><ul><li>Effect Size</li><li>Significance level (Probability of a Type 1 Error)</li><li>Power (Probability of a Type 2 Error)</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fgiphy.com%2Fembed%2FEkvB0eVUM9pja%2Ftwitter%2Fiframe&amp;url=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FEkvB0eVUM9pja%2Fgiphy.gif&amp;image=https%3A%2F%2Fmedia.giphy.com%2Fmedia%2FEkvB0eVUM9pja%2Fgiphy.gif&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=giphy" width="435" height="234" frameborder="0" scrolling="no"><a href="https://medium.com/media/911af3313b541c8813c2c6fd4b45e8fe/href">https://medium.com/media/911af3313b541c8813c2c6fd4b45e8fe/href</a></iframe><p>For the last two inputs, we can use recommended values that are used in academia (the rules can be more flexible for industry applications) which is a significance (alpha) value of 0.05 and a power of 0.8. So, that only leaves us with one value that we need to determine — effect size. Great, right? Maybe…</p><p>This particular input requires a little bit of thought (and some light calculations) since effect size will change depending on what you are trying to measure.</p><p>By definition, effect size represents the difference in means between treatment groups you want to detect. Keep in mind: the smaller the effect you want to measure, the more samples you will need. So, you’ll want to think about the minimum practical difference you care to measure, given the constraint of your budget.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/860/1*qE5-aIn9xpNNwgFxK664Og.png" /></figure><p>Cohen’s d represents the <em>standardized</em> version of effect size where values of 0.1, 0.3, and 0.5 correspond to small, medium, and large effects. But depending on your use-case, the standardized<em> </em>effect size you want to measure may be very small (even smaller than 0.1!).</p><p>For example, say you have a control group and a treatment group and you want to measure an absolute difference of 1%. Let’s also assume you know the baseline conversion rate to be 4% from historical knowledge or past experiments. Then the relative effect will be (5–4)/4 or 25% — and since it’s a binomial distribution, you can also calculate the standard deviation as sqrt(p*q) = 0.195 giving a standardized effect size of 0.01/0.195 or 0.05. Then the <em>absolute</em> effect size is 1%, the <em>relative</em> effect size is 25% and the <em>standardized</em> effect size will be about 0.05.</p><p>Armed with this information, you can now take the effect size, alpha, and power and plug into your favorite sample size calculator. Or you can use R by leveraging the pwr library and using the following or similar <a href="https://www.statmethods.net/stats/power.html">code</a>:</p><p><strong>pwr.anova.test(k = , n = , f = , sig.level = , power = )</strong></p><p>By filling in k (number of treatment groups), f (standardized effect size), sig.level (0.05) and power (0.8), it will spit back out to you the recommended sample size needed per variant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/330/1*gTzWis839Cc_nVpW7UPlgQ.gif" /></figure><p>Congrats! Now it’s on you to educate your company about why they shouldn’t constantly check the results of the test until you’ve reached at least that many in your sample.</p><p>Queue the end of the test. Now it’s up to you to evaluate the test, but I’ll save that for another day..</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=aefc35aa377f" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/you-say-you-want-statistical-significance-aefc35aa377f">You say you want statistical significance?</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>