<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lee Baker on Medium]]></title>
        <description><![CDATA[Stories by Lee Baker on Medium]]></description>
        <link>https://medium.com/@eelrekab?source=rss-792f52901c4b------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*JvotZsyewpD4m3s3pI5oNg.png</url>
            <title>Stories by Lee Baker on Medium</title>
            <link>https://medium.com/@eelrekab?source=rss-792f52901c4b------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 19:17:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@eelrekab/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Simulated Statistics is the New Black]]></title>
            <link>https://medium.com/@eelrekab/simulated-statistics-is-the-new-black-b8a5565d7948?source=rss-792f52901c4b------2</link>
            <guid isPermaLink="false">https://medium.com/p/b8a5565d7948</guid>
            <category><![CDATA[big-data]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Lee Baker]]></dc:creator>
            <pubDate>Tue, 26 May 2020 14:30:46 GMT</pubDate>
            <atom:updated>2020-07-09T15:28:30.517Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/data-science-in-the-real-world">DS in the Real World</a></h4><h4>What the experts tell us about the future of statistics</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nF468FYS3kFvNFlfEDzTkQ.jpeg" /></figure><p>Over the years I’ve often been asked by beginners where they should start in statistics, what they should do first, and which parts of statistics they should prioritise to get them to where they want to be (which is usually a higher paid job).</p><p>Now, as I’m almost completely self-taught I don’t really consider myself an authority in where one should get started, and I struggle to answer this question with any great conviction.</p><p>Sure, I have some thoughts about this subject, but they are coloured by my own experiences.</p><p>So I thought I’d reach out to some of our statistics friends to see what they can bring to the party.</p><p>Each of the statisticians in this post were asked the same question:</p><p><strong>If you had to start statistics all over again, where would you start?</strong></p><p>The answers were astounding — they turned out to be a roadmap of how to become a modern statistician from scratch.</p><p>In short, how to be a future statistician <em>without ever needing a single lesson</em>!</p><h3>Frequentist Statistics vs Bayesian Statistics</h3><p>There is a schism in statistics, and that is between the frequentists and the Bayesians.</p><p>Let’s see what the statisticians have to say about this debate.</p><p>We start with <a href="http://rocketdatascience.org/">Kirk Borne</a> (Twitter: <a href="https://twitter.com/KirkDBorne">@KirkDBorne</a>), astrophysicist and rocket scientist (well, rocket <em>data</em> scientist). Surprisingly, he tells me he’s never never had any interest in being an astronaut!</p><p>“I am not a statistician, nor have I ever had a single course in statistics, though I did teach it at a university. How’s that possible?”</p><p>Funnily enough, that was the same for me! So where did he get all his stats from?</p><p>“I learned basic statistics in undergraduate physics and then I learned more in graduate school and beyond while doing data analysis as an astrophysicist for many years. I then learned more stats when I started exploring data mining, statistical learning, and machine learning about 22 years ago. I have not stopped learning statistics ever since then”.</p><p>This is starting to sound eerily like my stats education. All you need to do is drop the ‘astro’ from astrophysics and they’re identical! So what does he think of starting stats all over again?</p><p>“I would have started with Bayesian inference instead of devoting all of my early years to simple descriptive data analysis. That would have led me to statistical learning and machine learning much earlier. And I would have learned to explore and exploit the wonders and powers of Bayesian networks much sooner”.</p><p>This is also what <a href="http://biostat.mc.vanderbilt.edu/wiki/Main/FrankHarrell">Frank Harrell</a>, author and professor of biostatistics at Vanderbilt University School of Medicine at Nashville thinks about hitting the reset button on statistics (Twitter: <a href="https://twitter.com/f2harrell">@f2harrell</a>). He told me:</p><p>“I would start with Bayesian statistics and thoroughly learn that before learning anything about sampling distributions or hypothesis tests”.</p><p>And Lillian Pierson, CEO of <a href="https://www.data-mania.com/">Data-Mania</a> (Twitter: <a href="https://twitter.com/Strategy_Gal">@Strategy_Gal</a>) also mentioned Bayesian statistics when I asked her where she would start:</p><p>“If I had to start statistics all over again, I’d start by tackling 3 basics: t-test, Bayesian probability &amp; Pearson correlation”.</p><p>Personally, I haven’t done very much Bayesian stats, and it’s one of my biggest regrets in statistics. I can see the potential in doing things the Bayesian way, but as I’ve never had a teacher or a mentor I’ve never really found a way in.</p><p>Maybe one day I will — but until then I will continue to pass on the messages from the statisticians in here.</p><p>Repeat after me:</p><p>Learn Bayesian stats.</p><p>Learn Bayesian stats.</p><p>LEARN BAYESIAN STATS!</p><h3>Statistical Recipes vs Calculus vs Simulated Statistics</h3><p>As I was reaching out and gathering quotations I got a rather cryptic response from Josh Wills (Twitter: <a href="https://twitter.com/josh_wills">@josh_wills</a>), software engineer at Slack and founder of the <a href="http://crunch.apache.org/">Apache Crunch project</a> (he also describes himself as an ‘ex-statistician’):</p><p>“Computation before calculus is the pithy answer”, he told me.</p><p>This intrigued me, so I asked him if he could elaborate a little, and here is his reply:</p><p>“So I think stats can be and is taught in three ways:</p><p>1. a set of recipes</p><p>2. from the perspective of calculus — mostly integrals and what not, and</p><p>3. computationally (like the bootstrap as a fundamental thing)”</p><p>“Most folks do the recipes approach, which doesn’t really help with understanding stuff but is what you do when you don’t know calculus”.</p><p>Ah, I understand the ‘set of recipes approach’, but I didn’t know anyone was still doing the calculus approach. He went further:</p><p>“I was a math major, so I did the calculus based approach, because that’s what you did back in the day. You mostly do some integrals with a head nod to computational techniques for distributions that are too hard to do via integrals. But the computational approach, even though it was discovered last, is actually the right and good way to teach stats”.</p><p>Whew, thank God for that — I thought he was saying that we should all learn the calculus approach!</p><p>“The computational approach can be made accessible to folks who don’t know calculus, and it’s actually most of what you use in the hard parts of real world statistics problems anyway. The calculus approach is historically interesting, but (and I feel heretical for saying this) it should be relegated to a later course on the history of statistical thought — not part of the intro sequence”.</p><p>It’s interesting to see the evolution of statistics in this light and shows just how far we’ve come — and in particular how much computers and computing power have developed over the past couple of decades.</p><p>It’s truly mind-blowing to think that when I was doing my PhD 20 years ago it was difficult getting hold of data, and when you did get some, you had to network computers together to get enough computing power. Now we’re all swimming in data and err, well, we still struggle to get enough computing power to do what we want — but it’s still way more than we used to have!</p><h3>Simulated Statistics is the New Black</h3><p>I also got a really interesting perspective from Cassie Kozyrkov, Head of Decision Intelligence at Google (Twitter: <a href="https://twitter.com/quaesita">@quaesita</a>), who told me that she’d:</p><p>“Probably enjoy making a bonfire out of printed statistical tables!”</p><p>Well, amen to that, but seriously though, where would you start again with stats?</p><p>“Simulation! If I had to start all over again, I’d want to start with a simulation-based approach to statistics”.</p><p>OK, I’m with you, but why specifically simulation?</p><p>“The ‘traditional’ approach taught in most STAT101 classes was developed in the days before computers and is unnecessarily reliant on restrictive assumptions that cram statistical questions into formats you can tackle analytically with common distributions and those nasty obsolete printed tables”.</p><p>Got you. So what exactly have you got against the printed tables?</p><p>“Well, I often wonder whether traditional courses do more harm than good, since I keep seeing their survivors making ‘Type III errors’ — correctly answering the wrong convenient questions. With simulation, you can go back to first principles and discover the real magic of statistics”.</p><p>Statistics has magic?</p><p>“Sure it does! My favorite part is that learning statistics with simulation forces you to confront the role that your assumptions play. After all, in statistics, your assumptions are at least as important as your data, if not more so”.</p><p>And when it came to offering his advice, Gregory Piatetsky, founder of <a href="https://www.kdnuggets.com/">KDnuggets</a> (Twitter: <a href="https://twitter.com/kdnuggets">@kdnuggets</a>), suggested that:</p><p>“I would start with Leo Breiman’s paper on <a href="https://projecteuclid.org/download/pdf_1/euclid.ss/1009213726">Two Cultures</a>, plus I would study Bayesian inferencing”.</p><p>If you haven’t read that paper (which is open access), Leo Breiman lays out the case for algorithmic modelling, where statistics are simulated as a black box model rather than following a prescribed statistical model.</p><p>This is what Cassie was getting at — statistical models rarely fit real-world data, and we are left to either try to shoe-horn the data into the model (getting the right answer to the wrong question) or switch it up and do something completely different — simulations!</p><h3>And There’s More…</h3><p>This is an excerpt of my original post, which is quite long — too long to post here in its entireity (there are more than 30 world-class contributors!).</p><p>If you’re enjoying reading, you might be interested to hear what Dez Blanchfield had to say about domain experts, or what Michael Friendly and Alberto Cairo said about the past, present and future of data visualisation.</p><p>There’s also a <strong>free book to download</strong> detailing all the comments made by the contributors, including what Jacqueline Nolis and Kristen Kehrer had to say about starting their careers over.</p><p>And don’t get me started with the epic suggestions that Natalie Dean and Jen Stirrup had about Information Flow and Detective Work.</p><p>Awesome — you really don’t want to miss them!</p><p>Come on over and read the original post here:</p><p><a href="https://www.chi2innovations.com/blog/discover-stats-blog-series/experts-say-this-is-the-future-of-statistics/">The Future of Statistics - According to 30+ World Experts</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b8a5565d7948" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Hilarious Graphs (and Pirates) Prove That Correlation Is Not Causation]]></title>
            <link>https://medium.com/data-science/hilarious-graphs-and-pirates-prove-that-correlation-is-not-causation-667838af4159?source=rss-792f52901c4b------2</link>
            <guid isPermaLink="false">https://medium.com/p/667838af4159</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[analytics]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Lee Baker]]></dc:creator>
            <pubDate>Thu, 25 Jan 2018 12:31:07 GMT</pubDate>
            <atom:updated>2020-07-09T15:30:21.652Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/data-science-in-the-real-world">DS in the Real World</a></h4><h4>When it comes to storytelling, we have a problem.</h4><p>It’s not our fault though — as human beings <strong>we are hard-wired from birth to look for patterns and explain why they happen</strong>. This problem doesn’t go away when we grow up though, it becomes worse the more intelligent we think we are. We convince ourselves that now we are older, wiser, <em>smarter</em>, that our conclusions are closer to the mark than when we were younger (the faster the wind blows the faster the windmill blades turn, not the other way around).</p><p>Even really smart people see a pattern and insist on putting an explanation to it, even when they don’t have enough information to reach such a conclusion. They can’t help it.</p><p>This is the thing about being human. We seek explanation for the events that happen around us. If something defies logic, we try to find a reason why it might make sense. <strong>If something doesn’t add up, we make it up</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/0*-MSBlf7yVFSFJE_z.jpg" /></figure><h3>The Post Hoc Fallacy</h3><p>Ever heard the Latin expression <em>Post Hoc, Ergo Propter Hoc</em>, meaning ‘After this, therefore because of this’? It is the basis of the saying ‘<strong>Correlation Does Not Imply Causation</strong>’, also known in statistics as the <strong>Post Hoc Fallacy</strong> because it’s a very familiar trap that we all fall into from time to time. This is the idea that when things are observed to happen in sequence, we infer that the thing that happened first must have caused the thing that happened next.</p><p>The Post Hoc Fallacy is what causes a football manager to only wear purple socks on match days. He once wore them at a match and his team won. Obviously, it was the socks that did it. Now he fears that if <em>doesn’t</em> wear them to a match the team might lose. Damn those stinky purple socks (he also daren’t wash them for fear of the magic pixie dust washing out).</p><p>Post Hoc is also what made rain men indispensible to the tribe — they believed that their rain man can make it rain. Spotting the clouds brewing in the distance, the rain man dances until it pours it down. It doesn’t usually take more than three or four days of dancing until the inevitable happens. “Rain man dance, water fall from sky”. It’s just a good job for the rain man that the Indians couldn’t speak Latin, otherwise he’d have been in real trouble…</p><h3>The Post Hoc Piracy Postulation</h3><p>For a humorous view of the Post Hoc Fallacy, let’s take a look at Pastafarianism. It’s all the rage these days. Not heard of it? It’s one of the newest and fastest growing religions on the block. <strong>Pastafarian Sparrowism</strong>, to give it its full title, is a ‘vibrant religion that seeks to bring the Flying Spaghetti Monster’s fleeting affection to all of us, through the life of His Prophet, Captain Jack Sparrow’. Seriously, they’re not joking. Well, actually, they are. They promote a light-hearted view of religion and oppose the teaching of intelligent design and creationism in public schools. They also maintain that <strong>pirates are the original Pastafarians</strong>.</p><p>In an effort to illustrate that correlation does not imply causation, the founder, Bobby Henderson, presented the argument that global warming is a direct effect of the shrinking number of pirates since the 1800s, and accompanied it with this graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*Eo0L0KvmV81Z1qyA.jpg" /><figcaption>Pirates Caused Global Warming. Honest…</figcaption></figure><p>Wow, look at that straight line, I hear you all say — there’s clearly a correlation between the decline in the numbers of pirates and the rise in global temperatures, so there just <em>must</em> be a causal connection here, mustn’t there? Yup, you’ve all just fallen for the Post Hoc Fallacy (I just knew you would).</p><p>Just because there is a straight line on the graph <strong>it doesn’t necessarily follow that one thing caused the other</strong>, particularly when you’ve grabbed two seemingly unconnected variables at random and stuck them together to see whether there might be some sort of tenuous correlation between them.</p><p>In the case of pirates and global warming, take a closer look at the labels on the x-axis. Notice something strange? Apart from the fact that the proportions of neighbouring data points are all out of whack, there is also the issue that a couple of them have been humorously disordered to deliberately deceive.</p><p>I don’t know about you, but I’m a believer! As soon as I’ve finished writing this book I’m giving up stats for a life as a pirate on the open seas. I’ll stop global warming if it’s the last thing I do.</p><p>It probably will be…</p><p><strong>We interrupt this blog post to bring you the breaking news…</strong></p><p>This blog post is an extract from the witty new book <strong>Truth, Lies and Statistics</strong></p><p>Here’s the blurb:</p><p>Pirates, cats, Mexican lemons and North Carolina lawyers. Cheese consumption, margarine and drowning by falling out of fishing boats. This book has got it all. In this eye-opening book, award winning statistician and author Lee Baker uncovers the key tricks used by statistical hustlers to deceive, hoodwink and dupe the unwary. Written as a layman’s guide to lying, cheating and deceiving with data and statistics, there’s not a dull page in sight! A roller coaster of a book in 8 witty chapters, this might just be the most entertaining statistics book you’ll read this year.</p><p>Discover the exciting world of statistical cheating and persuasive misdirection.</p><p><strong>Get this book RIGHT NOW by clicking on the link below…</strong></p><p><a href="https://www.chi2innovations.com/truth-lies-and-statistics-book/">Truth Lies And Statistics</a></p><h3>The Organic Autism Correlation Conundrum</h3><p>If you look online there are all sorts of humorous graphs that prove the Post Hoc Fallacy. Over the past 20 years or so, there’s been a huge increase in the anti-vaccine movement, particularly in the US, and there have been all sorts of <strong>spurious correlations</strong> that have been ‘discovered’ that ‘prove’ that there is a causal link between vaccination programmes and autism. At the same time, to debunk the most crackpot of the theories, other — equally ridiculous — correlations have popped up too.</p><p>There was one that was published that showed the correlation between sales of organic food in the US and diagnosis of autism:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*CYAgWelxNlyGSl4s.jpg" /><figcaption>Organic Food Causes Autism. Oh My…</figcaption></figure><p>There is a very close correlation between the pair of plot lines, even accompanied by a <strong>very large r-value</strong> (close to 1) and a <strong>very small p-value</strong> (close to 0). The suggestion is that — if we trust that correlation <em>does</em> imply causation — a much closer correlation exists between organic food and autism than any other theory that currently exists, so therefore it <em>must</em> be the cause. Except that correlation <em>does not</em> necessarily imply causation, and organic food does not cause autism. That would be ridiculous. And that is the whole point of these graphs. All you need to do is find any pair of variables that increase over the same time period, plot them on a graph with the same x-axis and <em>different</em> y-axes, adjust the y-axis scales until the plot lines coalesce, and — BOOM — correlation! If, by some magic of coincidence and fate, there is a <em>statistical</em> correlation, then publish the p-value that goes along with it as additional proof. <strong>What this does is prove that the correlation exists, but it does not prove that one thing causes the other</strong>. It might, but then again it might not…</p><h3>The Lemon Fatality Correlation Convergence</h3><p>I also quite enjoyed the correlation that proved that Mexican lemons are a major cause of deaths on US roads. Wait, what? I must have missed the news that day — Mexican lemons are killing Americans? You bet!</p><p>Take a look at a plot of the number of fresh lemons imported into the USA from Mexico <em>versus</em> the total fatality rate on US highways between 1996 and 2000:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/0*VX_5U3tDZ-KeF8jD.jpg" /><figcaption>Mexican Lemons Kill Americans!</figcaption></figure><p>My, my, just look at the R-squared value — it really <em>must</em> be true. Although the graph seems to be telling us that the more Mexican lemons there are in the US the fewer road deaths there are, the inescapable conclusion is that MEXICAN LEMONS KILL AMERICANS! What should we do about it? Should we import more Mexican lemons (the correlation tells us that this is what we should do)? Or should we ban Mexican lemons altogether? After all, if there are no Mexican lemons on the streets then they can’t kill any more Americans.</p><p>What utter tosh! I don’t care if there <em>is</em> a correlation, <strong>there is nothing to suggest that lemons cause accidents</strong>. If there was, don’t you think that lemons would be causing accidents on Mexican roads before the trucks crossed into the US? What about Sicilian lemons? Do they cause road deaths in Italy and across Europe?</p><p>Oh, the power of correlations. As long as your audience doesn’t understand that correlation does not necessarily imply causation you can make them believe pretty much anything.</p><p>Did you forget your copy of the book? Not to worry, it’s waiting for you right here:</p><p><a href="https://www.chi2innovations.com/truth-lies-and-statistics-book/">Truth Lies And Statistics</a></p><p><em>Originally published at </em><a href="https://www.chi2innovations.com/blog/discover-stats-blog-series/graphs-prove-correlation-not-causation/"><em>chi2innovations.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=667838af4159" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/hilarious-graphs-and-pirates-prove-that-correlation-is-not-causation-667838af4159">Hilarious Graphs (and Pirates) Prove That Correlation Is Not Causation</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Statistics is Dead — Long Live Data Science…]]></title>
            <link>https://medium.com/@eelrekab/statistics-is-dead-long-live-data-science-c560fe999855?source=rss-792f52901c4b------2</link>
            <guid isPermaLink="false">https://medium.com/p/c560fe999855</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Lee Baker]]></dc:creator>
            <pubDate>Tue, 11 Jul 2017 10:29:41 GMT</pubDate>
            <atom:updated>2017-07-11T10:44:58.730Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*KkeDjfr94vj_93VQNxQFyQ.jpeg" /></figure><p>I keep hearing Data Scientists say that ‘Statistics is Dead’, and they even have big debates about it attended by the good and great of Data Science.</p><p>Interestingly, there seem to be very few actual statisticians at these debates.</p><p>So why do Data Scientists think that stats is dead?</p><p>Where does the notion that there is no longer any need for statistical analysis come from?</p><p>And are they right?</p><p>Is statistics dead or is it just pining for the fjords?</p><p>I guess that really we should start at the beginning by asking the question ‘What Is Statistics?’. <a href="http://chi2innovations.com/blog/discover-stats-blog-series/statistics-last-dark-art/">I’ve already written a blog post on this here</a>.</p><p>Briefly, what makes statistics unique and a distinct branch of mathematics is that statistics is the study of the uncertainty of <a href="http://chi2innovations.com/blog/discover-data-blog-series/what-data/">data</a>.</p><p>So let’s look at this logically. If Data Scientists are correct (well, at least some of them) and statistics is dead, then either (1) we don’t need to quantify the uncertainty or (2) we have better tools than statistics to measure it.</p><h3>Quantifying the Uncertainty in Data</h3><p>Why would we no longer have any need to measure and control the uncertainty in our data?</p><p>Have we discovered some amazing new way of observing, <a href="http://chi2innovations.com/blog/discover-data-blog-series/11-essential-tips-effective-data-collection/">collecting</a>, collating and analysing our data that we no longer have uncertainty?</p><p>I don’t believe so and, as far as I can tell, with the explosion of data that we’re experiencing — the amount of data that currently exists <em>doubles</em> every 18 months — the level of uncertainty in the data is on the increase.</p><p>So we must have better tools than statistics to quantify the uncertainty, then?</p><p>Well, no. It may be true that most statistical measures were developed decades ago when ‘Big Data’ just didn’t exist, and that the ‘old’ statistical tests often creak at the hinges when faced with enormous volumes of data, but there simply isn’t a better way of measuring uncertainty than with statistics — at least not yet, anyway.</p><p>So why is it that many Data Scientists are insistent that there is no place for statistics in the 21st Century?</p><p>Well, I guess if it’s not statistics that’s the problem, there must be something wrong with Data Science.</p><p>So let’s have a heated debate…</p><h3>What is Data Science?</h3><p>Nobody seems to be able to come up with a firm definition of what <a href="http://chi2innovations.com/blog/captains-blog/50-shades-grey-psychology-data-scientist/">Data Science</a> is.</p><p>Some believe that Data Science is just a sexed-up term for statistics, whilst others suggest that it is an alternative name for ‘Business Intelligence’. Some claim that Data Science is all about the creation of data products to be able to analyse the incredible amounts of data that we’re faced with.</p><p>I don’t disagree with any of these, but suggest that maybe all these definitions are a small part of a much bigger beast.</p><p>To get a better understanding of Data Science it might be easier to look at what Data Scientists <em>do</em> rather than what they <em>are</em>.</p><p>Data Science is all about <a href="http://chi2innovations.com/blog/discover-stats-blog-series/statistics-last-dark-art/">extracting knowledge from data</a> (I think just about everyone agrees with this very vague description), and it incorporates many diverse skills, such as mathematics, statistics, artificial intelligence, computer programming, visualisation, image analysis, and much more.</p><p>It is in the last bit, the ‘much more’ that I think defines a Data Scientist more than the previous bits. In my view, if you want to be an expert Data Scientist in Business, Medicine or Engineering then the biggest skill you’ll need will be in Business, Medicine or Engineering. Ally that with a combination of some/all of the other skills and you’ll be well on your way to being in great demand by the top dogs in your field.</p><p>In other words, if you want to call yourself a Data Scientist you really do need to be an expert in your field as well as having some of the other listed skills.</p><h3>Are Computer Programmers Data Scientists?</h3><p>On the other hand — as seems to be happening in Universities here in the UK and over the pond in the good old US of A — there are Data Science courses full of computer programmers that are learning how to handle data, use Hadoop and R, program in Python and plug their data into Artificial Neural Networks.</p><p>It seems that we’re creating a generation of Computer Programmers that, with the addition of a few extra tools on their CV, claim to be expert Data Scientists.</p><p>I think we’re in dangerous territory here.</p><p>It’s easy to learn how to use a few tools, but much <em>much</em> harder to use those tools intelligently to extract valuable, actionable information in a specialised field.</p><p>If you have little/no medical knowledge, how do you know which data outcomes are valuable?</p><p>If you’re not an expert in business, then how do you know which insights should be acted upon to make sound business decisions, and which should be ignored?</p><h3>Plug-And-Play Data Analysis</h3><p>This, to me, is the crux of the problem. Many of the current crop of Data Scientists — talented computer programmers though they may be — see Data Science as an exercise in plug-and-play.</p><p>Plug your dataset into tool A and you get some descriptions of your data. Plug it into tool B and you get a visualisation.</p><p>Want predictions? Great — just use tool C.</p><p>Statistics, though, seems to be lagging behind in the Data Science revolution. There aren’t nearly as many automated statistical tools as there are visualisation tools or predictive tools, so the Data Scientists have to actually <em>do</em> the statistics themselves.</p><p>And statistics is <em>hard</em>.</p><p>So they ask if it’s really, <em>really</em> necessary.</p><p>I mean, we’ve already got the answer, so why do we need to waste our time with stats?</p><p><em>Booooring….</em></p><p>So statistics gets relegated to such an extent that Data Scientists declare it dead.</p><p>Talk about the lunatics running the asylum…</p><h4>If you enjoyed this story, please recommend and share to help others find it! Feel free to leave a comment below.</h4><h4>We publish stories, videos, blogs, books and webinars about data and stats. You can subscribe to get them <a href="http://chi2innovations.com/newsletter-signup/">here</a>.</h4><p><em>Originally published at </em><a href="http://chi2innovations.com/blog/captains-blog/statistics-dead-long-live-data-science/"><em>chi2innovations.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c560fe999855" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>