<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jonny mcgurk on Medium]]></title>
        <description><![CDATA[Stories by Jonny mcgurk on Medium]]></description>
        <link>https://medium.com/@jonnymcgurk?source=rss-e6375cc717ce------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Jonny mcgurk on Medium</title>
            <link>https://medium.com/@jonnymcgurk?source=rss-e6375cc717ce------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 20:21:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jonnymcgurk/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Project 2: Linear Regression]]></title>
            <link>https://medium.com/@jonnymcgurk/project-2-linear-regression-4ae07325c2fd?source=rss-e6375cc717ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/4ae07325c2fd</guid>
            <dc:creator><![CDATA[Jonny mcgurk]]></dc:creator>
            <pubDate>Thu, 25 Jun 2020 12:11:29 GMT</pubDate>
            <atom:updated>2020-06-25T12:11:29.567Z</atom:updated>
            <content:encoded><![CDATA[<p>An analysis of the King County WA housing market</p><p><strong>Introduction</strong></p><p>The first step in this project was receiving a fictional “brief”and dataset to simulate the kind of real-world projects we will be doing as a data scientist.</p><p><em>“You’ll clean, explore, and model this dataset with a multivariate linear regression to predict the sale price of houses as accurately as possible.”</em></p><p>Within this file there was data on hundreds of thousands of houses and for each house there were various attributes, among them were price , sq foot area, number of bathrooms, latitude and longitude ect….</p><p><strong>Research Questions</strong></p><p>Using this data set we set out to answer the following questions:</p><p><strong><em>Question 1</em></strong><em>: What attributes affect price the most?</em></p><p><strong><em>Question 2: </em></strong><em>How to increase the price attributes</em></p><p><strong><em>Question 3: </em></strong><em>What areas are best to buy in?</em></p><p><strong><em>Question 4: </em></strong><em>Is King County a sellers market?</em></p><p><strong><em>Question 5: </em></strong><em>Which month is best to sell in?</em></p><p>However, our <em>primary objective</em> was to build a linear regression model that could predict the sale prices of houses accurately.</p><p>We broke down the exact data sets we would need to extract and clean for each question. This was not straightforward as the data in places was partly incomplete or would need a lot of work to make it functional for our project. I will cover some of the ways we refined and improved the data quality.</p><p><strong>Methodology</strong></p><p>I will take you through some of our methodology and findings for certain questions.</p><p>For question 3 we wanted to answer from the persective of an investor, someone who might be happy chosing a house in an area that delivers the best return on investment (ROI), rather than a family who might choose schools, crime rate over simple ROI.</p><p>For this, our data was incomplete and we had to web scrape some additional data, for those who arent familiar with webscraping please refer to my first blog.</p><p>Our new data included the 10 year annualised percentage increase of value by zipcode, this allowed us to plot the percentage increase ( over the past 10 years) against a map of the area.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ho9neXFo_JsYHjgcB43C4A.png" /><figcaption>Kings county: area by annualised % increase</figcaption></figure><p>This visualisation gave us a pretty good idea of which areas would generate a good ROI. You will notice the northwest is generating a 7% average return year on year, however we knew that this area was also the most exclusive and expensive to buy in.</p><p>In our original dataset we also had house prices and the zipcodes of where those houses are, so we plotted each house and their price on the same map.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3t-Ci7FI5ZTt7grbWMRRmg.png" /></figure><p>As you can see the north west area usually has houses in excess of 800k or more, if you have the money to invest then it is still a fantastic area for ROI, however we wanted to explore other viable areas for a lower cost.</p><p>To do this we compared the two maps we had just produced and we noted that the mid western area had some of the lowest prices in the county of between $450 -550,000 and also had some of the highest percentage increases of between 6.6–7% on average each year for the past 10 years.</p><p>We designated this area “up and coming” and that i would be a suitable investment for people that dont have enough cash for the north west, but with half the principle could get around the same ROI.</p><h3><strong>Linear Regression model</strong></h3><h4>Choosing Features for Accurate Modelling</h4><p>After feature engineering and analysis, the features that we chose for our model are as follows:</p><ul><li>Whether or not a home has a basement</li><li>Whether or not a home was a custom build</li><li>Whether or not a home is considered waterfront</li><li>How many times a home has been viewed by potential buyers</li><li>The average living area of the 15 closest neighbors</li><li>The grade &amp; condition of a home</li></ul><p>The model was a reasonable sucess for my first attempt, the margin of error was that most homes were priced accurately within $25,000, with some within a few thousands or even hundreds of dollars.</p><p>The main area of improvement for the second time around would be the scaling of the data, some homes were drastic outliers and it was hard to normalise the distributions of some features without completely transforming the dataset.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4ae07325c2fd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[My First Data Science Project.]]></title>
            <link>https://medium.com/@jonnymcgurk/my-first-data-science-project-49ea97843acf?source=rss-e6375cc717ce------2</link>
            <guid isPermaLink="false">https://medium.com/p/49ea97843acf</guid>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Jonny mcgurk]]></dc:creator>
            <pubDate>Fri, 08 May 2020 11:45:31 GMT</pubDate>
            <atom:updated>2020-06-04T11:28:15.231Z</atom:updated>
            <content:encoded><![CDATA[<p>An analysis of movies at the box office</p><p>I am now four weeks into my Data Science journey, and the following blog is a breakdown of my module project experience, it has been a steep learning curve overall.</p><p><strong>Introduction</strong></p><p>The first step in this project was receiving a fictional “brief” to simulate the kind of real-world projects we will be doing as a data scientist.</p><p><em>“Your team is charged with doing data analysis and creating a presentation that explores what type of films are currently doing the best at the box office. You must then translate those findings into actionable insights that the CEO can use when deciding what type of films, they should be creating.”</em></p><p>For this we were given a series of data files containing everything from user-ratings gross-profit and mark-up. The files covered the four most widely used movie-rating websites. IMDB, Bom Movies, TMDB and Rotten Tomatoes.</p><p>The overall aim was to leverage our newfound data science skills to manipulate the data, produce some insights and then visualise them for both a technical and non-technical audience.</p><p>This blog post will focus on some of the tools and methodology to answer one of our research questions, while also provided a glance at the other questions for the project.</p><p><strong>Research Questions</strong></p><p>After initially importing the data into Jupyter notebook me and my project partner had a brainstorm to decide what questions we were going to try an answer after having an initial look at the data.</p><p>With this project, we set out to answer the following questions:</p><p><strong><em>Question 1</em></strong><em>: Is there a Correlation between Budget and Revenue?</em></p><p><strong><em>Question 2</em></strong><em>: Does Return on Investment vary by Budget?</em></p><p><strong><em>Question 3:</em></strong><em> Is the performance of a movie affected by the month in which it is released?</em></p><p><strong><em>Question 4:</em></strong><em> How do Gross Revenue and Return on investment change across the genres?</em></p><p><strong><em>Question 5</em></strong><em>: Is there a correlation between ratings and runtime?</em></p><p>We broke down the exact data sets we would need to extract and clean for each question. This was not straightforward as the data in places was partly incomplete or would need a lot of work to make it functional for our project. I will cover some of the ways we refined and improved the data quality.</p><p><strong>Web Scraping</strong></p><p>For question 5 we wanted to explore if the runtime of a move affected how it was received by the audience and critics. To do this we obviously needed to compare run-time data with ratings data.</p><p>For this we wanted to analyse data from IMBD and Metacritic. The problem is that it wasn’t in our dataset, indeed it is a common problem that databases are not always whole or kept current.</p><p>Luckily all the data we needed was on the IMDB web page, which includes both the IMDB ratings and Metacritic ratings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*g-fifiVF5pzs4JEvBakXnQ.png" /><figcaption>IMDB webpage screenshot</figcaption></figure><p>The solution to getting this data is web-scraping. We scraped the site by using BeautifulSoup in python, this parsed the information we requested and gave us:</p><ol><li><em>Title of the movie</em></li><li><em>Year of release</em></li><li><em>IMDB rating</em></li><li><em>Metacritic rating</em></li><li><em>Gross of each movie</em></li><li><em>Runtime</em></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wljloySlM_TNADSL3PXdZw.png" /></figure><p><strong>Data Preparation</strong></p><p>The data scraped from IMDB was reasonably well formatted and so strenuous data cleaning was not necessary. However, there were some issues that had to be addressed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6jEsi1bRkl8c9nJnQxup8A.png" /></figure><p>As you can see, there were characters, and none values that needed to be removed in order to start visualising the data using matplotlib and seaborn; so with a bit of code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_xq-l1z1WAbgm1Q3j2BZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uKWz-n3jaF6YtAKX4IE9tg.png" /></figure><p>For any non-technical readers, this kind of work is fundamental data-science, two easy methods for cleaning data are .replace() and .dropna(). The .replace method has taken our list (dict_list) and replaced(.replace) the unwanted characters within it (\N) with nothing (‘ ‘). The result was the following:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1jkBKezatpOiX9iSo6AB5w.png" /></figure><p>While this had cleaned the data up, it was still not workable for visualisation, we needed to separate and remove more values, for instance, the column meta_score needed to be just a number or integer, this meant removing the word ‘Metascore’, which was relatively simple using the .replace code above, however this method could not be applied to the column that contained votes and gross together.</p><p>The only data re required from that column was the gross value, not the number of votes. It is not possible to use the .replace() method for this as the values for votes are different, well, technically it is possible, that’s if you wanted to scroll thorough every movie for the past decade and input the exact value of votes into the .replace() code. No, me neither. Luckily, we have a solution.</p><p>To solve this problem, we can split the column into two or more and delete the columns we don’t need any more using this code:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OCIHFZJ3NHhbStfH3lueiw.png" /></figure><p>This function applies a new variable and takes our column with the info we want movie_ratings[‘gross’] and uses the .apply and lambda function with .split to split the column at a point we choose, in this case the $.</p><p>.apply (lambda x: x .split(“$”)[-1]</p><p>The result looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FBlpXTKFs_Wwvi6ZT4kh9A.png" /></figure><p>*disclaimer, there was a lot more cleaning code before we get to this end version, but I don’t want to bore you with 100s of cells of my code. However, I hope you as a reader-especially if you’re new to code- have gained a small insight into some common functions in data science and will explore them more yourself, to recap we have seen:</p><p>1. .<strong><em>replace()</em> .</strong></p><p>2. <strong><em>isna()</em></strong> .</p><p>3. <strong>split()</strong></p><p><strong><em>4.</em></strong> <strong><em>lambda functions</em></strong></p><p><strong>Data Visualisation</strong></p><p>Now that we have our data cleaned, we can start manipulating it and putting it into graphs that visualise the insights we may have gained.</p><p>One of the simplest ways you can test data like this in python is the .corr() method, this will instantly produce a table that shows you the correlation coefficient between the data.</p><p>Here we take our variable that contains the data and apply the method: movie_ratings.corr().</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*aKbE6fdensgocolNaWsZrQ.png" /></figure><p>Now that we have an idea how our data sets interact with each other we can start to visualise it. Firstly I wanted to see how IMDB and Metascore are ratings are distributed on average, so I put them into histograms to compare.</p><p>For this I used matplotlib, wich is a comprehensive library for creating static, animated, and interactive visualizations in Python.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OpIxF3l7wKzJfv_PrBRlbw.png" /></figure><p>As you can see here, the distributions of scores are roughly in the same range for both between 50–80 out of 100, with IMDB having a distribution on the higher end, between 60–80 out of a hundred and Meta critic having a wider distribution. While this was an interesting insight for produces on how they should receive critic ratings from each institution respectively, it isn’t the most useful of insights. So we decided to compare critic ratings to runtime.</p><p>Again, using matplotlib, we plotted the data of runtime v critic ratings of IMDB and Metacritic respectively to see if there was a difference, or indeed if runtime affects ratings.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*knYAp22itC0STwc91scGbA.png" /></figure><p>As you can see below, there is not much of a correlation between runtime and ratings. The insight gained from this would be that movie production companies need not worry about run time too much, as it does not affect how the movie is recived.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BXMxCTo5nVV8yBAKiWjOwg.png" /></figure><p><strong>Conclusion</strong></p><p>I have really enjoyed my first Data Science project, and I hope I have given you a brief insight as to how it went and some of the methods I used. For the non-technical reader I hope you may have taken some points of curiosity and will look into some of the methods discussed on here, moreover, it is important to note that the methods here were examined on a very surface level and that there was much more code running behind the scenes. for the technical user- I hope you were largely in agreement with my methodology and did not have too many flash backs of the hours you have undoubtably spent data cleaning also.</p><p>Thank you!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=49ea97843acf" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>