<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Megan Lyons on Medium]]></title>
        <description><![CDATA[Stories by Megan Lyons on Medium]]></description>
        <link>https://medium.com/@meganlyons_79212?source=rss-cdc2d0a96b7f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*dmbNkD5D-u45r44go_cf0g.png</url>
            <title>Stories by Megan Lyons on Medium</title>
            <link>https://medium.com/@meganlyons_79212?source=rss-cdc2d0a96b7f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 20:15:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@meganlyons_79212/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Confusion Matrices 101]]></title>
            <link>https://medium.com/@meganlyons_79212/confusion-matrices-101-22b4486bfa01?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/22b4486bfa01</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Sat, 12 Sep 2020 11:24:24 GMT</pubDate>
            <atom:updated>2020-09-12T11:24:24.225Z</atom:updated>
            <content:encoded><![CDATA[<h4>A breakdown of confusion matrices and evaluation metrics</h4><p>In Classification tasks, a confusion matrix is a table that allows you to visualise the performance of the algorithm. Each row of the matrix represents the instances in a predicted class while each column represents the instances in an actual class (or vice versa). The name stems from the fact that it makes it easy to see if the system is confusing two classes (i.e. commonly mislabeling one as another)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/816/0*yZ2Rqpqh3vCaclow" /></figure><p>Let us use an example where we are trying to classify whether or not a lump is benign or malignant</p><p><strong>True Positives (TPs) </strong>are instances where we predicted they were class 1, and they were in fact class 1. Predicted lump was malignant and it was malignant</p><p><strong>True Negatives (TNs)</strong> are instances where we predicted class 0 , and they were in fact class 0. Predicted lump was benign and it was benign.</p><p><strong>False Positives (FPs) </strong>are instances where we predicted class 1 and they were class 0. Predicted lump was malignant but it was benign.</p><p><strong>False Negatives (FNs) </strong>are instances where we predicted class 0 and they were class 1. Predicted lump was benign but it was malignant.</p><p>Based on the specific scenario, we will want to reduce our instances of FN or FP. Continuing the example where we are trying to classify whether or not a lump is benign or malignant. We would rather have a False Positive, that is we say that the lump is malignant when it is benign, and send them for further testing, than a False Negative, where we say the lump is benign when it is in fact malignant, and they are sent away without treatment.</p><h3><strong>Evaluation Metrics</strong></h3><p>There are 4 main evaluation metrics used in classification tasks.</p><p><strong>Accuracy score: </strong>It is the number of correct predictions made divided by the total number of predictions made, multiplied by 100 to turn it into a percentage</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/922/1*EJeX4A4S3HD91MvPd4IH9Q.png" /></figure><p>Accuracy allows us to measure the total amount of predictions the model got right. It is the most common metric for classification and provides a solid holistic view of the overall performance of our model.</p><p><strong>Precision score: </strong>measures the False Positive Rate. From the above example, it tells us out of all the times the model said the patient’s lump was malignant, how many times did the patient in question actually have a malignant lump?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/598/1*NVVLqjdGfSbWINAFqsd4EQ.png" /></figure><p>Unless true positive equals 0, High Precision score means that False Positives are very low. This means, if you want to minimise False Positives, you should optimise your precision score.</p><p><strong>Recall score: </strong>measures the False Negative Rate. Out of all the patients we saw that actually had a malignant lump, what percentage of them did our model correctly identify as being malignant?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/554/1*Pzxj8D1S-4XVYS0NSTtWeg.png" /></figure><p>Unless True Positive rate is zero, High Recall score means False Negatives are very low. If you want to minimise False Negatives, then you should optimise your recall score.</p><p><strong>F1 score: </strong>represents the harmonic mean of Precision and Recall</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*zMYaSKP3pUjmY5Cvwu2Row.png" /></figure><p>F1 score penalises models heavily if it skews too hard towards either precision or recall. For this reason, F1 score is generally the most used metric for describing the performance of a model.</p><p>Now that you understand all the evaluation metrics, you will know how to decide which metric to optimise during your next classification task.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22b4486bfa01" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Music Information Retrieval]]></title>
            <link>https://medium.com/@meganlyons_79212/music-information-retrieval-8a3facacf88e?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/8a3facacf88e</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Mon, 07 Sep 2020 19:33:51 GMT</pubDate>
            <atom:updated>2020-09-07T19:33:51.714Z</atom:updated>
            <content:encoded><![CDATA[<h4>Part 2: Extracting features from audio files using Librosa</h4><p>If you haven’t already, please make sure you have read part 1 of this series before continuing.</p><blockquote>Chroma</blockquote><p>The term <strong><em>chroma feature</em></strong> or <strong><em>chromagram</em></strong> closely relates to the twelve different pitch classes. Chroma-based features, which are also referred to as “pitch-class-profiles”, are a powerful tool for analysing music whose pitches can be meaningfully categorised (often into twelve categories) and whose tuning approximates to the equal-tempered-scale.</p><p>A <strong><em>chroma vector</em></strong> is a typically a 12-element feature vector indicating how much energy of each pitch class, {C, C#, D, D#, E, …, B}, is present in the signal</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Eg4SS4KS9oxFmOy6bJf_Q.png" /><figcaption>Chroma stft</figcaption></figure><p>The <strong><em>constant-Q transform</em></strong> transforms a data series to the frequency domain. Similiar to the mel scale, uses a logarithmically spaced frequency axis</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ttU--FJdvwH60f-j3a1j8A.png" /></figure><blockquote><strong>C</strong>hroma <strong>E</strong>nergy <strong>N</strong>ormalized <strong>S</strong>tatistics.</blockquote><p>The main idea of CENS features is that taking statistics over large windows smooths local deviations in tempo, articulation, and musical ornaments such as trills and arpeggiated chords. CENS are best used for tasks such as audio matching and similarity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BeEIbv2jPuvD-6HZenW2Aw.png" /><figcaption>CENS</figcaption></figure><blockquote>Spectral Features</blockquote><p>The <strong><em>spectral centroid</em></strong> is a measure used in digital signal processing to characterise a spectrum. It indicates where the center of mass of the spectrum is located. Perceptually, it has a robust connection with the impression of brightness of a sound.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LUIuarfvLUA4DX4f-0xsUA.png" /><figcaption>Spectral Centroid</figcaption></figure><p>The <strong>Spectral Bandwidth </strong>is the Wavelength interval in which a radiated spectral quantity is not less than half its maximum value. It is a measure of the extent of the Spectrum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XweamO8GIVVG91pr6l0ShA.png" /><figcaption>Spectral Bandwidth</figcaption></figure><p>The <strong>Spectral Contrast </strong>is defined as the level difference between peaks and valleys in the spectrum.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rZHVtxF6XgUSd_c7_w5APw.png" /><figcaption>Spectral Contrast</figcaption></figure><p>The <strong>Spectral Rolloff a</strong>s the frequency below which a specified percentage of the total spectral energy lies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RddRnL4EhhMAt5F-yFhoBA.png" /><figcaption>Spectral Rolloff</figcaption></figure><p>The <strong>Zero Crossing Rate </strong>is the rate of sign-changes along a signal, i.e., the rate at which the signal changes from positive to zero to negative or from negative to zero to positive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-x5CZIzFGgGJHGPRMuZAyQ.png" /><figcaption>ZCR</figcaption></figure><p>In musical tuning and harmony, the <strong>Tonnetz</strong> (German: tone-network) is a conceptual lattice diagram representing tonal space first described by Leonhard Euler in 1739. Various visual representations of the <strong>Tonnetz</strong> can be used to show traditional harmonic relationships in European classical music.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hXDhn0MQB83hcjnzkcIo5g.png" /><figcaption>Tonnetz</figcaption></figure><p>The <strong>energy</strong> of a signal corresponds to the total magnitude of the signal. For audio signals, that roughly corresponds to how loud the signal is.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oI4wrj5NnAzt35K9g_kqwg.png" /><figcaption>RMSE</figcaption></figure><blockquote>MFCC</blockquote><p><strong>Mel-frequency cepstral coefficients </strong>(<strong>MFCCs</strong>) are coefficients that collectively make up an MFC. They are derived from a type of cepstral representation of the audio clip (a nonlinear “spectrum-of-a-spectrum”). MFCCs of a signal are a small set of features (usually about 10–20) which concisely describe the overall shape of a spectral envelope.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-khInZj12wI5Z_VDlcp6Wg.png" /><figcaption>MFCCs</figcaption></figure><p>These features will be used to build Random Forests, Support Vector Machines, XGBoost and Neural Networks in order to classify audio files by genre.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8a3facacf88e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Music Information Retrieval]]></title>
            <link>https://medium.com/@meganlyons_79212/music-information-retrieval-a7bf19eec3b4?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/a7bf19eec3b4</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Mon, 07 Sep 2020 19:32:41 GMT</pubDate>
            <atom:updated>2020-09-07T19:32:41.369Z</atom:updated>
            <content:encoded><![CDATA[<h4>Part 1: Visualising audio files with Librosa</h4><p>This blog post will give a step by step guide to music information retrieval using Librosa.</p><p>Firstly, you need to make sure that you have the Librosa library installed. To this use:</p><blockquote>! pip install Librosa</blockquote><p>Next you need to import the following into your notebook:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/588/1*bpm5prfUUpvHZwQvD13wjA.png" /><figcaption>Required libraries and imports</figcaption></figure><p>You are able to play an audio file in your jupyter notebook. You must specify where the file is saved, load it in using librosa, and then play it using Ipython.display.</p><p>An <strong><em>audio signal</em></strong> is represents the fluctuation in air pressure caused by the vibration as a function of time. Use <strong>librosa.load</strong> to load an audio file into an audio array. This returns the audio array, y, as well as the sample rate, sr. The <strong>sampling rate, sr, </strong>is the rate at which a computer captures audio data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/872/1*HGoVFOvuCSBYa9vf1rAOFg.png" /><figcaption>Loading and playing audio in jupyter notebook</figcaption></figure><p>The change in air pressure at a certain time is graphically represented by a <strong><em>waveform</em></strong>.</p><p>To plot a waveform, use <strong>librosa.display.waveform</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9-cPZsRVq5ULNiOGCFxKww.png" /><figcaption>Waveplot for audio signal</figcaption></figure><p>A <strong><em>spectrogram</em></strong> is a visual representation of the spectrum of frequencies of a signal as it varies with time. When applied to an audio signal, spectrograms are sometimes called sonographs, voiceprints, or voicegrams. A spectrogram is usually depicted as a heatmap, i.e., as an image with the intensity shown by varying the colour or brightness.</p><p><strong><em>Fourier Transform </em></strong>is a function that gets a signal in the time domain as input, and outputs its decomposition into frequencies.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*yTKPBumzMkGQL9Ng.png" /></figure><p>The <strong><em>Short-time Fourier transform</em></strong><em> (</em><strong><em>STFT</em></strong><em>)</em>, is a Fourier-related transform used to determine the sinusoidal frequency and phase content of local sections of a signal as it changes over time.</p><p>The spectrogram shows the the intensity of frequencies over time. It is simply the squared magnitude of the STFT. The human perception of sound intensity is logarithmic in nature. Therefore, we are often interested in the log amplitude:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YN_4k4BGKVGYb1-vRXAokQ.png" /><figcaption>Spectrogram</figcaption></figure><p>Studies have shown that humans do not perceive frequencies on a linear scale. We are better at detecting differences in lower frequencies than higher frequencies. In 1937, Stevens, Volkmann, and Newmann proposed a unit of pitch such that equal distances in pitch sounded equally distant to the listener. This is called the <strong>Mel Scale. </strong>We perform a mathematical operation on frequencies to convert them to the mel scale.</p><p>A <strong><em>Mel spectrogram </em></strong>is merely a spectrogram that uses frequencies that have been converted to the mel scale.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fah-WTGGe5yQsVIkWXXk3w.png" /><figcaption>Mel Spectrogram</figcaption></figure><p>We will use these spectrogram images to create a Convolutional Neural Network, to predict music genres.</p><p>Now that we know how to visualise various audio frequencies, please see my next blog where we will extract various features from the audio files.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a7bf19eec3b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Time Series Forecasting]]></title>
            <link>https://medium.com/@meganlyons_79212/time-series-forecasting-6c90db40f004?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6c90db40f004</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Mon, 17 Aug 2020 17:20:58 GMT</pubDate>
            <atom:updated>2020-08-17T17:20:58.559Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>Home Value Forecasting for greatest return</blockquote><p>For my Module 4 project, I chose a Time Series forecasting project where I was required to forecast real estate prices of various zip codes using data from <a href="https://www.zillow.com/research/data/">Zillow</a>. For this project, I had to act as a consultant for a fictional real-estate investment firm. The firm has asked you what seems like a simple question:</p><blockquote>What are the top 5 best zip codes for us to invest in?</blockquote><p>The dataset contained the median house sales for 14,723 Zip Codes, spanning 1996 to 2018. To initially decrease the amount of data, I decided to focus on Zip Codes in the Los Angeles-Long Beach-Anaheim Metropolitan Area.</p><p>The Home value in 2018 varied greatly among the different Zip Codes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FhN-OjDZe5wlwmxVyAO3Ow.png" /></figure><p>Home values in Beverly were reaching nearly 5.6 million on average, where as they went as low as 226,600 in areas such as Lancaster. The average Home value was $931,603.20.</p><p>The deliverables for the investment were as follows:</p><ol><li>Home Value is below $500,000</li></ol><p>I wanted to focus on home values which were below the average price for the metropolitan area. This would allow the investor to diversify their investment by purchasing many homes in different zip codes. With a potential investment pool of $5,000,000 — they could purchase approximately 10 homes for their portfolio.</p><p>2. Home Value is no more than +/- $20,000 from 2006 peak prices.</p><p>I also wanted to focus on homes that were about to or had just reached the peak prices of the housing bubble. This was due to the belief that homes that have just reached past peak prices will have a better future earning potential than homes which have way surpassed their value in 2006. Also leaves less room to drop in event of another crisis.</p><p>3. minimum 10% 3 year Return on Investment</p><p>In order to ensure these homes have been yielding a good return, we looked at homes with a 3 year ROI of greater than 10%. This took into account of inflation.</p><p>36 Zip codes fitted into these criteria. The average price of these homes was $433,739 and they had an average 3-year ROI of 20%</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*j4MTYrF5CvYCQXL78tmZeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BGs2Jd8ZAmSNwKPlrPFBIg.png" /></figure><p>These homes all followed a similar trend throughout time.</p><p>They peaked in 2006–2007, before crashing in 2009, slowly climbing back to 2006 levels by 2018.</p><p>Due to the non-stationarity of the data, I chose to use a SARIMAX model.</p><p>Optimal parameters were found using itertools to find all possible combinations for p,d,q:</p><p>p = q = (1,2,3)</p><p>d = (1,2)</p><p>The combination of parameters which yielded the lowest AIC were chosen for the model.</p><p>SARIMAX was then run on all 36 zip codes, to predict house prices in 3 years time.We then chose the zip codes with the highest ROI,</p><p>Top 5 Zip Codes were chosen based on the highest 3 year ROI and RMSE under 1100.</p><p>ROI was calculated assuming cumulative 3 year inflation of 4.68%</p><blockquote>The Top Zip Codes are as follows:</blockquote><ul><li>90037: ROI = 20.35%</li><li>90304: ROI = 17.79%</li><li>90813: ROI = 14.99%</li><li>90001: ROI = 10.56%</li><li>91758: ROI = 4.96%</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JBeox5I6RJFAHVrMTJhhrQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ne6Kp4NN0teTfiFhcya2Dw.png" /></figure><p>For future work, I would like to gather more information about the house prices in the Zip Codes. That is more up to date information, given I only had prices up to April 2018. I would also like yo explore amenities in area such as schools, hospitals etc and their effect on Home values. Demographic information, such as age, families etc would also be interesting to explore.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6c90db40f004" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Customer Churn Analysis]]></title>
            <link>https://medium.com/@meganlyons_79212/customer-churn-analysis-5c7c9e3036e7?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c7c9e3036e7</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Mon, 13 Jul 2020 10:52:54 GMT</pubDate>
            <atom:updated>2020-07-13T10:52:54.169Z</atom:updated>
            <content:encoded><![CDATA[<p>For my Module 3 Data Science Project, I was tasked with building a Classification model to predict customer churn of a telecom company. The Project followed the CRISP-DM Process</p><h3><strong>1. Business Understanding</strong></h3><p>Predicting and preventing customer churn represents a huge additional potential revenue source for every business.</p><p>Customer churn is when a customer ceases their relationship with a company. The full cost of churn includes both lost revenue and the marketing costs involved with replacing those customers with new ones. Therefore reducing churn is a key business goal of every business.</p><p>The goal is to identify key causes of churn. This is crucial because a company that is churning customers owing to a poor pricing strategy will need a different improvement strategy than one that is churning customers due to a dysfunctional customer support program.</p><h3>2. Data Understanding</h3><p>In order to better understand the dataset, full exploratory data analysis was performed. This enabled me to explore different factors that affect churn as well as the similarities and patterns in customers that do churn.</p><p>The first aspect I wanted to investigate was whether or not the churn rate varied by state. This is important to understand as if the churn is different state to state, the strategies will vary based on customer location.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/700/1*D3KsnRNA2F4ck-Xcz8w34g.png" /><figcaption>Churn Rate by State</figcaption></figure><p>New Jersey, California and Texas have the highest churn rates, up at 25%. This is five times higher than the lowest churning states of Hawaii, Alaska and Arizona, whose churn rates are only 5%.</p><p>The next feature I investigated was monthly charge, and whether or not there is a relationship between increases in monthly charge and higher churn rates. The average monthly charge for customers who remain with the company is $59, compared with $67 for customers who churn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/919/1*uHrCvOKhTt4mzfJ0cPQKVA.png" /><figcaption>Monthly Charge vs Churn</figcaption></figure><p>From the above it is clear that customers who churn come in on a cheap plan, approx $50 per month, which after a year jumps up to $65 per month. This would lead us to believe that these customers come in on some sort of promotional plan. Compared with customers who remain with the company, the average charge stays very constant.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/962/1*kXeYbfFTsraBt1TYo_Rc1Q.png" /></figure><p>The last thing of note from my EDA, is the churn rate of customers with an international plan.</p><p>Nearly 50% of customers who have an international plan terminate their contract. This shows that customers are very unhappy with this plan.</p><h3>3. Data Preparation</h3><p>Very little data preparation was needed. The dataset had no missing values or duplicates. Redundant columns were dropped and some extra columns added.</p><p>Class Imbalance is an issue in this data set. Only 15% of customers churned , so for this reason it is likely that our model will be bias towards predicting customers don’t churn. To solve this issue we stratified our train-test split, so that the ratio remained, and I also used SMOTE on my train set.</p><h3>4. Modelling</h3><p>In the case of churn, A False Negative means we predicted that the customer would not churn, but they did in fact churn. A False Positive means that we predicted the customer would churn but they stayed. We are more concerned with reducing False Negatives, so the models will focus on optimising the Recall score.</p><p>The first model ran was a Logistic Regression Model. GridSearchCV was used to find the optimal parameters which were as follows:</p><pre>{&#39;logreg__C&#39;: 5.0, &#39;logreg__fit_intercept&#39;: False, &#39;logreg__penalty&#39;: &#39;l2&#39;}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jWljvK-zvy4a3zXQb4-Hhw.png" /></figure><p>This yielded an accuracy score of 0.8 for the train set, and 0.75 for the test set. The big difference in scoring for the test and training data shows that the model is overfitting.</p><p>The next model was a Random Forest model. Again, GridSearchCV was again used to find the optimal parameters.</p><pre>{&#39;RF__criterion&#39;: &#39;entropy&#39;,<br> &#39;RF__max_depth&#39;: 5,<br> &#39;RF__min_samples_leaf&#39;: 3,<br> &#39;RF__min_samples_split&#39;: 10}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zAx7pxf2ECK8CQc2HhGLEA.png" /></figure><p>The accuracy score improved on this model, 0.87 and 0.89 for the train and test sets respectively. However, overall the model is still not up to scratch, as the scores are not consistent on the train and test sets.</p><p>Finally, I decided to use an XGBoost Classifier. The optimal parameters were:</p><pre>{&#39;xgb__learning_rate&#39;: 0.05,<br> &#39;xgb__max_depth&#39;: 7,<br> &#39;xgb__n_estimators&#39;: 250,<br> &#39;xgb__subsample&#39;: 1}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6_IsbkegvjlvKfqwwol_uQ.png" /></figure><p>This model performed much better, with an accuracy score of 0.99 and 0.98 for the train and test datasets respectively. It is worth noting the drop in Recall score. This is because Recall is measured as</p><p>TP/(TP+FN)</p><p>Because the train set was smoted, there are far more True Positives, relative to the test set, resulting in a lower recall score.</p><p>Because of the SMOTE issue, I then compared the test data with the un-smoted train data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MITXpDaiQUlhxdmOtAx2pA.png" /></figure><p>The Train data which hasn’t been smoted had a recall score of 0.898 and had False negatives equal to 1.46% of the data</p><p>The Test data had da recall score of 0.907 and had False negatives equal to 1.35% of the data.</p><p>The full dataset had a recall score of 0.901 and False negatives equal to 1.44% of the data set.</p><p>This is a very good result. Our model correctly predicts 90% of people that will churn.</p><h3>5. Evaluation</h3><p>Our XGBoost model performed the best. Our model correctly predicts 90% of people that will churn.</p><p>The most important features of our model were as follows:</p><ul><li>State</li><li>International PLan</li><li>Total Charge</li><li>Customer Service Calls</li><li>Voicemail Plan</li></ul><p>Thank you for reading this blog and I hope you found my process interesting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c7c9e3036e7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating Maps with Folium]]></title>
            <link>https://medium.com/@meganlyons_79212/creating-maps-with-folium-baa6ff0f5c44?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/baa6ff0f5c44</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Thu, 04 Jun 2020 20:10:42 GMT</pubDate>
            <atom:updated>2020-06-04T20:10:42.824Z</atom:updated>
            <content:encoded><![CDATA[<h4>A Step by Step Guide.</h4><p>This blog will walk through a step by step guide to use folium.</p><p>First we have to make sure we have the correct imports in our Jupyter notebook.</p><p>You need to do the following imports</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/752/1*1yMC3NLhd-bu76s0FQQD0w.png" /></figure><p>Firstly, we will build a map of Seattle using the centre coordinates for the City. The code can be seen below</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4PEHRbalHRPbG5Pf83B4vQ.png" /></figure><p>You simply pass in the longitude and latitude of the location you want the map to be centred on. The zoom_start input chooses the zoom level for the initial map.</p><p>Next, given a data set of longitude and latitude points, I will show you how to plot all locations on an interactive map with Marker Clusters. Given a DataFrame df, containing longitude and latitude points . We will use a loop to create a Marker for each point.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9lLX-z0R-71XPG28DnRhxw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HoVYq0JpWSgzf0YfdN4IKw.png" /></figure><p>Next, if you want to show a distance from a surrounding area, say for example Bellevue, you can use the Circle function to draw a circle of a particular radius from your desired location. The below code creates a circle which shows a 15km radius surrounding Bellevue.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2F0H3Oa56u8WqhtaNRicCw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z5KugyI490NH_uHDsbI34w.png" /></figure><p>Now we will discuss how to use a Choropleth map to show information about zip codes. Suppose given a dataset of Houses in Kings County. You can use a geojson and a Choropleth map to colour code a map based on certain characteristics.</p><p>You can download the required geojson at the following link</p><p><a href="https://gis-kingcounty.opendata.arcgis.com/datasets/e6c555c6ae7542b2bdec92485892b6e6_113">Zipcodes for King County and Surrounding Area / zipcode area</a></p><p>First we need to open the geojson:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/924/1*wnePBgn2BH6cjQPucOcbjA.png" /></figure><p>Given that we have a dataset containing housing data in the Kings County area, we want to create a new geojson so it only contains the zip codes that we have in our data. To do this we first need to create a new DataFrame, containing our data grouped by each zip code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8-RlCrXuU8lKuMWhA6VMQg.png" /></figure><p>Once we have our new zip code DataFrame, we can create a loop that will find the zip codes in the geojson that match zip codes in our dataset, to create a new json file that only has the zip codes we need.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-j5a_P3ku0LZzLmXZ107SA.png" /></figure><p>Now we will create a function that takes in our zip code DataFrame, a column to colour code on, and a legend that will label the colour coded bar.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eURtbqfop3Y6sGDJZVA_6g.png" /></figure><p>Now, we can pass in our a count column to find a colour coded map of the county by the distribution of houses.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jJhTToPkAEwp5nk7zRuuNw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JSuVqadQZhtfNddoAXQj5A.png" /></figure><p>Similarly, we can pass in the median price column to the average house prices by zip code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6UEqW46wGVjTZRtRDiFXXA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sOGXyIPFSFdIvI9yGGtYbw.png" /></figure><p>I hope you found this helpful. And that you will have fun experimenting and making your own folium maps.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=baa6ff0f5c44" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Analysis of Movie Success at the Box Office]]></title>
            <link>https://medium.com/@meganlyons_79212/an-analysis-of-movie-success-at-the-box-office-2147a984af68?source=rss-cdc2d0a96b7f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2147a984af68</guid>
            <dc:creator><![CDATA[Megan Lyons]]></dc:creator>
            <pubDate>Fri, 08 May 2020 11:24:16 GMT</pubDate>
            <atom:updated>2020-05-08T11:24:16.211Z</atom:updated>
            <content:encoded><![CDATA[<blockquote><strong>An Analysis of Movie Success at the Box Office</strong></blockquote><p>Throughout this blog , I will discuss how things have changed in the movie industry of the decades.</p><p>The Relationship between Budgets and Revenue</p><p>In this day and age, the hallmark for success of a movie is how much it grossed at the box office. So, let’s see how the production budget is related to the Gross Revenue at the Box Office. At first glance one would assume that the bigger the budget, the bigger the Revenue. For the most part this is true. With a correlation coefficient of 0.75, budget and revenue have a strong relationship. Interestingly, this has changed massively over the decades. In 1980s, the correlation between budget and revenue was 0.35, by the 2000s, this had more than doubled to 0.7579.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*q_ggCqSUz39GWzRqzScS_g.png" /><figcaption>Evolution of the Relationship between Production Budget and Worldwide Gross</figcaption></figure><p>Ok so a bigger budget yields a bigger revenue. Sounds great right? Wrong. What about mark-up? High Revenues are great, but if mark-up is low, what’s the point. Choosing the right production budget is essential when looking at getting the greatest return on your investment. With over 93% of films having a budget of less than $100 million, I will explore this category in more detail.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/620/1*9tRCJi48C29Y6gA4KMyUwQ.png" /></figure><p>The 80–90 million range has the highest mark-up up, of over 1.2 times. It is worth noting there is little difference in return between the 60–70,80–90 and 90–100 categories. This is far more interesting than the correlation between budget and revenue. For example, if you have a budget of 100 million, you should choose to only spend 80–90 to get a better return. You should carefully choose your budget when looking for the best return. While a bigger budget inherently implies a bigger gross revenue, it does not constitute a greater return on investment.</p><p>What about the day and month a movie is released? Does the day of the week affect the gross revenue? 78% of films are released on a Friday. Presumably the rationale behind this is that people are more likely to go to the cinema on a weekend than on a weekday. Interestingly enough, Friday is not the most profitable day, on average, to release a film. In the graph below it is clear that Wednesday is the highest grossing day</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*jvH1v-OSwoUyzDjbrzkefw.png" /></figure><p>It is interesting to see how the average revenue by release month has changed over the past 4 decades. While some of the differences could be down to the increase in films being released year-round, it is interesting to see how society has changed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/896/1*VQBZtvKya9IKmvxP3xfCvQ.png" /></figure><p>It comes as no surprise that the summer months, notably July and August, have always been at the top. December, however, has become more and more unprofitable as the decades have progressed, with average revenue falling from $63 million in the 1990s to only $9 million in 2010s. January and February have picked up too. Perhaps the most noticeable trend, is that the spread of revenue is across far more months.</p><p>Genres play a huge part in the success of movies, and we see trends of genre popularity through the years. Of course, some genres will have a greater return on investment than others. Animation is top of the list, with an average return of investment of over 187%, followed closely by Adventure with 163%. Documentaries are a terrible genre to invest in, with an average return on investment of -57%. This is presumably down to the fact that not many Documentaries are released in the cinema, leading to decreased revenue opportunities. It would be interesting to delve into the world of Netflix, where documentaries are hugely popular, to see how their model works, and if the way in which they produce documentaries yields them a higher return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YsE4V4o_YmVcnt3gtXBC6g.png" /></figure><p>Ratings play a huge part in how a movie is perceived. Think about it, if you see a movie get slammed by critics, how likely are you to go and pay to watch it? Whether it be rotten tomatoes, IMDB ratings, or Metacritic, these all play a huge role in how we react to movies. They are like asking a friend if they would recommend a movie. Ratings can have an impact on our psychology too, if you read a movie has poor ratings and you go to see it with low expectations, you may come out pleasantly surprised. Contrarily, how many times have you heard that a movie is the next big thing, only to watch and it be left whole heartedly disappointment and deflated as you leave the cinema. Let’s explore these ratings in more detail.</p><p>Metacritic have perfected their scoring system. They collate a group of the most respected critics, and assign their scores to their reviews, before applying a weighted average in order to summarise the range of their opinions. This weighted average is important, ensuring more respected critics and publications are given a higher voting power.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*OugKAvqcOtx7yJekROOCJQ.png" /></figure><p>The Metascore has been perfectly created for easy interpretation. A colour coded traffic light system allows at quick glance to see how the movie is rated.</p><p>Metacritic have created a “Must-See” award, which is designated to movies with a Metascore of 81 or higher and has been reviewed by at least 15 publications. Only 5% of movies on the Metacritic’s database achieve this elite status. These are highly acclaimed movies that have been reviewed by a broad cross-section of the best critics.</p><p>Now that we know how the Metascore is calculated, let’s see how the top 250 grossing films at the US Box Office for the last decade scored. The average Metascore for these films is only 62. It is interesting that only 60% of these films received positive reviews. With 17 of these films receiving unfavourable reviews. Only 22 films gained a Metascore of over 81, achieving must see status.</p><p>Let’s look at the Grown-ups franchise, created by Adam Sandler. The first film, released in 2010, received a Metascore of only 30, meaning it received generally unfavourable reviews from the critics. This didn’t stop its success at the box office though, taking home $162m at the US Box office, ranking 148th for the decade. The sequel, Grown-ups 2, received even worse reviews, with a Metascore of 19, meaning critics voiced their overwhelming dislike for the film. Despite this, it still grossed $133 million at the US box office, ranking 189th for the decade.</p><p>Why has his happened? Is it because ratings have no influence on consumers? Doubtful. It is more likely due to a combination of factors. Firstly, high profile actors have large loyal followings, who will go see the movie based on the actors involved, even if reviews are poor. This is seen even more franchises, where people will want to follow the story to the finish.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2147a984af68" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>