<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Joe McHugh on Medium]]></title>
        <description><![CDATA[Stories by Joe McHugh on Medium]]></description>
        <link>https://medium.com/@teamastertoast?source=rss-25e20e25c2aa------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*t5M3lGXIMWQpdJa2dCNQNw.png</url>
            <title>Stories by Joe McHugh on Medium</title>
            <link>https://medium.com/@teamastertoast?source=rss-25e20e25c2aa------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 02:29:34 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@teamastertoast/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[An Overview of Different Time Series Models for Forecasting]]></title>
            <link>https://medium.com/@teamastertoast/an-overview-of-different-time-series-models-for-forecasting-c06c071c7684?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c06c071c7684</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[forecasting]]></category>
            <category><![CDATA[time-series-data]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Fri, 11 Sep 2020 04:04:30 GMT</pubDate>
            <atom:updated>2020-09-11T04:04:30.239Z</atom:updated>
            <content:encoded><![CDATA[<p>When one wants to start predicting the future, it must come as no surprise that one must deal with time. This is the essence of time series modeling. A time series model can be thought of as a function <em>F(t) = y </em>that takes a given point in time (t) and produces some output (y).</p><p>The difficult arises when we want to estimate what <em>y </em>will be during a point in time that hasn’t occurred yet. The model then changes to <em>F(t+h) = y, </em>where <em>h </em>is some amount of time into the future.</p><p>There are several time series models that exist and today I will give an overview of the most common.</p><h4><strong>Seasonal Decomposition:</strong></h4><p>Oftentimes, time series data can demonstrate clear seasonality, ie certain patterns or trends that emerge over consistent periods of time and repeat themselves (these could be daily, weekly, monthly, yearly, etc). A seasonal decomposition breaks the time series model into three parts:</p><p><em>F(t) = S(t) + T(t) + R(t)</em></p><p>where <em>S(t) </em>is a seasonal component, <em>T(t) </em>is a trend component, and <em>R(t) </em>is the remainder component. During a seasonal decomposition, the trend will be able to explain a certain proportion of the observations, the seasonality another portion, and then the remainder of the observations are by caused by some unknown factor or series of factors (“the remainder” <em>R(t)</em>). Below is an example of a seasonal decomposition with the original data on the top, and the trend, seasonal, and remainder components below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/672/1*fgzgpp89Y9FPOmZZWTB-Fw.png" /></figure><p><strong>Exponential Smoothing</strong></p><p>Exponential smoothing is a simple model that proposes that the model’s output at time <em>t</em> is a weighted average of the values of observations at t-1, t-2, all the way to t=1, with the weights assigned to each observation reducing exponentially the further back in time they are. Put more simply, t-1 will have the highest weight in the average, t-2 the second highest weight, and so on.</p><p>As time proceeds, the model values the information provided by an observation at a single point in time less and less.</p><p>The weights are calculated by some defining some beginning weight for the most recent observation (alpha), and the remaining weight are derived from this alpha as such:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/493/1*MgwNzoSrpg9VGXD7vLxOkg.png" /></figure><p>Below is a table of examples demonstrating how weights assigned to each observation vary depending on the initial alpha value:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/578/1*fkMZm4fk7LaT2m39rr0H2Q.png" /></figure><p>Note that for any value 0 &lt; alpha &lt;1, the sum of the weights will approximately sum to 1.</p><h4>AutoRegressive Integrated Moving Average (ARIMA)</h4><p>Combined with exponential smoothing, the ARIMA group of models is one of the most common forecasting techniques used by practitioners.</p><p>An ARIMA models combines two approaches to explaining future observations. The first is the autoregressive portion (AR) which postulates that a portion of the observation is some linear combination of the past observations. The second is the moving average portion (MA) which postulates that a portion of the observation is some linear combination of the past <em>errors</em> in the forecast.</p><p>These models require the time series to be stationary (ie there is no trend in the data), thus differencing or integrating (I) is a necessary step in most cases as a trend usually does exist in time series data.</p><p>The autoregressive portion of the model is expressed as such:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/458/1*75Yee-0_dLRuDOTC3rWeGQ.png" /></figure><p>With <em>p </em>being the number of lags that the model is assuming influences the outcome (this is also known as the <em>order</em> of the AR portion).</p><p>The moving average portion of the model is expressed as such:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*2Ebc_DyDnCdn3QgOBsaWJA.png" /></figure><p>With <em>q </em>being the number of error term lags that the model is assuming influences the outcome (this is also known as the <em>order </em>of the MA portion).</p><p>Together, the expression for the ARIMA model becomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/525/1*n1L7fd9ARzNylpibNNYKTA.png" /></figure><p>In order to determine which orders or <em>p </em>and <em>q </em>are needed ACF and PACF plots are required. ACF plots shows the autocorrelations between yt and all observations from yt-1 to yt-k. PACF plots shows the <em>partial </em>autocorrelations which is the correlation between yt and yt-k that isn’t explained by their mutual correlations with another specified set of variables.</p><p>In a future post I will demonstrate how to create all three of these models in python and how to use them to perform simple forecasts.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c06c071c7684" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Heart Failure Part II: Preprocessing the Data]]></title>
            <link>https://medium.com/@teamastertoast/predicting-heart-failure-part-ii-preprocessing-the-data-c07f720a7e93?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/c07f720a7e93</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-preprocessing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Tue, 08 Sep 2020 19:18:47 GMT</pubDate>
            <atom:updated>2020-09-08T19:22:09.176Z</atom:updated>
            <content:encoded><![CDATA[<p>Last week we created a simple classification model and tuned it to predict whether or not a patient would suffer heart failure, without too much tinkering, the model correctly predicted the occurrence of heart failure 83% of the time. This is definitely a good start, but there is a lot we can do to improve this score. In this post, we are going to try and improve the model’s score by performing a step known as <em>preprocessing</em>.</p><p>Preprocessing, is the step in which the data is encoded, transformed, scaled, or brought into some state that allows the model to interpret it more effectively. Last week, no preprocessing was performed, we simply entered the raw data into the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/0*bP10c9SgSurJpkQX.jpeg" /><figcaption>Data Types</figcaption></figure><p>In this specific case we have two types of data:</p><h4>Interval Numerical Data:</h4><pre>[&#39;age&#39;,<br> &#39;creatinine_phosphokinase&#39;,<br> &#39;ejection_fraction&#39;,<br> &#39;platelets&#39;,<br> &#39;serum_creatinine&#39;,<br> &#39;serum_sodium&#39;,<br> &#39;time&#39;]</pre><p>These data types is <strong><em>numerical</em></strong><em> </em>because their raw values are members of the set of real numbers (these values can either be integers or continuous numbers). Furthermore, these data types are <strong><em>interval</em></strong><em> </em>because their values have a defined unit of measurement and differences between them are meaningful.</p><h4>Nominal Categorical Data:</h4><pre>[&#39;anaemia&#39;,<br> &#39;diabetes&#39;,<br> &#39;high_blood_pressure&#39;,<br> &#39;sex&#39;,<br> &#39;smoking&#39;]</pre><p>These data types are <strong><em>categorical</em></strong><em> </em>since their raw values are members of a defined set of values. Furthermore, these data types are <strong><em>nominal</em></strong><em> </em>since their values do not have any implied order, they are simply booleans. (Days of the week on the other hand would be <strong><em>ordinal categorical</em></strong> data. <strong><em>Categorical </em></strong>because their raw values are members of the defined set of values Monday through Sunday, <strong><em>Ordinal</em> </strong>because their raw values have an implied order to them, ie. <em>tuesday </em>comes after <em>monday </em>and before <em>wednesday</em>).</p><h4>Encoding:</h4><p>Categorical data in its raw form is almost always stored as a string. Unfortunately, most machine learning models cannot understand strings (there is a branch of machine learning called NLP that seeks to address this very issue, but that’s a subject for another post). In order for a machine learning model to understand a categorical data type, it needs to be encoded.</p><p>For nominal categorical data, the encoding is known as <em>one hot encoding:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/994/1*EakQ6Ue_poVAUozyTnNxMA.png" /></figure><p>For ordinal categorical data, the encoding is known as <em>ordinal encoding:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/870/1*WWfS-B1C6kEvHOkRSBMAqw.png" /></figure><p>Below is a simple python script that uses the scikit learn library to encode features that are categorical data:</p><pre>from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder<br>from sklearn.compose import ColumnTransformer<br>import pandas as pd<br>import numpy as np</pre><pre>data = pd.read_csv(C:\some_file_path\data.csv)<br>X = data.drop(&#39;target_variable&#39;, axis=1)</pre><pre>#Collecting columns and storing them in a new data frame if they #aren&#39;t numbers (which generally indicates categorical data):</pre><pre>X_categorical = df.select_dtypes(exclude=&#39;number&#39;)</pre><pre>#At this point the data frame of categorical features will have to be #inspected to determine which columns are ordinal and which are #nominal. Once this is done, they need to be separated manually into #two additional data frames:</pre><pre>X_cat_nominal = X_categorical[[nominal_col_1, ..., nominal_col_n]]<br>X_cat_ordinal = X_categorical[[ordinal_col_1, ..., ordinal_col_n]]</pre><pre>#ColumnTransformer, OneHotEncoder, and Ordinal Encoder is now used #to encode the columns:</pre><pre>ord_enc = OrdinalEncoder()<br>one_hot_enc = OneHotEncoder()</pre><pre>preprocessor = ColumnTransformer(transformers = [<br>                     (&#39;ordinal_encoder&#39;,<br>                      ord_enc,<br>                      X_cat_ordinal.columns.to_list()<br>                     ),<br>                     (&#39;nominal_encoder,<br>                      one_hot_enc,<br>                      X_cat_nominal.columns.to_list()<br>                     )])</pre><pre>X_categorical_encoded = preprocessor.fit_transform(X_categorical)</pre><h4><strong>Scaling/</strong>Normalization/<strong>Standardization:</strong></h4><p>Depending on the type of model being used, the numerical data will have to be transformed.</p><p><strong>Scaling: </strong>The data is transformed such that the data is within a specific range of values. This is required for KNN and Support Vector models.</p><p><strong>Normalization: </strong>The data is transformed such that the distribution of the dataset can be described as normal. Any model that assumes a normal distribution will require normalization of the dataset. A classic example of a model that requires normalization is an OLS regression.</p><p><strong>Standardization: </strong>The data is transformed such that the distribution of the dataset has a mean of 0 and a standard deviation of 1 (ie can be described by a standard normal distribution). Many machine learning models require standardization. Examples include SVMs, neural networks, and logistic regressions.</p><p>Below is a simple python script that can be used to scale, normalize, and standardize numerical data:</p><pre>from sklearn.preprocessing import StandardScaler, RobustScaler,\ Normalizer, MinMaxScaler, MaxAbsScaler<br>import pandas as pd<br>import numpy as np</pre><pre>data = pd.read_csv(C:\some_file_path\data.csv)<br>X = data.drop(&#39;target_variable&#39;, axis=1)</pre><pre>#Collecting columns and storing them in a new data frame if they #contain numbers (indicating numerical data):</pre><pre>X_numerical = df.select_dtypes(include=&#39;number&#39;)</pre><pre>ss = StandardScaler()<br>norm = Normalizer()<br>rs = RobustScaler()<br>minmax = MinMaxScaler()<br>maxabs = MaxAbsScaler()</pre><pre>def transformation_function(transformer):<br>   return pd.DataFrame(transformer.fit_transform(X_numerical))</pre><p>After these transformations are made, we re-run the model from last week:</p><pre>rf = RandomForestClassifier()<br>gb = GradientBoostingClassifier()</pre><pre>rf_param_grid = {<br>    &#39;max_depth&#39;:range(2,7),<br>    &#39;n_estimators&#39;:range(10,30)<br>}</pre><pre>gb_param_grid = {<br>    &#39;loss&#39;:[&#39;deviance&#39;,&#39;exponential&#39;],<br>    &#39;max_depth&#39;:range(2,7),<br>    &#39;max_features&#39;:[&#39;sqrt&#39;,&#39;log2&#39;]<br>}</pre><pre>def model_fitter(model):<br>    model.fit(X_train, y_train)<br>    print(&#39;Training Score: {}&#39;.format(model.score(X_train, y_train)))<br>    y_pred = model.predict(X_test)<br>    print(&#39;Test Score {}:&#39;.format(accuracy_score(y_test, y_pred)))<br>    <br>def grid_search(model,param_grid):<br>    grid_search_rf = GridSearchCV(<br>        estimator=model,<br>        param_grid=param_grid,<br>        scoring=&#39;accuracy&#39;,<br>        n_jobs=-1,<br>        cv=5<br>    )<br>    grid_search_rf.fit(<br>        X_train,<br>        y_train<br>    )<br>    print(&#39;Best Parameters were: {}&#39;.format(grid_search_rf.best_params_))<br>    print(&#39;Best CrossVal Score was: {}&#39;.format(grid_search_rf.best_score_))</pre><p>After re-running the model with the preprocessed data the following scores were achieved:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/511/1*iURKvNmARB3uGavnSFOvzw.png" /></figure><p>After encoding and scaling the data, we have boosted the accuracy of the model by 3.66%. What’s clearly evident is that preprocessing is a simple yet crucial step in the modeling process, and can dramatically impact model performance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c07f720a7e93" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Heart Failure: Hyperparameter Tuning with GridSearch in Python]]></title>
            <link>https://medium.com/@teamastertoast/predicting-heart-failure-hyperparameter-tuning-with-gridsearch-in-python-973c35ce5a1a?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/973c35ce5a1a</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[tutorial-python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[gridsearchcv]]></category>
            <category><![CDATA[hyperparameter-tuning]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Mon, 24 Aug 2020 19:50:04 GMT</pubDate>
            <atom:updated>2020-08-24T19:52:42.706Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1uK6sM5AhiXprkQCy3DUVg.jpeg" /></figure><p>In this tutorial, I will demonstrate how to hyperparameter tune a model. The model has been trained to predict whether or not a patient will suffer heart failure based on a series of medical features that patient presents with.</p><p>Hyperparameter tuning is an essential step when building a predictive model. But… what is a hyperparameter? In ML modeling, a hyperparameter is a parameter that can be altered (or <em>tuned</em>) in order to change the way the model attempts to fit the data.</p><p>For instance, observe the following tree-based model used to predict a type of animal:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*LNGXsvrQR15ntHH8fOyFtA.png" /></figure><p>The first node in the tree splits into two, the second two nodes split into four, and so on. Each sequential layer of splits represents the “depth” of the model. We can tune the “depth” hyperparameter by instructing the model to stop splitting after reaching a depth of <em>k </em>layers.</p><p>Why would we want to do this? The short answer is to prevent overfitting. If the tree is allowed to run to an unlimited depth, it will continue to split until every single observation can be explained. Once this happens, the model loses any external validity. In its quest to explain every single observation, the model loses its ability to generalize the data. When an overfitted model is presented with new data, it will perform very poorly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UCd6KrmBxpzUpWt3bnoKEA.png" /><figcaption>Visualization of Overfitting</figcaption></figure><p>The data being used today is a dataset obtained from kaggle: <a href="https://www.kaggle.com/andrewmvd/heart-failure-clinical-data">https://www.kaggle.com/andrewmvd/heart-failure-clinical-data</a></p><p>The dataset contains observations from 299 patients in which 96 died from heart failure and 203 did not. Each observation has data for the following 12 variables:</p><pre>&#39;age&#39;,<br>&#39;anaemia&#39;,<br>&#39;creatinine_phosphokinase&#39;,<br>&#39;diabetes&#39;,<br>&#39;ejection_fraction&#39;,<br>&#39;high_blood_pressure&#39;,<br>&#39;platelets&#39;,<br>&#39;serum_creatinine&#39;,<br>&#39;serum_sodium&#39;,<br>&#39;sex&#39;,<br>&#39;smoking&#39;,<br>&#39;follow-up time&#39;</pre><p>The first step as always, import the necessary python libraries:</p><pre>import pandas as pd<br>import numpy as np<br>from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier<br>from sklearn.metrics import classification_report, accuracy_score, recall_score, precision_score, f1_score<br>from sklearn.model_selection import train_test_split, cross_val_score, GridSearchCV</pre><p>The next step is to split the data into two parts. The first part (the <em>train </em>portion) will be used to train the model. The second part (the <em>test </em>portion) will be used to test whether or not the model has external validity. 239 observations will be used to train the model, 60 observations will be used to test the model (8:2 split):</p><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)</pre><p>The next step will be to create a vanilla Random Forest Classification model with no hyperparameter tuning. We will then check the accuracy score on each portion of the data:</p><pre>rf_vanil = RandomForestClassifier()<br>rf_vanil.fit(X_train, y_train.values.ravel())<br>print(&#39;Train Score: {}&#39;.format(rf_vanil.score(X_train, y_train.values.ravel())))<br>y_pred = rf_vanil.predict(X_test)<br>print(&#39;Test Score: {}&#39;.format(accuracy_score(y_test, y_pred)))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/642/1*ue1oMJV5I2rI4ilKwdzP2A.png" /></figure><p>As you can see the model has clearly overfitted the training data, with an accuracy score of 100%! However it’s accuracy on the training data is 25% lower!</p><p>Now we can hyperparameter tune the model. The most comprehensive way to do this in using scikit-learn’s GridSearchCV method. First we establish the estimator, which in this case is the Random Forest Classifier, then we create the grid of hyperparameter types to iterate through:</p><pre>estimator = RandomForestClassifier()<br>param_grid = {<br>    &#39;max_depth&#39;:range(2,7),<br>    &#39;n_estimators&#39;:range(10,30)<br>}</pre><p>In this case the hyperparameters we’re tuning are depth and number of estimators. Depending on the model, these could change. What makes scikit-learn’s GridSearchCV so useful is that it always takes a dictionary, so no matter what the model is, GridSearchCV can be used.</p><pre>grid_search_rf = GridSearchCV(<br>    estimator=est,<br>    param_grid=param_grid,<br>)</pre><pre>grid_search_rf.fit(X_train, y_train)</pre><pre>print(&#39;Best Parameters were: {}&#39;.format(grid_search_rf.best_params_))<br>print(&#39;Best CrossVal Score was:{}&#39;.format(grid_search_rf.best_score_))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/574/1*6byMnx-W1yAOCek7YjcfdQ.png" /></figure><p>Now we can see which values of the hyperparameters produced the most optimal cross validation score. The accuracy score of 87% is much more reasonable than 100%! We can now apply these values of the hyperparameters into a new Random Forest Classifier:</p><pre>rf_clf = RandomForestClassifier(<br>    max_depth = 3,<br>    n_estimators = 26<br>)<br>rf_clf.fit(<br>    X_train,<br>    y_train.values.ravel()<br>)<br>y_pred_rf = rf_clf.predict(X_test)<br>print(&#39;Random Forrest Accuracy: {}&#39;.format(accuracy_score(y_test, y_pred_rf)))</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/492/1*qInF2WX8H7xFM3JrDKwo-w.png" /></figure><p>Now when we apply the trained model to the test portion of the data, it performs much better. By preventing the model from overfitting, it has been able to generalize the data much better and maintain external validity. We can say with some confidence now that if a new patient presented, the model would be able to correctly identify whether or not they were at a high risk of heart failure 83% of the time.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=973c35ce5a1a" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Smoothing Time Series in Python: A Walkthrough with Covid-19 Data]]></title>
            <link>https://medium.com/@teamastertoast/smoothing-time-series-in-python-a-walkthrough-with-covid-19-data-b4ccfb980a61?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4ccfb980a61</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[covid19]]></category>
            <category><![CDATA[data-visualization]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Tue, 18 Aug 2020 00:52:17 GMT</pubDate>
            <atom:updated>2020-08-18T19:49:14.230Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QIXqKaKHE5qVxuyRuGs-GA.jpeg" /></figure><p>This will be a brief tutorial highlighting how to code moving averages in python for time series. More complicated techniques such as Hodrick-Prescott (HP) filters and Loess smoothing will not be covered.</p><p>Being able to smooth out volatile time series data is a crucial tool in a data scientist’s tool box. When volatile data is smoothed, long term trends become clearer. To demonstrate, here is a time series before an after smoothing:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1009/1*4n46ihLjZz_1rIPKHR5LSw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1003/1*84l4juaXR_WdM0E85RoNNQ.png" /></figure><p>When one reviews the Covid-19 data, what becomes evident is that a sinusoidal pattern exists in the daily new cases data. Whilst baffling at first, the cause is quite intuitive: habitually, fewer individuals leave the house on the weekends and thus fewer people are being tested on the weekends. This is why we see a drop and subsequent rise in new cases every seven days:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TuaVj2FBPutB5uTNB8plyQ.png" /><figcaption>Daily New Covid-19 Cases</figcaption></figure><p>This data series is a prime example of when data smoothing can be applied. With the constant “jitteriness” in the data, it can be difficult to discern emerging trends in the number of new Covid-19 cases. Smoothing solves this problem.</p><p>The following code will demonstrate how to do this with a moving average. As always, the first thing I do in python is import all the packages I’m going to use:</p><pre>import pandas as pd<br>import numpy as np<br>import matplotlib.pyplot as plt<br>%matplotlib inline<br>import seaborn as sns<br>from datetime import datetime</pre><p>The next step is to read the data into python using pandas. The data is stored as a csv file that I’ve downloaded and saved on my local hardrive:</p><pre>df_nat = pd.read_csv(&#39;covid-19-data/us.csv&#39;)</pre><p>I do some brief data cleaning by converting the date column and cases column (which are both strings) into a datetime object and numeric object respectively. I then feature engineer two columns by calculating the first and second derivative of the number of cases:</p><pre>df_nat.date = pd.to_datetime(df_nat.date)<br>df_nat.cases = pd.to_numeric(df_nat.cases)<br>df_nat[&#39;new_cases&#39;] = df_nat.cases.diff()<br>df_nat[&#39;growth_new_cases&#39;] = df_nat.new_cases.diff()</pre><p>When I plot new_cases it looks like the image I showed earlier, very jittery and volatile:</p><pre>sns.set(rc={&#39;figure.figsize&#39;:(11.7,8.27)})<br>sns.lineplot(<br>    x=&#39;date&#39;,<br>    y=&#39;new_cases&#39;,<br>    data= df_nat)<br>plt.show()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TuaVj2FBPutB5uTNB8plyQ.png" /></figure><p>I calculate the moving average by feature engineering a new column using panda’s built-in <em>rolling</em> method. I chose a window of seven days because the wavelength of the sinusoidal pattern in the data is one week (since new cases rise and fall around the weekends):</p><pre>df_nat[&#39;mov_avg&#39;] = df_nat[&#39;new_cases&#39;].rolling(7).sum()</pre><p>Now when I graph the smoothed data calculated with the moving average, the series looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/724/1*hp6hoi7KwKjnFefRJVNpsQ.png" /></figure><p>We can now see clearly how the number of new cases trended downward during the lockdown, accelerated rapidly during the reopening, and now appear to be trailing off again.</p><p>We can also perform this smoothing on the second derivative, ie. examining the <em>growth </em>in the daily new cases to discern any emerging trends:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*Bp4PUc1T8Pq1ePELbqSqxw.png" /><figcaption>Growth in Daily New Cases</figcaption></figure><p>As one can see, the graph of the second derivative of Covid-19 cases looks a mess. There are huge spikes above and below zero, with the series looking almost like white noise. However, once smoothing is applied with the same 7-day moving average the data becomes much clearer:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/723/1*HhmLoMpLTMiIxBabsLIIOg.png" /><figcaption>Smoothed Growth in Daily New Cases</figcaption></figure><p>We can now see how the growth in daily new cases (a crucial leading indicator for public health officials) is changing over time. We can see there is a huge period of new case growth during march, a relatively stable period of growth during the lockdown, another spike in growth during the reopening, followed by another drop. What’s encouraging is that the current growth in new cases has fallen <em>below </em>the point at which it was during the lockdown.</p><p>The moving average is a simple and powerful data smoothing technique. One can go much further an implement more complex methods that are more robust and can address certain problems that the moving average can’t. These include HP filters, Loess smoothing, and various others. However, for those who are looking for a quick and effective method without too much code or calculation, the moving average is a great way to get started.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4ccfb980a61" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Syntax for Seaborn]]></title>
            <link>https://medium.com/@teamastertoast/syntax-for-seaborn-7e0a15c4c6b4?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/7e0a15c4c6b4</guid>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Fri, 01 May 2020 18:05:05 GMT</pubDate>
            <atom:updated>2020-05-01T18:05:05.271Z</atom:updated>
            <content:encoded><![CDATA[<p>Historically, Matplotlib is the typical visualization package used in python, but there’s another package which I personally never use, which is why it’s imperative that I write a blog post on how to use it.</p><p>Seaborn has more built-in functions, requires fewer lines of code, and produces aesthetically more pleasing visualizations. This begs the question why I don’t normally use it. The answer is I didn’t know the syntax. However, once it’s understood, it is actually a simpler method of producing visualizations.</p><p>I will cover briefly the standard syntax and good practices for using the Seaborn library, then provide a few examples of the different visualizations seaborn can produce.</p><p>First things first, import the libraries into python, it is convention to import seaborn as <em>sns</em>:</p><pre>import pandas as pd<br>import seaborn as sns<br>import matplotlib.pyplot as plt<br>%matplotlib inline</pre><p>The data I’m going to use is the movie data from mod-1 project which is a pandas dataframe which I’ve named <em>df</em> (by convention). Most seaborn functions will take a “<em>data =” </em>argument, which allows you to pass a pandas dataframe. This method of passing a dataframe is recommended over passing individual series as arguments.</p><p>It’s good practice to briefly check the dataframe visually with the following line:</p><pre>df.head()</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PV2vrGU4j5pfZzGreYFNwg.png" /></figure><p>Beautiful!</p><p>Now I want to use Seaborn to create a bar-chart showing movie genre by profit. In matplotlib the code would have been:</p><pre>x = df.index<br>y = df[&#39;Profit&#39;]</pre><pre>fig = plt.figure(figsize=(10, 6))<br>ax = plt.subplot()</pre><pre>plt.bar(x,y)</pre><pre>ax.set_xticks(range(len(x)))<br>ax.set_xticklabels(x)<br>plt.xticks(rotation=90)<br>plt.title(&#39;Mean Profit by Genre&#39;)<br>plt.xlabel(&#39;Genres&#39;)<br>plt.ylabel(&#39;Profit (Hundreds of Millions)&#39;)<br>plt.show()</pre><p>In seaborn the code is a little more straightforward:</p><pre>sns.barplot(x=genres_profit.index, y=genres_profit[&#39;Profit&#39;], data=df)<br>sns.set(rc={&#39;figure.figsize&#39;:(10,8)})<br>plt.xticks(rotation=90)</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E1jqkDcGYCYxethpTAemAA.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7e0a15c4c6b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Credit Risk Modeling: An Application for Machine Learning]]></title>
            <link>https://medium.com/@teamastertoast/credit-risk-modeling-an-application-for-machine-learning-b4ed53a78d7?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/b4ed53a78d7</guid>
            <category><![CDATA[risk-management]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[banking]]></category>
            <category><![CDATA[finance]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Tue, 28 Apr 2020 13:10:24 GMT</pubDate>
            <atom:updated>2020-04-28T13:10:24.397Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Og7cCK3XJU-_5y1LlRFvmw.jpeg" /></figure><h3>The Need for Improved Credit Risk Modeling:</h3><p>Nothing exposed the chronic mishandling of credit risk by lenders than the 2008 housing market crash and subsequent recession. In the span of a just several months, the banking sector came within an inch of total collapse due to massive overexposure to credit defaults.</p><p>What went wrong?</p><p>Contrary to popular belief, the banking sector didn’t <em>knowingly</em> plot to dangerously overexpose themselves to credit risk in order to bring down the entire world economy. The rapid expansion of credit during the housing market boom of the early-mid 2000s didn’t bat any eyelids amongst lenders because, according to their conventional credit risk modeling standards, everything was above board.</p><p>Despite using statistical methods and models to calculate credit risk, none raised any alarm bells to lenders that they were dangerously overexposed. In the wake of the financial crisis two things became clear:</p><ol><li>There was a new type of risk dimension: <em>Model </em>risk</li><li>The models being used to calculate credit risk needed overhaul.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/910/1*78z8AAbO0B-1_oTmavcYqA.png" /></figure><h3>A Brief History of Conventional Credit Risk Modeling:</h3><p><strong>Basel I:</strong></p><p>The idea of a sudden and complete collapse of a bank (or several banks) due to risk overexposure was not something that was outside the realm of imagination before 2008. In 1974, following the collapse of the German bank <em>Herstatt </em>due to insufficient capitalization to cover a catastrophic depreciation in the US dollar, central bank representatives from the G10 met in Basel Switzerland to set a standard for risk management that all member banks had to adhere to. These standards were called Basel I. These were a series of capital requirements for different types of risk. For credit risk, banks had to hold enough capital to cover at least 8% of all outstanding credit. This initial credit risk management strategy was simple to say the least and was only expanded 30 years later.</p><p><strong>Basel II:</strong></p><p>In 2004, the Basel committee created a whole new revised and updated list of banking standards. Included in these was the first major component of modern credit risk modeling: Internal Rating Based modeling (IRB).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*IdWllsFc3q4mG2hGplhGPQ.png" /></figure><p><strong>The 4 Risk Parameters:</strong></p><p>To calculate capitalization requirements, IRB introduced 4 <em>Risk Parameters</em> to be used to model credit risk:</p><ul><li>Probability of Default (PD): The likelihood of default for a credit instrument.</li><li>Exposure at Default (EAD): The amount of money that hadn’t been paid back to the lender.</li><li>Loss Given Default (LGD): The amount of money that hadn’t been paid back to the lender after all recovery attempts are exhausted.</li><li>Effective Maturity (M): The amount of time left before a credit instrument is fully paid off.</li></ul><p>The last three are relatively easy to calculate. The risk parameter that was of most importance, and required the most modeling effort from banks was the first parameter, Probability of Default (PD).</p><p><strong>Old School (Qualitative) PD Modeling:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dW6t76AQTp50aL_2dLHXQw.png" /></figure><p>To calculate the probability of default, lenders would adopt an approach called the <em>Judgmental Method</em> wherein a loan officer would review an application and use his or her judgement to determine whether someone would be approved for a loan.</p><p>The judgmental approach revolved around the 5 “C’s”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*R9WK-Sb4h3LFlS82qnMezQ.png" /></figure><p><strong>Character</strong>: Has the applicant paid back loans in the past?</p><p><strong>Capital</strong>: What is the applicants assets minus their liabilities?</p><p><strong>Collateral</strong>: What assets does the applicant have that can be promised to the bank in the event of a default?</p><p><strong>Capacity</strong>: By what means can the applicant pay back the loan? What is their income? Do they have a cosigner?</p><p><strong>Condition: </strong>What additional internal or external conditions could affect the applicant’s ability to repay the loan?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EHchbbjrBZCXA0AG4aL7Fw.jpeg" /></figure><p>The last “C,” <em>Condition</em>, proved to be the most problematic of the five. The vagueness and interpretability of the fifth “C” (as opposed to the quantifiable nature of the other four) offered the potential for a wide degree of variance in PD values from lender to lender. It also allowed for bias and prejudice to infect the loan application process. Indeed, it is well-documented that minorities have historically found it harder to have credit applications approved.</p><p><strong>The Statistical Method:</strong></p><p>Realizing the inadequacy of qualitative credit risk modeling, many banks began to switch over to statistical methods. Probably the most popular of which being the <em>Value at Risk </em>(VaR) model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1006/1*pGZhmkAqGQMalwtuDCtdrg.png" /></figure><p>VaR models fundamentally measure risk by calculating the standard deviation of a loan portfolio’s value. The higher the standard deviation of the loan portfolio, the riskier than portfolio was. The PD value would often be calculated by taking the number of defaults a loan portfolio would experience in a given time frame, divided by the total number of loans in that portfolio. Thus, as long as enough borrows didn’t default, the value of the portfolio wouldn’t experience significant volatility, the standard deviation would remain low, and the credit risk exposure would appear tenable.</p><p>Credit ratings agencies like Moody’s, Fitch, and S&amp;P would also use standard deviation as a key risk metric determining whether loans would be rated from AAA to C.</p><p><strong>The Vulnerabilities of Conventional Credit Risk Modeling Exposed:</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*-lwYOYkN5o0gc2ryGT5Hnw.jpeg" /></figure><p>In 2008, a series of rapid and simultaneous defaults in both the subprime and non-subprime housing market combined with a concurrent collapse in housing prices, revealed the true extent of the credit risk lenders and banks were exposed to. To be fair, the problem was made exponentially worse by the rampant sale of complex financial instruments such as credit default swaps (CDS) and collateralized debt obligations (CDO) in the years leading up to the crisis. These aggravating factors notwithstanding, there had been a fundamental failure in the ability to model credit risk.</p><h3>Post-crisis Credit Risk Modeling: Machine Learning &amp; Deep Learning Models</h3><p>Despite the catastrophic failure in risk management exposed by the financial crisis, the fundamental task for credit risk modeling ultimately hasn’t changed. That is, how can the PD parameter be accurately determined so that Banks know how much capital is required to prevent insolvency?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/597/1*00ctjOLY04Yvm3X0FnCOOQ.jpeg" /></figure><p>In recent years, Machine learning classification models have become increasingly popular methods for calculating PD. The machine learning algorithms proving to have the most success are <em>Elastic Net </em>(an extension of logistic regression), <em>Gradient Boosting</em>, and <em>Random Forrest</em>.</p><p>These are all supervised learning tasks requiring sufficiently large and balanced labelled data. However, this isn’t a problem for banking institutions, who have access to a myriad of transaction and financial data from their own customer bases.</p><p>Crucially, these machine learning classifiers allow for the consideration of macroeconomic data as well as individual loan applicant data. The probability values derived from a machine learning algorithm can be modified by adding a penalization term according to current macroeconomic conditions. These probabilities can also be “stress-tested” by modifying the penalization term according to worst-case-scenario macroeconomic conditions.</p><p>Macroeconomic conditions that can be used to calculate a penalization term include, but certainly aren’t limited to:</p><ul><li>GDP</li><li>Index of Industrial of Production</li><li>Interest rates</li><li>Unemployment</li><li>Case-Shiller Home Price Index</li><li>Inflation</li><li>Net exports</li><li>Foreign exchange rates</li></ul><p><strong>Deep Learning Methods:</strong></p><p>As data becomes increasingly complex, large and unstructured, deep learning models are increasingly being used for credit risk modeling. Principal component analysis for dimensionality reduction, clustering techniques, and neural networks have all shown promise in their ability to model credit risk.</p><p>Several high-profile “alternate-lending” start-ups emerged following the financial crisis implementing deep learning methods as their primary means of risk management, the most notably being SoFi, Prosper, and LendingClub.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*yOqLuR5NW7vzC1LBJfJOXg.jpeg" /></figure><p>The primary drawback of deep learning models is the problem of explainability. The features extracted from PCA and clustering techniques are difficult to identify, whereas the methodology used by black box techniques such as neural networks are by their very nature impossible to know. This makes regulatory bodies (who normally need to know methodologies to ensure proper oversight) extremely wary.</p><p>In all, the opportunities for machine learning in credit risk modeling are vast, and development is still in its infancy. Almost all large banks currently use proprietary machine learning algorithms for their credit risk modeling. What remains to be seen is how much further the field can be advanced using machine learning as its primary tool.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b4ed53a78d7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Predicting Hotel Cancellations]]></title>
            <link>https://medium.com/@teamastertoast/predicting-hotel-cancellations-af0f9b594b47?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/af0f9b594b47</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[classification-algorithms]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[hospitality]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Wed, 08 Apr 2020 18:13:02 GMT</pubDate>
            <atom:updated>2020-04-08T18:13:02.757Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Hj8W3BBCMFR6i6eIjg8I-w.jpeg" /></figure><p>We all know the feeling, after borrowing tens of millions of dollars to start our five-star hotel chain and painstakingly setting up a series of over-leveraged luxury hotel resorts, we get hit with the big kahuna: almost half of all our reservations get cancelled.</p><p>This number isn’t unique, according to a study conducted by D-edge hospitality solutions in 2018, almost 40% of hotel reservations are cancelled worldwide. Leaving hotel managers with a huge opportunity cost.</p><p>The conventional solution has been to blindly overbook. Unfortunately this brings with it a whole slew of new problems. If the number of cancellations dips just slightly, a hotel can be stuck with more guests than rooms.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/612/1*1UWmMV77TE8R7Ep-iKYIrQ.jpeg" /></figure><p>But what if there was a way you could predict which guests were likely to cancel and adjust the overbook rate accordingly? That would be great wouldn’t it? Luckily for you that is what we’re going to do today using the magic of machine learning in Python.</p><p>First thing’s first, we’re going to import our machine learning libraries and (previously cleaned) dataset:</p><pre>import pandas as pd<br>from sklearn.preprocessing import StandardScaler<br>from sklearn.model_selection import train_test_split<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.neighbors import KNeighborsClassifier<br>from yellowbrick.classifier import ConfusionMatrix<br>from sklearn.metrics import classification_report<br>from sklearn.dummy import DummyClassifier<br>pd.read_csv(&#39;cleaned_csv&#39;</pre><p>This dataset has a combination of categorical and numerical variables. For the machine learning algorithm to understand the data we need to dummy encode the categorical variables and scale the numerical variables.</p><p>To do this we define the categorical and numeric column names and run the following code:</p><pre><em>#defining categorical variables:<br></em>df_cat = df[[&#39;hotel&#39;,&#39;meal&#39;,&#39;market_segment&#39;,&#39;distribution_channel&#39;,\<br>&#39;reserved_room_type&#39;,&#39;assigned_room_type&#39;,&#39;deposit_type&#39;,\<br>            &#39;customer_type&#39;,&#39;reservation_status&#39;]]</pre><pre><em>#defining numeric variables:<br></em>df_num = <br>df[[&#39;lead_time&#39;,&#39;adults&#39;,&#39;children&#39;,&#39;babies&#39;,\<br>&#39;previous_cancellations&#39;,&#39;previous_bookings_not_canceled&#39;,\             &#39;booking_changes&#39;,&#39;days_in_waiting_list&#39;,\<br>&#39;required_car_parking_spaces&#39;,&#39;total_of_special_requests&#39;]]</pre><pre><em>#Dummy encoding categorical data:</em><br>df_dummies = pd.get_dummies(df_cat,drop_first=True)</pre><pre><em>#Scaling numeric data:</em><br>ss = StandardScaler()<br>df_num_scaled = pd.DataFrame(ss.fit_transform(df_num), columns = [&#39;lead_time&#39;,&#39;adults&#39;,&#39;children&#39;,&#39;babies&#39;,&#39;previous_cancellations&#39;,\                                                             &#39;previous_bookings_not_canceled&#39;,&#39;booking_changes&#39;,\<br>&#39;days_in_waiting_list&#39;,\                                                          &#39;required_car_parking_spaces&#39;,&#39;total_of_special_requests&#39;])</pre><pre><em>#concatenating the two dataframes:</em><br>df_final = pd.concat([df_num_scaled,df_dummies,df[&#39;is_canceled&#39;]],axis=1)</pre><pre><em>#dropping missing values:</em><br>df_final = df_final.dropna()</pre><p>We want to split the data into a training set and a test set so that we can test our model for external validity:</p><pre>X = df_sample.drop(columns=[&#39;is_canceled&#39;])<br>y = df_sample[&#39;is_canceled&#39;]</pre><pre>X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)</pre><p>We’re first going to fit a dummy model that simply returns the frequency of non-cancellations as a baseline for our classification model:</p><pre>#Fitting a Dummy Model:<br>dummy = DummyClassifier(strategy=&#39;most_frequent&#39;)<br>dummy.fit(X_train, y_train)<br>dummy.score(X_test, y_test)</pre><p>The dummy model returns a score of<strong> 0.6355 </strong>which matches up with what the research from D-edge hospitality solutions.</p><p>Now we’re going to fit a logistic regression model as the classifier:</p><pre>logreg = LogisticRegression()<br>logreg.fit(X_train,y_train)<br>logreg.score(X_train,y_train)</pre><p>The logistic regression model returns a score of <strong>1.0</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*yfFcYhwvqXg9MswrxaMH8A.jpeg" /></figure><p>This immediately should raise some eyebrows. A score of 1.0 essentially means that the model is perfectly predicting whether or not someone will cancel a reservation and making zero errors. A quick look at the confusion matrix demonstrates this visually:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/948/1*OLMtCzR0jzLS94TuAga6dw.png" /></figure><p>There is likely to be a variable that is perfectly correlated with the target variable that’s causing this, and sure enough, after inspecting the “reservation_status” regressor we can see:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/593/1*GAj-tclWkmn1XLZNCBxMFA.png" /></figure><p>So we should drop this column from the dataset in order to remove this “leakage” source from the data. We run logistic model again without this column and we get a much more reasonable score of <strong>0.812</strong></p><p>Let’s look at the new confusion matrix:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/966/1*cJvZHVkrexjbtY8agF29Og.png" /></figure><p>As we can see the recall is much better than the precision. In other words, the model is much better at correctly predicting whether someone is going to cancel (A True Positive) than it is at minimizing incorrect predictions that people will cancel but wont (A False Positive). From a profitability stand-point, it’s much better to correctly predict the majority of cancellations and adjusting overbooking accordingly, than it is to minimize the few false positives that we’re going to encounter.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=af0f9b594b47" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A method to using Git for Learn.co lessons]]></title>
            <link>https://medium.com/@teamastertoast/a-method-to-using-git-for-learn-co-lessons-37c533449db4?source=rss-25e20e25c2aa------2</link>
            <guid isPermaLink="false">https://medium.com/p/37c533449db4</guid>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[terminal]]></category>
            <category><![CDATA[git]]></category>
            <dc:creator><![CDATA[Joe McHugh]]></dc:creator>
            <pubDate>Wed, 18 Mar 2020 12:13:44 GMT</pubDate>
            <atom:updated>2020-03-18T12:13:44.403Z</atom:updated>
            <content:encoded><![CDATA[<p>The easiest way to learn the mystery that is Git is to create a flow, or a method to repeat mindlessly over and over again. Hopefully, if repeated enough times, the method itself will impart the knowledge necessary to use Git successfully.</p><p>I won’t be defining what Git is with any metaphors or attempting to explain the maze of git commands with colorful charts, you’ve all seen enough of those. This is simple a down and dirty checklist to using Git successfully for learn.co lessons.</p><p>You might ask, “why can’t we just use the learn.co jupyter notebook ?” The answer is, you definitely can, it just won’t save anywhere once you’re done. If you ever wanted to go and look back at your work, it won’t be saved at learn.co; you’ll just see a blank new jupyter notebook where all your previous work once was.</p><p><strong>The method:</strong></p><p>Start by navigating to the learn.co website and finding the lesson you want to complete. In this example I will be using the <em>Getting Started with Numpy Lab. </em>Once you’ve chosen your lesson, click on the GitHub symbol (it looks a big like a cat in a bubble).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q--Xfh7duYWjHbSim1OQWQ.png" /></figure><p>This will take you to the GitHub page that hosts the repository of the lesson. Once there, click on the “fork” button to create your own version of the repository (this will allow you to make your own changes and edits).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XneTlQdpU9nDolyzMZjViQ.png" /></figure><p>Once that is done, be sure to check that the file path has changed from <em>learn-co-students/lesson_name </em><strong>to</strong> <em>your_git_hub_username/lesson_name. </em>Once that is done, click on the “Clone or download” button in green to get a copy of the git repository url. Then click on the clipboard symbol to copy the url to your clipboard.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rq3MaHG0v3IK5jPyQL6qQA.png" /></figure><p>Now here is where things get a little spicy. In order to move the repository to your computer, you need to open up your terminal and navigate to the folder you want to clone to. If you want, you can create a new folder by typing:</p><pre>mkdir whatever_folder_name</pre><p>Navigate to your folder by typing:</p><pre>cd whatever_folder_name</pre><p>now to clone the repository type in the follow line of code (you can simply type control/command-v to paste the repository url:</p><pre>git clone <a href="https://github.com/Joe-Bit-lab/dsc-getting-started-with-numpy-lab-dc-ds-021720.git">https://github.com/Joe-Bit-lab/dsc-getting-started-with-numpy-lab-dc-ds-021720.git</a></pre><p>If successful, you’ll see the following message:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1010/1*EABmkuYtl4Z0iOC8Vd-sRw.png" /></figure><p>Now the repository should be cloned to your computer. You can now open the class in jupyter by typing the following:</p><pre>jupyter notebook </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GhoIFJFBGZxuCViT8N0bMQ.png" /></figure><p>The lesson is now running locally on your jupyter notebook. Once you’ve finished working on the notebook you’ll need to save it. We can do this by using the g<em>it add </em>&amp; <em>git commit </em>commands. To use <em>git add</em> we simply type in the git add command and the file name:</p><pre>git add &lt;file_name&gt;</pre><p>This stages the file to be committed. Once this is complete we can type in the git commit command followed by -m and a message (which is required):</p><pre>git commit -m &#39;Whatever message&#39;</pre><p>Now the changes we’ve made have been saved but only on your computer. If we type:</p><pre>git status</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*irUD-tBy_ThQ8eeesDs00w.png" /></figure><p>We can see that our “branch” (essentially our version) is “ahead” (different) from the repository we’ve cloned by 1 commit. To save our changes to GitHub, we can publish our local commits by typing the git push command:</p><pre>git push</pre><p>The changes have now been published on GitHub and saved on the cloud as well as our local computer. We’ll be able so see our changes on GitHub by navigating to our forked repository and clicking on the file we’d been working on.</p><p>Repeat this method over and over again until it becomes second nature.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=37c533449db4" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>