<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Andrew Sprague, CFA® on Medium]]></title>
        <description><![CDATA[Stories by Andrew Sprague, CFA® on Medium]]></description>
        <link>https://medium.com/@spraguesy?source=rss-71235cf30811------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Vk8UCy0SksrvdHUFOYeH1A.jpeg</url>
            <title>Stories by Andrew Sprague, CFA® on Medium</title>
            <link>https://medium.com/@spraguesy?source=rss-71235cf30811------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 06:48:44 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@spraguesy/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Crashing the Crash Course: A Look at Google’s Machine Learning Course — Part III]]></title>
            <link>https://medium.datadriveninvestor.com/crashing-the-crash-course-a-look-at-googles-machine-learning-course-part-iii-ad38d70b0ae8?source=rss-71235cf30811------2</link>
            <guid isPermaLink="false">https://medium.com/p/ad38d70b0ae8</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Andrew Sprague, CFA®]]></dc:creator>
            <pubDate>Tue, 30 Oct 2018 03:49:11 GMT</pubDate>
            <atom:updated>2018-10-30T03:49:11.564Z</atom:updated>
            <content:encoded><![CDATA[<h3>Crashing the Crash Course: A Look at Google’s Machine Learning Course — Part III</h3><p>Welcome to part III of my journey through Google’s Machine Learning crash course. In my last post (part <a href="https://medium.com/@spraguesy/crashing-the-crash-course-a-look-at-googles-machine-learning-crash-course-part-ii-f5c11d1cddde">II</a>), I walked through two sections of the course, <em>Descending into ML</em> and <em>Reducing Loss</em>. These topics cover basic linear regressions and how they are applied to the world of ML. I talked about Mean Squared Errors and how to reduce loss. In order to minimize loss, we use an iterative approach. The learning rate tells us how big of a step we take to make our next guess, and the negative gradient tells us the direction. The smaller the step, the more computations it may take to find the minimum. A big step means we may overshoot the minimum. Since we potentially have a lot (think numbers that start with Ms or Bs) of data points, it is often best practice to use mini-batch stochastic gradient descent, where hundreds of data points, chosen at random, are used instead of computing the entire data in order to reduce noise but be more efficient.</p><p>The next section is titled <em>First Steps with TF. </em>We get a short video which contains some basic information about TensorFlow. TensorFlow is built with a hierarchical structure. The higher level APIs build on those on lower-levels and allow users to use predefined architectures without having to write a lot of code. They abstract away a lot of the code. The API used most often in this course is the tf.estimator. Using the higher-level does have drawbacks. The more flexibility you require, the lower down you will need to go with the APIs you use.</p><p>After the video and the toolkit that is summarized in the previous paragraph, we get our first set of exercises. These exercises run on the <a href="https://colab.research.google.com/">Colaboratory</a> platform, another Google product. Colab runs in the browser, but there is a desktop version if you get the desire to run the exercises locally. We are to the point in the course where you need to start setting up your local environment (because you can’t use Colab in the real world, duh). If you haven’t ever developed in Python, I suggest <a href="https://www.youtube.com/watch?v=dNFgRUD2w68">this</a> video to help you get started on setting up your environment if you use VS code like me. Now onto the exercises.</p><p><em>Quick Introduction to pandas</em></p><p><em>Note: Click the brackets on the left in Colab to run the code. You will need to run the brackets to make the examples further along work.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/259/1*LVFAdMT8qAYpjgSpwbR0Pw.png" /></figure><p>Pandas is a column-oriented data analysis API. Pandas data structures are supported by many ML frameworks. There is a more comprehensive look at Pandas <a href="http://pandas.pydata.org/pandas-docs/stable/index.html">here</a>. The primary data structures of pandas are Series and DataFrame. A DataFrame is essentially a relational table (think SQL). A series is a single column. Here is an example of using Series and DataFrame:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*n8sJQyXmbSfjAykwXadskg.png" /></figure><p>You can run all of this in your command line by following <a href="https://www.youtube.com/watch?v=cQvwhjvd9Rc">this</a> video. Once you install Python and pandas, you can go into your command line, enter you’re the Python environment by typing python and then pasting the above. Should look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/579/1*fraoSQBkgtLagLJ10h1P3g.png" /></figure><p><em>First Steps with TensorFlow</em></p><p>At the time of this post, Python 3.7 is out, but TensorFlow support goes up to 3.6. So if you are wanting to implement this outside of Colab (and you should if you are taking this course), I suggest making sure you are using Python 3.6 until TF gets compatibility with Python 3.7.</p><p>When you do set this up in your own environment, remember to install the packages using pip (or conda) and import everything (like tensorflow and pandas) from the Setup section.</p><p>Now we are going to build a model using TF. There are six steps in building a model. The first step is to define features and configure feature columns. A feature is “ an individual measurable property or characteristic of a phenomenon being observed”. Essentially, a feature is the category that you are going to place data in that helps you make a prediction. Features come in two forms, Categorical and Numerical. Sticking with their housing example, an example of a categorical feature is the words in a real estate ad or a style of a house. An example of a numerical data type would be the square footage of a house. The feature type is indicated in the feature column, which only stores the data type and no actual data. Below, the feature <em>number of rooms</em> is defined along with the data type in the feature column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/545/1*OMRnyLaaJXxDtNpd8_v4pQ.png" /></figure><p>The next step is to define the target. The target is the data point that you want to predict. Defining it looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/491/1*yDUnBbCbOvAT7O1xMFbOag.png" /></figure><p>Step three is to configure the linear regression. If you recall from previous posts, here we input the learning rate, i.e. the gradient step. The optimizer used here is the GradientDescentOptimizer, which also calls on what we learned previously, Mini-Batch Stochastic Gradient Descent.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/1*v8_gUkNPu9ygdH7hwJISEg.png" /></figure><p>my_optimizer is set to train using the GDO with a learning rate of 0.0000001. my_optimizer, along with the featured columns, is then used to generate a <em>LinearRegression </em>estimator.</p><p>The next step is the input function. This function may look a little scary so I’ll try and break it down. The functions take potentially 5 total arguments, two of which are required. The first argument is <em>features</em>, these are the features that we are using to predict the target (in this case “total rooms”). The <em>targets</em> argument is the thing we are trying to predict. <em>Batch size</em> is the number of data points to be analyzed, here is defaults to one. <em>Shuffle</em>, which defaults to true, does exactly what it says. We shuffle so we get a random batch of data vs how the data is currently ordered. The last arguments is <em>num_epochs</em>. Epochs are essentially how many times the neural network is fed your data. Too few epochs can lead to underfitting your data, but too many can lead to overfitting. Take a look at <a href="https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9">this</a> article to get a better understanding of epochs.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/623/1*x3rh-9Rn1wLAI6eCXrv_MQ.png" /></figure><p>So we have defined all of our features, written the function, and now we just need to train the model and analyze its output. Training is fairly simple:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/419/1*DHlKMEFVh-8rNE7x5LcwqQ.png" /></figure><p>We then analyze the output. There are many ways to do that. First, can we look at the MSE and the Root MSE. Root MSE can be interpreted in the same scale as the data, so this summary statistic is preferred. We can then also look at our predictions vs the target. They look like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*ZCuOcApKFIviVzuGO-_MUQ.png" /></figure><p>And we realize that something looks really wrong. Our predictions and targets are way off. Here they are plotted against each other.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/536/1*90qQmcJthjEXRQ082j_yrw.png" /></figure><p>Terrible. But that doesn’t mean there is no relationship between these two data points. We can see that there is a better line. This means we need to tweak the model hyperparameters. Check out the section on Tweaking the hyperparameters and attempt to make the line a better fit.</p><p>The goal of the task is to get an RMSE below 180. Here is what it looks like when it is close. As you can tell that is a much better prediction than the line above (the final line is in red).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/543/1*HHfeCwvkSqYi0CLc3nKNOw.png" /></figure><p>There are a few rules of thumb to follow when tweaking hyperparameters. Ultimately, the “effects are data dependent”, so you should stay flexible. Training error should decrease steadily and eventually plateau. If it doesn’t converge, run it for longer. If it takes too long to decrease, increase the learning rate. If the TE is highly volatile, then decrease the learning rate. Use larger batch sizes at first, and the decrease until training error starts to become less stable.</p><p><em>Synthetic Features and Outliers</em></p><p>These exercises teach you how to make synthetic features (features that are ratios of each other) and how to handle outliers (like that one dot way to the right on the above chart). Creating a ratio is fairly simple using <em>pandas.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*nBrXEAGLm3ZG4VL9zCQfbA.png" /></figure><p>You then feed that into the model that they have built. Taking out the outliers is fairly easy as well. You then can run the model again and see the improvement that it makes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*gKfvcxHMm-sPzYXhNaUZFg.png" /></figure><p><em>Summary</em></p><p>We took a basic look at pandas and showed how TF takes datasets like pandas. We learned how to set up the environment (in Colab or in Windows). We then took a look at the six steps to set up a model and how to use the TF APIs to do so. Finally, we were able to view the analytics of our ML model and tweak them in order to get a better fit. In my next post, I’ll go through the <em>Generalization </em>and<em> Training and Test Sets </em>section of the course.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ad38d70b0ae8" width="1" height="1" alt=""><hr><p><a href="https://medium.datadriveninvestor.com/crashing-the-crash-course-a-look-at-googles-machine-learning-course-part-iii-ad38d70b0ae8">Crashing the Crash Course: A Look at Google’s Machine Learning Course — Part III</a> was originally published in <a href="https://medium.datadriveninvestor.com/">DataDrivenInvestor</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Crashing the Crash Course: A Look at Google’s Machine Learning Course — Part II]]></title>
            <link>https://medium.com/@spraguesy/crashing-the-crash-course-a-look-at-googles-machine-learning-crash-course-part-ii-f5c11d1cddde?source=rss-71235cf30811------2</link>
            <guid isPermaLink="false">https://medium.com/p/f5c11d1cddde</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Andrew Sprague, CFA®]]></dc:creator>
            <pubDate>Thu, 27 Sep 2018 03:51:56 GMT</pubDate>
            <atom:updated>2018-09-27T14:55:43.146Z</atom:updated>
            <content:encoded><![CDATA[<p>The next two parts of <a href="https://developers.google.com/machine-learning/crash-course/">Google’s Machine Learning Crash Course</a> are on our plate. In my last <a href="https://medium.com/@spraguesy/crashing-the-crash-course-a-look-at-googles-machine-learning-course-part-i-abd3e8c533f0">post</a>, I covered briefly what ML was, who would do this course, what the prerequisites are, and wrote briefly about framing. These two sections, <em>Descending into ML</em> and <em>Reducing Loss, </em>go into linear regressions and minimizing loss. The basis of ML is to use data to help make a prediction. These predictions are done through regressions. The better your data fits into the model, the higher the likelihood you will be able to make an accurate prediction on unlabelled data points.</p><h3>Descending Into ML</h3><p>There is a brief video in this section which uses a simple model to explain the idea of <em>loss</em>. They use square footage of a house as an independent variable, and housing prices as the dependent variable. We can fit a line on the data (more on how to calculate that later). The line predicts the value of the house given it square footage. What we are trying to do is make that prediction as best we can, so we want that line to go through our data in the most optimal way. The equation for this simple line is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/237/1*0nMf6cymg8CBmTMRTptWTg.png" /></figure><p>As a quick aside, I am not going to get the syntax on equations correct 100% of the time, but I will do my best.</p><p>The <em>w</em> refers to the factor weight and is the slope of our line. <em>b </em>is known in ML as the bias. It is the intersection of the line and the y-axis when x is equal to zero. Here is a look at our data and our line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eXj4E2whqjHiRLFWpYJg8Q.png" /></figure><p>Now the question begins, how do we know the line (the prediction) is the best prediction? First, we must understand loss. Loss is the difference between the predicted value and the actual value. A very common way to define loss is to square the difference between the actual value and the predicted value.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/166/1*xkUb8D2eHm2KhggA-spOJg.png" /></figure><p>In order to get the average loss across the data set, we sum all of the losses calculated with each data point and prediction and divide it by the number of data points to get the mean. This is known as Mean Squared Errors (MSE) and is one type of loss function used to get the best fit. To get the best fit, we try and get the smallest MSE.</p><h3>Reducing Loss</h3><p>So now we know what loss is, the difference between the predicted value and the actual value, and that we need to minimize that loss in order to get the best prediction. But how do we do that? What we do is computing the gradient. The derivative of the loss function tells us how our loss is changing. We first compute the loss, then take a step in a certain direction and see if that new loss is less than our previous one. The steps are governed by the learning rate. The learning rate tells you how big or small of steps you need to take in the direction towards the minimum (known as the negative gradient). Going back to our squared loss function, we can see that it is a parabola. We use this iterative approach (compute the loss, take a step, compute the loss) to find the minimum for this parabola.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/590/1*Q3q5J_zgLB6qxEMioEE6rA.png" /></figure><p>The learning rate is how big of a step we take in the direction of the negative gradient. If we take a large step, we wind up on the other side of the parabola. If we take a small step, we might have to take many steps to actually find the minimum. Now, this is a simple example. The number of computations to find the minimum of a parabola, regardless of where one starts and how big or small the learning rate is, is fairly small, especially when using a computer. However, in a much more complex algorithm, where one may have a billion (literally) data points, the starting point is pretty important. So we don’t have to make calculations on the entire data set, we use a “mini-batch gradient descent” where a small amount of the data is used to determine the gradient and the steps. In the housing example, there is also only one dependent variable. The gradient is simply the derivative of one dependent variable, ie the change in housing prices with respect to the change in square feet of a house. However, if you were to add another variable to try and make a better prediction about housing prices, you would begin to calculate the regression using partial derivatives based on the weights of each variable.</p><h3>Next Post</h3><p>On my next post, I’ll step into the next two sections about TensorFlow and Generalizations. TensorFlow is the software library for which this course is based on. Generalization is a topic about overfitting your model to the data.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f5c11d1cddde" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Crashing the Crash Course: A look at Google’s Machine Learning Course - Part I]]></title>
            <link>https://medium.com/@spraguesy/crashing-the-crash-course-a-look-at-googles-machine-learning-course-part-i-abd3e8c533f0?source=rss-71235cf30811------2</link>
            <guid isPermaLink="false">https://medium.com/p/abd3e8c533f0</guid>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Andrew Sprague, CFA®]]></dc:creator>
            <pubDate>Thu, 13 Sep 2018 04:02:40 GMT</pubDate>
            <atom:updated>2018-09-13T04:07:28.031Z</atom:updated>
            <content:encoded><![CDATA[<p>In this series, I will work my way through the <a href="https://developers.google.com/machine-learning/crash-course/">Google Machine Learning (ML) crash course</a>. Here are a few good articles to give those who are not familiar with ML a taste of what it is. <a href="https://www.forbes.com/sites/forbestechcouncil/2017/08/07/what-is-machine-learning/#54b8204c79a7">I</a>, <a href="https://medium.com/iotforall/the-difference-between-artificial-intelligence-machine-learning-and-deep-learning-3aa67bff5991">II</a>, <a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12">III</a>. If you don’t want to read those articles here is my short summary. Machine Learning creates prediction models by using data instead of code. An algorithm is fed data in order to “learn” how an outcome is created instead of being explicitly told how to get there by code. The course is divided into three groups: ML concepts, ML, engineering, and ML real-world examples. The bulk of the course is contained within the ML concepts group, and I will try and cover a few sections per part.</p><p>Now, why would we go through this course? There are a few reasons, but this course goes beyond “I want to learn what Machine Learning is”. You will go through this course if you want to learn the concepts behind making an ML model and how to use Google’s <a href="https://www.tensorflow.org/">TensorFlow</a> API to feed data to a prediction model. The course is designed to teach you the fundamentals of ML and artificial intelligence.</p><p>There are a few prerequisites for this course, and they may seem a little daunting. You should have a solid foundation of intro-level algebra. This includes concepts such as variable, coefficients, functions, linear equations, and matrix multiplication. Statistical measures will also show up, so your means and medians and standard deviation. Calculus will show up (think derivatives), but the material is optional. You will also need to know the basics of python (defining and calling functions, loops, etc.) and pandas (a library for manipulating data sets). I am going to skip the pre-work, but you may want to work through it if you feel shaky on the prerequisites.</p><h3>Section I: Introduction to ML</h3><p>The shortest section. You get an introduction by <a href="https://en.wikipedia.org/wiki/Peter_Norvig">Peter Norvig</a> who talks through the benefits of ML from a software engineers perspective. Benefits of the tool include reducing time programming, customizing and scaling products, and solving “unprogrammable” tasks. These three reasons are the “practical” reasons why one should master ML. He goes on to say there is a philosophical reason: turning software engineers into natural scientists vs. mathematical scientists. Software engineers understand how to tackle problems in a logical and mathematical way. However, ML changes the perspective to solve problems through observations in an observable world. Take the word predictor on your smartphone’s keyboard. You could write thousands of lines of code to take into account the words that have been typed. Or you could feed examples into a model and have the problem learned from the data.</p><h3>Section II: Framing</h3><p>So now that we have a low-level understanding of ML and the skills it takes to start down the path of ML let’s jump right in.</p><p>There are a few different ways that ML is accomplished, but this course looks at supervised machine learning. See the third article from above for other types. Supervised machine learning takes inputs to produce predictions on new data. To train a model, you feed it data. The outcome that we are trying to predict is the label, also known as the dependent variable. The input variables that describe the data are known as features, these are the independent variables. These models stem from simple, with one, two or a handful of variables, to ultra-complex that contain millions of variables. One particular piece of data is known as an example, and that can be labeled (has x and y) or unlabelled (has x but y is unknown).</p><p>The last piece of framing is the model itself. How do the independent variables relate to the dependent variable? Let’s use housing prices. The selling price of a house is the dependent variable. We use that price and along with a handful of other variables to get that price. The train the model by using known x’s and y’s so that we can infer the y’ from x variables that we have. In the case of housing price, our x variables might be square feet of the house, square feet of the lot, year built, total bedrooms, total bathrooms, and proximity to urban business centers. We use those factors about a house to try and estimate the value of the house.</p><p>There are two different of types of models we may try and predict. A regression model and a classification model. A regression model predicts continuous values. The previously discussed housing model is an example of this type. Classification models predict discrete values. An example of this is a spam filter. An e-mail is labeled as either spam or not spam, and features such as sender, content, and time of day help to predict if the e-mail is or not.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*yjVIv6g8-pHK-G3io83u6Q.gif" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=abd3e8c533f0" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>