<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by chris stevens on Medium]]></title>
        <description><![CDATA[Stories by chris stevens on Medium]]></description>
        <link>https://medium.com/@chrisstevens1?source=rss-64b8e1fc9a07------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*wSWSK5k31LceIVJA.</url>
            <title>Stories by chris stevens on Medium</title>
            <link>https://medium.com/@chrisstevens1?source=rss-64b8e1fc9a07------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 17:23:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@chrisstevens1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Approaching a competition on Kaggle: Avito Demand Prediction Challenge (Part 3 — linear regression)]]></title>
            <link>https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-3-linear-regression-115312500ff4?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/115312500ff4</guid>
            <category><![CDATA[sklearn]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Fri, 18 May 2018 20:51:16 GMT</pubDate>
            <atom:updated>2018-05-18T20:51:16.363Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zLk3dSJLhVveRJS3Pt9ynA.jpeg" /><figcaption>Infinite Regression — Alan Levine — Flickr</figcaption></figure><p>After spending the past 2 articles (<a href="https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-1-9bc4e6299ba4">here </a>and <a href="https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-2-a1afbe063bb8">here</a>) playing with the data in the Kaggle competition, today I will start to build a basic model. I think I will start with something simple like a linear regression model. Linear regression should be a perfect fit here as we have a number of independent variables all trying to describe our dependent variable deal_probability. What linear regression tries to do is form a linear equation that describes the pattern in our independent variable. See graph below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*QjxunW3kDtR2GhiwM_-owQ.png" /><figcaption>By Oleg Alexandrov (self-made with MATLAB) [Public domain], via Wikimedia Commons</figcaption></figure><p>Take price as the independent variable here, each red dot represents our mapping of price to deal_probability. And the blue line is what the linear regression formula tries to calculate so as to reduce the distances from our points, the green lines. This is often done using the least squares formula. Linear regression always makes a straight line (hence linear) of the form y = mx + c. As such it may not always be the best formula to use if relationships are more complicated than linear. We will give it a go though and see what we get. As always take a look at my <a href="https://github.com/chr15stevens/donorschooseapplicationscreening">Github</a> to follow along with the code, today I will be using file “main_day3.py”.</p><p>The thing about linear regression is in theory it should only work for numerical data. As such I shall purely include this to start with. I am using <a href="http://scikit-learn.org/stable/index.html">Sklearn</a> another fantastic machine library (probably targeted more at beginners) and it’s LinearRegression function. I will exclude all non-numerical data for now leaving just 3 columns ‘price’, ‘item_seq_number’, ‘image_top_1’. Running this linear regression I achieve a root mean squared error score (the scoring metric used in the competition) of ~0.257. Not impressive by any means but also not bad for 5 mins work. This was run locally on my 50000 row subset, so now to produce a kernel on Kaggle and run this algorithm against the entire dataset.</p><p>Running on the full dataset on Kaggle I achieve a score of 0.2631. Putting me near, but not completely at the bottom an achievement I would say. We at least have a working kernel that we can build upon. You can see my Kaggle script in my Github repository, file kaggle_main_day3.py. The script differs slightly as you must output results to a file for Kaggle’s competition scoring to pick up and process.</p><h4>A Per Category Model</h4><p>Now to improve upon this. My first idea which will allow me to keep operating with simple linear regression models is to split the data by category and build a ‘per category’ linear regression. Let’s see how this fairs. Let’s start with parent category as there are only 10 or so of these, this means less models and less intensive on my system.</p><p>Again I am using Pandas groupby function to split my data. Then code is not so different to how it was before. But instead of one model we now have to check the parent_category of each row before we choose which of our 10 models to use. Doing this I get ~0.246 root mean square error. A significant improvement, testing this on Kaggle I get 0.2524 score and I jumped 25 places up. It is clear to me there is still significant improvement to be made here. And I will probably begin by looking at all the different category columns:</p><p>region = 0.256865922681442<br>city = 0.2630428696751874<br>category_name = 0.24182267480235786<br>user_type Category = 0.2555981529232477</p><p>Every category variable apart from city seems to give a better result than the standard linear regression. In theory if I swap out my Kaggle kernel for one that looks at category_name rather than parent_category_name I will achieve a marginally better result. But I think I can do better than this.</p><h4>Ensemble Models</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*3BzbVgkgj5mziyVJNf8f2g.jpeg" /></figure><p>An ensemble model (also blending, also stacking) is essentially combining multiple models to give a superior result to just one. There are a number of ways to do this, in fact you can do it any way you like but obviously you are aiming for good results. Take my current scenario here, I have a collection of linear regression predictions for each test row. Now I can combine these results to maybe make a better model. One simple way I have seen before is to simply take the mean of your predictions. I will try this now. All I have to do is add up each prediction for each row and divide by the number of predictions.</p><p>Doing this I achieved ~0.256 not an improvement but I will run the code on Kaggle just in case. I received a score of 0.2624, a backwards step but at least now we know this will not be a successful avenue to explore.</p><p>Finally one more thing I wanted to try, what if we built a linear regression of our linear regression predictions? Could work. So let’s go ahead and do that. Unfortunately due to my shortsightedness this won’t work as I don’t have target variables on the test set. I feel like there will be some way to linearly combine these predictions to make a better prediction. I will do some research for tomorrow.</p><p>Additionally I aim to include a lot of the data I have been neglecting so far. I haven’t considered the wording or images at all. There is plenty of easy improvements to be made to my model and these will likely be in ensemble form.</p><p>Until then.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=115312500ff4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Approaching a competition on Kaggle: Avito Demand Prediction Challenge (Part 2)]]></title>
            <link>https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-2-a1afbe063bb8?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/a1afbe063bb8</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pandas]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Wed, 16 May 2018 19:31:46 GMT</pubDate>
            <atom:updated>2018-05-18T20:53:18.172Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JAkiWow_6Kjq_TUToXbGQg.jpeg" /></figure><p>So <a href="https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-1-9bc4e6299ba4">yesterday</a> I began to make some strides in understanding and visualising my data. Today I will continue down this road. To split up my code from each day I will make a new file main_day2.py, which you can as usual view on my <a href="https://github.com/chr15stevens/avito-demand-prediction-analysis/tree/master">github</a>.</p><p>One thing I forgot to mention yesterday is that I am not doing this analysis with the full dataset. Unfortunately the only hardware I have to hand is not able to handle this much data in a timely manner. I would spend 10mins+ producing a single graph. As such I split the data and am working with 50,000 rows for now. You may ask doesn’t this give you an incomplete picture of the data and you would be right. However as we are just gathering ideas and intuition for now, this is ok. When i progress to designing models I will begin to use Kaggle’s servers more and more, this is what they are there for. I can’t overstate the usefulness this brings to the community. If you wish to split your data you can use split_train.py in the git repo.</p><p>I set myself 2 preliminary goals yesterday:</p><ul><li>Understand the categorisation of data within our ‘parent_category_name’, ‘category_name’, ‘user_type’, ‘image_top_1’, ‘region’, ‘city’ columns.</li><li>Understand what ‘param_1’, ‘param_2’, ‘param_3’ contain.</li></ul><p>Some deeper intuition may not be straight forward as most of the writing is in Russian but I will give it my best.</p><h4>Understanding Data Categorisation Columns</h4><p>Running a simple <a href="http://pandas.pydata.org/pandas-docs/stable/groupby.html">dataframe.groupby</a> of the famous pandas library I can see how many different categories each column contains. I get the following results:</p><ul><li>region = 28</li><li>city = 1022</li><li>parent_category_name = 9</li><li>category_name = 47</li><li>user_type = 3</li><li>image_top_1 = 2774</li></ul><p>Some of these things are obvious and expected, region and city for example. Categories I would have expected to be a lot greater than 9 and 47. This is good for us, it means that our data is succinctly labelled which should mean we have a good amount of data for each category. Then, in theory, we could make some sort of individual ML models for each category in the hopes to improve accuracy. My intuition is telling me, as in all markets, market forces can not only be described at the macro level but we must also look at the micro (as micro as possible)to fully understand/describe our data.</p><p>Traditionally it would be easy to interpret the data in these categories but our data here is in Russian. So to continue to build understanding and intuition of Avito I will plug these into google translate:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IfLi1XgB_8hKU_ptHKvlng.png" /><figcaption>The parent categories we are looking at.</figcaption></figure><p>It can never hurt to understand your data better even if the models you are using are language agnostic. Everything here looks pretty standard as compared to an English equivalent site.</p><p>The last 2 categorical columns are user_type and image_top_1. I will need to go and do some research on these. To Kaggle and their great community, hopefully I’m not let down, I found a post discussing image_top_1 <a href="https://www.kaggle.com/c/avito-demand-prediction/discussion/56079">here</a>. Unfortunately it seems nobody knows for sure but they are guessing it is either a quality assessment score or some sort of category classifier. I may do some of my own analysis later. As for user_type there seems to be no discussion on this, it is likely as there are only 3 that it is something like personal, company, internal. I will look at this in greater detail later.</p><p>Now to look at our categories in greater detail, I will look at the spread of data across each category. I made this nice graph:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mXNuo2cEhasnAQMY6zO1tw.png" /><figcaption>The spread of data in each categorical column.</figcaption></figure><p>One thing this immediately taught me is that the user_type is actually labelled and was not 0, 1 or 2. It’s company, private and shop. It’s always a bonus to make a certainty out of one of your assumptions. While we can see activity is fairly spread across regions, it is much less so for parent categories and categories. Additionally a large percentage of user_type is private.</p><p>Now I will add the mean of the deal_probability to each graph to see how this fluctuates across each category.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EH9t9yEJYF29Palki49Snw.png" /><figcaption>Left y-axis is count in category and right y-axis is mean deal_probability.</figcaption></figure><p>Not sure what happened with my labels here but I suppose they were all in Russian anyway and the graph is still entirely readable. What this shows is that there isn’t a huge correlation between a category and the likelihood of a deal being made. This continues to lead me on the thought path of a per category model or possibly some sort of neural net to manage these complex relationships.</p><p>This is all I have time for today, come back tomorrow and hopefully I will begin to try some basic models. Note that I still need to consider ‘param_1’, ‘param_2’, ‘param_3’ and images and the other datasets included with the problem.</p><p><a href="https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-3-linear-regression-115312500ff4">(part 3)</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a1afbe063bb8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Approaching a competition on Kaggle: Avito Demand Prediction Challenge (Part 1)]]></title>
            <link>https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-1-9bc4e6299ba4?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/9bc4e6299ba4</guid>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[pandas]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Tue, 15 May 2018 21:02:41 GMT</pubDate>
            <atom:updated>2018-05-16T19:45:58.859Z</atom:updated>
            <content:encoded><![CDATA[<p>I have wrote a little about <a href="https://www.kaggle.com/">Kaggle</a> before and I consider it to be one of the greatest resources for someone trying to get into machine learning. Not only can you find real ML problems there is also a great open community who love to talk about what they are doing and the ideas they are trying to incorporate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JNVulBIT2UxBI1gTuvYbCw.png" /></figure><p>Today, as an example, I will focus on the <a href="https://www.kaggle.com/c/avito-demand-prediction">Avito Demand Prediction Challenge</a>. This is a problem posed by a Russian classified ads site that wants to provide their users with better data on how well their ads will fair when using the site. Avoiding frustration for users who can’t sell their items is stated as a key goal.</p><p>As this competition has been running for a short while at time of writing there has already been ~1000 entries into the competition with the top result achieving a mean squared error of 0.2180. It is likely if you are new to ML as I am that you will not be able to touch this result but don’t worry we all have to start somewhere. Lets look at the dataset first.</p><h4>The Data</h4><p>What makes this an interesting challenge is the range of data you get to work with. Basic ad data is made up of a region, city, parent category, category, title, description, price, activation date, user type, an optional image, an image classification code and (our target) deal probability.</p><p>The most important thing when beginning a new ML problem is to get a handle on the data. Understand what you have and what you need to get to. This instantly gives you extremely targeted information on what sort of approach can work. Additionally you should start to form a picture of which datapoints will be key, the ones that will have the greatest impact on your predictions. In addition to looking at the raw datapoints have a look at how your data appears naturally to users:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/736/1*wI_qZHm88Fg31N5HXy6JWQ.png" /><figcaption>Screenshot of an Avito listing.</figcaption></figure><p>This may give further insights into how the users may use this site which can help to seed your ideas on biggest impacts to deal probability (our target). My takeaways from this are the image, title and price all have pride of place on the site. Therefore it is likely these are larger influencers than the description. This could turn out to be wrong after some deeper analytics. We also need to consider listing pages that resemble these:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*AWADPS-DmryJeFA-TtGgWQ.png" /></figure><p>Having a good understanding of how users are creating the data you will be using is invaluable to your algorithm design, even if you can’t see it at first.</p><p>Download a copy of the <a href="https://www.kaggle.com/c/avito-demand-prediction/data">data</a> even if you plan to use Kaggle’s cloud resources as there is no better interactivity than having it on your own PC.</p><p>Now we can play with our data to see if we can get some further intuitions on what sort of model might be best or even just to understand our data better. As usual I will be using Pandas as there is no better tool for quickly processing and working with your data. The first thing I did was to look at our numerical columns: price, deal_probability, item_seq_number. I will use Pandas built in visualisation tools to get a high level look. I first used <a href="https://pandas.pydata.org/pandas-docs/stable/visualization.html#scatter-matrix-plot">pandas.plotting.scatter_matrix</a> to make:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GiTYnvzNYFr8Qo0iKwZZ5Q.png" /></figure><p>This plots each variable against each other variable and outputs a nice graph matrix type thing. I will remove the outliers and re-plot to give more shape to our data:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E1k0a3n9a2qWiQs9HRFgiA.png" /></figure><p>Some things that I notice straight away are:</p><ul><li>There are lots of cheap items.</li><li>Deal probability seems to reduce as users post more items (item_sequence_number). This could be due to poor quality of items or posts or something else. Something to explore later possibly.</li><li>Low price items have a higher deal_probability.</li><li>There is a slight correlation between a lower price and the number of items a user posts.</li></ul><p>While not immediately solving our problem I feel I have gained some further intuition into my data and thus users behaviour which is invaluable.</p><p>I will continue down this road tomorrow. Primarily I want to look at the categorised data how many and of what types of: ‘parent_category_name’, ‘category_name’, ‘user_type’, ‘image_top_1’ exist and what they might mean in the wider context of our data. Additionally I need to understand what ‘param_1’, ‘param_2’, ‘param_3’ contain.</p><p>As always you can find my current code on my <a href="https://github.com/chr15stevens/avito-demand-prediction-analysis">Github</a>. This will show you how to make scatterplots and how I am starting to explore my data.</p><h4>The Forums</h4><p>The other immeasurably valuable Kaggle resource is the forums or Discussion connected to each competition. Often in these users will post benchmark solutions or kernels that you can just fork and work with as a base point. There is all sorts of discussion around the competition for ideas also, it really is a great collaborative community despite being framed as a competition. I would highly recommend reading through the forums before beginning any challenge.</p><p>Thanks for reading. Tomorrow I will continue to explore the base data and start to think about the different algorithms I can implement.</p><p><a href="https://medium.com/@chrisstevens1/approaching-a-competition-on-kaggle-avito-demand-prediction-challenge-part-2-a1afbe063bb8">Part 2</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9bc4e6299ba4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Donors Choose Application Screening — TFIDF logistic regression for beginners day 2]]></title>
            <link>https://medium.com/@chrisstevens1/donors-choose-application-screening-tfidf-logistic-regression-for-beginners-day-2-ef560a7339f5?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef560a7339f5</guid>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[sklearn]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Mon, 14 May 2018 19:27:11 GMT</pubDate>
            <atom:updated>2018-05-14T19:27:54.173Z</atom:updated>
            <content:encoded><![CDATA[<p>So <a href="https://medium.com/@chrisstevens1/kaggle-competition-donors-choose-application-screening-ml-from-scratch-tfidf-logistic-59eefcdd0ef0">previously</a> I set up a basic logistic regression function to predict whether a project would be greenlit for funding. I achieved an error rate of 12%. Not bad, i will aim to improve this today. Firstly I will go through the algorithm in further detail to build some understanding.</p><p>Logistic regression is usually used to build a discrete classifier because its output is constrained between 0 and 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*UaglN8b18ILHTcZVo2pHpA.png" /><figcaption>The logistic function.</figcaption></figure><p>In our case the X that we are passing are our TFIDF matrix values. And we are asking Scikitlearn to build an equation of this form that maximises the accuracy of the training set. If it is successful we should get an equation that outputs close to a 1 when the project is greenlit and close to a 0 when it is not.</p><p>The form this equation takes is X = b₀x₀ + b₁x₁ + b₂x₂ + … . Where x₀ is our first piece of data (the first TFIDF column) and b₀ is a calculated coefficient that aims to maximise the results from the training set examples. Essentially our LogisticRegression algorithm aims to say which words seem to have the most impact on whether a project is chosen. Relatively simple but can be hard to understand without good examples.</p><h3>How to improve upon this?</h3><p>One thing that I am aware of is that I am using the TFIDF of every word that appears in every project_description. I feel like taking a subset of the words to only include the most popular may make an improvement. I will begin with this. Sklearn has this feature built right into their <a href="http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html">TfidfVectorizer</a> so I can simply set max_features=2000. There is no particular reason to choose 2000 I will take it as a first benchmark and work from there. This gave me a 0.125166828808 error or 12.5%, so this actually proved to be worse. I will play around with the number 2000 a bit see if there is any improvement to be found here. 1000 gives 12.4%, as does 500. Seems that it is best to leave it considering all words.</p><p>Other things I can do are: use word vector embeddings, review the additional data and see if there are more columns to use, lemmatize my data and use another ml algorithm. I may explore these over the next few days.</p><p>Until tomorrow!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef560a7339f5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Kaggle Competition: Donors Choose Application Screening — TFIDF logistic regression for beginners]]></title>
            <link>https://medium.com/@chrisstevens1/kaggle-competition-donors-choose-application-screening-ml-from-scratch-tfidf-logistic-59eefcdd0ef0?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/59eefcdd0ef0</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[sklearn]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Sat, 12 May 2018 20:27:43 GMT</pubDate>
            <atom:updated>2018-05-12T20:43:40.645Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*pVYWMLWtMiMlcPxp2xFs2Q.png" /><figcaption>Donors choose support teachers who need additional funding. They recently had a kaggle competition to help allocations.</figcaption></figure><p>In my past 2 articles I worked on top of tensorflow tutorials, this is great for getting into ML fast but not so great for building deeper intuition. So today I will try to build an ML algorithm from scratch and I have chosen an old Kaggle competition. The aim of this competition is to predict whether a project will be greenlit for funding given a description of the project.</p><p>If you are unfamiliar with Kaggle it is a site for Machine Learning challenges. You are given a set of data and users compete to achieve the greatest accuracy on some given ML task. It is a great site for the just beyond beginner as there are usually great, open communities who are happy to post and discuss their solutions. You can also see how your solution matches up to others. If that wasn’t enough you can also execute your ML algorithms on their servers completely alleviating the need for any expensive home ML setup.</p><p>Find the code on my <a href="https://github.com/chr15stevens/donorschooseapplicationscreening">GitHub</a>. Now one thing about Kaggle is that you don’t have access to all the test data to prevent cheating during competitions. As such for me to proceed with this task I have to split the current training set provided into a training and test set. 80:20 is a commonly used ratio so I will follow this tradition.</p><p>I am using <a href="https://pandas.pydata.org/">Pandas</a> library to load and manage my data it really takes all the heavy lifting out of gathering and pre-processing your data.</p><p>In terms of pre-processing I would like to drop a lot of columns from the data set. I think things like id, teacher_id, teacher_prefix etc won’t carry much weight upon our final result. I will instead concatenate all the wordings columns into a single column which we will then apply our ML algorithm to. The idea is we have a single input of our concatenated columns (project_description) and a single output “project_is_approved”. Additionally I will remove all punctuation. This aims to simplify the inputs so that our algorithm can focus on what is most important — the wording of the proposal.</p><p>I will be making a simple logistic regression model for this first part of my article. This essentially takes in a number of columns of data and tries to say how each one affects the outcome. So in this case it will look at the counts of words appearing in each project_description and try to find some that have an influence on whether a project is approved. I am using another great high level library <a href="http://scikit-learn.org/stable/">scikitlearn</a> which does all the heavy maths for you.</p><p>Finally I use the mean squared error formula to compare the results I get from my predictions to the actual results. And as a final result on a subset of the data, I get 0.123678792151 or about a 12% error.</p><p>Not bad for a first try. I plan to analyse this deeper tomorrow and see if I can make any improvements. I will also try to explain in further detail the algorithm used.</p><p>Thanks for reading, until tomorrow!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=59eefcdd0ef0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are Convolutional Neural Networks (CNNs)? For beginners — Tensorflow MNIST CNN]]></title>
            <link>https://medium.com/@chrisstevens1/what-are-convolutional-neural-networks-cnns-for-beginners-tensorflow-mnist-cnn-739538b9bc96?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/739538b9bc96</guid>
            <category><![CDATA[convolution-neural-net]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mnist]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Fri, 11 May 2018 18:35:34 GMT</pubDate>
            <atom:updated>2018-05-11T18:38:00.256Z</atom:updated>
            <content:encoded><![CDATA[<p>Today I will give a brief overview of Convolutional Neural Networks to hopefully inspire some simple intuition of how they work.</p><p>I will also be implementing and doing a brief explanation of the <a href="https://www.tensorflow.org/versions/r1.1/get_started/mnist/pros">Tensorflow MNIST CNN tutorial</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/231/1*QPXOBD4NluZ3rJmPz0q33Q.jpeg" /></figure><p>What is a CNN? A CNN is a deep learning algorithm that is largely used to classify images. They are inspired by certain insights we’ve gained on how the human eye and brain work together. Like most machine learning algorithms they take training data as input and build a network of weights to classify a test item. This network of weights is built of a number of layers that work to progressively classify higher level features. They start with basic things like lines and curves and as you go deeper in the network move to more complex and useful things such as faces and people.</p><p>Say we wanted to build an algorithm that classifies a person (no small task).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lUdqxBsHQJ8xtjaBXVmCrQ.png" /></figure><p>How could we do this? Well first we have to appreciate the complexity of the human form. While we could potentially build something that recognises the whole human form CNNs work by identifying smaller components first. So rather than a entire person we look at the individual components. We look for 2 arms, 2 legs, a torso and a head.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*y7VgtZ1bikrDQ6ZBNMI57A.png" /></figure><p>But arms are still such high level features. How do we recognise an arm? Well they are made of fingers, a forearm, an elbow etc. We then do the same process again and again until we are at the smallest of curves and lines that they would be unrecognisable on their own. This is in essence how a CNN works.</p><p>Note that the above is a simplification and the image used is not an accurate depiction of a CNN. Stop reading now and have a look at this incredible video of how a real CNN works.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FAgkfIQ4IGaM%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DAgkfIQ4IGaM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FAgkfIQ4IGaM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/389a146798014c5fc55a27c487b8a658/href">https://medium.com/media/389a146798014c5fc55a27c487b8a658/href</a></iframe><p>This should show you the incredible power of the CNN tool.</p><p>One important thing to understand at this point is that you don’t tell a CNN to look for an arm or a leg instead it learns what it thinks are good features to look for based on the training data provided. It may be that it arrives at an armpit being a better way to identify a human than anything we would normally consider.</p><p>Another important concept featured in the video above is pooling. Pooling is used for two reasons, firstly to reduce the size of the data to more efficiently calculate the network but also to reduce overfitting and allow the network to better generalise to new data. Think of it as our “arm recogniser” becoming a “maybe arm recogniser”, the accuracy of it is reduced slightly so that more different arms can be recognised.</p><p>The final step of a traditional CNN is a set of “fully connected” layers. This essentially means a standard neural network is strapped onto the output of our CNN so that recognised features can be converted into predictions. I like to think of it as the CNN part being synonymous with the human eye and the standard NN being the brain that interprets what the eye is seeing and says “hey, yeah! That is the sandwich I lost last week”.</p><p>This concludes part one I hope this helped give you some high level intuition that you can take forward. I will now take a slightly more technical look at the <a href="https://www.tensorflow.org/versions/r1.1/get_started/mnist/pros#build_a_multilayer_convolutional_network">tensorflow MNIST tutorial</a> on CNNs.</p><p>First here is a more technical view of a typical CNN:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3BRLw4lsANPEfGgimG3YVQ.png" /></figure><p>The different feature maps (f.maps) at each stage are similar to our arm, leg etc recognisers, notice every feature layer has multiple filters/recognisers. Subsampling is roughly the same as pooling.</p><p>I will be going through the code at tensorflow\examples\tutorials\mnist\mnist_deep.py. By running this program I achieve an accuracy of 0.9926. Quite the improvement over pure softmax regression featured last article.</p><p>For anybody doing this tutorial and struggling with understanding the code I added some additional comments <a href="https://github.com/chr15stevens/Tensorflow-Deep-MNIST-for-experts-explained">here</a> that might help.</p><p>Until tomorrow for day 3.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=739538b9bc96" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[An Implementation A Day 1: Introduction and Tensorflow MNIST]]></title>
            <link>https://medium.com/@chrisstevens1/an-implementation-a-day-1-introduction-and-tensorflow-mnist-f02cd8e5c41e?source=rss-64b8e1fc9a07------2</link>
            <guid isPermaLink="false">https://medium.com/p/f02cd8e5c41e</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[beginner]]></category>
            <dc:creator><![CDATA[chris stevens]]></dc:creator>
            <pubDate>Thu, 10 May 2018 20:55:49 GMT</pubDate>
            <atom:updated>2018-05-10T20:59:14.912Z</atom:updated>
            <content:encoded><![CDATA[<p>I am (or I was) a software developer who has worked on a range of web and native applications typically using Microsoft technologies. For me the work grew a little stale and repetitive so I am hoping to make a slight career deviation into the much more currently evolving field of data science. I will document my attempts to do this here. Initially I would like to do an implementation (what this exactly means I hope to define as I go) each day for a month.</p><p>Full disclosure, I have been playing with some machine learning algorithms for a few months but would like to have a place to document my work other than the My Documents folder on my PC. However, this is the first time I have played with Tensorflow.</p><p>To the code. For my first (rather unremarkable) trick I will simply implement the MNIST tutorial from tensorflow. This can be found <a href="https://www.tensorflow.org/versions/r1.1/get_started/mnist/beginners">here</a>.</p><p>This project was simple enough to setup. Get the code, run in Python and voila! 30 seconds later you have the ~92% promised accuracy.</p><p>While this was simple, reviewing and understanding what is going on behind the scenes was not so simple. Previous work with sklearn and Keras I found a lot easier to follow mainly due to the fact that everything is not hidden behind Tensors. I understand this is by design.</p><p>Tensors seem to be the main container class used within TF. They remind me a lot of asynchronous programmings promises whereby you set up a whole slew of operations to define a flow of data through an app. The name TensorFlow does not seem so mysterious and clever with this insider knowledge.</p><p>Anyway this concludes my first foray into Medium and Tensorflow. To any unlucky readers, apologies for the deeply unexciting article. One take away I can give is if you are just starting out avoid Tensorflow and look into sklearn or Keras instead. If you insist on going ahead your first priority should be to understand the <a href="https://www.tensorflow.org/api_docs/python/tf/Tensor">Tensor class</a>.</p><p>Until tomorrow!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f02cd8e5c41e" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>