<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nehal Udyavar on Medium]]></title>
        <description><![CDATA[Stories by Nehal Udyavar on Medium]]></description>
        <link>https://medium.com/@nehaludyavar?source=rss-b392b4018948------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*vS60WyWrV_VM3zevRknO6g.jpeg</url>
            <title>Stories by Nehal Udyavar on Medium</title>
            <link>https://medium.com/@nehaludyavar?source=rss-b392b4018948------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 22:00:12 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nehaludyavar/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[A Beginner’s Guide to Neural Networks: Part Two]]></title>
            <link>https://medium.com/data-science/a-beginners-guide-to-neural-networks-part-two-bd503514c71a?source=rss-b392b4018948------2</link>
            <guid isPermaLink="false">https://medium.com/p/bd503514c71a</guid>
            <category><![CDATA[computer-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Nehal Udyavar]]></dc:creator>
            <pubDate>Mon, 27 Mar 2017 19:37:53 GMT</pubDate>
            <atom:updated>2017-04-10T23:52:43.228Z</atom:updated>
            <content:encoded><![CDATA[<h4>Bias, activation functions, hidden layers, and building a more advanced feed-forward neural network architecture.</h4><p>In <a href="https://medium.com/luminous-ai/a-beginners-guide-to-neural-networks-b6be0d442fa4#.24wy4o50n">Part One,</a> I explained how a perceptron takes in inputs, applies a linear combination, and if that linear combination is greater or lesser than some threshold value, produces an ouput of 1 or 0 respectively. In the language of mathematics, it looks like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/356/1*7uqFwiPG5_fvnW8tjaYCjA.gif" /></figure><p>In the formula above, the Greek letter Sigma ∑ is used to represent <strong>summation</strong>, and the subscript <em>i </em>is used to iterate over input (<em>x)</em> and weight (<em>w) </em>pairings.</p><p>To make things a little simpler for <em>training</em> later, let’s make a small readjustment to the above formula. Let’s move the <em>threshold </em>to the other side of the <a href="https://www.mathsisfun.com/algebra/inequality.html">inequality</a>, and replace it with what’s known as the neuron’s <em>bias. </em>Now we can rewrite the equation as:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/338/1*C7AkWHjMWgNan5WQ3IJKDw.gif" /></figure><p>Effectively, <em>bias = — threshold. </em>You can think of <em>bias </em>as how easy it is to get the neuron to output a 1 — with a really big bias, it’s very easy for the neuron to output a 1, but if the bias is very negative, then it’s difficult.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/684/1*7c-ZVZ7h-AwjapAhQgeKdQ.png" /><figcaption><a href="http://smbc-comics.com/index.php?id=4105">Source: SMBC Comics</a></figcaption></figure><p>There’s a couple of reasons to make this small change in the definition (there will be a few more changes later on as well; it’s good to see the intuition behind making these changes). Like I mentioned in Part One, we randomly assign numbers for weights (we don’t know the right ones beforehand), and as the neural network <em>trains</em>, it makes incremental changes to those weights to produce more accurate outputs. Similarly, we don’t know the right <em>threshold</em>, and like the weights the network will need to make changes to the threshold to produce more accurate outputs. Now with <em>bias</em>, we only need to make changes to the left side of the equation, while the right side can remain constant at zero, and soon, you’ll see why this is useful.</p><p>A function that transforms the values or states the conditions for the decision of the output neuron is known as an <strong>activation function</strong>. The math formula above is just one of several activation functions (and the simplest) used in deep learning, and it is called the <strong>Heaviside step function.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*rdSZijQqNoiZPImMI4ENeg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/176/1*LWWzDCgUONGAKko67oMzcw.gif" /><figcaption>General formula (<em>h </em>being the condition) and graph of the Heaviside step function. The solid circle is the y-value you take (1.0 for x=0 in this case) , not the hollow circle. The line the hollow and solid circles are on is the ‘step’. (Source: Udacity)</figcaption></figure><p>Other activation functions include the <em>sigmoid</em>, <em>tanh</em>, and <em>softmax</em> functions, and they each have their own purpose. (In this post, I’ll just explain the sigmoid function, but I’ll get into the others whenever they pop up).</p><h4>Sigmoid Function</h4><p>Even when dealing with absolutes (1s and 0s; yeses and nos), it’s beneficial to have the output give an intermediary value. It’s a bit like answering “maybe” when asked a yes-or-no question you have no idea about, instead of guessing. This is essentially the advantage of the sigmoid function over the Heaviside.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*UzqjTkSHgkrU55P71NKNMA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/204/1*7fiuQEWfNRUCDQBAUZz6Rw.gif" /><figcaption>The graph and formula of the sigmoid function. Source: Udacity</figcaption></figure><p>Like the Heaviside step function, the sigmoid has it’s values between 0 and 1. But this time, there’s no <em>step</em>; it’s been smoothed out to create a <em>continuous </em>line. Thus, the output can be considered as a <em>probability</em> of being a success (a 1), or a yes. In everyday terms, an output of 0.5 means the network has no idea whether it’s a yes or no, while an output of 0.8 means the network is ‘pretty sure’ it’s a yes.</p><p>Having this feature is particularly important for the network’s learning capabilities, which we’ll see in a future post. For now, just think of it as being easier to <em>teach</em> a network to move incrementally towards the right answer, rather than just jump directly from a 0 to a 1 (or vice-versa).</p><h4>Hidden Layers</h4><p>So far, we have explored the architecture of the perceptron, the simplest neural network model, and seen two activation functions: the Heaviside step function and the sigmoid function. (To learn about how perceptrons can be used to compute logic functions such as AND, OR, and NAND, check out <a href="http://neuralnetworksanddeeplearning.com/chap1.html">Chapter 1 in Michael Nielsen’s online book</a>).</p><p>Now let’s make our network a little more complex. Here, and in all neural network diagrams, the layer on the far left is the input layer (i.e. the data you feed in), and the layer on the far right is the output layer (the network’s prediction/answer). Any number of layers in between these two are known as <strong>hidden layers</strong>. The more the number of layers, the more <em>nuanced</em> the decision-making can get.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*hGT-tQnvJ76fqzr8sLhaqQ.jpeg" /><figcaption>(Source: <a href="http://cs231n.github.io/neural-networks-1/">Stanford CS231n</a>)</figcaption></figure><p>The networks often go by different names: <em>deep feedforward networks, feedforward neural networks, </em>or <em>multi-layer perceptrons (MLP). </em>(To make things less confusing, I’ll stick with feedforward neural network). They’re called <em>feedforward </em>networks because the information flows in one general (forward) direction, where mathematical functions are applied at each stage. In fact, they’re called ‘networks’ <em>because </em>of this chain of functions (the output of the first layer function is the input to the second layer and that output is the input to the third layer and so on…). The length of that chain gives the <em>depth</em> of the model, and this is actually where the term ‘<em>deep’ </em>in deep learning comes from!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/600/1*MgJbWBmn9kB0pKi_f5kX4Q.jpeg" /><figcaption>Not quite the same ‘hidden’. (Source: Bill Watterson)</figcaption></figure><p>Adding hidden layers can allow the neural network to make more complex decisions, but more on that, and how neural networks learn through a process known as <em>backpropagation, </em>in Part Three (coming soon)!</p><h4>Resources</h4><ol><li><a href="http://neuralnetworksanddeeplearning.com/chap1.html"><em>Using neural nets to recognize handwritten digits</em></a><em> </em>by Michael Nielsen.</li><li><a href="http://cs231n.github.io/neural-networks-1/"><em>Neural Networks Part One: Setting Up the Architecture</em>,</a> Stanford CS231n.</li><li><a href="http://www.deeplearningbook.org/"><em>The Deep Learning Book</em></a> by Ian Goodfellow, Yoshua Bengio and Aaron Courville.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bd503514c71a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/a-beginners-guide-to-neural-networks-part-two-bd503514c71a">A Beginner’s Guide to Neural Networks: Part Two</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Beginner’s Guide to Neural Networks: Part One]]></title>
            <link>https://medium.com/data-science/a-beginners-guide-to-neural-networks-b6be0d442fa4?source=rss-b392b4018948------2</link>
            <guid isPermaLink="false">https://medium.com/p/b6be0d442fa4</guid>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Nehal Udyavar]]></dc:creator>
            <pubDate>Wed, 22 Mar 2017 20:44:17 GMT</pubDate>
            <atom:updated>2017-04-10T23:52:37.435Z</atom:updated>
            <content:encoded><![CDATA[<h4>The motivation behind neural networks, and the architecture behind the most basic one: the perceptron.</h4><p>Humans are incredible pattern-recognition machines. Our brains process ‘inputs’ from the world, categorize them (that’s a spider; that’s ice-cream), and then generate an ‘output’ (run away from the spider; taste the ice-cream). And we do this automatically and quickly, with little or no effort. It’s the very same system that <em>senses </em>that someone is angry at us, or involuntarily reads the stop sign as we speed past it. Psychologists call this mode of thinking ‘System 1’ (coined by Keith Stanovich and Richard West), and it includes the innate skills — like perception and fear — that we share with other animals. (There’s also a ‘System 2’, and if you’d like to know more about this, check out <a href="https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555"><em>Thinking, Fast and Slow </em>by Daniel Kahneman</a>).</p><p>So what does this have to do with neural networks? I’ll get there in a second.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/312/1*QwYAzEP8uBRF4Zk6Q4uPbg.png" /></figure><p>You effortlessly recognized the digits above, right? You just <em>knew</em> that the first digit is 5; you didn’t really have to think about it. Surely your brain didn’t go, “Ah, that looks like two orthogonal lines connected to a rotated, baseless semi-circle, so that’s a 5.” Designing rules to recognize a handwritten digit is unnecessarily complicated, which is why, historically, it’s been quite difficult for computer programs to recognize them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/900/1*DFPeFenZ2ZDzJpB9dsMYhw.gif" /><figcaption>Oh, Apple Newton. (Source: Gary Trudeau for Doonesbury)</figcaption></figure><p>Neural networks loosely mimic the way our brains solve the problem: by taking in inputs, processing them and generating an output. Like us, they <em>learn </em>to recognize patterns, but they do this by <em>training</em> on labelled datasets. Before we get to the learning part, let’s take a look at the most basic of artificial neurons: the <strong>perceptron</strong>, and how it processes inputs and produces an output.</p><h3>The Perceptron</h3><p>Perceptrons were <a href="https://books.google.ca/books/about/Principles_of_neurodynamics.html?id=7FhRAAAAMAAJ">developed</a> way back in the 1950s-60s by the scientist Frank Rosenblatt, inspired by earlier work from Warren McCulloch and Walter Pitts. While today we use other models of artificial neurons, they follow the general principles set by the perceptron.</p><p>So what are they, anyways? A perceptron takes several binary inputs: <em>x1, x2, …, </em>and produces a single binary output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/280/1*QExGFzBqS3u_rO12AIFH_g.png" /><figcaption>Perceptron with 3 inputs. (Source: <a href="http://neuralnetworksanddeeplearning.com/chap1.html">Michael Nielsen</a>)</figcaption></figure><p>Let’s understand this better with an example. Say you bike to work. You have two factors to make your decision to go to work: the weather must not be bad, and it must be a weekday. The weather’s not that big a deal, but working on weekends is a big no-no. The inputs have to be binary, so let’s propose the conditions as yes or no questions. Weather is fine? 1 for yes, 0 for no. Is it a weekday? 1 yes, 0 no.</p><p>Remember, I cannot tell the neural network these conditions; it has to learn them for itself. How will it know which information will be most important in making its decision? It does with something called <strong>weights</strong>. Remember when I said that weather’s not a big deal, but the weekend is? Weights are just a numerical representation of these preferences. A higher weight means the neural network considers that input more important compared to other inputs.</p><p>For our example, let’s purposely set suitable weights of 2 for weather and 6 for weekday. Now how do we calculate the output? We simply multiply the input with its respective weight, and sum up all the values we get for all the inputs. For example, if it’s a nice, sunny (1) weekday (1), we would do the following calculation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/566/1*78nQnb6zWC5UelDmL1Rugw.gif" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/268/1*aq8TnLcfwMes-e1cqwX2Rw.gif" /></figure><p>This calculation is known as a <strong>linear combination</strong>. Now what does an 8 mean? We first need to define the <strong>threshold value. </strong>The neural network’s output, 0 or 1 (stay home or go to work), is determined if the value of the linear combination is greater than the threshold value. Let’s say the threshold value is 5, which means that if the calculation gives you a number less than 5, you can stay at home, but if it’s equal to or more than 5, then you gotta go to work.</p><p>You have just seen how weights are influential in determining the output. In this example, I set the weights to particular numbers that make the example work, but in reality, we set the weights to random values, and then the network adjusts those weights based on the output errors it made using the previous weights. This is called <strong>training </strong>the neural network.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/413/1*LqQEd7iO2ECwEzW2vG0OBw.png" /><figcaption>(Source: Panagiotis Peikidis. Based on <a href="https://xkcd.com/303/">XKCD comic ‘Compiling’</a>)</figcaption></figure><p>Going back to the handwritten digit recognition problem, a simple one-node network like the one above will not be capable of making such complex decisions. To enable that, we need more complicated networks, with more nodes and <em>hidden layers</em>, using techniques such as a <em>sigmoid activation function </em>to make decisions, and <em>backpropagation </em>to learn. All that in <a href="https://medium.com/luminous-ai/a-beginners-guide-to-neural-networks-part-two-bd503514c71a">Part Two</a>!</p><h4>Next:</h4><p><a href="https://medium.com/@nehaludyavar/a-beginners-guide-to-neural-networks-part-two-bd503514c71a">A Beginner’s Guide to Neural Networks: Part Two</a></p><h4>Resources</h4><ol><li><a href="http://neuralnetworksanddeeplearning.com/chap1.html"><em>Using neural nets to recognize handwritten digits</em></a><em> </em>by Michael Nielsen (more detailed explanations at the same level).</li><li><a href="http://www.deeplearningbook.org/"><em>The Deep Learning Book</em></a> by Ian Goodfellow, Yoshua Bengio and Aaron Courville (more advanced &amp; technical, assumes undergrad level math knowledge).</li><li><a href="https://www.udacity.com/course/deep-learning-nanodegree-foundation--nd101"><em>Udacity Deep Learning Nanodegree Foundation</em></a> (17-week project-based course; difficulty is somewhere in the middle of 1. and 2.)</li><li><a href="http://playground.tensorflow.org/"><em>TensorFlow Neural Network Playground</em></a><em> </em>(fun, interactive visualization of neural networks in action)</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b6be0d442fa4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/a-beginners-guide-to-neural-networks-b6be0d442fa4">A Beginner’s Guide to Neural Networks: Part One</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction]]></title>
            <link>https://medium.com/an-fpl-exploration/introduction-674260327e8b?source=rss-b392b4018948------2</link>
            <guid isPermaLink="false">https://medium.com/p/674260327e8b</guid>
            <category><![CDATA[soccer]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[fantasy-sports]]></category>
            <category><![CDATA[football]]></category>
            <category><![CDATA[premier-league]]></category>
            <dc:creator><![CDATA[Nehal Udyavar]]></dc:creator>
            <pubDate>Fri, 19 Aug 2016 21:35:51 GMT</pubDate>
            <atom:updated>2016-08-19T21:35:51.677Z</atom:updated>
            <content:encoded><![CDATA[<p>At the beginning of every Premier League Season, I get engrossed in the creation of my <a href="https://fantasy.premierleague.com/">Fantasy Premier League (FPL)</a> team. I do a bit of research, check a few stats, make some biased and statistically inconclusive predictions and form my team, incredibly confident that this is the team that will bring me success. I then devoutly check my results over the first few weeks of the football season, only to forget about it altogether and log back in one year later, at the start of next season. This cycle has been going on for the last 3 years, and surely if I follow the same routine this time around, I’ll end up with the same results.</p><p>So this time, I have a new idea — one that will take at least a few years to accomplish, but one that I’m very interested in. This time, I’m playing not just to contest friends in mini-leagues or follow my national ranking, but also to explore techniques and strategies that deliver consistent results, with the aim of building algorithms that will process the data for me. I believe the best results come the combination of proper statistical analysis and sound tactical knowledge, both of which I’m aiming to computerize. My grand goal is to predict (or provide probability distributions for) football matches in a pretty detailed manner, but the complicated nature of such a goal (and my currently limited ability) means that is likely very, very far away.</p><p>Nonetheless, I have to start somewhere, and FPL provides a splendid (and fun) framework to do so. For the 2016/17 Premier League season, I will likely do most of the data analysis manually, and I’ll discuss the techniques and strategies I’m employing, the parameters I’m taking into account and the ones I’m not or can’t, and the inherent biases of some decisions I make. By the end of it, I will have documented what’s working and what’s not, which areas I struggled to compute manually but would be effortless for a computer, and I’ll identify dozens of aspects that I’m not aware of right now.</p><p>Sports, especially football, are not easy to predict. There are thousands of parameters, some of which are difficult to define with numbers, and complex tactics with innumerable nuances that are tedious, at best, to interpret mathematically. It’s a game of probabilities — there is no way to achieve perfection, but surely there are ways to make it more and more accurate. This is what I wish to explore over the coming years, beginning with this year’s Fantasy Premier League.</p><p>If you have any suggestions/tips/articles/any form of guidance, or would like to get in touch, feel free to e-mail or tweet me (see About page).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=674260327e8b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/an-fpl-exploration/introduction-674260327e8b">Introduction</a> was originally published in <a href="https://medium.com/an-fpl-exploration">An FPL Exploration</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>