<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Oksana Kurylo on Medium]]></title>
        <description><![CDATA[Stories by Oksana Kurylo on Medium]]></description>
        <link>https://medium.com/@oksanakurylo?source=rss-36d1fc6700e5------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*qJO9uRk2aTQKyvK-U4r0BA.jpeg</url>
            <title>Stories by Oksana Kurylo on Medium</title>
            <link>https://medium.com/@oksanakurylo?source=rss-36d1fc6700e5------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 13:55:01 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@oksanakurylo/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Breaking Down the Complexity of Neural Network Training]]></title>
            <link>https://medium.com/@oksanakurylo/breaking-down-the-complexity-of-neural-network-training-b72f3df75852?source=rss-36d1fc6700e5------2</link>
            <guid isPermaLink="false">https://medium.com/p/b72f3df75852</guid>
            <category><![CDATA[training]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[gradient-descent]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Oksana Kurylo]]></dc:creator>
            <pubDate>Thu, 25 Jul 2024 19:53:19 GMT</pubDate>
            <atom:updated>2024-07-25T19:53:19.933Z</atom:updated>
            <content:encoded><![CDATA[<p>Understanding the training process of a neural network is essential for anyone diving into the world of deep learning. Although training a neural network may initially seem complex, it can be broken down into a series of straightforward steps. In this post, we will:</p><ol><li><strong>Define the Concept of Training:</strong> Explain what training a neural network entails at a high level.</li><li><strong>Explore Linear Regression:</strong> Discuss how linear regression forms the foundation of neural networks.</li><li><strong>Understand Neural Networks:</strong> Explain how neural networks extend the principles of linear regression.</li><li><strong>Introduce Gradient Descent:</strong> Discuss the optimization algorithm used in training, including the role of derivatives and the training loop.</li></ol><p>By the end of this post, you’ll have a clear understanding of how neural network training works, making this seemingly difficult concept much easier to grasp.</p><h3><strong>Defining the Concept of Training</strong></h3><p>The primary goal of training a neural network is to find the optimal set of parameters — known as weights and biases — that allow the network to perform a specific task effectively. This task could be classification, regression, pattern recognition, or any other problem.</p><p>So, what training is first at a high level? Training just means that we’re minimizing a function. In the context of ML, what we’re minimizing is the model’s error, specifically the error on the training data. Let’s say we’re building a model to predict a house’s selling price, and our three inputs are the house’s square footage, the number of bedrooms, and the age of the house. Our dataset might have this information for a few thousand houses, along with their actual selling prices.</p><p>The model’s error is essentially the difference between the model’s prediction for a house’s selling price and the actual selling price in the dataset. The error function can vary depending on the ML model, but ultimately, we’re trying to minimize this error. If the model’s error on the training data is minimized, it suggests the model might generalize well to new data and accurately predict the selling prices of houses it hasn’t seen before.</p><h3>Linear Regression</h3><p>Linear regression is the simplest AI model possible. Let’s use our example of predicting a house’s selling price. The linear regression equation is:</p><figure><img alt="Linear Regression" src="https://cdn-images-1.medium.com/max/286/1*L_jfs8kXuqfRDO9TSScxHQ.png" /><figcaption>Linear Regression</figcaption></figure><p>Here, <strong><em>y</em></strong> is the model’s output (predicted selling price), and <strong><em>X1, X2</em></strong><em>, </em>and<em> </em><strong><em>X3</em></strong> are the inputs (square footage, number of bedrooms, and age of the house, respectively). The model parameters are: the<strong> <em>Ws </em>— weights</strong> and <strong><em>b</em></strong> is the <strong>bias</strong> term that we learn just like weights. Before training, these parameters are initialized to <strong>random values</strong> and adjusted during training to minimize the model’s error. The weights represent the significance of each input in predicting the output. A higher weight means the corresponding input is more important in making the prediction. For example, a higher weight for square footage would indicate that it has a stronger influence on the selling price compared to the number of bedrooms or the age of the house.</p><h3>Understand Neural Networks</h3><p>Neural networks are just an extension of linear regression. The input layer of a neural network is where we pass our inputs <strong><em>X1, X2</em></strong><em>, </em>and<em> </em><strong><em>X3</em></strong>. Each node in the hidden layer is fully connected to all three input nodes in the input layer. These 5 nodes in the hidden layer use the<strong> linear regression</strong> <strong>equation</strong> to get the output number <strong><em>y </em></strong>(also known as the <strong>activation</strong>) for <strong><em>X1, X2</em></strong><em>, </em>and<em> </em><strong><em>X3</em></strong> for their respective nodes. Each node learns its own set of parameters: <strong><em>W1, W2, W3 </em></strong>and <strong><em>b,</em></strong> so each node has 4 parameters.</p><p>The <strong>two output nodes</strong> in the <strong>output layer</strong> are fully connected to the <strong>five nodes</strong> in the previous (<strong>hidden</strong>) <strong>layer</strong>. Therefore, the outputs from the hidden layer (<strong><em>y1, y2, y3, y4, y5</em></strong>) become the inputs to the output layer. Given that we have five outputs in the hidden layer, the output layer’s equation is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/418/1*X9UERKwSBMojrla0XumIiA.png" /></figure><p>This means <strong><em>each node</em></strong> in the output layer has 5 parameters. The output layer then produces two output numbers (<strong><em>z1</em></strong>​ and <strong><em>z2</em></strong>). We can take the average of these two outputs to get a single prediction from the model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/976/1*Ci_luOd2LbJi3LVW3zM-6A.png" /><figcaption><strong>fully-connected neural network</strong></figcaption></figure><h3>Parameter Optimization with Gradient Descent</h3><p>Gradient descent is the algorithm that updates all the weights and biases of a neural network: <strong><em>Ws </em></strong>and<strong><em> b </em></strong>during training. Our goal is to minimize the error, and that’s exactly what gradient descent does. We decide beforehand how many iterations to run this algorithm. The more iterations, the better approximation we’ll get for minimizing the error. This process forms <strong>our training loop</strong>. Here’s how it works:</p><ol><li><strong>Get Model Predictions:</strong> First, we obtain the model’s predictions using the current weights and biases. We’ll pass in X (our training dataset) of size N×3, where N (e.g., 10) is the number of houses, and 3 is the number of attributes each house has. The output <strong>y</strong> will be an N×1 array, representing the model’s prediction for each house selling price.</li><li><strong>Calculate Current Error:</strong> Next, we calculate the current error by comparing the model’s predictions with the true values. We pass in the truth — an N×1 array of the actual heights from our training dataset — and the model’s predictions.</li><li><strong>Compute Derivatives:</strong> Using derivatives, which measure the rate of change of a function for some variable, we calculate the derivatives of the error function concerning each weight. For instance, we want the derivative of the error for <strong><em>W1, W2</em></strong>, and so on. And the derivatives depend on the value of the function (i.e., the error).</li><li><strong>Update Parameters:</strong> Adjust the weights and biases to reduce the error. The formula for updating the weights is:</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/178/1*y5rMGRihAKOKyFrilD2Mxg.png" /></figure><p>Here, α is the learning rate, a small number like 0.01 or 0.1, and</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/30/1*okzxYsWI33WRdRyw3hP7tQ.png" /><figcaption>is a derivative of the error for the weight</figcaption></figure><p><strong>5. Repeat:</strong> We then go back to the top of the loop with these updated parameters. With each iteration, we hope to get better model predictions and smaller errors. If we repeat this process for enough iterations, the model will learn the relationships between data points, and the error will be minimized.</p><h3>Conclusion</h3><p>I hope you found this post helpful. Understanding these fundamental concepts — training, linear regression, neural networks, and gradient descent — is crucial for anyone getting started with machine learning. These concepts form the backbone of more complex models and algorithms you’ll encounter as you dive deeper into the field.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b72f3df75852" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>