<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Thonix on Medium]]></title>
        <description><![CDATA[Stories by Thonix on Medium]]></description>
        <link>https://medium.com/@thonix?source=rss-089e52820324------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*5443Sh6VrnN56PeW5l-RTw.png</url>
            <title>Stories by Thonix on Medium</title>
            <link>https://medium.com/@thonix?source=rss-089e52820324------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 21:42:53 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@thonix/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding Matrix Multiplication Through Neural Networks]]></title>
            <link>https://grain.saltindex.com/understanding-matrix-multiplication-through-neural-networks-5e150cd0299a?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/5e150cd0299a</guid>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Sat, 21 Nov 2026 11:01:01 GMT</pubDate>
            <atom:updated>2026-11-21T11:01:01.671Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>You do not need to memorize pages of equations to understand a neural network. Start with one customer, two neurons, and the calculation that connects them.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YVoSvvvOv9pVK7zIoL8YDQ.png" /></figure><p>A neural network diagram can look intimidating. There are circles, arrows, and layers of numbers, and someone tells you that the computer is “learning.” But when you open the code, you see a line such as X @ W.</p><p>What does that multiplication actually do?</p><p>It takes information from one layer and combines it into the values used by the next. If you understand how a row of numbers meets a column of weights, you understand a large part of what happens during a neural network’s <strong>forward pass</strong>. You can also tell whether the dimensions in your code make sense before trying to debug a mysterious error.</p><p>Let’s work through one small online-store example, then expand it to several customers. All numbers in this article are invented to teach the calculation. They are not weights learned from a real dataset or reliable spending forecasts.</p><h3>First, what do vectors and matrices represent?</h3><p>Imagine you’re trying to estimate how much a customer might spend during an upcoming visit. For a simple demonstration, you know how many orders they placed recently and how many products they viewed. Customer A placed <strong>2 orders</strong> and viewed <strong>3 products</strong>, so we represent their information as a <strong>vector</strong>: [2, 3].</p><p>A vector is an ordered collection of numbers. In our example, the first position means orders and the second means products viewed. These meanings matter. Swapping the numbers without swapping the meaning of the model’s weights would change the calculation.</p><p>Now suppose we want a small neural network with <strong>two hidden neurons</strong>. Each neuron needs a weight for each input feature. We can store those weights in a <strong>matrix</strong>, which is simply a rectangular arrangement of numbers.</p><p>We’ll use this weight matrix:</p><pre>W = [[10,  1],<br>     [ 5, -2]]</pre><p>In this article, each <strong>row of W</strong> corresponds to an input feature, and each <strong>column</strong> corresponds to a hidden neuron. So the first neuron uses weights 10 and 5, while the second uses 1 and -2. A weight is a number that controls how strongly an input contributes to a neuron. During actual training, the network adjusts these values; for now, we&#39;re choosing them ourselves so the arithmetic is easy to follow.</p><h3>How a row and a column produce one number</h3><p>Here is the essential rule: <strong>multiply matching positions and add the products</strong>. That’s called a <em>dot product</em>. Matrix multiplication repeatedly performs this row-by-column operation.</p><p>For our first neuron, take the customer’s vector [2, 3] and the first weight column [10, 5]. The result is:</p><p><strong>(2 × 10) + (3 × 5) = 20 + 15 = 35</strong></p><p>For the second neuron, use the second weight column [1, -2]:</p><p><strong>(2 × 1) + (3 × -2) = 2–6 = -4</strong></p><p>The output of this matrix multiplication is therefore [35, -4]. We started with two customer features and created two weighted combinations, one for each hidden neuron.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8W-x2UuWx2tLqTEepvoQew.png" /></figure><p>This is why the matrix’s <strong>shape</strong> matters. The input has shape 1 × 2: one customer and two features. The weight matrix has shape 2 × 2: two input features and two hidden neurons. The inner dimensions, both 2, match, so the multiplication works and produces a 1 × 2 result.</p><p>If the model expected two input features but you accidentally supplied three, the multiplication would fail. That error isn’t mysterious once you see how the shapes connect.</p><h3>Why neurons also need a bias and an activation function</h3><p>Matrix multiplication gives us weighted sums, but a typical neural network layer has two more steps. It adds a <strong>bias</strong>, then applies an <strong>activation function</strong>.</p><p>A bias is a learnable offset. It lets a neuron shift its output even when the input values are unchanged. We’ll use the bias vector [5, 6], one value for each hidden neuron.</p><p>Our calculation becomes:</p><p><strong>[35, -4] + [5, 6] = [40, 2]</strong></p><p>Now apply <strong>ReLU</strong>, short for <em>rectified linear unit</em>. It keeps positive values as they are and replaces negative values with zero. Both values here are positive, so the result remains [40, 2].</p><p>If the pre-activation value had been -3, ReLU would change it to 0. This matters because an activation function allows successive layers to represent relationships that aren&#39;t just one big linear calculation. Without a nonlinear activation between layers, stacking linear transformations alone cannot give the model that extra flexibility.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fNa-Kjb838LP9XgvQc7oaA.png" /></figure><p>At this point, our two hidden neurons have produced the numbers <strong>40</strong> and <strong>2</strong>. Those are not yet dollar amounts or probabilities. They’re intermediate values the next layer can use.</p><h3>One more multiplication produces a prediction</h3><p>Let’s give the final output neuron a weight of 2 for the first hidden value and 1 for the second, plus a bias of 10.</p><p>Its calculation is:</p><p><strong>(40 × 2) + (2 × 1) + 10 = 80 + 2 + 10 = 92</strong></p><p>For this fictional model, we might interpret <strong>$92</strong> as an estimated spending amount. That interpretation comes from how we define the model’s target, not from matrix multiplication itself. Because we’ve invented the weights, we haven’t demonstrated that the estimate is accurate.</p><p>Notice something useful: the network never needed to perform a special calculation called “predict spending.” It just carried numbers through two weighted layers, with biases and an activation between them. Training is what would eventually make those numbers useful for spending prediction.</p><h3>Why matrices are especially useful when you have many customers</h3><p>So far, we’ve calculated one customer’s output. But an online store could have thousands of customer records. Repeating the same arithmetic by hand for each customer would be tedious.</p><p>Instead, place customers into a matrix, one row per customer:</p><pre>X = [[2, 3],   # Customer A<br>     [1, 5],   # Customer B<br>     [4, 2]]   # Customer C</pre><p>This input matrix has shape <strong>3 × 2</strong>, because it contains three customers and two features. Multiply it by our <strong>2 × 2</strong> weight matrix. The result has shape <strong>3 × 2</strong>: two weighted sums for each customer. Add the same bias vector to every row, then apply ReLU.</p><p>The hidden layer now produces:</p><pre>H = [[40, 2],<br>     [40, 0],<br>     [55, 6]]</pre><p>To see why Customer B has a zero, consider the second hidden neuron. Its weighted sum is (1 × 1) + (5 × -2) = -9. Adding the bias 6 gives -3, and ReLU turns that into 0.</p><p>Now multiply each hidden row by the final weight vector [2, 1] and add the final bias 10. Customer A gets <strong>$92</strong>; Customer B gets (40 × 2) + (0 × 1) + 10 = $90; Customer C gets (55 × 2) + (6 × 1) + 10 = $126.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZY8gnpfN5f21nZaAs-IHlA.png" /></figure><p>The same weights are reused for every customer. That is an important idea in neural networks: a layer applies one learned transformation to many input examples. In practice, optimized array libraries can perform these calculations efficiently, often on GPUs.</p><h3>Try the complete calculation in Python</h3><p>Here’s the entire example using NumPy. The @ operator performs matrix multiplication, and np.maximum applies ReLU to every hidden value.</p><pre>import numpy as np<br># Three customers, two features each: [orders, products_viewed]<br>X = np.array([[2, 3], [1, 5], [4, 2]], dtype=float)<br># Two inputs connected to two hidden neurons<br>W1 = np.array([[10, 1], [5, -2]], dtype=float)<br>b1 = np.array([5, 6], dtype=float)<br># Hidden layer: weighted sums, bias, then ReLU<br>H = np.maximum(0, X @ W1 + b1)<br># Two hidden neurons connected to one output<br>W2 = np.array([[2], [1]], dtype=float)<br>b2 = np.array([10], dtype=float)<br>predictions = H @ W2 + b2<br>print(&#39;Hidden layer:&#39;)<br>print(H)<br>print(&#39;Predicted spending:&#39;)<br>print(predictions.ravel())<br>print(&#39;Shapes:&#39;, X.shape, W1.shape, H.shape, predictions.shape)</pre><p>The output is:</p><pre>Hidden layer:<br>[[40.  2.]<br> [40.  0.]<br> [55.  6.]]<br>Predicted spending:<br>[ 92.  90. 126.]<br>Shapes: (3, 2) (2, 2) (3, 2) (3, 1)</pre><p>Try changing Customer A’s order count from 2 to 3 and see what happens. Then change one weight, such as 10 to 8. You&#39;ll see immediately that a weight is not just a number in a formula: it controls how input information affects the layer&#39;s output.</p><p>These weights remain fixed in our example. A genuine training loop would compare predictions with actual labeled spending data, calculate a loss, and adjust weights and biases to reduce that loss. Matrix multiplication carries out the forward calculations, while the training process determines which weights to use.</p><h3>The one detail that often confuses beginners</h3><p>Different books and frameworks sometimes arrange their weight matrices differently. Here we put input features in rows of the weight matrix and neurons in columns, so our forward calculation is X @ W + b.</p><p>PyTorch’s Linear layer stores its weight parameter with shape <strong>output features × input features</strong> and therefore computes the equivalent operation using the weight&#39;s transpose. Both conventions describe the same kind of layer; the storage arrangement is different. When comparing a diagram with library code, always check the documented shapes before assuming the weights are &quot;backward.&quot;</p><h3>What should you learn next?</h3><p>If you can explain why [2, 3] multiplied by our weight matrix becomes [35, -4], you&#39;ve understood the key operation. The useful next step isn&#39;t memorizing larger matrices. It&#39;s learning to track shapes, add biases, apply activation functions, and recognize how one layer feeds into the next.</p><p>For a beginner, I’d start by changing the numbers in this tiny NumPy example and predicting the result by hand before running it. Once that feels natural, try implementing the same network in a neural-network library and inspect its layer shapes.</p><p><strong>Matrix multiplication is not the entire story of how a neural network learns. But it is the arithmetic that lets each layer turn input information into new representations, and eventually into predictions.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5e150cd0299a" width="1" height="1" alt=""><hr><p><a href="https://grain.saltindex.com/understanding-matrix-multiplication-through-neural-networks-5e150cd0299a">Understanding Matrix Multiplication Through Neural Networks</a> was originally published in <a href="https://grain.saltindex.com/">Salt Index</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Linear Algebra for AI: Why Vectors and Matrices Matter]]></title>
            <link>https://grain.saltindex.com/linear-algebra-for-ai-why-vectors-and-matrices-matter-a231bea3a3d2?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/a231bea3a3d2</guid>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Sat, 21 Nov 2026 07:01:02 GMT</pubDate>
            <atom:updated>2026-11-21T07:01:02.792Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Vectors and matrices may look intimidating, but they are simply practical ways to organize information and calculate predictions. Here’s what you need to understand before going deeper into AI.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9DMPQKsdNZjnWCfADTcnzw.png" /></figure><p>Imagine you’re building a small AI feature for an online store. You want to estimate how much a customer might spend based on their recent orders, the products they’ve viewed, and the time since their last purchase. You have the information, and you know what you want to predict. The next question is how to give those numbers to a model.</p><p>You could keep each value in a separate variable, but that becomes awkward when you’re dealing with thousands of customers. <strong>Linear algebra gives you a consistent way to represent those customers and perform the same calculation across all of them.</strong> You don’t need to master abstract proofs before your first model. Understanding vectors, matrices, shapes, and dot products is a much more useful starting point.</p><h3>Start With a Vector: One Customer, Several Numbers</h3><p>A <strong>vector</strong> is an ordered collection of numbers. In Machine Learning, it can represent one example, such as a customer, a product, or a document after its relevant information has been converted into numerical features.</p><p>Suppose we describe one customer using three values: two recent orders, five product views, and 30 days since the last order. We can write the customer as <strong>x = [2, 5, 30]</strong>. The position of each number matters. The first entry always means orders, the second means views, and the third means days.</p><p>This is also why the word <strong>dimension</strong> comes up so often. Our vector has three components, so it is a three-dimensional feature vector. That doesn’t mean the customer exists in some mysterious three-dimensional digital world. It just means we’re recording three different measurements.</p><p>If another customer placed four orders, viewed three products, and last ordered ten days ago, their vector would be <strong>[4, 3, 10]</strong>. Now both customers use the same format, and the computer can apply the same operation to either one.</p><h3>A Matrix Is What Happens When You Have More Than One Customer</h3><p>A <strong>matrix</strong> is a rectangular arrangement of numbers organized into rows and columns. If one vector describes one customer, a matrix can store many customer vectors together.</p><p>Our first four fictional customers might be represented as:</p><pre>X = [ [2, 5, 30],<br>      [4, 3, 10],<br>      [1, 2, 45],<br>      [3, 4, 20] ]</pre><p>Each row represents one customer. Each column represents one feature. This matrix has <strong>four rows and three columns</strong>, so we call its <strong>shape</strong> 4 × 3.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oWDPFU7nk-LUHePqRFTVjQ.png" /></figure><p><em>One customer is a vector. Four customers, each described by three features, form a 4 × 3 matrix.</em></p><p>The shape matters because it tells you which calculations are possible. If you accidentally provide three numbers when a model expects four, you’ll usually get a shape or dimension error. Learning to recognize shapes early makes Python libraries such as NumPy and scikit-learn much less confusing.</p><h3>The Dot Product: How a Model Combines Features</h3><p>Now we have a customer vector. How might a model turn it into a spending estimate?</p><p>One simple approach is to give each feature a <strong>weight</strong>. A weight describes how strongly that feature contributes to the calculation. For this fictional example, let’s choose weights of <strong>[20, 5, -0.5]</strong> and an additional starting value, called a <strong>bias</strong>, of <strong>52.5</strong>.</p><p>The first weight adds $20 for each recent order. The second adds $5 for each product view. The negative weight subtracts $0.50 for each day since the last order. These numbers are deliberately chosen to demonstrate the arithmetic; a real model would normally estimate suitable weights from training data.</p><p>The operation that combines matching features and weights is called a <strong>dot product</strong>. You multiply the entries in matching positions, then add the results:</p><p><strong>(2 × 20) + (5 × 5) + (30 × -0.5) = 40 + 25–15 = 50</strong></p><p>Finally, add the bias:</p><p><strong>Prediction = 50 + 52.5 = $102.50</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*he00wdDPl4IN4vMQXHa8hA.png" /></figure><p><em>The dot product adds the weighted contribution of each feature. The bias then shifts the result.</em></p><p>That’s the central calculation behind a basic linear prediction: <strong>prediction = x · w + b</strong>. The symbol <strong>·</strong> means dot product, <strong>w</strong> represents the weights, and <strong>b</strong> represents the bias. No magic is happening inside the multiplication. The useful behavior comes from choosing or learning weights that fit the task.</p><p>A dot product also appears in other AI systems. For example, it can compare numerical representations of items, although real similarity systems often normalize vectors or use additional transformations. The key idea stays the same: turn many numbers into a single meaningful score.</p><h3>Matrix Multiplication: Repeat the Calculation for Everyone</h3><p>Calculating one customer’s estimate by hand is manageable. Doing it for 10,000 customers would be tedious, especially if each customer has hundreds of features.</p><p>This is where <strong>matrix multiplication</strong> becomes useful. Put all the customers into a matrix <strong>X</strong> and keep the weights in a vector <strong>w</strong>. Then calculate <strong>X @ w + b</strong>. In Python, the @ operator performs matrix multiplication or a compatible matrix-vector product.</p><p>Our 4 × 3 customer matrix can multiply a three-entry weight vector because the shared dimension is three. The result is one score for each of the four customers:</p><p><strong>[102.50, 142.50, 60.00, 122.50]</strong></p><p>For the second customer, you can check the arithmetic yourself: <strong>(4 × 20) + (3 × 5) + (10 × -0.5) + 52.5 = 142.5</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zqJlB2CLT4ARprCbHpVDLw.png" /></figure><p><em>Four customers multiplied by one shared weight vector produce four illustrative predictions.</em></p><p>This is one reason matrices matter so much in AI. They let a computer organize work into larger operations that optimized numerical libraries can handle efficiently. More advanced systems use similar operations over much larger arrays, though the mathematics and architecture become more involved.</p><h3>Try It in Python With NumPy</h3><p>You can reproduce every calculation above with a few lines of Python. Install NumPy first if needed using pip install numpy, then run the following complete example:</p><pre>import numpy as np<br># Fictional customer features: [orders, items_viewed, days_since_order]<br>X = np.array([[2, 5, 30], [4, 3, 10], [1, 2, 45], [3, 4, 20]], dtype=float)<br># Illustrative weights chosen by hand, not learned from real data<br>w = np.array([20, 5, -0.5], dtype=float)<br>bias = 52.5<br># One customer&#39;s prediction<br>first_prediction = X[0] @ w + bias<br>print(f&quot;First customer: ${first_prediction:.2f}&quot;)<br># The same calculation for every customer at once<br>predictions = X @ w + bias<br>print(&quot;All predictions:&quot;, predictions)<br>print(&quot;Matrix shape:&quot;, X.shape)<br>print(&quot;Weights shape:&quot;, w.shape)</pre><p>The output is:</p><pre>First customer: $102.50<br>All predictions: [102.5 142.5  60.  122.5]<br>Matrix shape: (4, 3)<br>Weights shape: (3,)</pre><p>Notice the two shapes at the end. The data matrix contains four customers with three features each. The weight vector contains three numbers. Because those three features line up with the three weights, the multiplication works.</p><p>Also notice what the code <strong>doesn’t</strong> do. It doesn’t train a model. We’ve supplied the weights ourselves. In a real supervised learning project, training would use historical features and known spending amounts to find weights that reduce prediction error. This example isolates the linear algebra so you can see what the model eventually calculates.</p><h3>Where Matrices Show Up Beyond Simple Predictions</h3><p>Once you recognize vectors and matrices, you’ll start noticing them throughout AI. In image processing, pictures can be represented as arrays of pixel values. In text systems, numerical vectors can represent words, tokens, or larger pieces of text. Recommendation systems may represent users and products as vectors and calculate scores from their relationships.</p><p>Neural networks also perform many matrix multiplications. A layer may take a batch of input vectors and multiply them by a matrix of learned weights, then add biases and apply a nonlinear function. That nonlinear step is important: a modern neural network isn’t just one giant multiplication.</p><p>You don’t need to learn all of those applications at once. Start by understanding how a single vector becomes a score and how a matrix lets you perform the same work on many examples.</p><h3>What Should You Learn Next?</h3><p>If you’re learning Machine Learning with Python, I’d begin with four practical skills: reading a vector, identifying the rows and columns of a matrix, calculating a dot product by hand, and checking whether the dimensions match for multiplication. Once those feel natural, learn how to use NumPy arrays and the @ operator.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qghRcRoBWO5WBNNsM5cexQ.png" /></figure><p><em>The most useful learning order is vector → matrix → dot product → matrix multiplication.</em></p><p>You can go further into vector lengths, angles, transposes, projections, and eigenvectors later, especially if you’re studying embeddings, recommendation systems, or neural networks more deeply. But those topics will make more sense once you’ve worked through a few small calculations yourself.</p><h3>The Takeaway</h3><p>You don’t need to memorize pages of matrix formulas before building your first AI project. <strong>You need to understand how information becomes numbers, how those numbers are organized, and how simple operations combine them into useful results.</strong></p><p>The next time you see a NumPy array, model weights, or a neural-network layer, ask two questions: <em>What does each number represent?</em> And <em>what is the shape of this data?</em> Those two checks will take you surprisingly far.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a231bea3a3d2" width="1" height="1" alt=""><hr><p><a href="https://grain.saltindex.com/linear-algebra-for-ai-why-vectors-and-matrices-matter-a231bea3a3d2">Linear Algebra for AI: Why Vectors and Matrices Matter</a> was originally published in <a href="https://grain.saltindex.com/">Salt Index</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Math You Actually Need for Machine Learning]]></title>
            <link>https://grain.saltindex.com/the-math-you-actually-need-for-machine-learning-e5344f1a8c4c?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5344f1a8c4c</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Fri, 20 Nov 2026 11:01:02 GMT</pubDate>
            <atom:updated>2026-11-20T18:28:13.053Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>You can build a model before mastering advanced math. The useful question is which mathematical ideas to learn first, and when you will actually need them.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A6sfQ0nF9VXegjX0JT1bOQ.png" /></figure><p>You open a machine learning course, and the first lesson seems manageable. Then the next screen fills with equations, matrices, probability, and derivatives. It starts to feel as if you need a mathematics degree before writing a line of Python.</p><p>You don’t. For a beginner who wants to train models with tools such as scikit-learn, <strong>basic algebra and statistics are the best place to start</strong>. Linear algebra becomes more useful when you work with many features, while calculus helps when you want to understand how certain models learn.</p><p>The mistake is treating all that mathematics as one enormous prerequisite. It is much easier to learn the part that answers a question in the project you are already building.</p><h3>Start With the Question Your Model Is Trying to Answer</h3><p>Imagine running a small online store. You want to predict how much a customer might spend based on the number of orders they place. Historical customers provide examples: the order count is the input, and spending is the value you are trying to predict.</p><p>You can train a simple model with a few lines of Python. But knowing <em>why</em> the prediction changes when the order count changes requires some algebra. Knowing whether the predictions are trustworthy requires statistics. You do not need to study everything at once.</p><h3>1. Algebra: The First Math to Learn</h3><p><strong>Algebra</strong> is about relationships between quantities. It lets us describe how one value changes when another changes. For a simple spending prediction, you might see the equation y = mx + b, where x is the number of orders, y is predicted spending, m is the slope, and b is the starting value.</p><p>Let’s calculate one prediction. Suppose the model uses y = 12x + 20, where x is the number of orders and y is spending in dollars. For five orders, replace x with 5: y = 12 × 5 + 20 = 60 + 20 = $80. For six orders, the prediction becomes 12 × 6 + 20 = $92. The difference is $12, which is the slope: each additional order increases this model&#39;s prediction by $12. It does not mean every real customer spends exactly $12 per order.</p><p>This is why variables, basic functions, graphs, and rearranging equations are worth learning early. You don’t need to solve difficult equations by hand, but you should feel comfortable asking what a number in a formula represents.</p><h3>2. Statistics and Probability: Is the Prediction Useful?</h3><p>A model can produce a number without producing a good answer. <strong>Statistics</strong> helps you summarize data and judge the size of mistakes. Start with the mean, median, spread, and the difference between a prediction and the actual value.</p><p>Here’s a small <strong>statistics calculation</strong>. Say three customers actually spend $74, $90, and $65, while your model predicts $80, $87, and $74. The absolute errors are |80 − 74| = $6, |87 − 90| = $3, and |74 − 65| = $9. Their <strong>mean absolute error (MAE)</strong> is (6 + 3 + 9) ÷ 3 = $6. In this fictional group, predictions miss the actual spending by an average of $6. That gives you more useful information than simply saying the model “works.”</p><p><strong>Probability</strong> is related but answers a different question. Instead of predicting spending, suppose you want to estimate the chance that a customer will stop buying. A predicted probability of 0.8 means the model estimates an 80% chance, not that the event is certain. To know whether those probabilities are reliable, you need to check them against real outcomes.</p><p>You can also calculate a simple <strong>observed probability</strong> from historical data. If 8 out of 40 similar customers became inactive, the observed fraction is 8 ÷ 40 = 0.20, or <strong>20%</strong>. That gives you a useful starting estimate, although it does not prove a particular new customer has exactly a 20% chance of leaving. A model&#39;s predicted probabilities still need to be checked against future outcomes. Understanding averages, variation, and probabilities is more useful at first than memorizing advanced proofs.</p><h3>3. Linear Algebra: When You Have More Than One Input</h3><p>Our first example uses one feature, the number of orders. Real models may also consider days since the last purchase, total spending, customer tenure, and other information.</p><p><strong>Linear algebra</strong> gives us a convenient way to work with many numbers together. A <strong>vector</strong> is an ordered collection of values. For a customer, the feature vector might be [5, 12], meaning five orders and 12 days since the last purchase. A <strong>matrix</strong> arranges many customers&#39; feature vectors into rows and columns.</p><p>Here’s a small calculation. Imagine a model assigns weights of $10 per order and −$0.50 per day since the last purchase, plus a starting value of $20. Multiply each input by its weight and add the results: 5 × 10 + 12 × (−0.50) + 20 = 50 − 6 + 20 = $64. In linear algebra, the part 5 × 10 + 12 × (−0.50) is a <strong>dot product</strong> between the feature vector [5, 12] and the weight vector [10, −0.50].</p><p>Real libraries handle these calculations across many customers at once. You don’t need to multiply large matrices by hand, but understanding this one small example explains how a model combines multiple inputs. The numbers are illustrative and describe a different toy model from the Python regression below.</p><h3>4. Calculus: How Some Models Improve During Training</h3><p><strong>Calculus</strong> becomes helpful when you want to understand how a model adjusts its parameters to reduce mistakes. A <strong>derivative</strong> describes how much a quantity changes when another quantity changes. In many machine learning algorithms, derivatives tell an optimizer which direction will reduce a loss function.</p><p>Picture walking downhill in fog. You cannot see the whole landscape, but you can sense which direction slopes downward. <strong>Gradient descent</strong> uses a related idea to update a model’s parameters in steps that aim to lower its error.</p><p>Here’s an intentionally tiny <strong>calculus calculation</strong>. Suppose a toy model has one adjustable number, w, and its loss is L(w) = (w − 12)². The minimum loss occurs at w = 12. The derivative is dL/dw = 2(w − 12), which tells us the direction of change. If w = 10, the derivative is 2 × (10 − 12) = −4. With a learning rate of 0.1, gradient descent updates it to 10 − 0.1 × (−4) = 10.4. The loss falls from (10 − 12)² = 4 to (10.4 − 12)² = 2.56. One small update made the model&#39;s objective better, although repeated updates are usually needed.</p><p>This toy loss illustrates the idea rather than the exact training procedure used by the Python example. Not every model uses gradient descent. Decision trees, for example, are typically trained differently. For beginner scikit-learn projects, understanding the direction of improvement is enough; detailed derivatives become more important when studying neural networks and optimization.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KoV2AJGLQ5DwjSm0HkWGPg.png" /></figure><h3>A Practical Example: Predicting Customer Spending</h3><p>Let’s connect the math to a working Python program. We’ll use fictional customer records. Each number of orders is paired with how much that customer spent, and we’ll keep the last two examples separate to test predictions on data the model did not train on.</p><pre>import numpy as np<br>from sklearn.linear_model import LinearRegression<br>from sklearn.metrics import mean_absolute_error<br># Fictional records: order count and spending in dollars<br>orders = np.array([1, 2, 3, 4, 5, 6, 7, 8]).reshape(-1, 1)<br>spending = np.array([32, 44, 58, 65, 83, 90, 108, 115])<br># Train on the first six customers<br>model = LinearRegression()<br>model.fit(orders[:6], spending[:6])<br># Evaluate on two customers the model has not seen<br>predictions = model.predict(orders[6:])<br>error = mean_absolute_error(spending[6:], predictions)<br>print(&quot;Test predictions:&quot;, np.round(predictions, 2))<br>print(&quot;Test MAE:&quot;, round(error, 2))<br>print(&quot;Prediction for 9 orders:&quot;, round(model.predict([[9]])[0], 2))</pre><p>The output is:</p><pre>Test predictions: [103.4  115.23]<br>Test MAE: 2.41<br>Prediction for 9 orders: 127.06</pre><p>The code learns approximately predicted spending = 11.83 × orders + 20.60 from the first six fictional customers. For a customer with seven orders, the algebra gives 11.83 × 7 + 20.60 ≈ $103.40. The actual spending is $108, so the absolute error is $108 − $103.40 = $4.60. For eight orders, the model predicts about $115.23 against an actual $115, an absolute error of about $0.23. The test MAE is therefore ($4.60 + $0.23) ÷ 2 ≈ $2.41.</p><p>That connects the four topics: algebra describes the prediction line, statistics measures the two errors, and linear algebra helps the library process the input arrays. Calculus explains parameter updates in many learning algorithms, although this particular scikit-learn linear regression fit does not require you to implement gradient descent or calculate derivatives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OAMTJgmZMUIXsGP4-h1fXA.png" /></figure><p>The model’s test error looks small here, but <strong>two held-out customers are not enough to establish real-world accuracy</strong>. The data is invented to demonstrate the mechanics, not to make a business claim. On real customer data, you would need more representative examples and a careful evaluation plan.</p><h3>So What Should You Study First?</h3><p>If you are just beginning, my recommendation is to learn basic Python and algebra together. Build a small regression model, then learn enough statistics to understand whether its predictions make sense. Once you work with several features, introduce vectors and matrices. Save detailed derivatives and optimization for when you start asking how the training algorithm updates its parameters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kcmUYI1V4cabmbMWBicolA.png" /></figure><p>This order is a learning strategy, not a strict syllabus. If you’re interested in deep learning research, linear algebra and calculus will move up your priority list. If your main goal is analyzing business data and building practical models with established libraries, statistics and evaluation deserve more attention early on.</p><p>You do not need to finish a mountain of mathematics textbooks before your first project. Start with the simplest model you can explain, measure where it fails, and learn the mathematical idea that helps you answer the next question. That is a more useful way to build both confidence and understanding.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5344f1a8c4c" width="1" height="1" alt=""><hr><p><a href="https://grain.saltindex.com/the-math-you-actually-need-for-machine-learning-e5344f1a8c4c">The Math You Actually Need for Machine Learning</a> was originally published in <a href="https://grain.saltindex.com/">Salt Index</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can You Learn Machine Learning Without a Computer Science Degree?]]></title>
            <link>https://medium.com/h7w/can-you-learn-machine-learning-without-a-computer-science-degree-bc15a5fb57a1?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc15a5fb57a1</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai-for-beginners]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Fri, 20 Nov 2026 07:01:02 GMT</pubDate>
            <atom:updated>2026-11-20T07:01:02.551Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Yes, you can start learning and building useful projects without a CS degree. The harder question is which skills you need for the goal you actually have.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Z_k-d2ngs-tXg1YiuBK4fg.png" /></figure><p>Imagine you’re interested in machine learning, but your degree is in business, design, biology, or something completely unrelated to computers. You find a course that promises to teach you how to train an AI model. Before you enroll, one question keeps coming back: should you study computer science first?</p><p><strong>You don’t need a computer science degree to begin learning machine learning.</strong> You do need patience with Python, data, basic mathematics, and the habit of testing whether a model works on examples it hasn’t seen. A degree can provide structure and make some career paths easier, but it is not a prerequisite for building your first working model.</p><p>The distinction matters because learning machine learning as a practical skill is different from qualifying for a specialized machine learning engineering job. Let’s look at what each path actually requires.</p><h3>What a Computer Science Degree Gives You, and What It Doesn’t</h3><p>A computer science program commonly teaches programming, algorithms, data structures, mathematics, and how software systems work. Those foundations are useful when projects become large or technically demanding. But taking a degree doesn’t automatically make someone good at choosing data, spotting a misleading accuracy score, or solving a business problem.</p><p>You can learn many of the same practical foundations outside a university. Google’s Machine Learning Crash Course emphasizes familiarity with Python, NumPy, pandas, and mathematical ideas rather than requiring a particular degree. The fast.ai practical course assumes some coding experience and high-school mathematics. Neither is a course for someone who has never written code, but both show that there is a structured learning route outside a CS program.</p><p>The real question is not “Do I have the correct certificate?” It’s “Can I understand the task, write and check the code, and explain the results?”</p><h3>The Skills That Matter Before Your First Model</h3><p><strong>Python</strong> is a good starting point because many beginner-friendly machine learning tools use it. You should be comfortable with variables, lists, functions, and reading error messages. You don’t need to build a complex application before you try a model.</p><p><strong>Data handling</strong> comes next. A model learns from examples, so you need to understand rows, columns, missing values, and the difference between <strong>features</strong> (information used to predict) and <strong>labels</strong> (the known answers during supervised training). A model can execute perfectly and still produce unreliable results if those examples are misleading.</p><p><strong>Basic mathematics</strong> helps you reason about patterns and mistakes. Start with averages, percentages, graphs, simple algebra, and probability. More linear algebra and calculus becomes valuable as you explore neural networks or develop algorithms, but you can learn that depth alongside practical projects.</p><p>Finally, you need <strong>evaluation</strong>. Training is when a model fits patterns in its examples. Testing means checking its performance on held-out examples. If you only check the records the model trained on, you may mistake memorization for real learning.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sYO2lsU7CCCs0MKF4EAbMg.png" /></figure><h3>Learn by Building One Small Project</h3><p>Suppose you run an online store and want to predict whether a customer might stop buying. For a simple exercise, we’ll represent customers using two features: the number of recent orders and days since their last order. Our label is 1 if the customer later became inactive and 0 otherwise.</p><p>A model learns from historical examples where the label is already known. For a new customer, it uses the two inputs to predict the label. This doesn’t replace a real retention strategy, but it gives you a complete small project: define a question, prepare data, train a model, and inspect a prediction.</p><p>Here is working Python code using scikit-learn. The records are fictional, deliberately tiny, and designed for learning only.</p><pre>import numpy as np<br>from sklearn.model_selection import train_test_split<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.metrics import accuracy_score<br># [recent_orders, days_since_last_order]<br>X = np.array([<br>    [8, 3], [6, 7], [7, 4], [5, 10],<br>    [1, 40], [0, 55], [2, 35], [1, 48],<br>    [9, 2], [4, 12], [0, 60], [2, 32],<br>    [6, 8], [5, 6], [1, 42], [0, 50]<br>])<br># 0 = active, 1 = inactive<br>labels = np.array([0, 0, 0, 0, 1, 1, 1, 1,<br>                   0, 0, 1, 1, 0, 0, 1, 1])<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X, labels, test_size=0.25, random_state=42,<br>    stratify=labels<br>)<br>model = DecisionTreeClassifier(max_depth=2, random_state=42)<br>model.fit(X_train, y_train)<br>predicted = model.predict(X_test)<br>print(&#39;Test accuracy:&#39;, accuracy_score(y_test, predicted))<br>print(&#39;New customer prediction:&#39;, model.predict([[1, 38]])[0])</pre><p>Notice what the code does beyond calling fit(). It separates records into training and testing groups, then evaluates the trained model on examples kept out of training. That separation matters much more than producing an impressive number from the same records you used to teach it.</p><p>The test score here can change with the split and is not evidence of real-world accuracy. Sixteen invented customers cannot tell us how actual shoppers will behave. In a real project, we would need more representative data, careful time-based evaluation, and a clear definition of customer inactivity.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*voBydUL6PKqUXjSdSKQluw.png" /></figure><h3>A Practical Learning Path Without Going Back to University</h3><p>If you are completely new to coding, begin with Python before trying to master machine learning. Write small scripts that read a CSV file, calculate averages, and print a useful summary. Next, use pandas to explore a dataset and learn why some records need cleaning.</p><p>Once you can explain features and labels, try one classification project and one regression project. Start with scikit-learn rather than building an algorithm from scratch. After each experiment, explain what the model predicted, what metric you used, and why the result might be unreliable.</p><p>Then make a small portfolio. A strong beginner project is more than a screenshot of code that ran. Include the problem, where the data came from, how you split it, the baseline you compared against, mistakes you found, and what you would improve. That documentation shows you understand the process instead of merely copying a tutorial.</p><h3>Learning the Skill Is Not the Same as Landing the Job</h3><p>If your goal is to automate reports, understand how AI works, or build small prototypes, you may not need a formal degree at all. A focused sequence of courses and projects can take you a long way.</p><p>If your goal is a full-time machine learning engineer role, the requirements can be much broader. Employers may expect strong programming, software engineering, statistics, deployment, and production monitoring. Some roles require a relevant degree or advanced study; others place greater weight on demonstrated experience. You should check actual job descriptions in the market you’re targeting rather than assuming a certificate or a portfolio guarantees employment.</p><p>That difference should change your learning plan. Someone who wants to predict store sales doesn’t need to prepare for every infrastructure interview. Someone who wants to build machine learning systems at scale should expect to study software engineering more deeply.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3fqa0Ll5E1C-KddhCa8DWg.png" /></figure><h3>So, Should You Study Computer Science First?</h3><p>For most beginners who want to discover whether machine learning is right for them, <strong>start with Python and one small project before committing to another degree</strong>. You will quickly learn whether you enjoy working with datasets, debugging mistakes, and investigating surprising results.</p><p>If you later find that your goals require more mathematics, algorithms, or software architecture, you can study those subjects deliberately. A university program may be the right choice for some people, especially if they want a broad foundation, research opportunities, or roles with formal education requirements. It simply isn’t the only starting point.</p><p>The useful test is straightforward: can you explain what your model learned, what its test score means, and why someone should or shouldn’t trust its prediction? If you can do that with a small project, you’ve begun learning machine learning, regardless of what’s printed on your diploma.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc15a5fb57a1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/h7w/can-you-learn-machine-learning-without-a-computer-science-degree-bc15a5fb57a1">Can You Learn Machine Learning Without a Computer Science Degree?</a> was originally published in <a href="https://medium.com/h7w">T3CH</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Supervised vs. Unsupervised Learning: Explained With Real Examples]]></title>
            <link>https://ai.plainenglish.io/supervised-vs-unsupervised-learning-explained-with-real-examples-dbc55f94e282?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/dbc55f94e282</guid>
            <category><![CDATA[ai-for-beginners]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[unsupervised-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 23:01:02 GMT</pubDate>
            <atom:updated>2026-11-19T23:01:02.698Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>One predicts an answer you already know how to measure. The other helps you discover patterns you did not label in advance. Here is how to decide which one your project needs.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mIX1HEigxQvd8jnjdzQSJw.png" /></figure><p>Imagine you run a small online store. You have thousands of customer records, including past orders, spending, and how recently people visited. You want to use those records to make better business decisions, but where do you start?</p><p>You might want to <strong>predict which customers will stop buying</strong>. Or you might want to <strong>discover different types of shoppers</strong> so you can understand their habits. Both sound like machine learning projects, and both can use the same customer records. Yet they ask the computer to do two different jobs.</p><p>That is the difference between supervised and unsupervised learning. <strong>If you have examples with the answers you want to predict, supervised learning is usually the starting point. If you want to explore patterns without those answers, unsupervised learning is often more appropriate.</strong></p><h3>First, Understand the Keywords</h3><h3>Supervised Learning: Learning With Known Answers</h3><p>In <strong>supervised learning</strong>, a model studies examples that contain both input information and a known target answer. The input information is called the <strong>features</strong>. The known answer is called the <strong>label</strong>, or <strong>target</strong>.</p><p>Suppose you look at historical customer records from your store. For each customer, you know how many orders they placed during the previous month, how long it has been since their last purchase, and whether they became inactive during the following 90 days. The first two pieces of information are features. The final outcome is the label.</p><p>The model uses these examples to learn relationships that might help it predict whether a new customer will become inactive. It is not given a handwritten rule such as “everyone who hasn’t bought in 30 days will leave.” It learns a relationship from labeled examples and then applies that relationship to new records.</p><p>Two common supervised tasks are <strong>classification</strong>, which predicts a category such as active or inactive, and <strong>regression</strong>, which predicts a number such as next month’s spending. The distinction matters because the target determines how you train and evaluate the model.</p><h3>Unsupervised Learning: Looking for Structure Without Answers</h3><p>In <strong>unsupervised learning</strong>, the model examines data without the specific target labels that supervised learning uses. Instead of learning to predict a known answer, it looks for useful structure in the inputs.</p><p>Imagine you have order counts and spending totals for every customer, but no predefined labels such as “budget shopper,” “frequent buyer,” or “occasional customer.” You could use an unsupervised method to find customers with similar behavior. This is called <strong>clustering</strong>.</p><p>A clustering algorithm might discover one group with frequent small purchases, another with infrequent expensive orders, and a third somewhere in between. Those descriptions are your interpretation of the groups afterward. <strong>The algorithm does not automatically know what a group means for your business.</strong></p><p>Another unsupervised technique is <strong>dimensionality reduction</strong>, which helps summarize data with many features into fewer dimensions. That can make complicated information easier to visualize or process. But clustering is the easiest place to start if you want to see unsupervised learning in action.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pG6sjZF1cmeeZnJ3KtZyuw.png" /></figure><h3>Why the Same Dataset Can Be Used in Two Ways</h3><p>This is where the distinction becomes interesting. You do not necessarily need two separate collections of customers. You might use the same order count and days-since-last-purchase features for both projects. What changes is your question, and whether you supply a target answer.</p><p>For the inactivity prediction project, you train on records where the future inactivity outcome is known. Later you compare predictions against real outcomes on customers the model did not learn from. This gives you a way to measure mistakes, such as how many future inactive customers the model missed.</p><p>For customer grouping, you provide only the behavior features and ask the algorithm to organize similar records. You then inspect the groups, look at actual customer behavior, and decide whether the groups are stable and useful. A group of customers who behave similarly is not automatically a marketing segment worth targeting.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MMM5POwPfT2WLFk3HArZ6A.png" /></figure><h3>A Small Python Project Using Both Approaches</h3><p>Let’s try both methods on the same fictional online-store data. We’ll use <strong>scikit-learn</strong>, a Python library that provides tools for supervised and unsupervised learning.</p><p>Our simplified dataset contains only two features: the number of recent orders and the number of days since the last purchase. It also includes a historical inactivity label for the supervised experiment. These are made-up examples for learning, not a real customer study. <strong>Run the following three Python code blocks in order in the same Python session.</strong> The second and third examples reuse the X_scaled variable created in the first block. You will need NumPy and scikit-learn installed.</p><pre>import numpy as np<br>from sklearn.preprocessing import StandardScaler<br>from sklearn.linear_model import LogisticRegression<br>from sklearn.cluster import KMeans<br># Orders in last 30 days, days since last purchase<br>X = np.array([<br>    [8, 3], [1, 28], [5, 7], [0, 45],<br>    [6, 5], [2, 35], [9, 2], [1, 40],<br>    [7, 6], [3, 25], [10, 1], [2, 32]<br>])<br># Known historical outcomes: 0 = active, 1 = inactive<br>y = np.array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1])<br>scaler = StandardScaler()<br>X_scaled = scaler.fit_transform(X)</pre><p>The X array holds the features, and y holds the labels. We scale the features so that the larger numerical range of days since last purchase does not dominate distance-based clustering. In a real project, preprocessing should be fitted using training data only, not future test examples.</p><p><strong>Supervised example: Predicting inactivity.</strong> We train logistic regression with both X_scaled and y. It learns from the known outcomes and predicts a category for a new customer.</p><pre>classifier = LogisticRegression()<br>classifier.fit(X_scaled, y)<br>new_customer = scaler.transform([[1, 38]])<br>print(classifier.predict(new_customer))</pre><p><strong>Unsupervised example: Discovering groups.</strong> This time we do not pass y to the algorithm. K-means groups the customers based on similarities in the two features.</p><pre>clusters = KMeans(n_clusters=2, random_state=42, n_init=10)<br>clusters.fit(X_scaled)<br>print(clusters.labels_)</pre><p>Notice the difference. Logistic regression receives the historical answers because its task is to predict them. K-means receives only the features because its task is to group similar customers. Its numeric cluster IDs are identifiers, not predictions that a customer is active or inactive. Also, choosing two clusters was our decision, not a fact the algorithm discovered about the business.</p><p>Neither example proves that we have built a reliable system. Twelve fictional customers are far too few for business decisions, and we haven’t created an independent evaluation set. The point is to show how the learning goals and inputs differ.</p><h3>How Do You Know Whether Either Method Worked?</h3><p>For supervised learning, you usually have a clear way to check: compare predictions with known outcomes on held-out data. If you’re predicting inactivity, look beyond overall accuracy. <strong>Precision</strong> tells you how many customers flagged as likely inactive actually became inactive. <strong>Recall</strong> tells you how many of the truly inactive customers you managed to identify. Both matter when deciding whether contacting those customers is worthwhile.</p><p>Unsupervised learning is harder to evaluate because there is no single correct grouping supplied at the start. You can check whether clusters are well separated, whether similar customers stay grouped when the data changes, and whether the groups reveal something your business can act on. A pretty scatter plot alone is not proof of a useful discovery.</p><p>There is another practical warning: <strong>features matter to both approaches</strong>. If a customer has missing purchase dates or the data mostly represents one kind of shopper, either method can produce misleading results. Better data preparation is often more valuable than trying a more complicated algorithm.</p><h3>Which One Should You Choose First?</h3><p>Start with the business question rather than the algorithm name. If you need to predict a defined outcome, such as whether an order will be returned, whether an email is spam, or how much a product may sell next week, look for historical examples with that outcome recorded. That’s a supervised learning problem.</p><p>If you want to explore behavior without known categories, such as identifying customer groups or discovering similarities among products, unsupervised learning may help you find a place to start. But you will still need to interpret the patterns and decide whether they are useful.</p><p>These approaches can also work together. A store might first cluster customers to understand different shopping habits and later use labeled data to train a supervised model for a specific prediction. The clusters can inform the analysis, but they are not a replacement for real target labels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7hhpL9Xbbq1uMliR5v22LA.png" /></figure><h3>The Takeaway</h3><p>Supervised learning and unsupervised learning are not competing versions of the same solution. They solve different kinds of questions. <strong>Supervised learning asks, “Can we predict this known outcome?” Unsupervised learning asks, “What useful structure might be hiding in this data?”</strong></p><p>For your first project, write down the question you want answered and check whether you have historical answers. That one check will usually tell you which direction makes sense before you write a line of code.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=dbc55f94e282" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/supervised-vs-unsupervised-learning-explained-with-real-examples-dbc55f94e282">Supervised vs. Unsupervised Learning: Explained With Real Examples</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is Overfitting? Why AI Models Sometimes Fail on New Data]]></title>
            <link>https://ai.plainenglish.io/what-is-overfitting-why-ai-models-sometimes-fail-on-new-data-146927558635?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/146927558635</guid>
            <category><![CDATA[overfitting]]></category>
            <category><![CDATA[ai-for-beginners]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 19:01:02 GMT</pubDate>
            <atom:updated>2026-11-19T19:01:02.196Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>An AI model can score almost perfectly on the examples it learned from and still make poor predictions for real customers. Here is how to recognize that problem and what to change.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zq3KD-QKgBxS8OxoCf1eLw.png" /></figure><p>Imagine you have built a machine learning model for your online store. Its job is to predict which customers are likely to stop buying. You train it on thousands of past customer records, run the evaluation, and see 99% accuracy on the training data.</p><p>That looks like a success. But when the model sees customers it has never encountered before, its accuracy drops to 70%. These numbers are hypothetical, but the problem they illustrate is real: <strong>the model may have learned the training examples too closely instead of learning a pattern that works for new customers.</strong></p><p>This is called <em>overfitting</em>. The practical question isn’t how to get the highest possible score on old data. It’s how to build a model you can trust on the next batch of customers.</p><h3>What Overfitting Actually Means</h3><p>A machine learning model learns relationships from examples. In our store, it might notice that customers who haven’t purchased for a long time are more likely to become inactive. That’s a useful relationship if it also holds for customers the model hasn’t seen.</p><p>But training data contains more than useful patterns. It also contains unusual purchases, incorrect records, random coincidences, and one-off events. A flexible model can learn those details too. It becomes extremely good at matching its training records while losing the ability to make sensible predictions about new ones.</p><p>Think about a student who memorizes the answers to a particular set of exam questions. The student does wonderfully when those questions appear again, but struggles when the exam asks the same ideas in a new way. Overfitting is similar: remembering the examples isn’t the same as understanding the general pattern.</p><p>The opposite of overfitting is not memorizing less at any cost. It’s <strong>generalization</strong>, which means performing well on unfamiliar examples that represent the same real-world task.</p><h3>Three Keywords That Explain the Problem</h3><p><strong>Training error</strong> describes how wrong a model is on examples used to fit it. <strong>Validation error</strong> measures mistakes on separate examples used to choose the model or tune its settings. <strong>Test error</strong> is a final check on held-out examples that were not used to make those choices.</p><p>When training performance looks much better than validation performance, that’s a warning that the model may be overfitting. This difference is often called the <strong>generalization gap</strong>. A large gap isn’t proof of a single cause, but it’s a reason to investigate.</p><p><strong>Model complexity</strong> describes how flexible a model is. A shallow decision tree can learn a few broad rules. A very deep tree can split the data into many tiny groups and even isolate individual training examples. That extra flexibility can help with genuinely complicated relationships, but it can also capture noise.</p><p>Finally, <strong>underfitting</strong> is the opposite kind of problem. An underfitted model is too limited to learn the important relationships, so it performs poorly even on the examples used during training. You want neither extreme: not a model that misses the pattern, and not one that remembers every accident of the dataset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sEpjVqojGZPhWwfs9AJ5eA.png" /></figure><h3>Why a Perfect Training Score Can Be Bad News</h3><p>Suppose your online store has a few customers who bought one expensive product during a holiday sale and never returned. A complex model might decide that customers who purchase that exact product at that exact time always become inactive. That rule could fit the historical records perfectly, even though the coincidence may never repeat.</p><p>The model can make many rules like that. As its training accuracy rises, its performance on later customers may stop improving or start falling. Google’s Machine Learning Crash Course describes this using training and validation curves: the training loss may continue decreasing while validation loss begins to rise.</p><p>That’s why you shouldn’t select a model just because it scores 100% on its own training examples. The score that matters is how well its predictions hold up on data that didn’t influence its learning or selection.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uwQGMYlRl7IddJ-WW2F_-A.png" /></figure><h3>A Small Python Project: Watch a Decision Tree Overfit</h3><p>Let’s demonstrate this with a synthetic customer dataset. Instead of claiming these are real store records, we’ll generate example customers whose purchase frequency and days since their last order relate to inactivity, with a little randomness added.</p><p>We’ll compare a very deep decision tree with a smaller one. A tree is a good teaching example because you can control its complexity with max_depth.</p><pre>import numpy as np<br>from sklearn.datasets import make_classification<br>from sklearn.model_selection import train_test_split<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.metrics import accuracy_score<br># Synthetic customers: illustrative numbers, not real store data<br>X, y = make_classification(<br>    n_samples=500,<br>    n_features=6,<br>    n_informative=2,<br>    n_redundant=0,<br>    flip_y=0.12,<br>    random_state=42<br>)<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X, y, test_size=0.25, stratify=y, random_state=42<br>)<br>for depth in [None, 3]:<br>    model = DecisionTreeClassifier(<br>        max_depth=depth, random_state=42<br>    )<br>    model.fit(X_train, y_train)<br>    print(&quot;Depth:&quot;, depth)<br>    print(&quot;Training:&quot;, round(accuracy_score(<br>        y_train, model.predict(X_train)), 3))<br>    print(&quot;Testing: &quot;, round(accuracy_score(<br>        y_test, model.predict(X_test)), 3))</pre><p>With max_depth=None, the tree can grow until its other stopping rules apply. That often lets it fit the training data very closely. With max_depth=3, it must rely on broader splits, which can sometimes improve predictions on unseen data. Run the code and compare both scores rather than assuming one depth will always win.</p><p>There’s one important shortcut in this teaching example: we’re comparing two depths directly on a test split. In a real project, <strong>choose depth using a separate validation set or cross-validation</strong>, then reserve the untouched test set for the final evaluation. Otherwise, repeated experiments can overfit your choices to the test set itself.</p><p>The synthetic features in this program are abstract numerical inputs, not actual order counts. In a production customer project, you would replace them with carefully prepared historical features and label each customer using an outcome observed later.</p><h3>What Should You Change if Your Model Overfits?</h3><p>The first thing to try is usually a <strong>simpler model</strong>. For a decision tree, that could mean limiting its maximum depth or requiring more examples before creating a new leaf. The aim is to prevent the model from creating fragile rules for tiny groups of customers.</p><p>Another option is to improve the data. More representative examples can help, especially if the existing dataset is small, but more data is not a universal cure. Incorrect labels, accidental leakage from the future, and unrepresentative customers can all remain problems even in a large dataset.</p><p>Some algorithms support <strong>regularization</strong>, which discourages unnecessarily complicated solutions. Neural networks may also use <strong>early stopping</strong>, ending training when validation performance stops improving. These methods differ, but they share a practical goal: help the model learn patterns that survive beyond its training data.</p><p>You should also verify how the data was split. If a customer’s records appear in both training and testing in a way that reveals their outcome, or future purchase information sneaks into the features, the reported score may be misleading. And because our task predicts future customer behavior, an evaluation using later time periods can be more realistic than a random split.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rFZbLecCqS3rupc1IkpuUQ.png" /></figure><h3>How Do You Know the Fix Worked?</h3><p>Don’t judge the fix by whether training accuracy goes down. That may be exactly what should happen. A model that scores 94% on training data and 85% on appropriate validation data can be more useful than one that scores 100% on training data and only 70% on validation data. These numbers are illustrative, not a measured result.</p><p>For customer inactivity, accuracy alone isn’t enough. You also need to know how many truly inactive customers the model finds, and how many active customers it incorrectly flags. That’s where measures such as <strong>precision</strong> and <strong>recall</strong> come in. A smaller generalization gap is encouraging, but the actual business mistakes still matter.</p><p>Once you’ve chosen a model using validation results, evaluate it on the untouched test set. If that result is good enough for the decisions your store wants to make, you have stronger evidence that the model can handle new cases. It still needs monitoring after deployment because customers and shopping behavior change over time.</p><h3>The Lesson to Remember</h3><p>Overfitting is not a sign that the computer failed to learn. Often, it’s a sign that it learned <strong>too much about the particular examples</strong> and too little about what makes those examples representative.</p><p>When your next model gets a surprisingly high score, don’t celebrate the number immediately. Check the performance on truly unseen data, compare it with the training score, and ask whether your evaluation resembles the real situation where the model will be used.</p><p><strong>A model that gets a slightly lower score on familiar data but works reliably on new customers is usually the better model to keep.</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=146927558635" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/what-is-overfitting-why-ai-models-sometimes-fail-on-new-data-146927558635">What Is Overfitting? Why AI Models Sometimes Fail on New Data</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Training vs. Testing: How Do We Know an AI Model Really Works?]]></title>
            <link>https://grain.saltindex.com/training-vs-testing-how-do-we-know-an-ai-model-really-works-a2a499d66e2d?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/a2a499d66e2d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[model-evaluation]]></category>
            <category><![CDATA[ai-for-beginners]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 11:01:01 GMT</pubDate>
            <atom:updated>2026-11-20T18:28:26.740Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>An AI model can get almost every training example right and still fail on new data. Here’s how to tell the difference before trusting its predictions.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZrHJhqSjMM0kwRHzCyhVBQ.png" /></figure><p><em>Illustration: learning on one set of records and evaluating on another.</em></p><p>Imagine you build a machine learning model for your online store. Its job is to predict which customers might stop buying products during the next 90 days. You give it historical customer records, train it, and see a score of 98% accuracy.</p><p>That sounds like a success. But there’s one question you need to answer before using the predictions: <strong>Was that score measured on customers the model had already seen?</strong> If so, you still don’t know whether it can handle new customers.</p><p>That’s why machine learning separates <strong>training</strong> from <strong>testing</strong>. Training teaches a model patterns. Testing checks whether those patterns hold beyond the examples it learned from. The difference matters more than a high score on a dashboard.</p><h3>Training Data Is for Learning, Not Proving</h3><p>Think about preparing for an exam. You study a collection of questions and learn how to solve them. You might answer every practice question correctly by the end, but that doesn’t prove you’ll succeed when the exam gives you different questions.</p><p>A machine learning model faces the same challenge. During <strong>training</strong>, an algorithm examines examples and adjusts or constructs a model to capture relationships in them. In our online store, each example could include recent order count and days since the last purchase, along with a label showing whether that customer later became inactive.</p><p>The model might learn that people who haven’t purchased for a long time are more likely to stop shopping. That’s useful, but training performance alone cannot tell us whether the relationship will work for other customers. A model can fit the training data closely for reasons that don’t generalize.</p><p>The important keyword is <strong>generalization</strong>: performing well on examples the model did not learn from directly. That’s what testing is meant to investigate.</p><h3>Training, Validation, and Testing Have Different Jobs</h3><p>Beginners often hear about an 80/20 train-test split. That can be a useful first demonstration, but many projects benefit from three separate groups of data.</p><p><strong>Training data</strong> is used to fit the model. <strong>Validation data</strong> is used during development to choose between models, adjust settings, or decide when to stop training. <strong>Test data</strong> is kept untouched until you’re ready to evaluate the final approach.</p><p>For example, you might reserve 70% of a dataset for training, 15% for validation, and 15% for testing. Those percentages are only illustrative. The right split depends on how many examples you have and how the model will be used.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Lmx2xJJBdQJP8hdB7dPBRw.png" /></figure><p><em>Example percentages are illustrative; your project may require a different split.</em></p><p>Why not keep checking the test set while adjusting the model? Because eventually your decisions start adapting to those test results. The test set is no longer a truly independent check. Use validation results for iteration, then evaluate the final model on the held-out test set.</p><p>You may also encounter <strong>cross-validation</strong>, which evaluates a model across several training and validation splits. It’s useful when a single split would give a shaky estimate, but it still doesn’t remove the need for a genuinely separate final evaluation when one is required.</p><h3>Why 99% Training Accuracy Can Be Bad News</h3><p>Imagine two students. One understands the material. The other memorizes every answer in the workbook. Both score perfectly on the workbook, but only one handles unfamiliar questions confidently.</p><p>When a model learns details or noise that are too specific to its training examples, it’s called <strong>overfitting</strong>. You might see 99% training accuracy and only 72% test accuracy. Those figures are hypothetical, but the gap tells you to investigate whether the model learned patterns that don’t travel well.</p><p>The opposite is <strong>underfitting</strong>. A model may be so simple that it misses important relationships even in the training data. The aim isn’t the most complex model or the one with the highest training score. It’s the model that performs reliably on the kind of new data you’ll actually encounter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fbCs1mT5BKJy5qOdMtW3aQ.png" /></figure><p><em>Schematic comparison, not measured model results.</em></p><p>A useful habit is to compare training and validation results while developing a model. A growing gap can warn you about overfitting. But remember that a poor test score can also reflect a dataset that doesn’t represent real conditions, not only a model that’s too complicated.</p><h3>The Test Must Look Like Real Life</h3><p>Here’s a less obvious problem. Suppose your store’s customer data covers January through October, and you’re trying to predict inactivity in November and beyond. If you randomly mix future records into training and older records into testing, you could end up evaluating under conditions that don’t match how the model will be used.</p><p>For a future-facing prediction, a <strong>time-based split</strong> is often more meaningful: train on earlier periods and test on later ones. You also need enough time after each customer’s observation date to know whether they became inactive during the following 90 days.</p><p>Another danger is <strong>data leakage</strong>. Imagine the model uses the number of orders placed <em>after</em> the prediction date to predict whether the customer will stop buying. That future information might make test results look impressive, but it won’t be available when the model makes a genuine prediction.</p><p>Leakage can also happen during preprocessing. If you calculate statistics from the entire dataset before splitting it, information from the test set can influence training. The official scikit-learn guidance recommends fitting preprocessing steps on training data only, often by placing them in a pipeline.</p><p><strong>A trustworthy test is not simply data the model hasn’t seen. It’s data that fairly represents the decision you’re asking the model to make.</strong></p><h3>A Small Online-Store Project in Python</h3><p>Let’s make this concrete with a compact example. We’ll use a fictional dataset in which each customer has two features: orders in the last 30 days and days since the last purchase. The label is 1 if the customer later became inactive and 0 if they stayed active.</p><p>The following code creates illustrative customer records, separates them into training and testing sets, trains a decision tree, and checks predictions. It is deliberately small enough to follow, not a business-ready churn system.</p><pre>import numpy as np<br>from sklearn.model_selection import train_test_split<br>from sklearn.tree import DecisionTreeClassifier<br>from sklearn.metrics import classification_report<br># Fictional examples: [recent_orders, days_since_purchase]<br>X = np.array([<br>    [8, 3], [1, 28], [5, 7], [0, 45],<br>    [6, 5], [2, 35], [9, 2], [1, 42],<br>    [4, 12], [0, 60], [7, 4], [2, 31],<br>    [10, 1], [1, 50], [5, 9], [0, 39],<br>    [6, 8], [2, 47], [8, 6], [1, 33]<br>])<br><br># 0 = active, 1 = inactive<br># Labels are fabricated to demonstrate the workflow.<br>y = np.array([0, 1, 0, 1, 0, 1, 0, 1, 0, 1,<br>              0, 1, 0, 1, 0, 1, 0, 1, 0, 1])<br>X_train, X_test, y_train, y_test = train_test_split(<br>    X, y, test_size=0.25, stratify=y, random_state=42<br>)<br>model = DecisionTreeClassifier(max_depth=2, random_state=42)<br>model.fit(X_train, y_train)<br>predictions = model.predict(X_test)<br>print(classification_report(y_test, predictions,<br>                            zero_division=0))</pre><p>The split keeps some examples out of training. The stratify=y setting helps preserve the proportion of active and inactive examples in this small demonstration. Then .fit() learns from the training portion, while .predict() generates answers for customers held back for testing.</p><p>Because these labels were constructed to follow a simple pattern, the program may report a perfect test score on just five held-out examples. That is a property of this toy dataset, not evidence that a real customer model would perform perfectly.</p><p>Notice what we <strong>didn’t</strong> do: train on X_test or adjust the model repeatedly to chase its test score. Also, this random split is fine for illustrating the code, but a real churn project should be designed around time, customer identity, and when labels become known.</p><h3>Accuracy Is Not the Only Score That Matters</h3><p>Suppose your real online store has 1,000 customers and only 50 will become inactive. A model that predicts everyone will remain active gets 950 predictions right, giving it <strong>95% accuracy</strong>. But it finds none of the 50 customers your retention team wanted to help.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LFj3V1LiGRRv8zExbB1izA.png" /></figure><p><em>Illustrative scenario: 950 active customers, 50 inactive customers.</em></p><p>This is why you should also learn <strong>precision</strong> and <strong>recall</strong>. Precision asks, “Of the customers we flagged as likely to leave, how many actually did?” Recall asks, “Of all the customers who left, how many did we catch?” In the all-active example, recall for inactive customers is 0%.</p><p>The best metric depends on the consequences of being wrong. If contacting customers is expensive, false alarms matter. If losing an important customer is costly, missing people who are genuinely at risk may matter more. Compare the model with a simple baseline and choose metrics that reflect the decision the business will make.</p><p>Finally, even a good test score is not a lifetime guarantee. Customer behavior changes, sales campaigns come and go, and a model can become less reliable over time. Once deployed, it still needs monitoring against new outcomes.</p><h3>So, When Can You Trust the Result?</h3><p>You can’t prove an AI model will always be correct. You can, however, build convincing evidence that it works for a specific purpose under realistic conditions.</p><p>A sensible evaluation keeps model development separate from the final test, prevents future information from leaking into features, and measures the mistakes that matter to the user. For a customer-inactivity model, that means testing on genuinely later customer outcomes and checking how many at-risk customers it identifies without overwhelming the team with false alarms.</p><p><strong>The next time you see an AI model advertised with 99% accuracy, ask one question before anything else: 99% on what data?</strong> The answer tells you whether you’re looking at a model that learned its examples or one that has shown it can handle new ones.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a2a499d66e2d" width="1" height="1" alt=""><hr><p><a href="https://grain.saltindex.com/training-vs-testing-how-do-we-know-an-ai-model-really-works-a2a499d66e2d">Training vs. Testing: How Do We Know an AI Model Really Works?</a> was originally published in <a href="https://grain.saltindex.com/">Salt Index</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What Is a Dataset, and Why Does AI Need So Much Data?]]></title>
            <link>https://generativeai.pub/what-is-a-dataset-and-why-does-ai-need-so-much-data-0caa40037c1d?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/0caa40037c1d</guid>
            <category><![CDATA[dataset]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[ai-for-beginners]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Thu, 19 Nov 2026 07:01:02 GMT</pubDate>
            <atom:updated>2026-11-19T07:01:02.839Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>More data can help an AI model learn, but the quality, variety, and relevance of that data matter just as much as the number of examples.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lJdKF_gONAQ8SKIvuduW8w.png" /></figure><p>Imagine you’re building a system that recommends products in an online store. You have records from 100 customers and wonder whether that’s enough to train your first model.</p><p>You could start experimenting with those records. But whether they’re enough depends on what you want the model to predict, how different your customers are, and how accurate the results need to be.</p><p>That’s the central idea behind datasets. <strong>AI doesn’t need huge amounts of data simply because it’s AI. It needs enough useful examples to learn patterns that work beyond the examples it has already seen.</strong></p><p>Let’s understand the key terms first. Then we’ll use an online-store project to see how these ideas affect a real prediction system.</p><h3>First, What Exactly Is a Dataset?</h3><p>A <strong>dataset</strong> is an organized collection of information used to explore a problem, train a model, or test how well it works. It could be a spreadsheet of customer purchases, a folder of product images, or a collection of written reviews. The format changes, but the purpose stays the same: it gives the system examples from which it can learn or be evaluated.</p><p>Imagine a spreadsheet containing 1,000 customer records. Every row represents one customer’s activity during a particular period. One column might show the number of orders, another the days since the last purchase, and another whether the customer bought something in the following month.</p><p>A single record is called a <strong>sample</strong>, or sometimes an <strong>example</strong>. The information used to predict an outcome is called a <strong>feature</strong>. In supervised learning, the known outcome is the <strong>label</strong>. If we’re predicting future purchases, recent order count could be a feature and whether the customer purchased again could be the label.</p><p>Not every dataset needs labels. A business might also use <strong>unlabeled data</strong> to discover groups of customers with similar shopping behavior. The important point is that the dataset must suit the question you’re asking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HW1cdJGSAj1EOZWSjma_IA.png" /></figure><h3>What Do Training, Validation, and Test Data Mean?</h3><p>One dataset usually shouldn’t be used for every stage of a project. If a model learns from customer records and you then judge it using exactly those same records, good results could simply mean it has remembered the examples.</p><p><strong>Training data</strong> is used to fit the model. <strong>Validation data</strong> helps you compare approaches or adjust settings during development. <strong>Test data</strong> is held back for a final check on examples the finished model hasn’t trained on. This helps you measure <strong>generalization</strong>, meaning how well the model works on new situations.</p><p>For our online-store project, time matters too. A customer record from December shouldn’t be used to predict what happened in November. You might train with earlier months and evaluate with later months so your test better resembles the job the model will actually do.</p><p>A clean separation also helps prevent <strong>data leakage</strong>, where future information or information from the evaluation set improperly influences training.</p><h3>Why Do Some AI Models Need Millions of Examples?</h3><p>Imagine teaching a model to recognize cats in photographs. If you show it only ten pictures, all taken in bright sunlight, it may struggle with black cats, unusual camera angles, shadows, or partially hidden faces.</p><p>More varied examples can help it learn which visual patterns matter across different situations. The same principle applies to language models, speech recognition, and recommendation systems. Real-world information comes in many forms, and a model needs sufficient coverage of the patterns it will encounter.</p><p>Larger, more flexible models can learn complicated relationships, but they may also require substantial data to train effectively from scratch. That is one reason large general-purpose AI systems are often trained on very large collections of text, images, audio, or other material.</p><p>But <strong>there is no universal minimum dataset size</strong>. A small, clearly defined task might work with hundreds or thousands of suitable examples. A broad system expected to handle many languages and situations may need vastly more. Starting with a pretrained model can also reduce the amount of task-specific data needed.</p><h3>Why More Data Can Still Produce a Worse Model</h3><p>Suppose your store has 50,000 customer records, but most come from frequent buyers in one city. If you expand into new markets, that dataset might not represent your new customers very well. Adding another 50,000 nearly identical records doesn’t necessarily solve the problem.</p><p>This is where <strong>representativeness</strong> matters. Your examples should resemble the people, products, and situations the model will eventually encounter. <strong>Diversity</strong> matters too: a dataset should contain meaningful variations, not just repeated copies of the same pattern.</p><p>You also need to consider <strong>data quality</strong>. Missing purchase dates, incorrect labels, duplicate records, and inconsistent formats can teach the model the wrong relationships. Sometimes correcting those problems improves performance more than collecting more examples.</p><p>There’s also <strong>class imbalance</strong>. Imagine only 2% of customers become inactive. A model that predicts everyone will stay active could appear 98% accurate while identifying none of the customers your business cares about. Your dataset and evaluation should account for this imbalance.</p><p>In short, the goal isn’t to collect the largest dataset possible. It’s to collect enough accurate, relevant, varied examples to answer your question reliably.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2pJ1zrFsdPujgz-GYvEHDA.png" /></figure><h3>How Does the Model Use Those Examples?</h3><p>During supervised training, the model receives features and labels. It makes predictions, compares them with known outcomes, and its training algorithm adjusts what the model has learned. Different algorithms do this in different ways.</p><p>The model doesn’t store a perfect rule for every possible customer. It tries to learn relationships that are useful for making predictions about customers it hasn’t seen before. That is why both the examples and the evaluation process matter.</p><h3>A Practical Project: Predicting Which Customers Might Stop Buying</h3><p>Let’s make this concrete. Imagine your online store wants to predict whether customers will become inactive over the next 90 days. You have historical examples showing recent order activity and what happened afterward.</p><p>First, define the label. We’ll use 1 when a customer became inactive and 0 when they stayed active. Next, choose features available at prediction time, such as orders in the previous 30 days and days since the last order.</p><p>Here is a deliberately tiny dataset that demonstrates the structure. The values are fictional and aren’t sufficient for a real business prediction system.</p><pre>import pandas as pd<br><br>customers = pd.DataFrame({<br>    &quot;orders_30d&quot;: [8, 1, 5, 0, 6, 2],<br>    &quot;days_since_order&quot;: [3, 28, 7, 45, 5, 35],<br>    &quot;inactive_next_90d&quot;: [0, 1, 0, 1, 0, 1]<br>})<br>X = customers[[&quot;orders_30d&quot;, &quot;days_since_order&quot;]]<br>y = customers[&quot;inactive_next_90d&quot;]<br>print(&quot;Examples:&quot;, len(customers))<br>print(&quot;Features:&quot;, list(X.columns))<br>print(&quot;Labels:&quot;, y.tolist())</pre><p>X contains the features, and y contains the labels. Each row is one customer example. With a larger, properly prepared historical dataset, you could use those inputs to train a classifier and evaluate it on later customer records.</p><p>But before training, ask whether your data includes customers who shop rarely, frequent buyers, seasonal shoppers, and people from every market you intend to serve. If all six examples came from one type of customer, adding thousands of similar examples might still leave important gaps.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SziDFBMkUJH09KkuHSm4KA.png" /></figure><h3>How Much Data Should You Collect for Your First Project?</h3><p>Rather than starting with an arbitrary target like one million rows, start with the decision your model needs to support. Define the outcome, inspect the records you already have, and build a simple baseline. Then evaluate on genuinely unseen, representative examples.</p><p>If your model struggles, examine where it fails. Are there too few examples of inactive customers? Are some customer groups missing? Are labels incorrect? Is the problem itself difficult to predict from the information you’ve collected? The answers tell you whether to collect more data, improve existing data, change features, or reconsider the task.</p><p>When using customer records, keep privacy and permission in mind as well. Having access to information doesn’t automatically make every use of it appropriate. Collect only what the project needs and handle it responsibly.</p><h3>The Takeaway</h3><p>A dataset is more than a spreadsheet or a folder of files. It’s the collection of examples that shapes what a model can learn and what it may get wrong.</p><p>More data can help, especially when it brings useful variety. But inaccurate, repetitive, or unrepresentative data can leave you with a confident model that still makes poor predictions.</p><p><strong>If you’re starting a Machine Learning project, don’t ask only, “How much data do I have?” Ask, “Does this data represent the problem I want to solve?”</strong> That question is a better starting point for building something useful.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=0caa40037c1d" width="1" height="1" alt=""><hr><p><a href="https://generativeai.pub/what-is-a-dataset-and-why-does-ai-need-so-much-data-0caa40037c1d">What Is a Dataset, and Why Does AI Need So Much Data?</a> was originally published in <a href="https://generativeai.pub/">Generative AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How Does a Machine Actually Learn From Data?]]></title>
            <link>https://ai.plainenglish.io/how-does-a-machine-actually-learn-from-data-bc9f37b0fef0?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/bc9f37b0fef0</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 19:01:01 GMT</pubDate>
            <atom:updated>2026-11-18T19:01:01.763Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A beginner-friendly explanation of how models discover patterns, learn from mistakes, and make predictions, with a small online-store project you can try in Python.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N-Y3lTj1MyVs-xBr1LWaJA.png" /></figure><p>Imagine you run an online store and want to know which customers might stop buying. You have records of their past orders, but no obvious formula that tells you who will return next month. You could try writing rules yourself, such as flagging anyone who hasn’t ordered in 30 days. The trouble is that real customers don’t all behave the same way.</p><p>Machine Learning offers another approach. Instead of writing every rule, you give a computer historical examples and let a training algorithm find relationships in them. But what does it mean to say the computer <em>learns</em>? Does it remember each customer, discover rules, or somehow understand shopping habits?</p><p><strong>The useful answer is that training changes a model’s internal parameters so its predictions better match patterns in the examples.</strong> That is different from human understanding, and it doesn’t guarantee the predictions will work on new customers. Knowing the difference helps you decide when a model is worth trusting.</p><h3>First, Understand What “Learning” Really Means</h3><p>A traditional program follows instructions that a developer writes. If a shopping cart contains three products, you can tell the program to add their prices and calculate a total. The rules are already known.</p><p>Machine Learning becomes helpful when the rules are difficult to write precisely. A customer’s likelihood of buying again might depend on their purchase frequency, the time since their last order, and other signals that interact in complicated ways. Rather than specify every possible combination, we can show a model many examples of past customer behavior.</p><p>In <strong>supervised learning</strong>, each training example contains <strong>features</strong> and a <strong>label</strong>. Features are the information available to make the prediction. A label is the historical answer we want the model to learn to predict. For our store, the features might be the number of orders in the last 30 days and days since the last purchase. The label might indicate whether the customer made no purchases during the following 90 days.</p><p>An <strong>algorithm</strong> is the procedure used to learn from examples. The resulting <strong>model</strong> is the learned system that makes predictions. Different algorithms learn in different ways: some fit numerical coefficients, while decision trees learn a sequence of splits. They aren’t all secretly doing the same calculations.</p><h3>How a Model Learns From Mistakes</h3><p>Let’s use a particularly simple model to see the mechanism. Imagine predicting how much a customer will spend based on their number of orders. A straight-line model might calculate a prediction like this:</p><p>predicted spending = weight × number of orders + bias</p><p>The <strong>weight</strong> determines how much the prediction changes when the number of orders changes. The <strong>bias</strong> shifts the prediction up or down. At first, the values might not match the data very well. For instance, with a weight of 10 and a bias of 0, the model predicts $30 for three orders, even if the training example says $60.</p><p>The training process needs a way to judge that mismatch. A <strong>loss function</strong> measures prediction errors. For a numerical prediction, one possible loss is the squared difference between the predicted and actual value. An <strong>optimizer</strong> can then adjust the weight and bias to reduce the loss across the training examples.</p><p>Repeated adjustment is why people describe model training as learning from mistakes. The model isn’t feeling that it made a mistake. Its parameters are being changed by a numerical procedure. In gradient-based training, those changes are guided by how the loss responds to changes in the parameters. Other algorithms, including many decision-tree methods, use different procedures.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Pz_-sco_rBKeXtwJA8EpDA.png" /></figure><p>It’s also important to separate <strong>training</strong> from <strong>inference</strong>. Training is the process of learning parameters or decision rules from examples. Inference happens after that, when you give the trained model new inputs and ask it for a prediction. Most prediction requests do not automatically retrain the model.</p><h3>Why More Training Doesn’t Always Mean Better Learning</h3><p>Suppose you evaluate your model using the same customers it saw during training. It may perform beautifully because it has learned details specific to those records. That doesn’t prove it can predict the behavior of someone new.</p><p>This problem is called <strong>overfitting</strong>. The model fits the training data too closely, including quirks that don’t repeat in the real world. <strong>Underfitting</strong> is the opposite: the model is too limited to capture important patterns in the first place.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jAXSHJlQbqRiPeLxK-ECZg.png" /></figure><p>What you actually want is <strong>generalization</strong>, the ability to make useful predictions on examples the model has not seen. That’s why practitioners reserve data for evaluation. With customer behavior, an evaluation using later time periods can be more realistic than randomly mixing past and future records. You must also avoid <strong>data leakage</strong>, such as using a purchase made next month as an input for a prediction supposedly made today.</p><p>The practical lesson is simple: a model hasn’t demonstrated useful learning just because its training score looks good. It needs to perform on carefully separated examples that resemble how you’ll use it.</p><h3>A Small Project: Predicting Which Customers Might Leave</h3><p>Let’s make this concrete with a deliberately tiny example. We’ll label a customer <strong>inactive</strong> if they make no purchase during the next 90 days. The input features are the number of orders in the preceding 30 days and the number of days since their last purchase. In a real project, both would be calculated at a fixed prediction date.</p><p>We’ll train a <strong>decision tree classifier</strong>. A decision tree does not use the same weight-and-bias optimization we described earlier. It learns decision splits from labeled examples. That makes it a useful way to see how different algorithms can learn relationships without being manually given every rule.</p><pre>import numpy as np<br>from sklearn.tree import DecisionTreeClassifier<br># Each row: [orders in last 30 days, days since last order]<br>X = np.array([<br>    [8, 3], [1, 28], [5, 7],<br>    [0, 45], [6, 5], [2, 35]<br>])<br># 0 = stayed active, 1 = became inactive<br># during the following 90 days<br>y = np.array([0, 1, 0, 1, 0, 1])<br>model = DecisionTreeClassifier(max_depth=2, random_state=42)<br>model.fit(X, y)<br>new_customer = [[1, 40]]<br>prediction = model.predict(new_customer)[0]<br>print(&quot;Inactive&quot; if prediction == 1 else &quot;Active&quot;)</pre><p>The six records are fictional, chosen only to make the code easy to follow. The .fit() call learns splits that separate the two labels in this dataset. The .predict() call applies the learned tree to a customer with one recent order and 40 days since their last order. You can change those two inputs and see how the prediction responds.</p><p>Notice what you didn’t do: you never wrote a rule saying <em>if days since the last purchase exceed a certain number, predict inactivity</em>. The algorithm selected splits by examining the labeled examples. That’s the essential difference between supplying a rule and supplying data for a model to learn from.</p><p><strong>But this isn’t a deployable customer-retention system.</strong> Six fictional records cannot establish whether the model will predict real behavior. A serious project needs enough representative historical data, a carefully designed time-based evaluation, and checks for misleading features or missing information.</p><h3>How Do You Know the Model Has Learned Something Useful?</h3><p>For an actual store, you would evaluate the model on customers whose outcomes were not used to train it. You’d also compare it with a simple baseline, such as always predicting that a customer will remain active. If the trained model can’t beat that baseline on the business metric you care about, the extra complexity may not be justified.</p><p><strong>Accuracy</strong> counts the proportion of predictions that are correct, but it can hide important mistakes. If almost every customer remains active, a model that predicts “active” for everyone could score highly while identifying nobody who might leave. <strong>Recall</strong> tells you what fraction of truly inactive customers you found. <strong>Precision</strong> tells you what fraction of flagged customers really became inactive.</p><p>Which metric matters more depends on what your store will do with the predictions. If you plan to offer expensive discounts to customers flagged as at risk, false alarms cost money. If missing customers is your bigger concern, recall may deserve more attention. This is how understanding the learning process changes a practical decision: it tells you what to test before relying on the output.</p><p>And even a useful model can become outdated. Promotions, subscriptions, pricing changes, or new markets can change customer behavior. You may need to monitor the model and train a new version on newer labeled examples. That is another kind of improvement over time, but it requires an intentional process, not an automatic promise that every prediction makes the model smarter.</p><h3>The Real Answer</h3><p>A machine learns from data when a training algorithm uses examples to create or adjust a model that captures useful relationships. Depending on the algorithm, that might mean tuning numerical parameters or learning a set of decision rules. After training, the model applies what it learned to new inputs.</p><p>The harder question isn’t whether a computer can fit a pattern. It’s whether the pattern remains useful outside the data it learned from.</p><p><strong>If you’re building your first model, don’t start by chasing the most advanced algorithm. Start with a clear prediction, a small understandable dataset, and a fair test on unseen examples.</strong> Once you can explain what your model learned and how you checked it, the bigger Machine Learning ideas become much easier to understand.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bc9f37b0fef0" width="1" height="1" alt=""><hr><p><a href="https://ai.plainenglish.io/how-does-a-machine-actually-learn-from-data-bc9f37b0fef0">How Does a Machine Actually Learn From Data?</a> was originally published in <a href="https://ai.plainenglish.io/">Artificial Intelligence in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The 10 Concepts You Should Understand Before Learning Machine Learning]]></title>
            <link>https://medium.com/ai-engineering-simplified/the-10-concepts-you-should-understand-before-learning-machine-learning-bcc0f734ab2d?source=rss-089e52820324------2</link>
            <guid isPermaLink="false">https://medium.com/p/bcc0f734ab2d</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[technology]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[ai-for-beginners]]></category>
            <dc:creator><![CDATA[Thonix]]></dc:creator>
            <pubDate>Wed, 18 Nov 2026 11:01:01 GMT</pubDate>
            <atom:updated>2026-11-18T11:01:01.832Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Before training your first model, understand these 10 essential ideas, why they matter, and how they work together in a real project.</em></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1PU1XIsIeO3CShdzxrJbdQ.png" /></figure><p>Imagine you want to learn Machine Learning. You open a Python tutorial, copy a few lines of code, and train your first model. A few seconds later, your computer produces a prediction.</p><p>That seems surprisingly easy.</p><p>But then the tutorial starts mentioning datasets, features, labels, training, overfitting, and accuracy. You can run the code, but you’re not entirely sure what the computer is doing or why those terms matter.</p><p>This is where many beginners get stuck. They start learning algorithms before understanding the ideas that make those algorithms useful.</p><p><strong>You don’t need advanced mathematics or complicated neural networks to get started. But you should understand what a model learns, what information it uses, and how to tell whether its predictions are reliable.</strong></p><p>Let’s start with 10 essential concepts. Then we’ll use them together to build a small Machine Learning project for an online store.</p><h3>First, Understand These 10 Machine Learning Concepts</h3><h3>1. Problem Definition: What Do You Want the Computer to Predict?</h3><p>Before building a Machine Learning model, you need to decide exactly what problem you’re trying to solve.</p><p>Imagine you own an online store and want to identify customers who might stop buying products. That’s a reasonable business goal, but it isn’t specific enough for a model. Does a customer count as inactive after 30 days without a purchase, or 90 days? What if they normally shop only twice a year?</p><p>You need to define the outcome clearly. For example, you could ask the model to predict whether a customer will make no purchases during the next 90 days.</p><p>This brings us to two important keywords: <strong>Classification</strong> and <strong>Regression</strong>.</p><p>Classification means predicting a category, such as whether an email is spam, whether a transaction is fraudulent, or whether a customer will become inactive.</p><p>Regression means predicting a numerical value, such as a house price, next month’s sales, or how much a customer might spend.</p><p>The difference matters because your prediction target influences the data you need, the algorithms you might choose, and how you measure performance.</p><p>Before worrying about which model to use, make sure you know what answer you want it to produce.</p><h3>2. Dataset: What Does the Computer Learn From?</h3><p>Machine Learning needs examples to learn patterns. A collection of those examples is called a <strong>Dataset</strong>.</p><p>Suppose your online store has customer records containing purchase history, spending, and account activity. You could organize that information into a dataset where each row represents a customer and each column describes something about that customer.</p><p>One important keyword is <strong>Sample</strong>. A sample is an individual example in a dataset. In our case, one customer’s record is one sample.</p><p>Another is <strong>Data Quality</strong>, which describes how accurate, relevant, consistent, and suitable the information is for your task.</p><p>Imagine your store has 100,000 customer records, but thousands contain incorrect purchase dates. A model trained on those records might learn relationships that don’t reflect actual customer behavior.</p><p>That’s why having more data doesn’t automatically mean having better data.</p><p>A smaller dataset with accurate, representative examples can sometimes be more useful than a larger dataset full of errors. What matters is whether the information helps the model learn patterns that remain useful outside the training examples.</p><h3>3. Features and Labels: What Goes In and What Comes Out?</h3><p>Two of the most important Machine Learning terms are <strong>Features</strong> and <strong>Labels</strong>.</p><p>Features are the pieces of information the model uses to make predictions. Labels are the correct answers provided during supervised training.</p><p>Let’s return to our online store. If you’re predicting whether a customer will stop purchasing, useful features might include the number of previous orders, days since their last purchase, and total spending.</p><p>The label would tell you whether that customer actually became inactive during the following 90 days.</p><p>During training, the model receives the features together with their historical labels. It learns relationships between customer behavior and the outcome. Later, when a new customer arrives, the model receives their features and tries to predict an outcome that isn’t yet known.</p><p>There’s another important keyword here: <strong>Data Leakage</strong>.</p><p>Data leakage happens when information that wouldn’t be available at the time of prediction improperly enters the training or evaluation process.</p><p>For example, imagine using a customer’s purchases from next month to predict whether they will become inactive next month. The model is effectively receiving information from the future.</p><p>It might achieve excellent accuracy during testing, but that result won’t represent what it can do in a real application.</p><p><strong>The important rule is simple: only use information that would genuinely be available when making the prediction.</strong></p><h3>4. Data Preprocessing: Why Can’t You Use Raw Data Immediately?</h3><p>Real-world datasets are rarely ready for training.</p><p>Customer records might contain missing values, duplicate entries, inconsistent dates, or product categories written in different formats. One system might record a date as 2026-10-01, while another uses 01/10/2026.</p><p>Before training, you need to decide how to handle those differences. This process is called <strong>Data Preprocessing</strong>.</p><p>Preprocessing can involve correcting errors, handling missing values, converting categories into numerical representations, and scaling numerical features when an algorithm requires it.</p><p>Another useful keyword is <strong>Feature Engineering</strong>.</p><p>Feature engineering means creating useful inputs from information you already have. For example, instead of giving the model a customer’s last purchase date, you could calculate the number of days since that purchase.</p><p>That new feature may be more meaningful when predicting customer inactivity.</p><p>This is also why Machine Learning isn’t just about algorithms. A well-prepared dataset with useful features can make a simple model perform better than a complicated model trained on poorly prepared information.</p><h3>5. Training, Validation, and Test Data: How Do You Know the Model Works?</h3><p>Imagine studying for an exam by memorizing 100 practice questions. Then, during the real exam, you receive exactly those same questions and score 100%.</p><p>Does that prove you understand the subject?</p><p>Not necessarily. You might simply remember the answers.</p><p>Machine Learning can have the same problem. A model might perform extremely well on data it has already seen but struggle when given new examples.</p><p>That’s why developers separate data into different sets.</p><p><strong>Training Data</strong> is used to fit the model and learn patterns. <strong>Validation Data</strong> is used during development to compare models and adjust settings. <strong>Test Data</strong> is held back to evaluate the finished model on unseen examples.</p><p>This introduces another important keyword: <strong>Generalization</strong>.</p><p>Generalization describes how well a trained model performs on new data, rather than only the examples used during training.</p><p>For our customer prediction project, we also need to think about time. If we’re predicting future customer behavior, training on older records and evaluating on later records may provide a more realistic test than randomly mixing everything together.</p><p>The goal isn’t to build a model that remembers historical customers. It’s to build one that can make useful predictions about future customers.</p><h3>6. Model and Algorithm: What Is Actually Learning?</h3><p>Beginners often use the words <em>algorithm</em> and <em>model</em> interchangeably, but they describe different things.</p><p>An <strong>Algorithm</strong> is a procedure used to perform a task, such as learning patterns from examples.</p><p>A <strong>Model</strong> is the resulting system that uses learned parameters or patterns to make predictions.</p><p>Imagine using a decision tree algorithm to study historical customer records. During training, the algorithm examines relationships between customer behavior and whether customers became inactive.</p><p>The resulting decision tree is the model.</p><p>Once trained, you can give it information about a new customer and ask it to predict whether that customer might leave.</p><p>This introduces another important term: <strong>Inference</strong>.</p><p>Training is when the model learns from examples. Inference is when the trained model uses what it learned to produce a prediction.</p><p>The model doesn’t necessarily retrain itself every time you use it.</p><p>Understanding this distinction becomes increasingly important when working with larger models, where training can be expensive but inference may be performed thousands or millions of times.</p><h3>7. Loss Function: How Does the Model Know It’s Wrong?</h3><p>Suppose you’re building a model to predict customer spending. It predicts that someone will spend $100, but the actual amount is $70.</p><p>The prediction was wrong by $30.</p><p>A training algorithm needs a way to measure errors like this. That’s where a <strong>Loss Function</strong> comes in.</p><p>A loss function assigns a numerical penalty based on how poorly the model’s predictions match the expected outcomes.</p><p>Different problems use different loss functions. A model predicting spending might use mean squared error, while a classifier estimating the probability that a customer becomes inactive might use log loss.</p><p>Another important term is <strong>Optimization</strong>.</p><p>Optimization means finding parameters or settings that improve a mathematical objective, often by reducing loss.</p><p>Many neural networks use an optimization method called gradient descent. It helps adjust model parameters based on how changes affect the loss. Other algorithms, including many decision tree methods, use different training procedures.</p><p>You don’t need to understand every mathematical detail immediately. The important idea is that training needs some way to determine whether a model is improving.</p><p>Without understanding what the training process is optimizing, it becomes difficult to explain why changing an algorithm or setting affects the result.</p><h3>8. Overfitting and Underfitting: Why Does a Model Fail?</h3><p>Imagine your model achieves 99% accuracy on training data but only 70% on unseen test data.</p><p>Those are hypothetical numbers, but they illustrate an important problem.</p><p>The model may have learned details that are useful for remembering its training examples but don’t represent reliable patterns for new customers.</p><p>This is called <strong>Overfitting</strong>.</p><p>Think of a student who memorizes practice questions but struggles when the real exam asks something slightly different.</p><p>The opposite problem is <strong>Underfitting</strong>. This happens when a model is too limited or insufficiently trained to capture important relationships, so it performs poorly even on familiar training examples.</p><p>There’s another useful term: <strong>Generalization Gap</strong>.</p><p>This describes the difference between a model’s performance on training data and its performance on unseen evaluation data. A large gap can be a warning sign, although it doesn’t tell you the exact cause.</p><p>The goal is to find a model that learns meaningful patterns without becoming too dependent on the details of its training dataset.</p><p>Sometimes that means simplifying the model. Other times, it means improving the data or changing how training is performed.</p><h3>9. Evaluation Metrics: Does High Accuracy Mean a Good Model?</h3><p>Imagine a bank processes 10,000 transactions, and only 100 are fraudulent.</p><p>Now suppose a Machine Learning model predicts that every transaction is legitimate.</p><p>It correctly classifies 9,900 transactions.</p><p>That’s 99% accuracy.</p><p>Sounds impressive, right?</p><p>Except the model didn’t catch a single fraudulent transaction.</p><p>This is why <strong>Accuracy</strong> alone can be misleading.</p><p>Accuracy measures the proportion of predictions that were correct. But depending on the problem, you may care much more about particular types of mistakes.</p><p>Two other important metrics are <strong>Precision</strong> and <strong>Recall</strong>.</p><p>Precision tells you how many cases predicted as positive were actually positive. Recall tells you how many actual positive cases the model successfully identified.</p><p>The <strong>F1 Score</strong> combines precision and recall using their harmonic mean, providing another way to compare classification performance.</p><p>For fraud detection, missing fraudulent transactions may be especially costly. In another application, false alarms might be the bigger concern.</p><p>The best metric depends on the consequences of the model’s mistakes.</p><p><strong>A model with impressive accuracy isn’t necessarily a useful model.</strong></p><h3>10. Data Drift and Concept Drift: Why Can a Good Model Become Bad?</h3><p>Suppose your customer prediction model works well when you first deploy it.</p><p>Six months later, your online store changes its pricing, launches subscriptions, and starts selling in several new countries.</p><p>Customer behavior begins to change.</p><p>Someone who previously purchased every month might now receive products automatically through a subscription. Customers in new markets may also have different buying habits.</p><p>Your model was trained on historical data representing the old business. Its predictions may become less reliable even though the code hasn’t changed.</p><p>This introduces two important concepts.</p><p><strong>Data Drift</strong> happens when the distribution of the model’s input data changes.</p><p><strong>Concept Drift</strong> happens when the relationship between the inputs and the outcome changes.</p><p>For example, the relationship between purchase frequency and customer inactivity may change after the store introduces subscriptions.</p><p>This is why production Machine Learning systems need monitoring. Developers may need to collect newer examples, evaluate performance, investigate changes, and retrain models when appropriate.</p><p>Training a model is not necessarily the end of the project.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ArP7tFPxUO2EHkb7hqaipQ.png" /></figure><h3>Now Let’s Use These Concepts in a Real Project</h3><p>We’ve covered the important terms, but knowing their definitions isn’t the same as understanding how they work together.</p><p>Let’s return to our online store.</p><p>Imagine you have 10,000 customers and want to identify which ones are likely to stop purchasing so your team can decide whether to contact them.</p><p>We’ll build a simplified version of that project and connect it to the concepts we just learned.</p><h3>Step 1: Define the Problem</h3><p>First, decide exactly what the model should predict.</p><p>Suppose we define an inactive customer as someone who makes no purchases during the next 90 days. The model will predict one of two categories: <strong>0 for Active</strong> and <strong>1 for Inactive</strong>.</p><p>That makes this a classification problem.</p><p>Notice that we haven’t chosen an algorithm yet. We first defined the outcome, because everything else depends on knowing what we’re trying to predict.</p><h3>Step 2: Prepare the Dataset</h3><p>Now imagine collecting historical customer records. For each customer, we know how many orders they placed during the previous 30 days, how many days have passed since their last purchase, and whether they became inactive during the following 90 days.</p><p>Let’s use four fictional customers.</p><p>Customer A placed eight orders recently and last purchased three days ago. They remained active. Customer B placed one order and last purchased 28 days ago. They became inactive.</p><p>Customer C placed five orders and last purchased seven days ago. They remained active. Customer D placed no orders during the previous 30 days and last purchased 45 days ago. They became inactive.</p><p>In this example, the order count and days since the last purchase are our features. Whether the customer became inactive is the label.</p><p>Before training, we’d check whether the records are accurate, whether values are missing, and whether every feature would have been available at the time of prediction.</p><p>That’s where data quality, preprocessing, and data leakage become relevant.</p><h3>Step 3: Train Your First Model</h3><p>For this demonstration, we’ll use a decision tree.</p><p>A decision tree is a Machine Learning model that learns how to divide examples into groups based on their characteristics. For instance, it might discover that customers who haven’t purchased recently are more likely to become inactive.</p><p>Let’s see what that looks like in Python.</p><pre>import numpy as np<br>from sklearn.tree import DecisionTreeClassifier</pre><pre># Features:<br># [orders_last_30_days, days_since_last_order]<br>X = np.array([<br>    [8, 3],<br>    [1, 28],<br>    [5, 7],<br>    [0, 45],<br>    [6, 5],<br>    [2, 35]<br>])</pre><pre># Labels: 0 = active, 1 = inactive<br>y = np.array([0, 1, 0, 1, 0, 1])</pre><pre>model = DecisionTreeClassifier(<br>    max_depth=2,<br>    random_state=42<br>)</pre><pre>model.fit(X, y)</pre><pre># Predict for a new customer<br>prediction = model.predict([[1, 40]])</pre><pre>print(&quot;Predicted class:&quot;, prediction[0])</pre><p>The code creates a small dataset with features and labels, selects a decision tree algorithm, and trains the model using .fit().</p><p>Then .predict() uses the trained model to estimate the outcome for a new customer who placed one order during the previous 30 days and last purchased 40 days ago.</p><p>This is inference.</p><p>The example uses only six fictional records, so it demonstrates the process rather than a reliable business model. A real customer prediction system would require many more representative examples and careful testing.</p><h3>Step 4: Evaluate the Model Before Trusting It</h3><p>Now imagine you have thousands of genuine historical customer records.</p><p>You could use older records to train the model and later records to evaluate how well it predicts future customer behavior. You would also keep appropriate data for model selection and final testing.</p><p>Suppose 500 customers actually became inactive during the evaluation period, and your model successfully identified 400 of them.</p><p>Its recall would be:</p><p><strong>400 ÷ 500 = 80%</strong></p><p>That means it identified 80% of the customers who actually became inactive.</p><p>But there’s still something missing.</p><p>How many active customers did it incorrectly predict would leave?</p><p>If the model flags thousands of active customers, your retention team might waste considerable time contacting people who weren’t at risk.</p><p>That’s why we’d also examine precision, compare performance with a simple baseline, and consider the cost of false alarms and missed customers.</p><p>A model shouldn’t be judged by one attractive number.</p><h3>Step 5: Check for Overfitting and Future Changes</h3><p>Suppose your decision tree performs extremely well on training records but struggles when predicting outcomes for newer customers.</p><p>One possibility is overfitting. Another is that the newer customers behave differently from the people represented in the training dataset.</p><p>You’d need to investigate rather than immediately choosing a more complicated model.</p><p>Perhaps the features aren’t useful enough. Maybe the data contains errors, or the model is learning patterns specific to one period. You could compare it with a simpler baseline, adjust the training process, and evaluate again.</p><p>Now imagine the model is deployed successfully.</p><p>A few months later, your store introduces subscriptions. Customers begin receiving products automatically, changing the relationship between purchase frequency and inactivity.</p><p>The model may become less reliable.</p><p>That’s where monitoring, data drift, concept drift, and retraining become important.</p><p><strong>A model that works today still needs to prove it works tomorrow.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AqtcRWgpXs7_uysfMPhatg.png" /></figure><h3>How All 10 Concepts Connect</h3><p>Look at what happened throughout our customer prediction project.</p><p>We began by defining the problem and identifying the outcome we wanted to predict. Then we collected historical data, selected features and labels, and considered how to prepare the information correctly.</p><p>Next, we used a training algorithm to build a model. We discussed how training differs from inference and why the training procedure needs a way to improve its predictions.</p><p>We also learned why evaluation must use unseen examples, how overfitting can make a model unreliable, and why accuracy isn’t always the right measure of success.</p><p>Finally, we considered how changes in customer behavior can reduce performance after deployment.</p><p>These aren’t unrelated technical terms.</p><p>They’re different parts of the same process, and each one helps answer a question that matters when building a real Machine Learning system.</p><h3>The Most Important Lesson</h3><p>Machine Learning isn’t simply choosing an algorithm, writing Python code, and waiting for a prediction.</p><p>It’s understanding what you’re trying to predict, whether your data represents the problem, how the model learns, and whether its predictions remain useful beyond the training examples.</p><p>You can train a model in minutes. But knowing whether that model should be trusted requires understanding the process around it.</p><p><strong>Learn the concepts first. Then build a small project that gives each one a purpose.</strong></p><p>That’s a much stronger foundation than memorizing algorithms without understanding what problems they solve.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=bcc0f734ab2d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/ai-engineering-simplified/the-10-concepts-you-should-understand-before-learning-machine-learning-bcc0f734ab2d">The 10 Concepts You Should Understand Before Learning Machine Learning</a> was originally published in <a href="https://medium.com/ai-engineering-simplified">AI Engineering Simplified</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>