<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Lalana Gurusinghe on Medium]]></title>
        <description><![CDATA[Stories by Lalana Gurusinghe on Medium]]></description>
        <link>https://medium.com/@lalanagurusinghe?source=rss-4d3fe492a85f------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*XW9uQKB_vT-6KlNkH2Ojyw.png</url>
            <title>Stories by Lalana Gurusinghe on Medium</title>
            <link>https://medium.com/@lalanagurusinghe?source=rss-4d3fe492a85f------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 10:10:13 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@lalanagurusinghe/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Matrices: the Machines That Move Qubits]]></title>
            <link>https://medium.com/@lalanagurusinghe/matrices-the-machines-that-move-qubits-430512ccb8a9?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/430512ccb8a9</guid>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[qiskit]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Thu, 05 Nov 2026 11:50:38 GMT</pubDate>
            <atom:updated>2026-11-05T11:50:38.542Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>Subtitle:</strong><br> <strong>Quantum Computing &amp; QML, Part 3: gates, observables, eigenvalues and the tensor product, explained as machines that bend arrows</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*122N3bHdxKKpa2IgFvy-dQ.png" /></figure><p>In Part 2 we learned that a quantum state is a vector: an arrow of length 1. But an arrow that just sits there doesn’t compute anything. Something has to <strong>move</strong> it: turn |0⟩ into a superposition, flip it, rotate it by an angle you choose.</p><p>That something is a matrix. Every quantum gate you’ll ever use is a matrix. So is every quantity you can measure. Even the “results” a measurement can return are hiding inside a matrix as its eigenvalues. If vectors are the nouns of quantum computing, matrices are the verbs.</p><p>This is Part 3 of my series following IBM’s <strong>Qiskit Global Summer School 2021 on Quantum Machine Learning</strong>. Part 1 covered complex numbers, Part 2 covered vectors and the inner product. This post covers <strong>matrices</strong>, the last big piece of linear algebra before we meet the qubit properly.</p><p>A quick note on notation, because Medium can’t typeset matrices: I’ll write a 2×2 matrix row by row, with a semicolon between rows. So <strong>[a b; c d]</strong> means top row a, b and bottom row c, d. Column vectors are written like (x, y).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k4noVsy-ne-0A-5Dva4kXQ.png" /></figure><p><em>Caption: Six of the seven tools are now covered. Only probability is left.</em></p><h4>A matrix is a machine that moves arrows</h4><p>If vectors are arrows, a matrix is a <strong>machine that takes an arrow in and gives you a new arrow out</strong>. It might rotate it, stretch it, flip it or squash it. A Photoshop filter works like this: you feed in an image (a big vector of pixel values) and get a new image out.</p><p>In quantum computing, matrices show up in three roles:</p><ul><li>Every <strong>gate</strong> is a matrix, a special kind called <em>unitary</em> that only rotates and never stretches.</li><li>Every <strong>measurable quantity</strong> (energy, spin, “is it 0 or 1?”) is a matrix, a special kind called <em>Hermitian</em>.</li><li>Every <strong>noisy state</strong> is a matrix too, called the <em>density matrix</em>. That one gets its own post later.</li></ul><p>And in machine learning you already use matrices everywhere: the weights of a neural network layer, the covariance of your data, a kernel table. Learning matrices properly means learning the grammar of both fields at once.</p><h4>How a matrix acts on a vector</h4><p>An m×n matrix is a table with m rows and n columns. It takes an n-vector in and gives an m-vector out. For a 2×2 matrix:</p><p><strong>[a b; c d] · (x, y) = (ax + by, cx + dy)</strong></p><p>There are two ways to read that same multiplication, and both are worth having.</p><p><strong>Row view (dot products).</strong> Each output entry is one row of the matrix dotted with the input. Top row (a, b) dotted with (x, y) gives ax + by.</p><p><strong>Column view (a recipe).</strong> The output is a mix of the matrix’s <em>columns</em>, using the input’s entries as the amounts: x × (column 1) + y × (column 2).</p><p>The column view gives you the most useful rule in this whole post:</p><blockquote><strong><em>The columns of a matrix are where the basis vectors go.</em></strong></blockquote><p>Column 1 is what the matrix does to (1, 0). Column 2 is what it does to (0, 1). So a matrix isn’t a mysterious grid of numbers. It’s a list of answers to the question “where does each basis arrow land?” And because every other arrow is a mix of the basis arrows, knowing where those land tells you where <em>everything</em> lands. The whole grid bends with them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*SHWDCt2JtgZPBR1L4LRvxw.png" /></figure><p><em>Caption: A matrix bends the whole grid. Its columns are the new positions of (1, 0) and (0, 1).</em></p><p>This is also how you <strong>design a gate</strong>. Ask two questions: where should |0⟩ go, and where should |1⟩ go? Write the answers as columns, and that’s your gate.</p><p><strong>Worked example: the NOT gate.</strong> We want a gate that swaps the two components: (1, 0) should go to (0, 1), and (0, 1) should go to (1, 0). Those answers are the columns, so</p><p>X = [0 1; 1 0]</p><p>Check it on a state: X · (0.6, 0.8) = (0.8, 0.6). The amplitudes swapped places. This is the quantum NOT gate, called <strong>X</strong>.</p><p><strong>Worked example: rotation.</strong> To rotate the plane by angle θ, the arrow (1, 0) has to land on (cos θ, sin θ), and (0, 1) has to land on (−sin θ, cos θ), because the y-axis is always 90° ahead of the x-axis. Write those as columns:</p><p>R(θ) = [cos θ −sin θ; sin θ cos θ]</p><p>This is almost exactly the quantum <strong>Ry</strong> rotation gate. The only difference is that the quantum version uses θ/2 instead of θ. If you read the Part 2 discussion of the Bloch sphere, you already know where that ½ comes from.</p><h4>Multiplying matrices = running machines one after another</h4><p>If you apply B to a vector and then apply A to the result, that’s A(Bv). The combined machine is the product AB. Entry (i, j) of AB is row i of A dotted with column j of B.</p><p><strong>Worked example.</strong> Take A = [1 2; 3 4] and B = [0 1; 1 0]. Then</p><p>AB = [1·0 + 2·1 1·1 + 2·0; 3·0 + 4·1 3·1 + 4·0] = [2 1; 4 3]</p><p>The columns of A got swapped. Now the other order:</p><p>BA = [3 4; 1 2]</p><p>This time the rows got swapped. Different answer.</p><blockquote><strong><em>Order matters: in general AB ≠ BA. And the rightmost matrix acts first.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CErQkxH46Rt0ezj9pjW08A.png" /></figure><p><em>Caption: Shear-then-rotate and rotate-then-shear give different shapes. Same two machines, different order.</em></p><p>That second part trips up almost everyone. A quantum circuit diagram reads left to right: “apply H, then apply Z”. But the matrix for that circuit is <strong>ZH</strong>, not HZ, because the vector sits on the right and the first machine to touch it is the one nearest to it. Think socks then shoes: the maths reads Shoes · Socks · (foot).</p><p>Does it really change anything? Try it on |0⟩. Apply H then Z, which is ZH|0⟩, and you get (1, −1)/√2, the state called |−⟩. Apply Z then H, which is HZ|0⟩, and you get (1, 1)/√2, the state |+⟩. Same two gates, opposite results.</p><p>The <strong>commutator</strong> [A, B] = AB − BA measures exactly how much the order matters. If it’s zero, the matrices <em>commute</em> and order doesn’t matter. In quantum mechanics this has real consequences: two quantities whose matrices don’t commute can’t both have definite values at the same time. That is the uncertainty principle, written as algebra. For example, XZ − ZX = −2iY, which is not zero. (Y is the third Pauli matrix, which you’ll meet below.)</p><h4>Identity, inverse, transpose and dagger</h4><p>A few matrices and operations come up so often that they need names.</p><ul><li>The <strong>identity</strong> I has 1s on the diagonal and 0s everywhere else. Iv = v. It’s the “do nothing” machine.</li><li>The <strong>inverse</strong> A⁻¹ undoes A, so A⁻¹A = AA⁻¹ = I. For a 2×2 matrix, the inverse of [a b; c d] is (1/(ad − bc)) · [d −b; −c a]. It only exists if ad − bc ≠ 0. More on that number in a moment.</li><li>The <strong>transpose</strong> Aᵀ flips the matrix over its diagonal, so rows become columns. Watch the order: (AB)ᵀ = BᵀAᵀ.</li><li>The <strong>dagger</strong> A† (also called the conjugate transpose) transposes <em>and</em> conjugates every entry. Again (AB)† = B†A†. This is the same dagger from Part 2: for a column vector, |v⟩† = ⟨v|, which is how a ket becomes a bra.</li></ul><p><strong>Worked example.</strong> For A = [1 2i; 3 4−i], flip it over the diagonal and conjugate every entry: A† = [1 3; −2i 4+i].</p><p><strong>Worked example.</strong> For [2 1; 5 3], ad − bc = 6 − 5 = 1, so the inverse is [3 −1; −5 2]. Check the top-left entry of the product: 2·3 + 1·(−5) = 1. ✓</p><h4>The determinant: how much a matrix scales area</h4><p>Draw the unit square, then let a matrix act on it. The square turns into a parallelogram. The <strong>determinant</strong> is the area of that parallelogram, with a sign that tells you whether the shape got flipped over.</p><p>For a 2×2 matrix: <strong>det [a b; c d] = ad − bc</strong>, and det(AB) = det A · det B.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cj5s_99yfrk2PeMQqxsrMQ.png" /></figure><p><em>Caption: A rotation keeps the area (det = 1). [2 1; 1 2] triples it (det = 3). [1 2; 2 4] flattens the square onto a line (det = 0).</em></p><p>The determinant-zero case matters most. If det A = 0, the matrix squashes the whole plane flat onto a line. Points that used to be different end up in the same place, so information has been destroyed and nothing can undo it. That’s why there’s no inverse when ad − bc = 0.</p><p><strong>Worked examples.</strong> A rotation has det R(θ) = cos²θ + sin²θ = 1: area kept. The shear [1 1; 0 1] also has det = 1: the shape changes but the area doesn’t. And [1 2; 2 4] has det = 4 − 4 = 0: everything lands on the line y = 2x.</p><p>Quantum gates never destroy information, because every gate has to be undoable. So the determinant of a quantum gate always has size exactly 1.</p><h4>Eigenvalues and eigenvectors: a matrix’s favourite directions</h4><p>Most arrows get turned <em>and</em> stretched by a matrix. But a few special arrows only get stretched: they come out pointing the same way as they went in, or exactly the opposite way. These are the <strong>eigenvectors</strong> (German <em>eigen</em> = “own”, so they’re the matrix’s own directions). The stretch factor is the <strong>eigenvalue</strong>, written λ (lambda).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E5cJKims02sRt-_Jf4Va4g.png" /></figure><p><em>Caption: Under [2 1; 1 2], the unit circle becomes an ellipse. One eigenvector is stretched 3×, the other is left alone, and an ordinary vector gets turned as well as stretched.</em></p><p>Why does quantum computing care so much about this? Because <strong>when you measure a quantity, the only results you can get are the eigenvalues of its matrix</strong>, and right after the measurement, the state <em>is</em> the matching eigenvector. Energy levels of an atom, the “0 or 1” of a qubit, the answers QAOA finds all eigenvalues. In ML, PCA finds the eigenvectors of your data’s covariance matrix, which are the directions your data varies along most.</p><p>In symbols, v is an eigenvector of A with eigenvalue λ when</p><p><strong>A v = λ v</strong>, with v ≠ 0</p><p>Move everything to one side and you get (A − λI)v = 0. So the matrix A − λI squashes a non-zero vector all the way down to zero. That’s only possible if it flattens space, which means its determinant must be zero:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oKfPiQ7bTmfGUHxHfGczsg.png" /></figure><p><em>Caption: Av = λv ⟺ det(A − λI) = 0. Also: trace = sum of eigenvalues, determinant = product of eigenvalues.</em></p><p>That equation, det(A − λI) = 0, is called the <strong>characteristic equation</strong>. For a 2×2 matrix it always becomes λ² − (Tr A)λ + det A = 0, where Tr A (the <em>trace</em>) is the sum of the diagonal entries. So the recipe is:</p><ol><li>Solve λ² − (Tr A)λ + det A = 0 to get the eigenvalues.</li><li>For each λ, solve (A − λI)v = 0 to get its eigenvector.</li></ol><p><strong>Worked example: A = [2 1; 1 2].</strong><br> Step 1: Tr A = 4 and det A = 3, so λ² − 4λ + 3 = 0, which factors as (λ − 1)(λ − 3) = 0. The eigenvalues are 1 and 3.<br> Step 2, λ = 3: A − 3I = [−1 1; 1 −1], so −x + y = 0, which gives v = (1, 1)/√2.<br> Step 2, λ = 1: A − I = [1 1; 1 1], so x + y = 0, which gives v = (1, −1)/√2.<br> Check: A · (1, 1) = (3, 3) = 3 · (1, 1). ✓ Notice that the two eigenvectors are perpendicular. That’s not a coincidence: it happens because the matrix is symmetric, and we’ll see why below.</p><p><strong>Worked example: the Pauli X matrix.</strong> X = [0 1; 1 0] has Tr = 0 and det = −1, so λ² − 1 = 0 and λ = ±1. For λ = +1 the eigenvector is (1, 1)/√2 = |+⟩. For λ = −1 it’s (1, −1)/√2 = |−⟩. So measuring “X” on a qubit always returns +1 or −1, and leaves the qubit in |+⟩ or |−⟩.</p><p>This also explains a convention from Part 1. The Z matrix, Z = [1 0; 0 −1], has eigenvalues +1 for |0⟩ and −1 for |1⟩. That’s why quantum computing scores measurement result 0 as +1 and result 1 as −1: those are literally Z’s eigenvalues.</p><p><strong>Worked example: a rotation has complex eigenvalues.</strong> R(90°) = [0 −1; 1 0] gives λ² + 1 = 0, so λ = ±i. Of course no real arrow keeps its direction under a 90° turn. But complex arrows can: λ = i has eigenvector (1, −i)/√2. This is one more reason quantum mechanics uses complex numbers: with them, <em>every</em> matrix has eigenvectors. Notice also that |±i| = 1. Rotations always have eigenvalues on the unit circle.</p><p>The trace</p><p>The <strong>trace</strong> is the sum of the diagonal: Tr A = Σᵢ Aᵢᵢ. Two facts worth memorising:</p><ul><li>Tr A = the sum of the eigenvalues, and det A = the product of the eigenvalues. (Check with [2 1; 1 2]: 1 + 3 = 4 = trace, 1 × 3 = 3 = determinant.)</li><li>The trace is <strong>cyclic</strong>: Tr(AB) = Tr(BA), and Tr(ABC) = Tr(CAB) = Tr(BCA).</li></ul><p>That cyclic rule looks like a small curiosity, but later in the series it’s exactly what proves that quantum classifiers are linear models: ⟨ψ|M|ψ⟩ = Tr(M |ψ⟩⟨ψ|).</p><h4>The special matrices of quantum mechanics</h4><p>Hermitian matrices: the things you can measure</p><p>A matrix that equals its own dagger, A = A†, is called <strong>Hermitian</strong>. It’s the complex version of “symmetric”. Hermitian matrices have two superpowers: their eigenvalues are always <strong>real numbers</strong>, and their eigenvectors are always <strong>perpendicular</strong> to each other.</p><p>That’s exactly what physics needs. A meter in a lab shows real numbers. You never read “3 + 2i volts” off a dial. So every measurable quantity has to be described by a Hermitian matrix.</p><p>Why are the eigenvalues real? It takes three lines. Let A = A†, with A|v⟩ = λ|v⟩ and ⟨v|v⟩ = 1. Then λ = ⟨v|A|v⟩. Take the complex conjugate of both sides: λ* = ⟨v|A†|v⟩ = ⟨v|A|v⟩ = λ. A number equal to its own conjugate is real. ✓</p><p><strong>Worked example.</strong> The third Pauli matrix is Y = [0 −i; i 0]. Transpose it and conjugate every entry, and you get [0 −i; i 0] again. So Y is Hermitian, even though it contains imaginary numbers. Its eigenvalues come from λ² − 1 = 0, so they’re ±1: real, as promised.</p><p>Unitary matrices: the gates</p><p>A <strong>unitary</strong> matrix preserves length. Whatever arrow you put in, the arrow that comes out has exactly the same length. It’s a pure rotation, possibly with reflections and complex phases mixed in.</p><p>Since a quantum state must always have length 1 (total probability 1), <strong>every allowed quantum operation must be unitary</strong>. And there’s a bonus: the inverse of a unitary matrix is just its dagger, so every gate can be undone by “running it backwards”.</p><blockquote><strong><em>U†U = UU† = I ⟺ U⁻¹ = U† ⟺ ‖Uv‖ = ‖v‖ for every v</em></strong></blockquote><p>There’s an easy test too: a matrix is unitary exactly when <strong>its columns form an orthonormal basis</strong>. Its eigenvalues all have size 1, so they look like e^(iφ), the unit-circle arrows from Part 1.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4HmL0ztQ9Y4W2HOrG0eGbQ.png" /></figure><p><em>Caption: The Hadamard gate maps the unit circle to itself. The shear stretches it into an ellipse, so it can’t be a gate.</em></p><p><strong>Worked example.</strong> Is H = (1/√2)[1 1; 1 −1] unitary? Its columns are (1, 1)/√2 and (1, −1)/√2. Each has length 1, and their inner product is ½(1 − 1) = 0. Yes ✓. H is also Hermitian (H† = H), so H² = H†H = I: applying H twice does nothing.</p><p>Is the shear [1 1; 0 1] unitary? Its second column (1, 1) has length √2, not 1. No, so it can never be a quantum gate.</p><p>Projectors</p><p>For a unit vector |e⟩, the matrix P = |e⟩⟨e| is a <strong>projector</strong>. It keeps the part of a vector that lies along |e⟩ and throws away the rest, which is exactly the “shadow” from Part 2. Projectors satisfy P² = P (projecting twice is the same as projecting once) and P† = P, and their eigenvalues are 0 and 1. Measurement in quantum mechanics is described by projectors.</p><h4>Outer products: building matrices from kets and bras</h4><p>In Part 2, a bra times a ket, ⟨a|b⟩, gave a single number. Flip the order, a ket times a bra, and you get a whole <strong>matrix</strong>: |a⟩⟨b| has entries aᵢbⱼ*.</p><p>|0⟩⟨0| = [1 0; 0 0], |0⟩⟨1| = [0 1; 0 0], |1⟩⟨0| = [0 0; 1 0]</p><p>So any 2×2 matrix can be written as a sum of these building blocks. For example, X = |0⟩⟨1| + |1⟩⟨0|, which you can read out loud as “send 1 to 0, and send 0 to 1”. One more identity worth knowing, called <strong>completeness</strong>: for any orthonormal basis, Σₖ |eₖ⟩⟨eₖ| = I.</p><h4>The spectral theorem: a Hermitian matrix is a menu of directions</h4><p>This is the deepest result in the chapter, but what it says is simple. A Hermitian matrix is completely described by its favourite directions (its eigenvectors) and how much it stretches each one (its eigenvalues). It’s like a menu: “direction e₁, value λ₁; direction e₂, value λ₂; …”</p><blockquote><strong><em>A = Σₖ λₖ |eₖ⟩⟨eₖ| (for A = A†, with orthonormal eigenvectors |eₖ⟩)</em></strong></blockquote><p><strong>Worked example.</strong> Z = (+1)|0⟩⟨0| + (−1)|1⟩⟨1|, and X = (+1)|+⟩⟨+| + (−1)|−⟩⟨−|. Multiply them out with the outer products above and you’ll get exactly [1 0; 0 −1] and [0 1; 1 0].</p><h4>Functions of matrices, especially e to the power of a matrix</h4><p>Quantum gates are often written like <strong>e^(−iθX/2)</strong>: “e to the power of a matrix”. That sounds crazy, but there are two easy ways to make sense of it.</p><ol><li>Use the power series of eˣ from Part 1, with the matrix plugged in: e^A = I + A + A²/2! + A³/3! + …</li><li>Or, easier: apply the function to each eigenvalue, keeping the eigenvectors: f(A) = Σₖ f(λₖ)|eₖ⟩⟨eₖ|.</li></ol><p>Why do physicists write gates this way? Because when a physical interaction with energy matrix (Hamiltonian) H runs for time t, it produces the gate e^(−iHt). That’s literally how hardware <em>makes</em> gates: switch an interaction on for just the right amount of time.</p><p>There’s one identity you’ll see constantly. Whenever P² = I — which is true for X, Y, Z and any product of them — the power series splits neatly in two:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KT_CF5esMcZFrIdvrj4New.png" /></figure><p><em>Caption: e^(−iθP/2) = cos(θ/2)·I − i·sin(θ/2)·P whenever P² = I.</em></p><p>Why? In the series, every even power of P is P²ᵏ = I and every odd power is P. The even terms add up to 1 − (θ/2)²/2! + … = cos(θ/2). The odd terms add up to −i((θ/2) − (θ/2)³/3! + …)P = −i sin(θ/2)P. It’s Euler’s formula from Part 1, with a matrix in place of the number i.</p><p><strong>Worked example.</strong> For Z = diag(1, −1), apply the function to each eigenvalue: e^(−iθZ/2) = diag(e^(−iθ/2), e^(iθ/2)). That’s the <strong>Rz(θ)</strong> gate. The identity gives the same answer: cos(θ/2)I − i sin(θ/2)Z = diag(cos(θ/2) − i sin(θ/2), cos(θ/2) + i sin(θ/2)). ✓</p><h4>The tensor product: gluing qubits together</h4><p>How do we describe two qubits at once? We need to combine one 2-dimensional space with another 2-dimensional space to get a 4-dimensional one. The <strong>Kronecker (tensor) product</strong>, written ⊗, does that with one rule: every entry of the first times the whole of the second.</p><p>(a₀, a₁) ⊗ (b₀, b₁) = (a₀b₀, a₀b₁, a₁b₀, a₁b₁)</p><p>For matrices it works the same way: A ⊗ B is a big block matrix where block (i, j) is Aᵢⱼ × B.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*h-fMRpzYU9DoyRFUotn8eA.png" /></figure><p><em>Caption: Two 2-vectors make one 4-vector, one entry for each of |00⟩, |01⟩, |10⟩, |11⟩. X ⊗ I is every entry of X times a whole copy of I.</em></p><p>A few rules you’ll use constantly: (A ⊗ B)(C ⊗ D) = AC ⊗ BD, (A ⊗ B)† = A† ⊗ B†, Tr(A ⊗ B) = Tr A · Tr B, and sizes multiply (2×2 ⊗ 2×2 = 4×4).</p><p><strong>Worked example.</strong> X ⊗ I = [0·I 1·I; 1·I 0·I], which written out is the 4×4 matrix with rows (0 0 1 0), (0 0 0 1), (1 0 0 0), (0 1 0 0). It swaps the top half of the state vector with the bottom half. In other words, it flips the <em>first</em> qubit and leaves the second one alone.</p><h4>Why simulating qubits gets hard so fast</h4><p>Multiplying an N×N matrix by a vector takes about N² multiplications. For n qubits, N = 2ⁿ. So just <em>storing</em> the state takes 2ⁿ complex numbers at 16 bytes each, and storing one gate as a full matrix takes 4ⁿ of them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*fs1e9bT6qQ0M1ZoMMM0IrQ.png" /></figure><p><em>Caption: The state vector for 30 qubits needs about 17 GB. Writing a single gate as a full matrix hits 17 GB at just 15 qubits.</em></p><p>At 30 qubits, the state alone is about 17 GB. At 50 qubits it’s about 18 petabytes, which you’ll recognise from Part 1. A quantum computer does all of this “for free”, because it doesn’t store the vector. It <em>is</em> the vector.</p><h4>Key takeaways</h4><ul><li>A matrix is a machine that moves arrows. Its columns are where the basis vectors land, and that’s how you build a gate.</li><li>Matrix products act right to left and usually don’t commute: “H then Z” is ZH, and ZH ≠ HZ.</li><li>The determinant is the area scale factor. det = 0 means information is destroyed and there’s no inverse.</li><li>Eigenvectors are the directions a matrix only stretches. Measurement results are eigenvalues, which is why Z gives +1 and −1.</li><li>Trace = sum of eigenvalues, determinant = product of eigenvalues, and the trace is cyclic.</li><li>Hermitian (A = A†) = observables, with real eigenvalues and perpendicular eigenvectors.</li><li>Unitary (U†U = I) = gates. They preserve length, and their inverse is their dagger.</li><li>Every Hermitian matrix is a menu of directions: A = Σ λₖ|eₖ⟩⟨eₖ|.</li><li>e^(−iθP/2) = cos(θ/2)I − i sin(θ/2)P is how rotation gates are built.</li><li>The tensor product ⊗ glues qubits together, and its cost grows as 2ⁿ for states and 4ⁿ for full gate matrices.</li></ul><h4>References</h4><ul><li>IBM Research, “The Qiskit Global Summer School is returning with a focus on Quantum Machine Learning” (2021). The course this series follows.</li><li>M. A. Nielsen and I. L. Chuang, <em>Quantum Computation and Quantum Information</em>, Cambridge University Press, Chapter 2.</li><li>Gilbert Strang, <em>Introduction to Linear Algebra</em>, Wellesley-Cambridge Press — the classic “column picture” treatment of matrices.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=430512ccb8a9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Vectors: the Arrows That Hold a Qubit
 Quantum Computing & QML,]]></title>
            <link>https://medium.com/@lalanagurusinghe/vectors-the-arrows-that-hold-a-qubit-quantum-computing-qml-041e742e0c83?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/041e742e0c83</guid>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[ibm-qiskit]]></category>
            <category><![CDATA[linear-algebra]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Thu, 29 Oct 2026 17:04:59 GMT</pubDate>
            <atom:updated>2026-10-29T17:04:59.082Z</atom:updated>
            <content:encoded><![CDATA[<h3>Part 2: vector spaces, bases, the inner product and bra-ket notation, explained as arrows</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hvblYeVKjzMtAf3E3JEiqw.png" /></figure><p>In Part 1 of this series I kept writing a sentence like “a qubit’s state is cos(θ/2)|0⟩ + sin(θ/2)|1⟩” without ever saying what that notation actually means. Time to fix that.</p><p>Here is the one-line version of this whole post: <strong>a quantum state is a vector</strong>, and everything that makes vectors useful in high-school geometry adding them, scaling them, measuring how much two of them agree still works once the numbers inside them turn complex. You just have to be slightly careful about one thing: conjugates. Get that one thing right and the rest of quantum computing’s notation stops looking like a secret code.</p><p>Part 1 covered the basic toolkit and complex numbers. This post covers <strong>vectors and vector spaces</strong>: everything you need before the first qubit shows up on a Bloch sphere. If you missed Part 1, you can still follow this one I’ll recap the one fact from it that matters here.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kRlyXwbNpfeFq-u6TJNHCg.png" /></figure><p><em>Caption: Five of the seven tools are now covered. Matrices and probability are next.</em></p><h4>A vector is an arrow, and that’s the whole idea</h4><p>A vector is an arrow with a length and a direction. “Walk 3 km east and 1 km north” is a vector. You can add two walks do one, then the other and you can scale a walk walk twice as far. Those two moves, adding and scaling, are the entire definition. Everything else in this post is built from them.</p><p>Why does a quantum course spend a whole chapter on something this simple? Because <strong>a quantum state is a vector</strong>. A qubit’s state is an arrow in a 2-dimensional complex space, and n qubits give an arrow in a 2ⁿ dimensional space. “Superposition” just means adding vectors. “Measurement probability” means how much one arrow points along another. And in machine learning, every data point is a vector of features too. Vectors are the meeting point of quantum computing and ML, which is exactly why quantum machine learning is a real field and not a buzzword.</p><p>A vector is also a list of numbers</p><p><strong>Pick two perpendicular directions, east = ê₁ and north = ê₂. Any arrow in the plane is “some of ê₁ plus some of ê₂”, so we can write it as a column of two numbers, its components: v = (3, 1) = 3ê₁ + 1ê₂.</strong></p><p>In n dimensions, a vector is a column of n numbers. Nothing stops n from being 4, or 784, or 2⁵⁰. We can’t draw those, but the arithmetic is exactly the same as it is for an arrow on paper.</p><p>The two operations: adding and scaling</p><p>Adding and scaling work component by component: u + v = (u₁ + v₁, u₂ + v₂) and c·v = (c·v₁, c·v₂). With u = (3, 1) and v = (1, 2): u + v = (4, 3), 2u − v = (5, 0), and −v = (−1, −2) points the opposite way.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WHnSY_bHzAtinWyezMXstQ.png" /></figure><p><em>Caption: Adding is tip to tail. Scaling stretches, shrinks or flips.</em></p><p>A <strong>linear combination</strong> is “a bit of this arrow plus a bit of that arrow”: αu + βv. Think of mixing paint, some red plus some blue. In quantum mechanics, α|0⟩ + β|1⟩ is a linear combination of the two basis states, and <strong>that is all superposition means mathematically</strong>. The only twist, which we covered in Part 1, is that α and β can be complex: little arrows of their own, not just plain scaling factors.</p><h4>What counts as a vector space?</h4><p>Mathematicians noticed that lots of different things behave like arrows: lists of numbers, polynomials, functions, sound waves, quantum states. So instead of proving the same facts over and over for each one, they wrote one general definition. A <strong>vector space</strong> is any collection of objects you can add and scale, where ten common-sense rules hold:</p><ul><li>Adding or scaling never takes you outside the space.</li><li>Order and grouping don’t matter when adding: u + v = v + u, and (u + v) + w = u + (v + w).</li><li>There is a zero vector, and every vector has a negative that cancels it: v + (−v) = 0.</li><li>Scaling spreads over sums and behaves like normal multiplication: a(u + v) = au + av, (a + b)v = av + bv, a(bv) = (ab)v, and 1v = v.</li></ul><p>If something passes these rules, every theorem ever proved about arrows works for it automatically, free of charge. That’s why the same maths powers both quantum physics and ML kernels: nobody had to reinvent linear algebra for each one. For quantum computing, the space is <strong>ℂᴺ</strong>: columns of N complex numbers, with N = 2 for a single qubit and N = 2ⁿ for n of them.</p><h4>Span, independence, basis and dimension</h4><p>Four related ideas, and they all answer one question: which arrows do I really need to describe everything in this space?</p><ul><li>The <strong>span</strong> of some arrows is everything you can build by combining them. Two non-parallel arrows span the whole plane; a third arrow in that plane adds nothing new.</li><li>Arrows are <strong>linearly independent</strong> if none of them can be built from the others. No redundant arrow in the set.</li><li>A <strong>basis</strong> is an independent set that spans everything: a minimal toolkit from which every vector in the space can be built in exactly one way. It’s like an alphabet with no redundant letters.</li><li>The <strong>dimension</strong> is how many arrows a basis has. One qubit has dimension 2. n qubits have dimension 2ⁿ, which is the number from Part 1 that made 50 qubits need 18 petabytes.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LjXoImpoP66wknp8QdRycg.png" /></figure><p><em>Caption: Independent vectors span the whole space. Dependent ones collapse onto a line.</em></p><p>For example,</p><p><strong>(1, 0) and (1, 1) form a basis of the plane. To write (5, 3) in that basis, solve a(1, 0) + b(1, 1) = (a + b, b) = (5, 3), which gives b = 3 and a = 2. But (1, 2) and (2, 4) are not independent, because one is just twice the other multiply (1,2) by 2 and you get (2,4), so together they only ever reach points on a single line.</strong></p><p>Now here is the idea to actually remember: <strong>the same vector has different components in different bases</strong>. The arrow itself doesn’t change only the numbers you use to describe it change, depending on which basis you measured it against. In quantum computing, “measuring in a different basis” is exactly this idea, no more mysterious than that.</p><h4>Length and the dot product</h4><p>How long is an arrow? Pythagoras answers that. How much do two arrows agree with each other? The <strong>dot product</strong> answers that: multiply matching components and add them up. If two arrows point the same way, the dot product is big and positive. If they’re perpendicular, it’s zero. If they point in roughly opposite directions, it’s negative.</p><p>Geometrically, the dot product is a shadow: a · b = |a||b| cos θ, the length of a times the shadow that b casts onto a. Because of this, the dot product works as a similarity score which is exactly what a <strong>kernel</strong> is in machine learning, and exactly what an <strong>overlap</strong> is in quantum mechanics. In ML you already know it as cosine similarity: cos θ = (a · b)/(|a||b|).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*FyDMO6ZdXFWAhcdlLH11fQ.png" /></figure><p><em>Caption: The dot product measures how much one arrow lies along another.</em></p><p>For example, a = (3, 4) and b = (4, −3) give a · b = 12 − 12 = 0, so they’re perpendicular. And |a| = √(9 + 16) = 5.</p><p>A <strong>unit vector</strong> has length 1. To <strong>normalise</strong> a vector, divide it by its own length: (3, 4) becomes (0.6, 0.8), and indeed 0.36 + 0.64 = 1. Here’s a fact worth highlighting on its own: <strong>every quantum state is a unit vector</strong>, because the total probability of all its outcomes has to add up to 1.</p><p>A basis whose vectors all have length 1 and are all mutually perpendicular is called <strong>orthonormal</strong>. With one, finding components is easy: just take the dot product with each basis vector in turn.</p><p><strong>Take f₁ = (1, 1)/√2 and f₂ = (1, −1)/√2. For v = (1, 0), f₁ · v = 1/√2 and f₂ · v = 1/√2, so v = (1/√2)f₁ + (1/√2)f₂. In quantum notation, that line reads |0⟩ = (|+⟩ + |−⟩)/√2 — the standard basis state written in the diagonal basis.</strong></p><h4>Complex vectors and the inner product</h4><p>Quantum amplitudes are complex numbers, so we need a version of the dot product that works for complex vectors. The fix is the one thing I promised to recap from Part 1: <strong>conjugate the first vector</strong>. The result is called the <strong>inner product</strong>. It still measures overlap, and the inner product of a vector with itself is still its length squared a real number that is never negative.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HkeOiOZGV6KMgIH6Kzz6YQ.png" /></figure><p><em>Caption: |0⟩ is (1, 0) in one basis and (1/√2, 1/√2) in the |+⟩, |−⟩ basis. Same arrow, different description.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5OAy8bOGX8TF7flV1m61WQ.png" /></figure><p><em>Caption: </em><strong><em>⟨a, b⟩ = Σₖ aₖ</em> bₖ and ‖a‖² = ⟨a, a⟩ = Σₖ |aₖ|² ≥ 0*</strong></p><p>Why the conjugate?</p><p><strong>Take v = (1, i). Without conjugating, v₁² + v₂² = 1 + i² = 0, which would claim a non-zero vector has zero length. Nonsense. With the conjugate, |1|² + |i|² = 2, so the length comes out correctly as √2. One rule to remember: always conjugate one side, never both, never neither.</strong></p><p>A few properties worth knowing by heart, because they explain notation you’ll see constantly:</p><ul><li><strong>Order matters</strong>: ⟨a, b⟩ = ⟨b, a⟩*, unlike the real dot product, where order never mattered.</li><li><strong>Linear in the second slot</strong>: ⟨a, βb + γc⟩ = β⟨a, b⟩ + γ⟨a, c⟩.</li><li><strong>Conjugate-linear in the first slot</strong>: ⟨βa, b⟩ = β*⟨a, b⟩.</li><li><strong>Cauchy–Schwarz</strong>: |⟨a, b⟩| ≤ ‖a‖‖b‖. For unit vectors, this means |⟨a, b⟩|² is always between 0 and 1 — which is exactly why it’s allowed to be a probability.</li></ul><p>A nice surprise: a = (1, i) and b = (i, 1) give ⟨a, b⟩ = 1*·i + i*·1 = i − i = 0. They’re orthogonal, even though neither one has a zero entry. Complex orthogonality doesn’t look like the real kind.</p><h4>Dirac (bra-ket) notation</h4><p>Paul Dirac invented a neat shorthand that shows up on every slide of every quantum course you’ll ever see. It looks intimidating for about a week, and then it feels completely natural.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*REsETCi-D2c7uSdiqIXt1A.png" /></figure><p><em>Caption: Measurement is projection: the probability is the squared length of the shadow.</em></p><ul><li>A <strong>ket</strong> |v⟩ is a column vector: an arrow.</li><li>A <strong>bra</strong> ⟨v| is the same thing turned into a row and conjugated: a measuring stick for that arrow. Formally, ⟨v| = |v⟩†, where † (“dagger”) means transpose and conjugate together.</li><li>Put them side by side and you get a <strong>bra-ket</strong> — a bracket: ⟨a|b⟩ = Σₖ aₖ* bₖ, a single number. This is exactly the inner product from the last section, just written in physicist shorthand.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2b09IBW8qkFe3U35WwzyBQ.png" /></figure><p><em>Caption: </em><strong><em>Gram–Schmidt: keep the first vector, subtract its shadow from the rest, then normalise.</em></strong></p><p>The label inside a ket is just a name: |0⟩, |1⟩, |+⟩, |ψ⟩, even |cat⟩ if you want to be dramatic about it. A common trap worth flagging: <strong>|0⟩ is not the zero vector</strong>. It’s the name of the first basis vector, (1, 0). A state is normalised when ⟨ψ|ψ⟩ = 1, and two states are orthogonal when ⟨φ|ψ⟩ = 0.</p><p><strong>Worked example.</strong> Let |a⟩ = (1, i)/√2 and |b⟩ = (1, 0). Then ⟨a| = (1, −i)/√2, so ⟨a|b⟩ = (1·1 + (−i)·0)/√2 = 1/√2, and |⟨a|b⟩|² = ½. Hold on to that ½: it’s the probability of measuring 0 when the qubit is in state |a⟩, and it’s exactly the number the projection picture below computes geometrically.</p><p>Projection: <strong>a first look at measurement</strong></p><p>Projecting means casting a shadow. The projection of |ψ⟩ onto a unit direction |e⟩ is the part of |ψ⟩ that lies along |e⟩, written |e⟩⟨e|ψ⟩, and its squared length is |⟨e|ψ⟩|². In quantum mechanics, <strong>measurement is exactly this</strong>: the state gets projected onto one of the basis directions, and the probability of that outcome is the squared length of the shadow.</p><p>Take |ψ⟩ = 0.6|0⟩ + 0.8|1⟩. Projected onto |1⟩, the shadow is 0.8|1⟩, with squared length 0.64 a 64% chance of measuring 1. Now project onto |+⟩ = (|0⟩ + |1⟩)/√2 instead: ⟨+|ψ⟩ = (0.6 + 0.8)/√2 ≈ 0.99, which squares to about 0.98. In the |+⟩, |−⟩ basis, this particular state lies almost entirely along |+⟩, even though it looked evenly split in the |0⟩, |1⟩ picture. Same state, different question, different answer because which basis you measure in genuinely changes what “probability of this outcome” means.</p><p>Related tool: <strong>Gram–Schmidt</strong> turns any independent set of vectors into an orthonormal basis, one vector at a time. Keep the first vector, after normalising it. From the second, subtract its shadow on the first, then normalise what’s left. Starting from v₁ = (1, 1) and v₂ = (1, 0), you get e₁ = (1, 1)/√2 and e₂ = (1, −1)/√2 — the |+⟩ and |−⟩ directions again, built from scratch.</p><h4>Functions are vectors too</h4><p>Here’s a mind-stretcher worth planting now, for when it pays off later in the series. A function f(x) can be added to another function and scaled by a number, so functions form a vector space just like lists of numbers do. Their “components” are the values f(x) at every single x infinitely many of them. The “dot product” of two functions is ∫ f(x) g(x) dx, an integral standing in for a sum.</p><p>Kernel methods in machine learning secretly work inside these huge, sometimes infinite-dimensional function spaces. Quantum computers work in 2ⁿ dimensional spaces instead. It’s the same idea at a different scale, and it’s the bridge that connects classical kernel methods to quantum kernels later in this series.</p><h4>High dimensions are strange</h4><p>Our intuition for vectors comes entirely from 2D and 3D, where we can draw pictures. In very high dimensions, two random directions are almost always nearly perpendicular to each other, simply because there’s so much room for them to avoid lining up. In N dimensions, the typical squared overlap between two random unit vectors is about <strong>1/N</strong>. A quick simulation with random complex unit vectors lands right on that line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Yv-nd6xT8jFLyfMSUeXZgw.png" /></figure><p><em>Caption:</em><strong><em> Kets are columns, bras are conjugated rows, and a bra-ket is a number.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4u7X0q5uDts-S_ZT5z7V8g.png" /></figure><p><em>Caption: Average |⟨u|v⟩|² for random unit vectors falls as 1/N. With 12 qubits it’s already about 0.0002.</em></p><p>With N = ²⁵⁰, that overlap is essentially zero. This fact comes back to haunt quantum machine learning in two different disguises later in the series: as <strong>barren plateaus</strong>, where training gradients vanish because almost every direction looks orthogonal to the one you want; and as <strong>kernel concentration</strong>, where every pair of data points starts to look equally similar, because in high dimensions almost everything is equally far from everything else. Both get their own posts once we reach the quantum machine learning chapters.</p><h4>Key takeaways</h4><ul><li>A vector is an arrow; adding is tip to tail, scaling stretches or flips it.</li><li>Superposition is just a linear combination: α|0⟩ + β|1⟩.</li><li>A vector space is anything you can add and scale by the ten common-sense rules, which is why the same maths serves quantum states, feature vectors and even functions.</li><li>A basis is a minimal, independent set that spans everything. The same vector has different components in different bases.</li><li>The dot product measures a shadow, a · b = |a||b| cos θ, and is the same idea as a kernel in ML or an overlap in quantum mechanics.</li><li>Every quantum state is a unit vector, because total probability must equal 1.</li><li>The inner product ⟨a|b⟩ = Σ aₖ* bₖ conjugates the first vector; |⟨a|b⟩|² is between 0 and 1 for unit vectors, which is why it can be a probability.</li><li>Kets are columns, bras are conjugated rows, and bra-kets are numbers.</li><li>Measurement is projection: the probability of an outcome is the squared length of the state’s shadow on that outcome’s direction.</li><li>In high dimensions, random vectors are nearly orthogonal, with overlap² ≈ 1/N the seed of barren plateaus and kernel concentration later in the series.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=041e742e0c83" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Maths You Need Before Your First Qubit]]></title>
            <link>https://medium.com/@lalanagurusinghe/the-maths-you-need-before-your-first-qubit-2cb503ea9bf6?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/2cb503ea9bf6</guid>
            <category><![CDATA[quantum-computing]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[quantum-machine-learning]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Tue, 27 Oct 2026 15:34:26 GMT</pubDate>
            <atom:updated>2026-10-27T15:34:26.941Z</atom:updated>
            <content:encoded><![CDATA[<h4>Quantum Computing &amp; QML, Part 1: trigonometry, exponentials and complex numbers, explained as arrows, with the quantum reason behind each one</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ncVRcAQeeAEmKRhZJ8lEug.png" /></figure><p><em>A qubit is an arrow. This post builds the maths you need to say that precisely.</em></p><p>Here is a number that explains why people get excited about quantum computers. To write down the exact state of just <strong>50 qubits</strong>, a normal computer has to store 2⁵⁰ complex numbers. At 16 bytes each, that is about <strong>18 petabytes</strong> of memory. Add one more qubit and you need twice as much.</p><p>Most explanations stop at “qubits can be 0 and 1 at the same time”. That sounds deep, but you can’t calculate anything with it. The real explanation is maths, and it turns out to be friendlier than you’d expect: mostly arrows (vectors), tables of numbers (matrices), a special kind of number (complex numbers) and a little probability.</p><p>I’m working through IBM’s <strong>Qiskit Global Summer School 2021 on Quantum Machine Learning</strong>: 19 lectures, 5 labs and a kick-off. As I go, I’m writing my own notes in a simple rhythm: the idea in plain words first, then the maths, then a worked example you can do with a pencil. This series turns those notes into posts.</p><p>Part 1 covers the basic toolkit and <strong>complex numbers</strong>, the numbers quantum mechanics is built on. Vectors come in Part 2. If you know school maths and a bit of Python, you have everything you need. You don’t need any physics background.</p><h3>Maths is the explanation, not decoration</h3><p>Nobody can picture quantum mechanics the way you picture a ball rolling down a hill. So physicists stopped trying to picture it. Instead, they found maths that predicts every experiment correctly.</p><p>That changes how you should read this whole series. When someone says “<strong><em>a qubit is in superposition</em></strong>”, what they actually mean is <em>“</em><strong><em>a qubit is described by a vector with two complex numbers in it</em></strong><em>”</em>. Once the tools in this series feel comfortable, most of quantum computing becomes “just” using them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BKirPiqTGCoyvSGF5serFg.png" /></figure><p><em>Each tool has a job later in the course. This post covers the first four.</em></p><h3>The basic toolkit, at speed</h3><p>Most of this section is school maths. Skim what you know, but notice the quantum reason attached to each idea, because that’s the part that pays off later.</p><h4>Functions: machines with an input and an output</h4><p>A function is a little machine: drop a number in, get a number out. In machine learning you build functions from data to predictions. In quantum computing, a <strong>gate is a function from one state to another</strong>.</p><p>Doing one machine and then another is called <strong>composition</strong>, and a quantum circuit is exactly that: gates composed one after another. Order matters, just as putting on socks then shoes isn’t the same as shoes then socks.</p><p>Some functions can be undone. f(x) = 2x + 1 has the inverse f⁻¹(y) = (y − 1)/2. Others can’t: x² sends both 3 and −3 to 9, so from 9 you can’t tell which one you started with. Remember this: <strong>every quantum gate has an inverse</strong>. Quantum computation is reversible, and that matters a lot later on.</p><h4>Radians: the circle’s own unit</h4><p>Degrees are a human choice (360 per circle, thanks to the ancient Babylonians). Radians come from the circle itself: walk along the edge of a circle of radius 1, and the distance you walk is the angle. A full trip round is 2π ≈ 6.283, so <strong>360° = 2π radians</strong>, 180° = π and 90° = π/2.</p><p>Calculus formulas, and especially Euler’s formula later in this post, only come out clean in radians, so every quantum paper uses them. To convert, multiply degrees by π/180. One radian is about 57.3°, and when you meet an angle like π/8 in the <strong>QAOA </strong>algorithm, read it as 22.5°.</p><h4>Sine and cosine: shadows of a spinning arrow</h4><p>Put an arrow of length 1 at the centre of a circle and rotate it by an angle θ. Shine a light from above and its shadow on the floor has length <strong>cos θ</strong>. Shine a light from the side and its shadow on the wall has length <strong>sin θ</strong>. That’s all sine and cosine are. When the arrow keeps spinning, the shadows swing back and forth forever, which is why sin and cos are the maths of anything that oscillates or rotates. And a qubit, as you’ll see, is something that rotates.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*r4bB-8-KFZ_aQaPEVu7iCA.png" /></figure><p><em>The tip of a unit arrow at angle θ sits at (cos θ, sin θ).</em></p><p>The arrow has length 1, so Pythagoras gives the most useful identity in the course:</p><blockquote><strong><em>cos²θ + sin²θ = 1</em></strong></blockquote><p>Because of this line, a qubit written as cos(θ/2)|0⟩ + sin(θ/2)|1⟩ automatically has total probability 1. (The |0⟩ and |1⟩ notation gets a proper introduction in Part 2.)</p><p>A few more facts you’ll use constantly:</p><ul><li><strong>Special values</strong>: cos 0 = 1, sin(π/2) = 1, cos π = −1, and cos(π/4) = sin(π/4) = 1/√2 ≈ 0.707.</li><li><strong>Symmetry</strong>: cos(−θ) = cos θ, and sin(−θ) = −sin θ.</li><li><strong>Period</strong>: adding 2π changes nothing.</li><li><strong>Addition formulas</strong>: <strong>cos(a + b) = cos a cos b − sin a sin b, and sin(a + b) = sin a cos b + cos a sin b. </strong>You don’t need to memorise these. Complex numbers will hand them to you in one line later in this post.</li><li><strong>Half-angle formulas</strong>, which appear in almost every qubit calculation: cos²(θ/2) = (1 + cos θ)/2 and sin²(θ/2) = (1 − cos θ)/2.</li></ul><p><strong>Worked example.</strong> A qubit’s chance of being measured as 0 turns out to be p₀ = cos²(θ/2). What is the average result, p₀ − p₁?</p><ul><li>p₁ = 1 − p₀ = sin²(θ/2), by Pythagoras.</li><li>p₀ − p₁ = cos²(θ/2) − sin²(θ/2).</li><li>That is the double-angle formula with angle θ/2, so p₀ − p₁ = <strong>cos θ</strong>.</li></ul><h4>Waves and phase</h4><p>Anything that wiggles regularly looks like A cos(ωt + φ). That includes a guitar string, a light wave, and the microwave pulse that controls a superconducting qubit. There are three knobs: the <strong>amplitude</strong> A (how tall), the <strong>angular frequency</strong> ω (how fast; one full wiggle takes time 2π/ω) and the <strong>phase</strong> φ (where in its cycle the wave starts).</p><p>Phase is <em>the</em> word of quantum computing. Two waves of the same height can add up or cancel out, depending only on their phases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9dVTdyADpjKJgvPIr52Lzw.png" /></figure><p><em>Same waves, different phase. They add up (left) or cancel (right).</em></p><h4>Exponentials and logarithms: why qubits are a big deal</h4><p>Fold a sheet of paper: 2 layers, 4, 8, 16. Each fold multiplies by 2, so after n folds you have 2ⁿ layers. With paper 0.1 mm thick, 50 folds would make a stack about 113 million km tall, roughly three-quarters of the way to the Sun.</p><p>Qubits grow the same way. <strong>n qubits need 2ⁿ complex numbers to describe.</strong> That’s why quantum computers are interesting, and also why simulating them on a laptop becomes impossible so quickly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DlVljO_gVDzhHR78fCyxcg.png" /></figure><p><em>Every 10 extra qubits multiply the memory by about a thousand.</em></p><p>The exponent rules are worth having at your fingertips: aᵐaⁿ = aᵐ⁺ⁿ, (aᵐ)ⁿ = aᵐⁿ, a⁰ = 1, a⁻ⁿ = 1/aⁿ and a^(1/2) = √a. The first one does a lot of work later in this post.</p><p>The special number <strong>e ≈ 2.71828</strong> is the natural base for continuous growth and decay. Imagine a bank that pays 100% interest a year. Paid once a year, your money grows ×2. Paid monthly, about ×2.613. Paid every instant, you approach ×2.71828. It can also be written as a power series:</p><blockquote><strong><em>eˣ = 1 + x + x²/2! + x³/3! + x⁴/4! + …</em></strong></blockquote><p>Keep that series in mind. It’s how we’ll put an <em>imaginary</em> number in the exponent later. Two facts make eˣ special: its slope equals its height, and e^(−t/T) describes decay with a lifetime T.</p><p>Qubits use that second fact directly. A qubit in state |1⟩ slowly loses its energy. With a lifetime T₁ = 100 μs, the chance it’s still in |1⟩ after 50 μs is e^(−0.5) ≈ 0.61. After 200 μs it’s e^(−2) ≈ 0.14. That’s the noise lecture in one line.</p><p>A <strong>logarithm</strong> runs the exponential backwards: “2 to the what is 1024?” Answer: 10, so log₂ 1024 = 10. Logs turn multiplication into addition: log(ab) = log a + log b. In quantum computing they answer the question “how many qubits do I need to store N numbers?” A 28 × 28 image has 784 pixels, and log₂ 784 ≈ 9.6, so <strong>10 qubits</strong> can hold it (padded to 1024 amplitudes). A million numbers fit in about 20 qubits.</p><p>The natural log (ln, the log with base e) also gives machine learning its cross-entropy loss. If a model gives the correct class a probability of 0.9, the loss is −ln 0.9 ≈ 0.105. If it gives 0.1, the loss is −ln 0.1 ≈ 2.303. Confidently wrong answers are punished hard.</p><h4>Three small tools: Σ, XOR and slopes</h4><p>The <strong>Σ sign</strong> is a for-loop written in maths. If you’re a programmer, read every Σ like this:</p><pre>total = 0<br>for k in range(1, 5):<br>    total += k**2      # Σ k² for k = 1..4  →  1 + 4 + 9 + 16 = 30</pre><p>A 3-qubit state is a Σ with eight terms, one per bit string. One sum we’ll need for the Fourier transform is the geometric series: 1 + r + r² + … + r^(N−1) = (1 − rᴺ)/(1 − r), for r ≠ 1.</p><p><strong>Binary and XOR.</strong> Computers count in base 2, and with n bits you can write 2ⁿ different strings. The basis states of n qubits are labelled by these strings, so you need to be fluent: 101₂ = 4 + 0 + 1 = 5. <strong>XOR</strong> (⊕) is “addition mod 2”: <strong>0⊕0 = 0, 0⊕1 = 1, 1⊕0 = 1, 1⊕1 = 0.</strong> It gives 1 when the bits differ. Doing it twice undoes it, since a ⊕ a = 0. The first quantum algorithms (<strong>Deutsch–Jozsa, Bernstein–Vazirani</strong>) are built on this.</p><p>Finally, a <strong>derivative</strong> is a slope: how much the output changes for a tiny change in the input. Training any model, classical or quantum, means computing slopes and walking downhill. Take a loss L(θ) = (cos θ − 1)². The chain rule gives L′(θ) = 2(cos θ − 1)(−sin θ). At θ = π/2 that is 2(0 − 1)(−1) = 2. The slope is positive, so to reduce the loss you decrease θ. That’s one training step of a quantum classifier, done by hand.</p><h3>Complex numbers: the numbers quantum mechanics is made of</h3><p>Ordinary numbers answer “how much?”. Complex numbers answer a richer question: <strong>“how much, and in which direction?”</strong> A complex number is a little arrow on a flat page, with a length and an angle.</p><p>Here’s why that matters. In quantum mechanics, every possible outcome gets one of these arrows, called an <strong>amplitude</strong>. To get the probability of an outcome, you take the arrow’s length and square it. When two different paths lead to the same outcome, you add their arrows tip to tail. If they point the same way, the outcome becomes more likely. If they point opposite ways, they cancel and the outcome becomes impossible.</p><p>That cancelling is called <strong>interference</strong>, and it’s the main trick behind quantum algorithms. Ordinary positive numbers can’t do it. This is the most important part of the post.</p><h4>Why we needed a new kind of number</h4><p>For centuries, mathematicians refused to accept √−1. Then in the 1500s, Italian mathematicians solving cubic equations (<strong>Cardano, Bombelli</strong>) noticed something strange. Their formulas sometimes passed through √−1 in the middle and still came out with a perfectly real, correct answer. Throwing away the “impossible” numbers gave wrong answers, so they kept them and grudgingly called them “imaginary”. The name is misleading: negative numbers were once called absurd too (“how can you have −3 cows?”). Around 1800, <strong>Wessel and Argand</strong> showed you can draw complex numbers as points on a plane, and suddenly they made sense. In 1926, <strong>Schrödinger’s </strong>equation for the atom had an i built right into it, and complex numbers became the language of physics.</p><p>Here’s a picture that makes i feel natural. Multiplying by −1 means turning around, a 180° turn on the number line. Which number, applied <em>twice</em>, turns you around? A 90° turn, because two quarter-turns make a half-turn. That quarter-turn is <strong>i</strong>, and i × i = −1 just says “two quarter-turns make a half-turn”. A quarter-turn also takes you off the number line into a second direction, which is why complex numbers live on a plane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xJtyNkJFviMCttF3Jh7NLA.png" /></figure><p><em>Multiplying by i rotates by 90°. Doing it twice rotates by 180°, which is the same as multiplying by −1.</em></p><p>So the definition is i² = −1, and a complex number is z = a + ib, with a real part <strong>a = Re z</strong> and an imaginary part <strong>b = Im z</strong>. Powers of i go round in a cycle of four: i⁰ = 1, i¹ = i, i² = −1, i³ = −i, i⁴ = 1. To find i⁷, divide 7 by 4. The remainder is 3, so i⁷ = −i. For i¹⁰², the remainder is 2, so it’s −1.</p><h4>The complex plane</h4><p>Put the real part on the horizontal axis and the imaginary part on the vertical axis. Then 3 + 2i is just the point “3 right, 2 up”. Better still, think of it as an arrow from the origin to that point. This picture is called the <strong>complex plane</strong> (or Argand diagram).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2fFBJCze8K83HCUdQriPxw.png" /></figure><p><em>z = 3 + 2i as an arrow. Its mirror image in the real axis is the conjugate z</em>.*</p><h4>Adding, multiplying and the conjugate</h4><p><strong>Adding</strong>: add the real parts and the imaginary parts separately. (3 + 2i) + (1 − 5i) = 4 − 3i. In the picture, you put the arrows tip to tail, like adding walking directions. And (2 + i) − (2 + i) = 0: two equal and opposite arrows cancel perfectly. Hold on to that image. Adding amplitudes tip to tail <em>is</em> quantum interference.</p><p><strong>Multiplying</strong>: expand the brackets like ordinary algebra and replace i² by −1. (3 + 2i)(1 + 4i) = 3 + 12i + 2i + 8i² = −5 + 14i. And (1 + i)² = 1 + 2i + i² = 2i. What multiplying <em>looks like</em> is much clearer in polar form, coming up in a moment. Spoiler: it scales the length and rotates the angle.</p><p>The <strong>conjugate</strong> z* flips the sign of the imaginary part, so the conjugate of a + ib is a − ib. It reflects the arrow in the horizontal axis. It matters because z times z* is always a real, non-negative number: the length squared.</p><blockquote><strong><em>z z* = (a + ib)(a − ib) = a² + b² = |z|²</em></strong></blockquote><p>Probabilities in quantum mechanics are exactly these squared lengths. Whenever you see a * or a † in a quantum formula, a conjugate is quietly making sure the probabilities come out real and positive. For z = 3 + 2i, |z|² = 9 + 4 = 13 and the length (the <strong>modulus</strong>) is |z| = √13.</p><p>One rule about lengths explains interference in a nutshell: |zw| = |z||w| always, but |z + w| is usually <em>not</em> |z| + |w|. It can be anything from ||z| − |w|| (arrows cancelling) to |z| + |w| (arrows reinforcing).</p><p><strong>Dividing</strong> uses a trick: multiply the top and bottom by the conjugate of the bottom, which makes the bottom real. For example, 1/i = −i/(i·(−i)) = −i. And (2 + i)/(1 − i) = (2 + i)(1 + i)/2 = (1 + 3i)/2.</p><h4>Polar form and Euler’s formula</h4><p>“3 right, 2 up” (the Cartesian form a + ib) is good for adding. “Go √13 steps in the direction 33.7°” (the <strong>polar form</strong>, with length r and angle φ) is good for multiplying. Quantum computing multiplies by “phase factors” all the time, so polar form is the one you’ll use most. The two are linked by a = r cos φ and b = r sin φ, so z = r(cos φ + i sin φ).</p><p>Be careful with the angle. tan φ = b/a can’t tell 1 + i (at 45°) from −1 − i (at 225°), because both have b/a = 1. Always check the signs of a and b, or use <strong>atan2</strong>. For example, z = −√3 + i has r = √(3 + 1) = 2 and sits in the second quadrant, so φ = 180° − 30° = 150°.</p><p>Now the formula that makes everything click:</p><blockquote><strong><em>e^(iφ) = cos φ + i sin φ</em></strong></blockquote><p>This is Euler’s formula. It says that e^(iφ), “e to an imaginary power”, is simply <strong>the arrow of length 1 pointing at angle φ</strong>. As φ grows, e^(iφ) walks round the unit circle. It turns the abstract idea “exponential” into the concrete idea “rotation”. Since quantum gates are rotations, you’ll be writing e^(something) on every page from here on. Any complex number can now be written as z = r e^(iφ).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A-nGSeSvui_MflczdmvhQQ.png" /></figure><p><em>e^(iφ) is a point on the unit circle. The marked values come up again and again.</em></p><p>Why is it true? Take the power series eˣ = 1 + x + x²/2! + x³/3! + …, put in x = iφ, and use i² = −1, i³ = −i, i⁴ = 1. The real terms collect into the series for cos φ, and the imaginary terms into the series for sin φ. The length is always cos²φ + sin²φ = 1, so e^(iφ) is pure direction.</p><p>The values you’ll meet again and again are e^(i0) = 1, e^(iπ/2) = i, e^(iπ) = −1, e^(i3π/2) = −i, e^(i2π) = 1 and e^(iπ/4) = (1 + i)/√2, which is the phase of the T gate. Plug in φ = π and you get e^(iπ) + 1 = 0, which links five of the most famous numbers in maths.</p><h4>Multiplying means rotating and scaling</h4><p>Using the exponent rule eˣ eʸ = eˣ⁺ʸ:</p><blockquote><strong><em>(r₁e^(iφ₁)) (r₂e^(iφ₂)) = r₁r₂ e^(i(φ₁ + φ₂))<br> Multiply the lengths, add the angles.</em></strong></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nBlosilSrYAio8gTjF3pmw.png" /></figure><p><em>z·w has length √2 × 1.5 ≈ 2.12 and angle 45° + 30° = 75°.</em></p><p>This is the heart of it. Multiplying by e^(iφ), which has length 1, <strong>only rotates</strong>. It never changes the length. So multiplying an amplitude by a phase factor never changes the probability of that outcome on its own. What it changes is the arrow’s direction, and that changes how the amplitude will interfere with others later. That’s exactly what the phase gates Z, S and T do.</p><p>Here’s a bonus. Remember the trig addition formulas from earlier? Write e^(i(a+b)) = e^(ia)e^(ib) = (cos a + i sin a)(cos b + i sin b). Expand, and compare the real and imaginary parts with cos(a + b) + i sin(a + b). Both addition formulas drop out for free.</p><p>Powers become easy too. <strong>De Moivre’s </strong>theorem says (re^(iφ))ⁿ = rⁿe^(inφ). Since 1 + i = √2 e^(iπ/4), we get (1 + i)⁸ = (√2)⁸ e^(i2π) = 16. Try getting that by expanding brackets. Square roots work too: i = e^(iπ/2), so √i = e^(iπ/4) = (1 + i)/√2. That little fact is why IBM hardware has a √X (“square root of NOT”) gate.</p><h4>Roots of unity: the numbers behind the Fourier transform</h4><p>Cut a pizza into N equal slices. The points where the cuts meet the crust are the N-th roots of unity: ω = e^(2πi/N) and its powers ωᵏ for k = 0 to N − 1. Multiplied by itself N times, each one lands back on 1.</p><p>They’re spread evenly round the circle like spokes, so they add up to zero (for N &gt; 1). The geometric series proves it: 1 + ω + … + ω^(N−1) = (1 − ωᴺ)/(1 − ω) = (1 − 1)/(1 − ω) = 0. For N = 4, ω = i and the roots are 1, i, −1 and −i, and indeed 1 + i − 1 − i = 0.</p><p>That “spokes add up to zero” fact is why the Quantum Fourier Transform works, and the QFT sits inside phase estimation and Shor’s algorithm. We’ll come back to it in the algorithms part of the series.</p><h4>Interference: the whole secret</h4><p>Now put it together. An outcome’s probability is |amplitude|². Suppose two routes lead to the same result, with amplitudes z₁ and z₂. Nature adds the amplitudes first and then squares: P = |z₁ + z₂|². If you added probabilities instead, the way you would for ordinary coins, you’d get |z₁|² + |z₂|². The difference between the two is the <strong>interference term</strong>:</p><blockquote><strong><em>|z₁ + z₂|² = |z₁|² + |z₂|² + 2|z₁||z₂| cos(φ₁ − φ₂)</em></strong></blockquote><p>The last term depends only on the <strong>phase difference</strong> φ₁ − φ₂, and it can be negative:</p><ul><li>Phase difference 0: cos = 1, the strongest <strong>constructive</strong> interference.</li><li>Phase difference π: cos = −1, <strong>destructive</strong>. If the two sizes are equal, the outcome becomes impossible.</li><li>Phase difference π/2: cos = 0, no interference. The probabilities just add.</li></ul><p><strong>Worked example: a quantum coin.</strong> Two paths each have an amplitude of size ½. Then P = ¼ + ¼ + 2 · ½ · ½ · cos Δ = ½ + ½ cos Δ, where Δ is the phase difference. With Δ = 0 the outcome is certain (P = 1). With Δ = π/2, P = ½. With Δ = π it never happens (P = 0). The same formula comes back in the <strong>Hadamard–Hadamard</strong> circuit and in QAOA.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jLQ9BZhywlCraB8y6LtOJw.png" /></figure><p><em>Adding amplitudes before squaring lets the phase push the probability anywhere from 0 to 1.</em></p><h4>Global phase vs relative phase</h4><p>If you rotate <em>every</em> amplitude by the same angle, all the arrows turn together. Their relative directions don’t change, so no interference pattern changes and no experiment can ever notice. That’s a <strong>global phase</strong>, and we ignore it. In maths terms, |e^(iγ)z|² = |z|², and every phase difference stays the same.</p><p>If you rotate only <em>some</em> amplitudes, their relative angle changes and future interference changes. That’s a <strong>relative phase</strong>, and it’s physically real. Think of turning a whole photo versus turning one person in it.</p><p>So the amplitude pairs (1/√2, 1/√2) and (i/√2, i/√2) describe the same physical state, because they differ only by the global factor i. But (1/√2, 1/√2) and (1/√2, −1/√2) are different states, because the second amplitude has a relative phase of −1. You’ll meet that pair later as |+⟩ and |−⟩.</p><h4>A preview: complex numbers inside vectors</h4><p>In Part 2 we’ll put complex numbers into vectors and matrices. The conjugate comes along for the ride. The length of a complex list uses conjugates: ‖v‖² = Σ |vₖ|² = Σ vₖ* vₖ, never Σ vₖ².</p><p>Here’s why that matters. Take v = (1, i). Without conjugates, 1²+ i² = 1 − 1 = 0, which would say a non-zero list has length zero. That’s nonsense. With conjugates, |1|² + |i|² = 2, so the length is √2. The rule to remember: always conjugate one side.</p><h4>Does nature really need complex numbers?</h4><p>Fair question. Negative numbers already allow cancelling (+1 and −1), so why not stop there? There are three reasons:</p><ol><li><strong>Smooth change.</strong> A qubit’s state changes continuously in time. To go smoothly from + to − without passing through zero (which would mean the state vanished), an amplitude has to swing round through a second direction. Rotation needs two dimensions.</li><li><strong>The equation of motion</strong>. <strong>Schrödinger’s equation</strong>, iħ d|ψ⟩/dt = H|ψ⟩, has an i built into it. Its solutions are spinning arrows e^(−iEt/ħ), where the spin speed is the energy.</li><li><strong>Experiments.</strong> In 2021, <strong>Renou and colleagues </strong>showed that a version of quantum theory using only real numbers predicts different results from the complex version in certain entanglement experiments. Labs ran those experiments in 2022, and the real-number version lost. Nature really does use complex numbers; they aren’t just a convenient tool.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2cb503ea9bf6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Deep Learning? The Intuitive Guide to AI’s Most Powerful Tool]]></title>
            <link>https://medium.com/@lalanagurusinghe/what-is-deep-learning-the-intuitive-guide-to-ais-most-powerful-tool-99e87f180476?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/99e87f180476</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Mon, 22 Jun 2026 11:55:50 GMT</pubDate>
            <atom:updated>2026-06-22T11:55:50.976Z</atom:updated>
            <content:encoded><![CDATA[<p>You’ve heard the term everywhere. ChatGPT, image recognition, self-driving cars, recommendation systems that know what you want before you do. All powered by deep learning.</p><p>But what actually is it?</p><p>Most explanations throw you into neural networks and backpropagation without answering the fundamental question: Why does it work so differently from traditional machine learning? What makes it “deep”? And more importantly, when should you actually use it?</p><p>Let’s start from the ground up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ThUTfpiCRksFAVRGaMCHmQ.jpeg" /></figure><h3>Understanding AI, ML, and DL</h3><p>Here’s where it all lives in the ecosystem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4BsP1GlbSTpY4OmHOCEo1Q.jpeg" /></figure><p><strong>Artificial Intelligence (AI)</strong> is the broadest umbrella. It’s any technique that makes a machine exhibit intelligent behavior. Chess algorithms, chatbots, recommendation engines, self-driving cars, they’re all AI.</p><p><strong>Machine Learning (ML)</strong> is AI’s smarter sibling. Instead of humans hardcoding rules, the machine learns patterns from data. You show it examples, it finds the rules. Feed it 10,000 emails, it learns what spam looks like.</p><p><strong>Deep Learning (DL)</strong> is ML’s specialized powerhouse. It’s machine learning that uses neural networks with many layers (hence “deep”). Instead of you engineering the features by hand, the network learns its own features at each layer.</p><p>Think of it like this:</p><p>AI: “Make a smart system” ML: “Learn from data” DL: “Learn hierarchical features from data”</p><p>The key difference? Traditional machine learning needs you to do the heavy lifting. You decide which features matter. You normalize the data. You design the pipeline.</p><p>Deep learning says: “Give me raw data and a flexible architecture. I’ll figure out what features matter at each layer.”</p><h3>Why Deep Learning Won (And When It Didn’t)</h3><p>Deep learning dominated the last 15 years. Image recognition, natural language processing, speech synthesis, game-playing AI all conquered by deep learning.</p><p>But here’s the truth nobody talks about: <strong>deep learning is not always the right choice.</strong></p><p>When Deep Learning Crushes It:</p><p>You have massive amounts of data. Deep learning needs thousands or millions of examples to shine. With 10,000 images, it beats everything. With 100 images, it’s useless.</p><p>The problem is unstructured. Images, text, audio, video these are deep learning’s domain. CNNs learned to see, Transformers learned to understand language, RNNs learned to process sequences.</p><p>The relationships are hierarchical. A cat image is <strong>pixels → edges → textures → parts → cat</strong>. Each layer builds on the previous. Deep learning discovers this automatically.</p><p>Computational power is available. Deep learning demands GPUs. If you can’t afford the compute, traditional ML is your friend.</p><p>When Traditional Machine Learning Still Wins:</p><p>You have small datasets. Logistic regression, Random Forests, and XGBoost work beautifully with 1000 examples. Deep learning fails.</p><p>The data is tabular (spreadsheets). Your features are already extracted. Traditional ML algorithms like Gradient Boosting (XGBoost, LightGBM) beat deep learning here. Always.</p><p>Interpretability is critical. You need to explain to a regulator why the model made a decision. Deep learning is a black box. Tree-based models show feature importance.</p><p>Speed matters. A trained XGBoost model makes predictions in milliseconds without a GPU. A deep network requires a GPU and is slower.</p><p>The Real World Choice:</p><p>Start with XGBoost on tabular data. Start with deep learning on images, text, or audio. Not the other way around.</p><h3>The Deep Learning Stack: From Perception to Reasoning</h3><p>Why does deep learning work so well for perception tasks?</p><p>Because it learns features the same way your brain does: hierarchically.</p><p>Layer 1 learns simple edges and colors. Layer 2 combines those into textures. Layer 3 combines textures into shapes. Layer 4 combines shapes into parts (eyes, nose, ears). Layer 5 recognizes the object (cat, dog, person).</p><p>Each layer is a level of abstraction. Each level stands on the shoulders of the previous one. That’s why it’s called “deep” — not because of depth in space, but depth in understanding.</p><p>And that’s why it works for language too. The first layers learn grammar and syntax. Middle layers learn semantic meaning. Later layers learn reasoning and context.</p><p>It’s abstraction all the way up.</p><h3>A Quick Look at the Architecture Landscape</h3><p>Deep learning comes in flavors. Each is specialized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2X1VhakZg9pvBKIdh1P9Zg.jpeg" /></figure><p><strong>Fully Connected Networks (Dense Networks):</strong> Every neuron connects to every neuron in the next layer. They work but are massive and slow for images. Perfect for structured data.</p><p><strong>Convolutional Neural Networks (CNNs):</strong> Optimized for images. They use filters to detect patterns (edges, textures, shapes) at multiple scales. This is why they crush computer vision.</p><p><strong>Recurrent Neural Networks (RNNs): </strong>Memory-based. They process sequences. LSTMs and GRUs are the refined versions. They power language models, time series prediction, and anything sequential.</p><p><strong>Transformers:</strong> The newcomers. They use attention mechanisms to understand relationships between distant elements. They’re why ChatGPT understands context across paragraphs. They’re slower to train but devastatingly powerful.</p><p><strong>Autoencoders:</strong> Compression machines. They learn to compress data and reconstruct it. Useful for anomaly detection, dimensionality reduction, and unsupervised learning.</p><p><strong>Generative Adversarial Networks (GANs):</strong> Two networks in a duel. One generates fake data, one tries to catch it. The result? Synthetic images, deepfakes, and creative models that understand distribution.</p><p>You don’t need to know all of them yet. But know they exist. Different architectures solve different problems.</p><h3>The Training Loop: Intuition Before Math</h3><p>Here’s the process that makes deep learning “learn.”</p><p><strong>Step 1: Make a Guess</strong></p><p>You start with random weights. Feed an image of a cat to your network. It has no idea what it’s looking at. It might guess “dog” or “toaster.” Completely wrong.</p><p><strong>Step 2: Measure the Damage</strong></p><p>You compare the prediction to the correct answer. The difference is the error or loss. The bigger the error, the further you are from the truth.</p><p><strong>Step 3: Adjust and Repeat</strong></p><p>Here’s the magic: the network traces this error backward through all its layers. Each weight asks: “Did I contribute to this mistake?” If yes, adjust yourself. If no, stay the way you are.</p><p>This backward flow is backpropagation. It’s how the network learns.</p><p><strong>Step 4: Do It a Million Times</strong></p><p>You repeat this process with thousands of examples. Each iteration, the weights get slightly better. After a million adjustments, the network can recognize cats.</p><p>This is optimization. We’re sliding down a hill toward “less error.” The shape of that hill is the loss landscape, and it’s bumpy, chaotic, and sometimes full of dead ends (local minima).</p><p>That’s why this is hard. Not the concept. The implementation.</p><h3>Why Now? Why Is This Decade Different?</h3><p>Deep learning isn’t new. The math dates back to the 1980s. But three things changed in 2012.</p><ol><li><strong>Data Explosion</strong></li></ol><p>The internet created massive datasets. ImageNet released 14 million labeled images. Suddenly, deep learning had the fuel it needed. You can’t learn from 1000 examples. You need millions.</p><ol><li><strong>Computational Power</strong></li></ol><p>GPUs went mainstream. Graphics cards can perform billions of floating-point operations per second. A task that took weeks on a CPU takes hours on a GPU. The math didn’t change. The speed did.</p><ol><li><strong>Better Algorithms</strong></li></ol><p>Researchers figured out how to train deep networks. ReLU activation functions. Batch normalization. Dropout. Careful weight initialization. These techniques made training stable instead of chaotic.</p><p>Without all three, deep learning stays theoretical. With all three, it becomes practical.</p><h3>The Real Limitation: Data, Not Magic</h3><p>This is crucial. Deep learning is a data-hungry beast.</p><p>A CNN for image classification needs 10,000 images to be decent. 100,000 to be good. A million to be state of the art.</p><p>A language model needs billions of words.</p><p>A recommendation system needs millions of user interactions.</p><p>Deep learning doesn’t create knowledge from nothing. It finds patterns in the data you give it. No data, no learning. Garbage data, garbage patterns.</p><p>This is why traditional ML still dominates in industries with small datasets. Healthcare doesn’t have millions of labeled patient records. Finance can’t afford to expose data for training. In these domains, XGBoost and Logistic Regression are still king.</p><h3>One Last Thing</h3><p>Deep learning feels like magic because it works so well. But it’s not magic. It’s layers of simple operations, repeated millions of times, guided by a loss function that whispers: “You’re getting warmer.”</p><p>The magic is in the scale. In the depth. In the patience to adjust billions of weights until the pattern emerges.</p><p>Now you know what you’re getting into.</p><p>Let’s build some intuition. Episode 2 is waiting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=99e87f180476" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From Raw Data to Machine Learning Ready: The 5 Essential Steps of Data Prep]]></title>
            <link>https://medium.com/@lalanagurusinghe/from-raw-data-to-machine-learning-ready-the-5-essential-steps-of-data-prep-8e6cb4507042?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e6cb4507042</guid>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Tue, 26 May 2026 15:35:46 GMT</pubDate>
            <atom:updated>2026-05-26T15:35:46.199Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UqgairK1p-3c-0odUkpHEA.png" /></figure><p>You’ve got your dataset. You’re excited. You fire up Jupyter Notebook, import pandas, and you’re ready to build the next great machine learning model.</p><p>Then you look at the data.</p><p>Nulls everywhere. A salary value of $9,999,999. A column that says “Yes”, “No”, and “Maybe”. And feature values ranging from 0.001 to 850,000 sitting right next to each other.</p><p>Welcome to the real world of data science.</p><p><strong>The model is only as good as the data you feed it.</strong> Garbage in, garbage out always. The good news? There’s a clean, repeatable process for turning that messy raw data into something your model will actually thank you for.</p><p>In this post, we’re walking through the <strong>5 essential steps of data preprocessing</strong> the unsung hero of every successful ML project.</p><p>Let’s get into it.</p><p>🗺️ The Road Map</p><p>Here’s where we’re headed:</p><ol><li>Removing Missing Values</li><li>Removing Outliers</li><li>Feature Binning</li><li>Feature Encoding</li><li>Feature Scaling</li></ol><p>Each step builds on the last. By the end, your dataset will be clean, structured, and completely machine-readable. Let’s go.</p><h3>Step 1: Removing Missing Values Don’t Feed Your Model Empty Plates</h3><h3>Why Missing Data Is a Big Problem</h3><p>Imagine you’re trying to predict house prices, and 20% of your “square footage” column is blank. Your model doesn’t know what to do with a blank. It doesn’t shrug and move on it either crashes, produces NaN errors, or silently makes terrible predictions.</p><p>Missing values are landmines. And most real-world datasets are full of them.</p><h4>How to Find Them</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/442/1*36sN1wzjgez6-QI_4GAo9A.png" /></figure><h4>Strategy 1: Drop Them</h4><p>If a column is missing more than 40–50% of its values, it’s probably not saving you much. Cut it loose.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/466/1*QnpO0fDF2zv_N_rPVDAMTA.png" /></figure><p><strong>When to use it:</strong> When the missing data is random, and you have enough rows to spare.</p><h4>Strategy 2: Impute Them (Fill the Gaps Smartly)</h4><p>Dropping rows means losing data. Sometimes it’s smarter to fill missing values with a reasonable estimate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/589/1*6mjjM0ZlaxRsq0Kl444sVg.png" /></figure><p><strong>Pro tip:</strong> Use the <em>median</em> for numerical data, not the mean the median is resistant to outliers (which we’re about to tackle next).</p><h3>Step 2: Removing Outliers Evicting the Troublemakers</h3><h3>Why Outliers Are Dangerous</h3><p>An outlier is a data point that lives far, far away from the rest of the neighborhood. Think of a dataset of employee salaries where everyone earns between $40,000 and $120,000 and then there’s one entry for $47,000,000.</p><p>That one value will completely warp your model. Linear regression, for example, tries to minimize the distance between its predictions and actual values. One extreme outlier pulls the entire line toward it like a magnet.</p><p>The result? A model that’s great at predicting for that one weird data point, and terrible for everyone else.</p><h4>Method 1: The IQR Method (Interquartile Range)</h4><p>This is the most commonly used, intuitive approach. It defines a “normal” range based on where the middle 50% of your data lives.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*Y77Wodp-o9ikbfcwm4cU6Q.png" /></figure><p>Any value below Q1 - 1.5*IQR or above Q3 + 1.5*IQR gets flagged as an outlier. Clean and simple.</p><h4>Method 2: The Z-Score Method</h4><p>Z-scores measure how many standard deviations a data point is from the mean. If it’s more than 3 standard deviations away it’s almost certainly an outlier.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/398/1*UB3IzKykKKouioDnKoUhUA.png" /></figure><p><strong>When to use which?</strong></p><ul><li>Use <strong>IQR</strong> when your data is skewed or not normally distributed.</li><li>Use <strong>Z-score</strong> when your data is roughly bell-shaped (normal distribution).</li></ul><h3>Step 3: Feature Binning : Turning Numbers Into Meaningful Categories</h3><h3>What Is Binning?</h3><p>Here’s where things get interesting.</p><p>Binning (also called discretization) is the process of converting a continuous numerical variable into discrete categories or “bins.” Instead of working with a raw number like 724, you assign it to a category like &quot;Good&quot;.</p><p>Classic example — Credit Scores:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*tqrJS9czUD0pvoCSTuHW6A.png" /></figure><p>So 720 doesn&#39;t stay 720. It becomes &quot;Good&quot;. Clean, interpretable, and often more useful for certain models.</p><h4>Why Would You Do This?</h4><ul><li><strong>Reduces noise:</strong> Small fluctuations between 718 and 722 probably don’t matter. Binning treats them the same.</li><li><strong>Captures non-linear relationships:</strong> A model might not naturally know that the jump from 579 to 580 is significant (crossing from “Poor” to “Fair”). Bins make that explicit.</li><li><strong>Handles outliers gracefully:</strong> Extreme values just fall into the top or bottom bin, instead of skewing the whole model.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/516/1*DgOir8g1VnL3uN6n3vZ9Vg.png" /></figure><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/402/1*65Ap-ikFCV1nz1o5FvW3gg.png" /></figure><p><strong>But wait</strong> : you just turned numbers into text categories. Doesn’t the machine need numbers? Yes! And that’s exactly why Step 4 exists. This is intentional. Keep reading.</p><h3>Step 4: Feature Encoding : Translating Human Language Back to Machine Language</h3><h3>The Brilliant Loop: Numbers → Categories → Numbers (Again)</h3><p>Here’s the move that confuses a lot of beginners, so let’s break it down clearly.</p><p>In Step 3, we <em>deliberately</em> turned a number (720) into a category (&quot;Good&quot;). Why? Because that category carries <em>semantic meaning</em> it tells the model about quality and ordering, not just quantity.</p><p>But machines can’t process the word &quot;Good&quot;. They only speak numbers.</p><p>So in Step 4, we translate those meaningful categories back into numbers but in a <em>smarter</em> way than the original raw values. The numbers now carry structure and meaning.</p><p><strong>That’s the strategy:</strong> Raw number → Meaningful category → Structured number.</p><h4>First, Know Your Data Type</h4><p>Before encoding, you need to ask: <em>does order matter?</em></p><p><strong>Nominal data</strong> : Categories with no natural order. Examples: City, Color, Product Type → &quot;Red&quot; is not greater than &quot;Blue&quot;. They&#39;re just different.</p><p><strong>Ordinal data</strong> : Categories with a clear order. Examples: Credit Score Tier, Education Level, Satisfaction Rating → &quot;Excellent&quot; is genuinely better than &quot;Good&quot;.</p><h4>Encoding Ordinal Data: Label Encoding</h4><p>When order matters, we assign integers that reflect that order.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/665/1*9vtEd-T155dM4v-5PZeElw.png" /></figure><p>Output:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/506/1*olbzWJUWgEOU_yA8KlLilg.png" /></figure><p>The model now knows 4 &gt; 2 &gt; 0 which perfectly reflects the real-world meaning.</p><h4>Encoding Nominal Data: One-Hot Encoding</h4><p>When order does <em>not</em> matter, label encoding is dangerous. If you encode Red=0, Blue=1, Green=2, the model might incorrectly assume Green &gt; Blue &gt; Red. That&#39;s nonsense.</p><p>Instead, we use <strong>One-Hot Encoding</strong> each category becomes its own binary column.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/576/1*_pF4u-sM7PdruAmbIMLTpw.png" /></figure><p>Output (if cities were London, Paris, Tokyo):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/323/1*kgg27ff2PfMO7UEzuK8y0A.png" /></figure><p>drop_first=True removes one column to avoid multicollinearity (the &quot;dummy variable trap&quot;). London becomes the reference when both city_Paris and city_Tokyo are 0, the model knows it&#39;s London.</p><p><strong>Quick Decision Guide:</strong></p><ul><li>Order matters → Label Encoding (with a custom map)</li><li>Order doesn’t matter → One-Hot Encoding (pd.get_dummies)</li></ul><h3>Step 5: Feature Scaling Putting Everyone on a Level Playing Field</h3><h3>The Problem: Not All Features Are Created Equal</h3><p>Imagine your dataset has two features:</p><ul><li>bedrooms: values range from 1 to 6</li><li>square_footage: values range from 400 to 5,000</li></ul><p>To a machine learning algorithm, square_footage looks astronomically more important purely because its numbers are bigger. The model will spend most of its time optimizing for that feature and practically ignore bedrooms.</p><p>But bedrooms might be just as predictive! The model is being fooled by <em>scale</em>, not by actual importance.</p><p>Feature scaling fixes this by bringing all features to a comparable range.</p><h4>Method 1: Min-Max Scaling (Normalization)</h4><p>This squishes all values into a range between 0 and 1.</p><p><strong>Formula:</strong> X_scaled = (X - X_min) / (X_max - X_min)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/504/1*oMKGKNYKl0QrLg_Ybs-E6A.png" /></figure><ul><li>A house with the minimum square footage → 0.0</li><li>A house with the maximum square footage → 1.0</li><li>Everything else lands somewhere in between.</li></ul><p><strong>Best for:</strong> When you know the data has a fixed range, or when you’re using neural networks / image data.</p><h4>Method 2: Standardization (Z-score Scaling)</h4><p>This transforms features so they have a <strong>mean of 0</strong> and a <strong>standard deviation of 1</strong>.</p><p><strong>Formula:</strong> X_scaled = (X - mean) / std</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/509/1*7aWGsWMAkz6IZgZ9QHjF6Q.png" /></figure><p>Now both features are centered at 0. A value of 2.0 means &quot;2 standard deviations above average.&quot; The model treats them fairly.</p><p><strong>Best for:</strong> When your data has outliers, or when using algorithms that assume a normal distribution (like linear regression, logistic regression, SVM).</p><h4>When Do You Actually Need Scaling?</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/628/1*FB5GV8TEUO-HCYf3t0IuLw.png" /></figure><p>Tree-based models split on thresholds, so scale doesn’t affect them. Distance-based and gradient-based models are extremely sensitive to it.</p><h3>🎉 Your Dataset’s Journey: A Summary Checklist</h3><p>Your data just went through boot camp. Here’s everything it survived:</p><ul><li>✅ <strong>Missing Values Handled</strong> : Dropped empty columns, imputed missing values with median/mode</li><li>✅ <strong>Outliers Removed</strong> : Used IQR or Z-score to evict extreme values that would warp the model</li><li>✅ <strong>Feature Binning Applied</strong> : Converted continuous numbers into meaningful, interpretable categories</li><li>✅ <strong>Features Encoded</strong> : Translated categories back into numbers using Label Encoding (ordinal) or One-Hot Encoding (nominal)</li><li>✅ <strong>Features Scaled</strong> : Normalized or standardized all numerical features so no column bullies the others</li></ul><p>What started as a chaotic, unusable raw dataset is now clean, structured, and completely machine-readable.</p><h3>One Last Thing</h3><p>Data preprocessing isn’t a one size fits all checklist. Every dataset has its own personality quirks. Some steps you’ll use every time. Others you’ll skip entirely depending on your data and your algorithm.</p><p>But now you know <em>what</em> each step does, <em>why</em> it matters, and <em>how</em> to implement it. That puts you miles ahead of someone who just throws raw data at a model and wonders why the accuracy is terrible.</p><p>Go prep that data. Your model is waiting.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e6cb4507042" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Decision Trees & Ensemble Models — A Complete Theory Guide]]></title>
            <link>https://medium.com/@lalanagurusinghe/decision-trees-ensemble-models-a-complete-theory-guide-b92b104418e8?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/b92b104418e8</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[probability]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Tue, 05 May 2026 10:53:44 GMT</pubDate>
            <atom:updated>2026-05-05T10:53:44.928Z</atom:updated>
            <content:encoded><![CDATA[<p>If you have spent any time in machine learning, you have heard the phrase “tree-based models.” Random Forest, XGBoost, CatBoost, LightGBM — these names appear constantly in Kaggle leaderboards, production ML systems, and interview questions. They all share the same foundation: the humble decision tree.</p><p>Before you can truly understand why ensemble methods are so powerful, you need to understand what a single decision tree does, how it makes decisions mathematically, where it breaks down, and what that breakdown reveals about the nature of learning itself. This article goes deep on all of that.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YEiJ3X_mIwz6VHbEKo1kkQ.png" /></figure><h3>1. What is a Decision Tree?</h3><p>A decision tree is a model that makes predictions by asking a sequence of yes/no questions about the input features. Each question is called a split. The tree starts at the root node with the full dataset, splits it into two groups based on one feature and one threshold, then recursively splits each group again, until it reaches a stopping point. At that stopping point called a leaf node it makes a prediction. For classification, that prediction is the most common class in the leaf. For regression, it is the average value.</p><p>Here is a concrete example. Suppose you are predicting whether a student will pass or fail based on hours studied and assignments completed. The tree might first ask: “Did the student complete more than 6 assignments?” If yes, go right. If no, go left. On the left branch, it might ask: “Did the student study more than 40 hours?” And so on. Each path from root to leaf is a decision rule that you can read out loud and explain to a non-technical person that is one of the biggest strengths of this model family.</p><p>The algorithm that builds the tree is greedy. At every node, it searches through every possible feature and every possible threshold for that feature, evaluates how “good” each split would be using a mathematical criterion, and picks the best one. It does not look ahead it just makes the locally best decision at each step. This greedy nature is both its practical strength (it is fast) and its theoretical weakness (greedy solutions are rarely globally optimal).</p><h4>The structure of the algorithm</h4><p>To build a tree from scratch, the algorithm does the following. Start with the full training dataset at the root. Evaluate every possible binary split for each feature, sort the values, try every midpoint between consecutive values as a threshold. For each candidate split, compute an impurity score for the left child and right child, then compute the weighted average impurity of the two children. The split that gives the biggest reduction in impurity wins. Partition the data accordingly, then recursively repeat for each child node. Stop when the node is pure (all one class), when the depth limit is reached, when the node has too few samples, or when no split offers any improvement.</p><h3>2. The Mathematics of Splitting</h3><p>This is the heart of the whole thing. The word “impurity” keeps coming up but what does it mean formally? There are three quantities you absolutely must understand: Entropy, Gini Impurity, and Information Gain (and its Gini counterpart, Gini Gain).</p><h4>Entropy</h4><p>Entropy comes from information theory, specifically from Claude Shannon’s 1948 work on communication. It measures the amount of disorder or uncertainty in a set of labels. If every sample in a node belongs to the same class, there is zero uncertainty you already know the answer before looking at any feature. If the classes are perfectly balanced, you have maximum uncertainty.</p><p>The formula is:</p><h4><strong>H(S) = − ∑ pᵢ · log₂(pᵢ)</strong> for i = 1 to c</h4><p>where S is the current set of samples, c is the number of classes, and pᵢ is the proportion of samples belonging to class i.</p><p>Let’s understand every piece of this. The pᵢ · log₂(pᵢ) term measures the contribution of class i to the total uncertainty. The log₂ means we are measuring uncertainty in bits (binary digits of information). The negative sign in front makes the whole thing positive, because log₂ of a value between 0 and 1 is always negative.</p><p>For binary classification (two classes), entropy becomes:</p><h4><strong>H = − p · log₂(p) − (1 − p) · log₂(1 − p)</strong></h4><p>When p = 0.5 (equal classes), H = 1.0 maximum entropy. When p = 0 or p = 1 (pure node), H = 0. This makes perfect intuitive sense: a node where everything is the same class tells you nothing new, so it carries no uncertainty.</p><p><strong>Intuition:</strong> Imagine you have a box of 100 balls. If all 100 are red, picking one at random tells you nothing surprising entropy = 0. If 50 are red and 50 are blue, you have maximum suspense about what you will pull entropy = 1 The tree’s job is to create boxes (nodes) where the balls are as uniform as possible.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/565/1*5PiMyLMnyRSxTbD5q3CEVQ.png" /></figure><h4>Gini Impurity</h4><p>Gini Impurity is a second way to measure node impurity, introduced as a computationally cheaper alternative to entropy. It answers a slightly different question: if you randomly pick a sample from the node and randomly assign it a label according to the class distribution of that node, what is the probability you would label it wrong?</p><p>The formula is:</p><h4><strong>Gini(S) = 1 − ∑ pᵢ²</strong> for i = 1 to c</h4><p>For binary classification, this simplifies to:</p><h4><strong>Gini = 1 − (p² + (1−p)²) = 2p(1−p)</strong></h4><p>When p = 0.5, Gini = 0.5 (maximum impurity for binary). When p = 0 or p = 1, Gini = 0 (pure node). So Gini ranges from 0 to a maximum of (1 − 1/c), which for binary classification is 0.5.</p><p><strong>Why does squaring the probabilities work?</strong> The probability of picking a sample of class i and then randomly guessing class i is pᵢ × pᵢ = pᵢ². Summing over all classes gives the probability of a correct random label. Subtracting from 1 gives the probability of an incorrect random label which is exactly what impurity should measure.</p><h4>Entropy vs Gini</h4><p><strong>a practical comparison.</strong> In practice, both metrics produce very similar splits and very similar final trees. Gini is preferred in implementations like scikit-learn because it avoids computing logarithms, which are expensive. Entropy can sometimes produce slightly more balanced trees because the log function penalises unequal probabilities more severely near the extremes. For most problems, the difference in model accuracy is negligible pick Gini unless you have a specific reason not to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/602/1*cuHptPNLguYLdLq6CIkDnQ.png" /></figure><h4>Information Gain</h4><p>Now that you can measure the impurity of a single node, you need a way to measure how much a split improves things. Information Gain (IG) is that measure. It tells you how much the entropy of the dataset is reduced by splitting on a particular feature.</p><p>The formula is:</p><h4><strong>IG(S, A) = H(S) − ∑ᵥ (|Sᵥ| / |S|) · H(Sᵥ)</strong></h4><p>Let’s unpack this. H(S) is the entropy of the parent node before the split. The summation on the right computes the entropy after the split: for each possible value v of attribute A, you take the subset Sᵥ (all samples where A = v), compute its entropy H(Sᵥ), and weight it by the fraction of samples that went into that subset (|Sᵥ| / |S|). The weighted sum gives the average entropy of the children. Subtracting from the parent entropy gives the gain how much uncertainty was eliminated.</p><p>The tree always chooses the split with the highest information gain at each node.</p><p><strong>Why the weighted average?</strong> Because a split that creates one huge child and one tiny child is different from a split that creates two equal children, even if the child entropies are the same. Weighting by size correctly accounts for this a large, slightly-impure child contributes more to the total uncertainty than a small, slightly-impure child.</p><h4>Gini Gain</h4><p>Gini Gain applies the exact same logic but uses Gini impurity instead of entropy:</p><h4><strong>GiniGain(S, A) = Gini(S) − ∑ᵥ (|Sᵥ| / |S|) · Gini(Sᵥ)</strong></h4><p>At each node, the algorithm evaluates GiniGain for every possible split and picks the maximum. Everything else is identical to the information gain approach only the impurity measure changes.</p><h3>3. A Worked Example from Scratch</h3><p>Let’s put numbers to all of this so the formulas become real.</p><p><strong>Setup:</strong> You have 10 training samples at a node=6 labelled Pass, 4 labelled Fail. You are evaluating whether to split on “Hours Studied ≤ 45.”</p><p><strong>Step 1: Parent entropy.</strong> p(Pass) = 6/10 = 0.6, p(Fail) = 4/10 = 0.4 H(parent) = − 0.6 · log₂(0.6) − 0.4 · log₂(0.4) = − 0.6 · (-0.737) − 0.4 · (-1.322) = 0.442 + 0.529 = <strong>0.971 bits</strong></p><p><strong>Step 2: After the split.</strong> Left child (Hours ≤ 45): 4 Pass, 1 Fail → p(Pass) = 0.8, p(Fail) = 0.2 H(left) = -0.8·log₂(0.8) -0.2·log₂(0.2) = 0.264 + 0.464 = <strong>0.722 bits</strong></p><p>Right child (Hours &gt; 45): 2 Pass, 3 Fail → p(Pass) = 0.4, p(Fail) = 0.6 H(right) = -0.4·log₂(0.4) -0.6·log₂(0.6) = 0.529 + 0.442 = <strong>0.971 bits</strong></p><p><strong>Step 3: Weighted child entropy.</strong> (5/10) · 0.722 + (5/10) · 0.971 = 0.361 + 0.486 = <strong>0.847 bits</strong></p><p><strong>Step 4: Information Gain.</strong> IG = 0.971 − 0.847 = <strong>0.124 bits</strong></p><p>The algorithm repeats this for every feature and every possible threshold, then picks whichever combination gives the highest IG. Here it would compare 0.124 bits against the IG of splitting on “Assignments Completed ≤ 3”, “Score ≤ 70”, and so on.</p><p><strong>Cross-checking with Gini:</strong> Gini(parent) = 1 -(0.⁶² + 0.⁴²) = 1 -(0.36 + 0.16) = <strong>0.48</strong> Gini(left) = 1 -(0.⁸² + 0.²²) = 1 -0.68 = <strong>0.32</strong> Gini(right) = 1 -(0.⁴² + 0.⁶²) = 1 -0.52 = <strong>0.48</strong> GiniGain = 0.48 -[(5/10)·0.32 + (5/10)·0.48] = 0.48 -0.40 = <strong>0.08</strong></p><p>Both metrics agree that this split is worth making the specific numbers differ but the ranking of splits is almost always the same.</p><h3>4. Overfitting and Pruning</h3><h4>Why deep trees overfit</h4><p>A decision tree that is allowed to grow without any restrictions will eventually create one leaf per training sample. Every training example is classified perfectly. The training accuracy is 100%. But the test accuracy collapses sometimes dramatically.</p><p>This happens because a deep tree is not learning the underlying pattern; it is memorising the training data, including all the noise and random fluctuations in it. The model becomes so specific that it cannot generalise. This is the classic overfitting problem, and it is the central weakness of single decision trees.</p><p>Concretely: imagine a training set with one mislabelled example. An unrestricted tree will eventually create a leaf just for that example, carving out a tiny region of feature space to accommodate the noise. Any future sample that falls in that region will be misclassified in the exact same wrong way.</p><h4>Pruning strategies</h4><p><strong>Pre-pruning (early stopping)</strong> means you set constraints before the tree grows. Common constraints include a maximum depth (max_depth), a minimum number of samples required to attempt a split (min_samples_split), a minimum number of samples required in a leaf (min_samples_leaf), and a minimum improvement in impurity required for any split to happen. These hyperparameters are the first levers you tune when working with decision trees.</p><p><strong>Post-pruning (cost-complexity pruning)</strong> takes the opposite approach. Grow the full tree first, then work backwards, collapsing branches that do not contribute meaningfully to accuracy on a held-out validation set. Scikit-learn implements this through the ccp_alpha parameter (cost-complexity alpha). Higher alpha = more pruning = simpler tree.</p><p>The fundamental tension is that shallow trees underfit (too simple, miss real patterns) while deep trees overfit (too complex, memorise noise). Pruning tries to find the sweet spot. But here is the twist ensemble methods largely sidestep this tension by combining many trees, which lets them use deep trees as building blocks without the overfitting problem destroying the final result.</p><h3>5. The Bias-Variance Tradeoff</h3><p>This is one of the most important theoretical concepts in all of supervised machine learning, and it explains everything about why ensembles work.</p><p>Every model’s total expected error can be decomposed into three parts:</p><p><strong>Total Error = Bias² + Variance + Irreducible Noise</strong></p><p><strong>Bias</strong> is the error that comes from wrong assumptions in the model. A high-bias model systematically makes the same kind of mistake regardless of the training data. It is too simple to capture the real patterns.</p><p><strong>Variance</strong> is the error that comes from sensitivity to fluctuations in training data. A high-variance model changes dramatically when you retrain it on a slightly different dataset. It learns the noise along with the signal.</p><p><strong>Irreducible noise</strong> is the inherent randomness in the data that no model can remove.</p><p>For decision trees specifically:</p><ul><li>A shallow tree has high bias (it misses complex patterns) and low variance (its structure barely changes when you retrain on new data).</li><li>A deep tree has low bias (it can fit any complex pattern) and high variance (its structure changes wildly with different training samples, because it is memorising noise).</li></ul><p>You cannot have low bias and low variance simultaneously with a single tree. Push bias down by growing deeper-variance shoots up. Control variance by staying shallow -bias increases. This is the tradeoff.</p><p><strong>Ensemble methods solve this tradeoff.</strong> By combining many trees intelligently, you can achieve simultaneously low bias and low variance. Bagging reduces variance while keeping bias roughly the same. Boosting reduces bias at the cost of some variance increase-but carefully enough that the net effect is better overall performance. This is the entire theoretical justification for ensemble learning.</p><h3>6. Ensemble Learning: The Core Idea</h3><p>The intuition behind ensemble methods is sometimes called the “wisdom of crowds.” A single expert might be wrong. But if you ask 500 independent experts and take the majority opinion, the random errors cancel out and the correct signal dominates -as long as each expert is at least slightly better than random chance.</p><p>In machine learning terms, each “expert” is a weak learner -a model that is only slightly better than random on its own. Combining enough weak learners produces a strong learner that is significantly better than any individual component. The two main strategies for doing this are Bagging and Boosting.</p><h3>7. Bagging (Bootstrap Aggregating)</h3><h4>The algorithm</h4><p>Bagging was introduced by Leo Breiman in 1994. The procedure is conceptually simple:</p><ol><li>Draw B bootstrap samples from your training data. A bootstrap sample is drawn with replacement-meaning the same training example can appear multiple times in a single sample, and some examples will not appear at all. Each bootstrap sample is the same size as the original dataset (n samples), but because of replacement, roughly 63.2% of the original examples appear at least once, and 36.8% do not appear.</li><li>Train one full decision tree on each bootstrap sample independently. These trees can run in parallel they have no dependency on each other.</li><li>At prediction time, feed the input to all B trees. For classification, take the majority vote across all trees. For regression, take the average of all predictions.</li></ol><p>The prediction formula for regression is:</p><p><strong>f̂_bag(x) = (1/B) · ∑ fᵦ(x)</strong> for b = 1 to B</p><h4>Why does averaging reduce variance?</h4><p>Here is the mathematical reason. If you have B independent random variables each with variance σ², their average has variance σ²/B. Averaging reduces variance by a factor of B. Of course, the trees are not perfectly independent (they are all trained from the same original data), so the reduction is not quite 1/B. But it is still substantial.</p><p>The key condition is that the trees must be somewhat decorrelated. If all trees made identical predictions, averaging would accomplish nothing. Bagging achieves decorrelation by giving each tree a different bootstrap sample each tree sees a different version of the dataset, so it builds a somewhat different structure.</p><h4>Out-of-Bag (OOB) Error</h4><p>Because each tree is trained on roughly 63% of the data, the remaining 37% (the out-of-bag samples) can be used as a natural validation set for that tree. For each training example, you collect predictions from all trees that did not see that example during training, and use those predictions to compute an error estimate.</p><p>This gives you a free, unbiased estimate of generalisation error without needing to set aside a separate validation set. In practice, OOB error is very close to k-fold cross-validation error, at a fraction of the computational cost.</p><h3>8. Boosting: Sequential Error Correction</h3><h4>The fundamental idea</h4><p>Bagging builds trees in parallel and averages them to reduce variance. Boosting builds trees sequentially and focuses each new tree on correcting the mistakes of the previous ensemble. The goal is to reduce bias — to get the ensemble to learn patterns that are genuinely difficult.</p><p>The general boosting prediction formula is:</p><h4><strong>F_M(x) = ∑ αₘ · fₘ(x)</strong> for m = 1 to M</h4><p>where each fₘ is a weak learner and αₘ is its contribution weight, determined by its accuracy. Better models get higher weights in the final ensemble.</p><h4>AdaBoost (Adaptive Boosting)</h4><p>AdaBoost, introduced by Freund and Schapire in 1995, is the original and most intuitive boosting algorithm. It works with any weak learner, but is most commonly used with decision stumps -trees of depth 1 (a single question).</p><p><strong>The algorithm in full detail:</strong></p><p>Step 1: Initialise equal weights for all n training samples: wᵢ = 1/n for all i.</p><p>Step 2: For each boosting round m from 1 to M:</p><ul><li>Train a weak learner fₘ on the training data, weighted by the current weights wᵢ. Samples with higher weights are effectively more important to get right.</li><li>Compute the weighted error: εₘ = ∑ wᵢ · 𝟙[yᵢ ≠ fₘ(xᵢ)] / ∑ wᵢ. This is simply the weighted fraction of misclassified samples.</li><li>Compute the model weight: αₘ = ½ · ln((1 − εₘ) / εₘ). If εₘ is small (accurate model), αₘ is large and positive. If εₘ = 0.5 (random guessing), αₘ = 0. If εₘ &gt; 0.5 (worse than random), αₘ is negative — the model’s prediction gets flipped.</li><li>Update sample weights using the rule: <strong>wᵢ⁽ᵐ⁺¹⁾ = wᵢ⁽ᵐ⁾ · exp(αₘ · 𝟙[yᵢ ≠ fₘ(xᵢ)])</strong>. Misclassified samples get their weight multiplied by e^αₘ (greater than 1), making them more important next round. Correctly classified samples stay the same weight. Then renormalise all weights to sum to 1.</li></ul><p>Step 3: Final prediction is the sign of the weighted sum: F(x) = sign(∑ αₘ · fₘ(x)).</p><p><strong>The beautiful thing about AdaBoost</strong> is that it is provably correct the training error decreases exponentially as the number of rounds increases, as long as each weak learner does slightly better than 50% accuracy on the weighted data. In practice, it can also overfit if you run too many rounds on noisy data, but it tends to be more resistant to overfitting than you might expect.</p><h4>Gradient Boosting</h4><p>Gradient Boosting, developed by Jerome Friedman in 1999–2001, is a dramatic generalisation of AdaBoost. The key insight is to reframe boosting as gradient descent in function space.</p><p>Instead of adjusting sample weights (as AdaBoost does), Gradient Boosting fits each new tree to the residuals the differences between the current ensemble’s predictions and the true labels. More precisely, it fits the tree to the negative gradient of the loss function with respect to the current predictions, which Friedman called pseudo-residuals.</p><p><strong>The full algorithm:</strong></p><p>Step 1: Initialise with a constant model. For regression with mean squared error loss, this is simply the mean of the targets: F₀(x) = ȳ.</p><p>Step 2: For each round m from 1 to M:</p><ul><li>Compute pseudo-residuals: rᵢₘ = − [∂L(yᵢ, F(xᵢ)) / ∂F(xᵢ)] evaluated at F = Fₘ₋₁. For MSE loss, this simplifies to rᵢₘ = yᵢ − Fₘ₋₁(xᵢ) — literally the residuals.</li><li>Fit a regression tree fₘ to the pseudo-residuals (rᵢₘ, xᵢ) pairs.</li><li>Compute the optimal leaf weights by solving a line search within each leaf.</li><li>Update: Fₘ(x) = Fₘ₋₁(x) + η · fₘ(x), where η is the learning rate (typically 0.01 to 0.1).</li></ul><p>Step 3: The final model is F_M(x).</p><p><strong>Why pseudo-residuals = gradient descent?</strong> In regular gradient descent on parameters, you move in the direction of the negative gradient of the loss with respect to the parameters. In function space, you move in the direction of the negative gradient of the loss with respect to the function values which is exactly the pseudo-residuals. Each new tree points in the direction that most reduces the loss if you add it to the ensemble. The learning rate η controls the step size, and a smaller η requires more trees but usually gives better generalisation.</p><p>The power of Gradient Boosting over AdaBoost is its flexibility. By changing the loss function L, you get different behaviour: MSE loss for regression, log-loss for classification, absolute error for robustness to outliers, and many others -all without changing the core algorithm.</p><h3>9. Modern Algorithms: XGBoost and CatBoost</h3><h4>XGBoost</h4><p>XGBoost (eXtreme Gradient Boosting), released by Tianqi Chen in 2016, became the dominant algorithm in structured data competitions. It builds on Gradient Boosting with three major innovations.</p><p><strong>Second-order gradients.</strong> Standard Gradient Boosting uses only the first derivative (gradient) of the loss. XGBoost uses both the first derivative gᵢ and the second derivative hᵢ (Hessian) to get a more accurate quadratic approximation of the loss at each step. The objective it optimises per tree is:</p><p><strong>Obj = ∑ [gᵢ · fₘ(xᵢ) + ½ · hᵢ · fₘ(xᵢ)²] + Ω(fₘ)</strong></p><p>where gᵢ is the gradient and hᵢ is the Hessian of the loss for sample i.</p><p><strong>Built-in regularisation.</strong> The term Ω(fₘ) penalises tree complexity directly: Ω(f) = γT + ½λ · ∑wⱼ², where T is the number of leaves and wⱼ are the leaf weights. The hyperparameter γ controls how much you penalise having many leaves (tree depth), and λ controls L2 regularisation of the leaf weights. This is why XGBoost naturally resists overfitting without extensive manual tuning.</p><p><strong>Parallelisation and speed.</strong> XGBoost parallelises the computation of split gains across features (not trees-the trees are still sequential). It uses approximate algorithms for finding best splits, handles sparse data natively (missing values are automatically routed to the best child at each split), and implements column subsampling (like Random Forest’s feature randomness) as an additional regularisation technique.</p><h4>CatBoost</h4><p>CatBoost, developed by Yandex in 2017, solves a specific problem that plagues all other gradient boosting implementations: target leakage when encoding categorical features.</p><p>The problem is subtle. Standard techniques for encoding categorical features (like target encoding, which replaces a category with the average target value of that category) use information from the entire training set. When training on the full data, this means each sample’s own target value influenced its own encoding -creating subtle data leakage that inflates training accuracy and can hurt generalisation.</p><p>CatBoost solves this with Ordered Boosting. It processes the training data in a random permutation, and when computing statistics for sample i, it only uses the samples that appeared before position i in the permutation. This makes the encoding truly leak-free.</p><p>Beyond that, CatBoost uses oblivious trees (also called symmetric trees) where the same split criterion is applied at every node of the same depth level. This sounds like a restriction, but it dramatically speeds up prediction, enables efficient GPU training, and serves as a built-in regulariser. CatBoost also handles categorical features natively without any preprocessing, and performs especially well on small datasets where other algorithms tend to overfit aggressively.</p><h3>10. Comparing Bagging and Boosting Side by Side</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/1*mFcI-Lu54w2qsKckSqkhQA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RwF7ZF7x-6EvpnxqKHu6Iw.png" /></figure><h3>11. Practical Wisdom and When to Use What</h3><p><strong>Start with Random Forest.</strong> It is robust, works well out of the box with default hyperparameters, does not overfit easily, handles missing values and categorical features reasonably well, and gives you feature importance for free. Use it as your first baseline on any tabular dataset.</p><p><strong>Move to XGBoost or LightGBM when you need maximum accuracy.</strong> These almost always outperform Random Forest with proper tuning, especially on large datasets. The key hyperparameters to tune are: n_estimators (number of trees), learning_rate (smaller = more trees needed, usually better generalisation), max_depth (deeper trees = more complex patterns but more overfitting), subsample (fraction of samples used per tree), and colsample_bytree (fraction of features considered per tree).</p><p><strong>Use CatBoost when your data has many categorical features.</strong> It handles them natively, and the ordered boosting means you can feed raw categories directly without any preprocessing. It is especially strong on datasets with high cardinality categorical variables (categories with thousands of unique values).</p><p><strong>The practical workflow:</strong> start with Random Forest, note the baseline accuracy. Try XGBoost with early stopping (stop when validation loss stops improving) usually beats Random Forest. Run a hyperparameter search. If categorical features are dominant, try CatBoost. The extra tuning time is almost always worth it for production systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b92b104418e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Regularization & Generalization in Machine Learning: A Complete Guide]]></title>
            <link>https://medium.com/@lalanagurusinghe/regularization-generalization-in-machine-learning-a-complete-guide-6e85970c79c6?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e85970c79c6</guid>
            <category><![CDATA[regression]]></category>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[macine-learning]]></category>
            <category><![CDATA[logistic-regression]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Sun, 26 Apr 2026 15:57:24 GMT</pubDate>
            <atom:updated>2026-04-26T15:57:24.643Z</atom:updated>
            <content:encoded><![CDATA[<p>A beginner-friendly, in-depth guide covering everything from overfitting to Ridge, Lasso, Elastic Net, and how to diagnose your model like a pro.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/883/1*PK-ZFH89njOAx1QjwdFQ1Q.png" /></figure><h3>The Core Problem — Why Do Models Fail?</h3><p>Before we dive into solutions, we need to deeply understand the problem.</p><p>Imagine you are studying for an important exam. There are two very different ways you could fail:</p><p><strong>Scenario A — You barely studied.</strong> You only picked up the most surface-level information and lacked depth. When the exam comes, you don’t know enough to answer well. In machine learning, this is called <strong>underfitting</strong> — your model is too simple to capture the real patterns in the data.</p><p><strong>Scenario B — You memorized the practice tests word for word.</strong> You drilled every answer from last year’s papers, but on exam day the questions are phrased slightly differently. You can’t adapt, because you learned answers, not concepts. In machine learning, this is called <strong>overfitting</strong> — your model memorized the training data instead of learning the underlying pattern.</p><p>Machine learning models face this exact dilemma every time they train. Your job as a practitioner is to find the middle ground.</p><h3>What Is Overfitting, Exactly?</h3><p>When you train a model, you feed it data and it learns to find patterns. A simple model might find a broad, useful pattern — the kind that generalizes to new data. But a highly complex model starts chasing every tiny fluctuation in the training data, including the noise — the random, meaningless quirks that exist only in that specific dataset.</p><p>The result? A model that scores 99% accuracy on training data but crashes to 70% on new, unseen data. That gap is the red flag. The model has memorized the training set rather than truly learning.</p><p><strong>Signs you are overfitting:</strong></p><ul><li>Training accuracy is very high, but test accuracy is significantly lower — a large gap between the two.</li><li>Model weights or coefficients are unusually large in magnitude.</li><li>Performance gets worse the longer you continue training.</li><li>The model performs brilliantly on old data but poorly on new cases.</li></ul><h3>The Bias-Variance Tradeoff</h3><p>This is one of the most foundational ideas in all of machine learning. If you understand nothing else, understand this.</p><h4>Bias</h4><p><strong>Bias</strong> is the error that comes from your model making the wrong assumptions about the structure of the data.</p><p>A high-bias model is too simple. It cannot capture the real pattern regardless of how much data you throw at it. The classic example: trying to fit a straight line to data that follows a curve. No matter how many data points you have, the line will always be wrong in a systematic, predictable way.</p><p>High bias leads to underfitting.</p><h4>Variance</h4><p><strong>Variance</strong> is the error that comes from your model being overly sensitive to small changes in the training data.</p><p>A high-variance model is too complex. It bends and wiggles to hit every single training point perfectly — including the noisy ones. If you trained the same model on a slightly different sample of data, you would get very different predictions. The model has no stability.</p><p><strong>High variance leads to overfitting.</strong></p><h4>The Mathematical Decomposition</h4><p>The total error of any predictive model can be broken down into three components:</p><p><strong><em>Total Error = Bias² + Variance + Irreducible Noise</em></strong></p><p>Formally:</p><p><strong><em>E[(y — ŷ)²] = Bias[ŷ]² + Variance[ŷ] + σ²</em></strong></p><p>Where:</p><ul><li><strong>Bias²</strong> measures how far off your model’s average prediction is from the true answer.</li><li><strong>Variance</strong> measures how much your model’s predictions jump around across different training sets.</li><li><strong>σ² (sigma squared)</strong> is the irreducible noise — the randomness built into the real world that no model, no matter how perfect, can ever eliminate.</li></ul><h4>The Cruel Tradeoff</h4><p>Here is the fundamental tension that every machine learning practitioner must navigate:</p><blockquote><strong><em>As you reduce bias (by making the model more complex), variance increases. As you reduce variance (by simplifying the model), bias increases.</em></strong></blockquote><p>You cannot minimize both at the same time. Your goal is to find the sweet spot — the model complexity at which the sum of bias² and variance is at its minimum. This is what machine learning theory calls the “good fit.”</p><h4>A Visual Intuition</h4><p>Think of it like archery:</p><ul><li><strong>High bias, low variance:</strong> Every arrow hits the same spot, but that spot is far from the bullseye. Consistently wrong.</li><li><strong>Low bias, high variance:</strong> Arrows are scattered all over the target. Sometimes close, sometimes far — unpredictable.</li><li><strong>Low bias, low variance:</strong> Arrows cluster tightly around the bullseye. This is what you want.</li></ul><h3>What Is Regularization?</h3><p>Regularization is the family of techniques used to prevent overfitting by penalizing model complexity.</p><p>Here is the core idea: instead of simply minimizing prediction error (how wrong your model is), you also penalize the model for having large weights.</p><p>The new objective becomes:</p><p><strong><em>New Objective = Original Loss + Penalty for Complexity</em></strong></p><h4>Why Penalize Large Weights?</h4><p>Large weights are the fingerprint of overfitting. When a model overfits, it assigns enormous coefficients to features — sometimes even irrelevant ones — because doing so helps it memorize training examples. A well-generalized model should describe the real world with small, stable, interpretable weights that represent true signal, not massive ones chasing noise.</p><p>Regularization makes large weights expensive. Since the model wants to minimize total cost, it is now forced to ask: “Is this large weight really worth the penalty it costs me?”</p><h4>The Leash Analogy</h4><p>Think of regularization like putting a dog on a leash:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/683/1*p4j9VAWhCrmIcQlxXGAWig.png" /></figure><p>The <strong>regularization parameter λ (lambda)</strong> is the length of the leash:</p><ul><li><strong>λ too small:</strong> The leash is too loose. The model still chases noise and overfits.</li><li><strong>λ too large:</strong> The leash is too tight. The model is too constrained and underfits.</li><li><strong>λ just right:</strong> The model has enough freedom to learn real patterns but not enough to memorize noise.</li></ul><p>Finding the right λ is one of the central challenges of applied machine learning, and you find it through a process called cross-validation.</p><h3>L2 Regularization — Ridge Regression</h3><h4>The Formula</h4><p>Ridge Regression adds the sum of squared weights to the loss function:</p><p><strong><em>J(β) = Loss(β) + λ × Σ(βⱼ²)</em></strong></p><p>In full detail for linear regression with Mean Squared Error:</p><p><strong><em>J(β) = (1/n) × Σ(yᵢ — ŷᵢ)² + λ × Σ(βⱼ²)</em></strong></p><p>The model must now minimize two competing things simultaneously:</p><ol><li>Fit the training data well (minimize prediction error).</li><li>Keep all weights small (minimize the penalty term).</li></ol><h4>Why Squaring the Weights Matters</h4><p>The choice to square the weights — rather than just take their absolute value — has profound mathematical consequences.</p><p><strong>First, squaring makes everything positive.</strong> A weight of −8 is just as “large” and dangerous as a weight of +8. Squaring handles this naturally, since (−8)² = 64 = ⁸².</p><p><strong>Second, squaring creates an accelerating penalty.</strong> Consider what happens as a weight grows:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/479/1*DNByeCKhuviwBye4ucfJRA.png" /></figure><p>A weight of 10 is penalized 100 times harder than a weight of 1. A weight of 100 is penalized 10,000 times harder. This explosive growth in penalty means the model faces enormous resistance to growing large weights — the cost becomes prohibitive very quickly. At the same time, the pressure on small weights is gentle. A weight shrinking from 0.1 to 0 saves almost no penalty at all (0.01 → 0).</p><p>This proportional, quadratic structure is why Ridge is so effective: it applies maximum force precisely where the overfitting danger is greatest.</p><h4>What Happens During Training — Weight Decay</h4><p>To understand how Ridge actually works mechanically, consider the gradient update step.</p><p>Without L2 (ordinary regression):</p><p><strong><em>βⱼ ← βⱼ − α × (∂Loss/∂βⱼ)</em></strong></p><p>You update each weight by moving in the direction that reduces the loss.</p><p>With L2 (Ridge):</p><p><strong><em>βⱼ ← βⱼ − α × (∂Loss/∂βⱼ + 2λβⱼ)<br>βⱼ ← βⱼ(1 − 2αλ) − α × (∂Loss/∂βⱼ)</em></strong></p><p>Notice the factor <strong>(1 − 2αλ)</strong>. This is a number slightly less than 1 — say, 0.98. Every single update step, the weight is first multiplied by this shrinkage factor before the gradient correction is applied. The weights literally decay toward zero at every step.</p><p>This is why Ridge Regression is also known as <strong>weight decay</strong> — a term you will hear constantly in deep learning.</p><p>The critical consequence: no matter how many steps you apply, multiplying by 0.98 repeatedly approaches zero asymptotically but never actually reaches it (0.9⁸¹⁰⁰⁰ ≈ 0.00002, very small but nonzero). This is the mathematical reason Ridge almost never produces exactly zero weights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*ocYjHxbmUlrFnZv4BcdDug.png" /></figure><h4>The Geometric Interpretation</h4><p>This is where L2 regularization truly clicks. Picture the weight space in two dimensions — two weights, β₁ and β₂, on the x and y axes.</p><p><strong>The loss landscape</strong> forms elliptical contour lines, like a topographic map of a bowl viewed from above. The center of the innermost ellipse is the unconstrained optimal solution — where ordinary regression would land.</p><p><strong>The L2 constraint</strong> says that the weights must satisfy:</p><p><strong><em>β₁² + β₂² ≤ s</em></strong></p><p>This is the equation of a <strong>circle</strong>. The size of the circle (its radius) depends on s, which is inversely related to λ. A larger λ means a smaller, tighter circle.</p><p><strong>The Ridge solution</strong> is the point where the loss contour ellipses first touch the constraint circle — the innermost ellipse that can reach the circle boundary.</p><p>Because the circle is <strong>perfectly smooth with no corners</strong>, the touching point can land almost anywhere on the boundary. It almost never falls exactly on one of the coordinate axes, meaning β₁ or β₂ will almost never be exactly zero. Both weights are pulled toward zero, but neither is eliminated.</p><p>This stands in direct contrast to L1 (Lasso), where the constraint is a <strong>diamond</strong> shape with sharp corners sitting precisely on the axes. Ellipses are far more likely to first touch the diamond at one of those corners, forcing one weight to zero. That is the geometric reason Lasso performs feature selection and Ridge does not.</p><h4>Effects of Changing λ — The Coefficient Path</h4><p>As you increase λ from 0 toward infinity, all coefficients shrink smoothly toward zero:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/722/1*TRY2u7k4fkuzAzbxk-EMWw.png" /></figure><p>Three key observations from this path:</p><p><strong>1. All weights shrink together.</strong> Ridge does not pick favorites. Every coefficient gets smaller proportionally. Larger weights get pushed harder (because of the quadratic penalty), but none are ever eliminated.</p><p><strong>2. No coefficient hits exactly zero (until λ = ∞).</strong> Ridge is not a feature selector. If you need to know which features matter and which do not, Ridge will not give you that answer. It only tells you that all features are somewhat relevant, just at reduced scale.</p><p><strong>3. Correlated features share the weight smoothly.</strong> If two features are highly correlated — say, “hours studied” and “practice tests completed” — ordinary regression might arbitrarily assign all weight to one and ignore the other. Ridge distributes the weight between them in a stable, proportional way. This makes Ridge models significantly more robust when your input features overlap in what they measure.</p><h4>Why You Do Not Penalize the Intercept</h4><p>A subtle but important detail: the regularization penalty applies only to the slope coefficients βⱼ, not to the intercept β₀.</p><p>The intercept is the baseline prediction when all features equal zero. It shifts the model up or down but does not add any complexity or shape to the predictions. Penalizing it would force the model to make predictions centered around zero, which is often incorrect. Regularization targets complexity — and the intercept does not contribute to complexity.</p><h4>When to Use Ridge</h4><p>Ridge Regression is your best choice when:</p><ul><li>Your features are correlated with each other and you want stable weight distribution.</li><li>You believe most features genuinely contribute to the prediction.</li><li>You want to keep all features in the model but reduce their influence.</li><li>Interpretability of individual coefficients matters less than prediction stability.</li></ul><h3>L1 Regularization — Lasso</h3><h4>The Formula</h4><p>Lasso (Least Absolute Shrinkage and Selection Operator) replaces the squared penalty with the absolute value:</p><p><strong><em>J(β) = Loss(β) + λ × Σ|βⱼ|</em></strong></p><h4>The Power of Absolute Value: Exact Zeros</h4><p>The absolute value function |β| has a constant slope of 1 for all positive values of β, and −1 for all negative values. It does not accelerate like the quadratic does. This means that even the smallest weight faces the same constant pressure per unit to shrink toward zero.</p><p>Combined with the geometry of the diamond constraint, Lasso can drive weights to <strong>exactly zero</strong> — permanently eliminating those features from the model entirely.</p><h4>The Diamond Geometry</h4><p>Return to the weight space visualization. Lasso’s constraint is:</p><p><strong><em>|β₁| + |β₂| ≤ s</em></strong></p><p>This traces out a <strong>diamond</strong> shape (a square rotated 45 degrees) in two dimensions.</p><p>The diamond has four pointed corners sitting exactly on the coordinate axes — at points like (s, 0), (0, s), (−s, 0), and (0, −s).</p><p>When the loss function’s elliptical contours expand outward looking for the diamond, they are geometrically much more likely to first contact the diamond at one of these corners. At a corner, one of the coordinates is zero. This means one weight becomes exactly zero — that feature is completely removed from the model.</p><p>The more features you have, the more pronounced this effect becomes. In high-dimensional spaces, Lasso aggressively zeros out the irrelevant weights, leaving only a sparse set of truly important features.</p><h4>Automatic Feature Selection</h4><p>This is Lasso’s defining superpower. You feed it 100 features, and it might return a model that uses only 12 of them, with the other 88 set exactly to zero.</p><p>This matters enormously in practice:</p><ul><li><strong>Interpretability:</strong> A model with 12 features is far easier to explain and reason about than one with 100.</li><li><strong>Performance:</strong> Eliminating truly irrelevant features reduces noise and often improves predictions.</li><li><strong>Discovery:</strong> In scientific research, Lasso can reveal which variables actually matter — a form of automated feature discovery.</li></ul><h4>L1 vs L2: A Side-by-Side Comparison</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/763/1*xoo9dEjm38MJqtp_pjyTeA.png" /></figure><h4>When to Use Lasso</h4><p>Lasso is your best choice when:</p><ul><li>You have many features and suspect that most are irrelevant or redundant.</li><li>You want a sparse, interpretable model with automatic feature selection.</li><li>You are doing exploratory analysis and want to discover which variables actually matter.</li><li>Memory and computational efficiency matter (fewer features = simpler model).</li></ul><h3>Elastic Net — Best of Both Worlds</h3><h4>The Motivation</h4><p>Ridge handles correlated features gracefully. Lasso performs feature selection. What if you need both?</p><p>Elastic Net combines the L1 and L2 penalties in a single framework:</p><p><strong><em>J(β) = Loss(β) + λ₁ × Σ|βⱼ| + λ₂ × Σ(βⱼ²)</em></strong></p><h4>What Elastic Net Achieves</h4><p>By blending the two penalties, Elastic Net gets:</p><ul><li>The <strong>feature elimination</strong> capability of L1 (Lasso).</li><li>The <strong>correlated feature stability</strong> of L2 (Ridge).</li></ul><p>This makes Elastic Net particularly powerful in situations where you have many features, some of which are correlated with each other, and you also suspect that many are irrelevant.</p><p>A pure Lasso in this situation might arbitrarily pick one feature from each correlated group and zero out the others — even if both genuinely contribute to the prediction. Elastic Net instead tends to zero out the truly irrelevant features while distributing weight among the correlated ones.</p><h4>The Hyperparameters</h4><p>Elastic Net has two hyperparameters to tune:</p><ul><li><strong>Overall regularization strength</strong> (total λ) — how much you penalize complexity overall.</li><li><strong>L1 ratio (mixing parameter)</strong> — the balance between L1 and L2. In scikit-learn, this is called l1_ratio. A value of 1.0 gives pure Lasso; a value of 0.0 gives pure Ridge; values between give a blend.</li></ul><h4>When to Use Elastic Net</h4><p>Elastic Net is your best choice when:</p><ul><li>You have many correlated features AND many potentially irrelevant features simultaneously.</li><li>Pure Lasso gives unstable results due to correlations in your data.</li><li>You want the benefits of feature selection without fully sacrificing correlated feature handling.</li><li>You are working with genomic data, text data, or other high-dimensional, correlated datasets.</li></ul><h3>Learning Curves — Your Diagnostic Tool</h3><p>Once you have trained a model, how do you know whether you are suffering from underfitting or overfitting? <strong>Learning curves</strong> are the answer.</p><h4>What Is a Learning Curve?</h4><p>A learning curve is a plot with:</p><ul><li><strong>X-axis:</strong> The number of training examples used.</li><li><strong>Y-axis:</strong> Model error (or equivalently, accuracy).</li><li><strong>Two lines:</strong> One for training error, one for validation (test) error.</li></ul><p>You train your model multiple times, each time using a different number of training examples (from a small subset up to the full dataset), and record both errors each time.</p><p>As you add more training data, two things happen naturally:</p><ul><li><strong>Training error increases:</strong> More data is harder to memorize perfectly, so training error rises.</li><li><strong>Validation error decreases:</strong> The model generalizes better with more examples, so validation error falls.</li></ul><p>Eventually, the two curves converge toward each other. The shape and gap of this convergence tells you everything about your model’s problem.</p><h4>Pattern 1 — High Bias (Underfitting)</h4><p><strong>What you see:</strong></p><ul><li>Both training error AND validation error are high.</li><li>The two lines are close together — a small gap.</li><li>Both curves flatten out (plateau) early, even with more data.</li></ul><p><strong>What it means:</strong> Your model is too simple to capture the real patterns. Adding more data will not help much, because the problem is not the amount of data — it is the model’s capacity to learn.</p><p><strong>How to fix it:</strong></p><ul><li>Use a more complex model (add layers, increase polynomial degree, etc.).</li><li>Add more relevant features to the input.</li><li>Reduce the regularization strength (decrease λ).</li></ul><h4>Pattern 2 — High Variance (Overfitting)</h4><p><strong>What you see:</strong></p><ul><li>Training error is very low, but validation error is significantly higher.</li><li>There is a <strong>large gap</strong> between the two lines.</li><li>The validation error curve has not yet plateaued — it is still slowly declining.</li></ul><p><strong>What it means:</strong> Your model is too complex and is memorizing training data. It does not generalize well. The gap between the lines is the overfitting penalty.</p><p><strong>How to fix it:</strong></p><ul><li>Add or increase regularization (increase λ).</li><li>Collect more training data (the validation curve is still descending, meaning more data would help).</li><li>Simplify the model architecture.</li><li>Use dropout (in neural networks) or other complexity-reduction techniques.</li></ul><p>The Key Diagnostic Rule</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/758/1*EeXpbXNFn3HQJcXLyKQTBA.png" /></figure><h3>Validation Curves — Tuning Your Hyperparameter</h3><p>Learning curves vary the amount of data. <strong>Validation curves</strong> vary the hyperparameter — specifically, λ.</p><h4>What Is a Validation Curve?</h4><p>A validation curve is a plot with:</p><ul><li><strong>X-axis:</strong> The value of λ (regularization strength), usually on a logarithmic scale.</li><li><strong>Y-axis:</strong> Model error (or accuracy).</li><li><strong>Two lines:</strong> Training error and validation error.</li></ul><p>You train the same model repeatedly across a range of λ values and record both errors.</p><h4>Reading the Validation Curve</h4><p><strong>At low λ (left side of the curve):</strong> The model is barely regularized. It overfits. Training error is very low, but validation error is high — a large gap between the two lines.</p><p><strong>At optimal λ (middle of the curve):</strong> The model has the right amount of constraint. Both errors are at a reasonable level, and the gap between them is small. This is the sweet spot.</p><p><strong>At high λ (right side of the curve):</strong> The model is over-regularized. It underfits. Both training error and validation error are high — a small gap, but both are poor.</p><p>The validation error traces out a <strong>U-shape</strong>. The bottom of that U, where validation error is minimized, is the optimal λ to use.</p><h4>How to Find the Best λ in Practice</h4><p>In Python with scikit-learn, the standard approach is cross-validation:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*aRQX5RHvjhRePuALHNjKGg.png" /></figure><p><strong>Note:</strong> scikit-learn calls λ alpha in Ridge and Lasso. The concept is identical — only the variable name differs.</p><h3>More Data vs. Better Regularization?</h3><p>A common question in practice: should I collect more data or improve my regularization?</p><p>Both approaches address overfitting, but they work differently and suit different situations.</p><h4>When More Data Helps</h4><p>More data is the right answer when:</p><ul><li>Your learning curve shows validation error still declining (not yet plateaued) — the model would benefit from more examples.</li><li>You can realistically collect more data at a reasonable cost.</li><li>You have a high-variance problem where the model has learned training-specific noise.</li></ul><p>The logic: more data makes it harder for the model to memorize, because there is simply too much to memorize. The model is forced to find the true underlying pattern instead.</p><h4>When Better Regularization Helps</h4><p>Regularization is the right answer when:</p><ul><li>Collecting more data is expensive, slow, or impossible.</li><li>Your learning curves have already plateaued — more data would not significantly close the gap.</li><li>You need more interpretability (especially L1/Lasso).</li><li>Memory and computational resources are constrained.</li></ul><h4>The Real Answer: Use Both</h4><p>In professional practice, the best models use both strategies together. Regularization controls complexity; data quantity controls generalization. They are complementary, not competing.</p><p>The practical workflow:</p><ol><li>Start with a reasonable amount of data and a baseline model.</li><li>Use learning curves to diagnose your problem (bias vs. variance).</li><li>Apply the appropriate regularization technique and tune λ with cross-validation.</li><li>If validation error is still declining with data quantity, invest in more data collection.</li><li>Iterate.</li></ol><h3>Final Thoughts</h3><p>Regularization is not just a technical trick — it reflects a deep truth about learning itself. Whether you are a student preparing for an exam or a neural network processing images, the challenge is always the same: learn the concept, not just the examples. Generalize, do not memorize.</p><p>The tools in this guide — Ridge, Lasso, Elastic Net, learning curves, and validation curves — give you a complete framework for building models that work not just on data you have seen, but on data you have not.</p><p><strong>The key takeaways:</strong></p><ul><li>Overfitting is caused by models that chase noise. The symptom is a large gap between training and validation performance.</li><li>Regularization solves this by making complexity expensive. The model must justify every large weight.</li><li>Ridge keeps all features but shrinks them. Lasso eliminates irrelevant features entirely. Elastic Net does both.</li><li>Learning curves tell you whether your problem is bias or variance. Validation curves tell you the right λ.</li><li>In practice, use both regularization and sufficient data for the best results.</li></ul><p>Machine learning is not just about fitting data. It is about finding the truth hidden in the data, without being fooled by the noise. Regularization is your primary weapon for that.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e85970c79c6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Model Evaluation: From Confusion Matrix to ROC-AUC]]></title>
            <link>https://medium.com/@lalanagurusinghe/understanding-model-evaluation-from-confusion-matrix-to-roc-auc-fcb14c6865e1?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/fcb14c6865e1</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Wed, 22 Apr 2026 11:11:31 GMT</pubDate>
            <atom:updated>2026-04-22T11:11:31.772Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TXwKj0WmcdTXpOzINIY9DQ.png" /></figure><h3>Introduction</h3><p>Training a machine learning model is only half the battle. The real challenge — and the part most beginners underestimate — is knowing <em>how well</em> your model actually performs in the real world.</p><p>Many beginners rely on a single number: <strong>accuracy</strong>. But accuracy alone can be deeply misleading. Consider a medical model trained to detect a rare disease that affects only 1% of the population. A model that simply predicts “no disease” for every single patient will achieve 99% accuracy — yet it is completely useless and potentially dangerous.</p><p>This is why machine learning practitioners use a richer set of evaluation tools: the <strong>confusion matrix</strong>, <strong>True Positive Rate (TPR)</strong>, <strong>False Positive Rate (FPR)</strong>, the <strong>ROC Curve</strong>, and <strong>Area Under the Curve (AUC)</strong>. Together, these give you a complete, honest picture of your model’s strengths and weaknesses.</p><p>This article walks you through every one of these concepts, from the ground up, with clear explanations and real-world examples.</p><h3>1.What is a Confusion Matrix?</h3><p>A confusion matrix is a simple 2×2 table that summarizes the performance of a binary classification model. It compares what the model <em>predicted</em> against what was <em>actually true</em> in the dataset.</p><p>For any prediction a binary classifier makes, there are exactly four possible outcomes:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/969/1*7A5KG0Cwj4-lESTj-1SKlw.jpeg" /></figure><p>Let’s break each one down in detail.</p><h4>1.1 True Positive (TP)</h4><p><strong>Definition:</strong> The model predicted <em>positive</em>, and the actual label was also <em>positive</em>. This is a correct prediction.</p><p><strong>Simple example:</strong> A spam email filter labels an incoming email as “spam” — and that email really is spam.</p><p><strong>Medical example:</strong> A cancer screening model predicts that a patient has cancer — and after biopsy, the patient is confirmed to have cancer.</p><p><strong>Why it matters:</strong> True positives represent the model’s ability to correctly identify the things it’s designed to find. A high TP count is generally what you want, but it must be balanced against the other cells.</p><h4>1.2 False Positive (FP)</h4><p><strong>Definition:</strong> The model predicted <em>positive</em>, but the actual label was <em>negative</em>. The model raised an alarm when it shouldn’t have. This is also called a <strong>Type I Error</strong> or a <strong>false alarm</strong>.</p><p><strong>Simple example:</strong> A spam filter labels a legitimate email from your boss as “spam” — it incorrectly flagged something as positive when it was not.</p><p><strong>Medical example:</strong> A cancer screening model flags a healthy patient as having cancer. That patient then undergoes unnecessary, stressful follow-up tests.</p><p><strong>Why it matters:</strong> False positives erode trust in your model. In fraud detection, too many FPs means legitimate customers are blocked from making purchases. The cost of a false positive depends entirely on the domain.</p><h4>1.3 False Negative (FN)</h4><p><strong>Definition:</strong> The model predicted <em>negative</em>, but the actual label was <em>positive</em>. The model missed something it should have caught. This is also called a <strong>Type II Error</strong> or a <strong>miss</strong>.</p><p><strong>Simple example:</strong> A spam filter lets a phishing email pass through to your inbox because it didn’t detect it as spam.</p><p><strong>Medical example:</strong> A cancer screening model tells a patient they are healthy, when in reality they have cancer. This is extremely dangerous — the patient may not seek treatment until it is too late.</p><p><strong>Why it matters:</strong> False negatives are often the most dangerous type of error in high-stakes domains like medicine, fraud detection, or security. Missing a positive can have severe consequences. Whenever someone says “the model missed it”, they are describing a false negative.</p><h4>1.4 True Negative (TN)</h4><p><strong>Definition:</strong> The model predicted <em>negative</em>, and the actual label was also <em>negative</em>. This is a correct rejection.</p><p><strong>Simple example:</strong> A spam filter correctly lets a legitimate email through to your inbox.</p><p><strong>Medical example:</strong> A cancer screening model correctly identifies a healthy patient as having no cancer.</p><p><strong>Why it matters:</strong> True negatives confirm that your model is not over-predicting. A model that has many TNs correctly ignores the background noise and focuses only on genuine signals.</p><h4>A Worked Example</h4><p>Imagine you built a model to detect fraudulent bank transactions. You tested it on 1,000 transactions, of which 100 were actually fraudulent and 900 were legitimate.</p><p>Your model produced these results:</p><ul><li><strong>TP = 80</strong> → caught 80 of the 100 real frauds</li><li><strong>FP = 50</strong> → wrongly flagged 50 legitimate transactions as fraud</li><li><strong>FN = 20</strong> → missed 20 real frauds (let them through)</li><li><strong>TN = 850</strong> → correctly cleared 850 legitimate transactions</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/780/1*PKGGbYdJaSpZoHABjJ-iZg.png" /></figure><p>Total predictions = 80 + 50 + 20 + 850 = 1,000. The model got 930 right (80 + 850), so raw accuracy = 93%. But it missed 20 real frauds and annoyed 50 legitimate customers — context that raw accuracy hides completely.</p><h3>2. Derived Metrics — TPR, FPR, Precision, Recall, and F1</h3><p>From the four cells of the confusion matrix, we can compute a family of important rates. These normalize the raw counts so you can compare models fairly, even across datasets of different sizes.</p><h4>True Positive Rate (TPR) — Also Called Recall or Sensitivity</h4><p><strong><em>TPR = TP / (TP + FN)</em></strong></p><p><strong>Plain English:</strong> Of all the things that were <em>actually positive</em>, what fraction did the model correctly identify?</p><p><strong>Intuition:</strong> TPR measures your model’s ability to find the positive cases. If you have 100 cancer patients and your model catches 90 of them, your TPR = 90/100 = 0.90 (90%).</p><p><strong>Using our fraud example:</strong></p><p><strong><em>TPR = 80 / (80 + 20) = 80 / 100 = 0.80</em></strong></p><p>The model caught 80% of all real frauds.</p><p><strong>When to prioritize TPR:</strong> In any domain where <em>missing a positive is costly</em> — disease detection, fire alarm systems, quality control on critical parts. You want to catch as many real positives as possible, even if it means more false alarms.</p><p><strong>Also known as:</strong> Recall, Sensitivity, Hit Rate</p><h4>False Positive Rate (FPR) — Also Called Fall-out</h4><p><strong><em>FPR = FP / (FP + TN)</em></strong></p><p><strong>Plain English:</strong> Of all the things that were <em>actually negative</em>, what fraction did the model wrongly flag as positive?</p><p><strong>Intuition:</strong> FPR measures how often your model causes false alarms. If you have 900 healthy people and your model falsely flags 90 of them, your FPR = 90/900 = 0.10 (10%).</p><p><strong>Using our fraud example:</strong></p><p><strong><em>FPR = 50 / (50 + 850) = 50 / 900 ≈ 0.056</em></strong></p><p>The model wrongly flagged about 5.6% of legitimate transactions.</p><p><strong>When to prioritize low FPR:</strong> In domains where <em>false alarms are very costly</em> — spam filters where important emails must not be lost, or security systems where blocking legitimate users hurts the business.</p><p><strong>Also known as:</strong> Fall-out, 1 − Specificity</p><h4>Precision</h4><p><strong><em>Precision = TP / (TP + FP)</em></strong></p><p><strong>Plain English:</strong> Of everything the model predicted as <em>positive</em>, how much of it was actually positive?</p><p><strong>Intuition:</strong> Precision answers the question — <em>when my model raises an alarm, how trustworthy is that alarm?</em></p><p><strong>Using our fraud example:</strong></p><p><strong><em>Precision = 80 / (80 + 50) = 80 / 130 ≈ 0.615</em></strong></p><p>When the model flags a transaction as fraudulent, it’s correct only 61.5% of the time.</p><p><strong>When to prioritize Precision:</strong> When false alarms are expensive. A recommendation engine that suggests irrelevant products wastes user attention and damages trust. You want your positive predictions to be highly reliable.</p><h4>Recall vs. Precision Trade-off</h4><p>Precision and Recall are in constant tension with each other. This is one of the most fundamental trade-offs in machine learning:</p><ul><li><strong>To increase Recall:</strong> Lower your classification threshold — predict positive more aggressively. You’ll catch more real positives, but also flag more negatives incorrectly, hurting Precision.</li><li><strong>To increase Precision:</strong> Raise your threshold — only predict positive when you’re very confident. You’ll make fewer false alarms, but you’ll also miss more real positives, hurting Recall.</li></ul><p>There is no free lunch. You must decide which error type your application can tolerate more.</p><h4>F1 Score</h4><p><strong><em>F1 = 2 × (Precision × Recall) / (Precision + Recall)</em></strong></p><p><strong>Plain English:</strong> The F1 Score is the harmonic mean of Precision and Recall. It gives you a single balanced score that penalizes extreme imbalance between the two.</p><p><strong>Why harmonic mean?</strong> Unlike the regular average, the harmonic mean is low unless <em>both</em> values are high. A model with Precision = 1.0 and Recall = 0.0 would have an arithmetic average of 0.5, but an F1 Score of 0 — which correctly reflects that the model is useless.</p><p><strong>Using our fraud example:</strong></p><p><strong><em>Precision ≈ 0.615, Recall = 0.80<br>F1 = 2 × (0.615 × 0.80) / (0.615 + 0.80)<br>F1 = 2 × 0.492 / 1.415<br>F1 ≈ 0.696</em></strong></p><p><strong>When to use F1:</strong> When classes are imbalanced and you care about both precision and recall equally. It’s a good general-purpose metric for classification problems.</p><h4>Specificity (True Negative Rate)</h4><p><strong><em>Specificity = TN / (TN + FP)</em></strong></p><p>Note that: <strong>FPR = 1 − Specificity</strong></p><p><strong>Plain English:</strong> Of all the actual negatives, how many did the model correctly identify as negative?</p><p><strong>Using our fraud example:</strong></p><p><strong><em>Specificity = 850 / (850 + 50) = 850 / 900 ≈ 0.944</em></strong></p><p>The model correctly cleared 94.4% of legitimate transactions.</p><h4>Complete Metrics Summary</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/897/1*Hnk-1mDBFq-9O6Ra6xrrXA.png" /></figure><h3>3. The ROC Curve</h3><p>The <strong>ROC curve</strong> (Receiver Operating Characteristic curve) is a graph that shows the performance of a classification model at <em>all possible classification thresholds</em>. It plots:</p><ul><li><strong>Y-axis: True Positive Rate (TPR / Recall)</strong></li><li><strong>X-axis: False Positive Rate (FPR)</strong></li></ul><h4>Why Do We Need It?</h4><p>Most binary classifiers don’t output a label directly. They output a <strong>probability score</strong> between 0.0 and 1.0. For example, a fraud detection model might output:</p><ul><li>Transaction A → 0.92 (very likely fraud)</li><li>Transaction B → 0.61 (moderately suspicious)</li><li>Transaction C → 0.14 (probably legitimate)</li></ul><p>You then choose a <strong>threshold</strong>. If the score is above the threshold, you predict positive (fraud). Below it, you predict negative (legitimate).</p><p>The threshold is a dial you turn:</p><ul><li><strong>Low threshold (e.g. 0.1):</strong> Flag almost everything as positive → High TPR but also High FPR</li><li><strong>High threshold (e.g. 0.9):</strong> Only flag very certain positives → Low FPR but also Low TPR</li></ul><p>The ROC curve draws one point for every possible threshold from 0.0 to 1.0, connecting them into a continuous line. This lets you see the full performance profile of your model across all operating conditions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hBX7R6X8iP75nVsozFhp9w.png" /></figure><h4>What are TPR and FPR?</h4><p>Every prediction your model makes lands in one of four buckets:</p><ul><li><strong>True Positive (TP)</strong>: predicted positive, actually positive ✓</li><li><strong>True Negative (TN)</strong>: predicted negative, actually negative ✓</li><li><strong>False Positive (FP)</strong>: predicted positive, actually negative ✗ (a “false alarm”)</li><li><strong>False Negative (FN)</strong>: predicted negative, actually positive ✗ (a “miss”)</li></ul><p>From these, two rates are derived:</p><ul><li><strong>TPR (True Positive Rate)</strong> = TP / (TP + FN) — “of all actual positives, how many did I catch?” Also called <em>recall</em> or <em>sensitivity</em>. You want this <strong>high</strong>.</li><li><strong>FPR (False Positive Rate)</strong> = FP / (FP + TN) — “of all actual negatives, how many did I wrongly flag?” You want this <strong>low</strong>.</li></ul><h4>What does the curve actually represent?</h4><p>Your model doesn’t output just “yes/no” — it outputs a <strong>probability score</strong> (e.g. 0.73 = “73% chance this email is spam”). You then pick a <strong>threshold</strong> to convert scores into decisions.</p><ul><li><strong>High threshold (e.g. 0.95)</strong>: only very confident positives are flagged → low FPR, but also low TPR (you miss a lot).</li><li><strong>Low threshold (e.g. 0.10)</strong>: almost everything is flagged → high TPR, but also high FPR (many false alarms).</li></ul><p>The ROC curve plots every possible threshold as a single point on the graph. The <strong>curve is the full picture</strong> of your model’s behavior across all thresholds.</p><h4>Reading the key landmarks</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*GMej1yMORCpJhKW9WfxvAA.png" /></figure><h4>AUC — the single summary number</h4><p><strong>AUC (Area Under the Curve)</strong> collapses the entire curve into one number:</p><ul><li><strong>AUC = 1.0</strong> → perfect classifier</li><li><strong>AUC = 0.5</strong> → random guessing (the triangle under the diagonal)</li><li><strong>AUC = 0.0</strong> → perfectly wrong (a model that always predicts the opposite — perversely useful, just flip its predictions!)</li><li><strong>AUC &gt; 0.9</strong> → excellent; <strong>AUC &gt; 0.8</strong> → good; <strong>AUC &gt; 0.7</strong> → acceptable</li></ul><p>AUC has a nice probabilistic interpretation: it’s the probability that your model ranks a random positive example higher than a random negative one.</p><h3>4. The Recommended Evaluation Workflow</h3><p>Follow this step-by-step process every time you evaluate a classification model:</p><p><strong>Step 1 — Build the confusion matrix first.</strong> Always start here. It gives you the raw TP, FP, FN, TN counts and an immediate visual sense of where your model is failing.</p><p><strong>Step 2 — Compute all your derived metrics.</strong> Calculate Accuracy, TPR, FPR, Precision, Specificity, and F1. Look at them together, not in isolation.</p><p><strong>Step 3 — Plot the ROC curve.</strong> Visualize how your model trades off TPR against FPR across all thresholds. Look at the shape of the curve — does it rise steeply before moving right?</p><p><strong>Step 4 — Compare models using AUC.</strong> If you’re evaluating multiple candidate models, AUC gives you a clean, threshold-free ranking. Higher AUC = better discrimination.</p><p><strong>Step 5 — Select your operating threshold.</strong> Based on the cost of FP vs FN in your domain, pick a threshold from the ROC curve that gives you the right trade-off. Use the Youden Index (TPR − FPR is maximized) as a neutral starting point.</p><p><strong>Step 6 — Recompute final metrics at your chosen threshold.</strong> Compute your final Accuracy, Precision, Recall, and F1 at the selected threshold. These are the numbers you report and monitor in production.</p><h3>5. Common Mistakes and Misconceptions</h3><h4>“My model has 98% accuracy, it must be great.”</h4><p>Not necessarily. If 98% of your data is negative, a model that predicts negative for everything achieves 98% accuracy. Always check the confusion matrix.</p><h4>“AUC = 1.0 means my model is perfect.”</h4><p>More likely, it means there is <strong>data leakage</strong> — information from the future or from the target variable has accidentally leaked into your features. Investigate immediately.</p><h4>“I should always maximize Recall.”</h4><p>No. Maximizing Recall at all costs means lowering your threshold until you predict positive for everything, giving TPR = 1.0 but also FPR = 1.0. Always consider the Precision-Recall trade-off.</p><h4>“AUC below 0.5 means my model is useless.”</h4><p>Actually, an AUC below 0.5 means your model has learned to predict <em>backwards</em> — it consistently ranks negatives higher than positives. You can flip its predictions and get an AUC = (1 − original AUC), which may be quite good.</p><h4>“ROC-AUC is always the best metric.”</h4><p>Not for extremely imbalanced data. In datasets with 0.1% positive rate, even a random model can achieve a reasonably high AUC because the many negatives inflate the TN count and suppress FPR. Consider Precision-Recall AUC in such cases.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fcb14c6865e1" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Linear Regression, Logistic Regression, and Gradient Descent: A Complete…]]></title>
            <link>https://medium.com/@lalanagurusinghe/mastering-linear-regression-logistic-regression-and-gradient-descent-a-complete-e5d09db2d423?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/e5d09db2d423</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Mon, 20 Apr 2026 05:17:59 GMT</pubDate>
            <atom:updated>2026-04-20T05:17:59.211Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mastering Linear Regression, Logistic Regression, and Gradient Descent: A Complete Beginner-to-Intermediate Guide</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/898/1*D63qzLRsW4Oc-AHvn5e-MA.jpeg" /></figure><p>If you’ve ever wanted to predict house prices, classify spam emails, or build your first machine-learning model, you’ve probably heard of linear regression and logistic regression. But how do they actually learn? The secret sauce is Gradient Descent — the algorithm that turns messy data into smart predictions.</p><p>In this full-length, beginner-friendly yet deep-dive article, we’ll break everything down step by step with:</p><ul><li>Zero fluff</li><li>Real intuition</li><li>Clean math (in KaTeX)</li><li>Simple Python code snippets</li><li>Visual analogies</li><li>Common pitfalls and how to avoid them</li></ul><p>By the end, you’ll understand <strong>why</strong> these algorithms work, <strong>how</strong> they connect, and <strong>when</strong> to use each one. Let’s dive in.</p><h3>1. Linear Regression: Predicting Continuous Values</h3><h3>What is it?</h3><p>Linear Regression is the simplest supervised learning algorithm. It predicts a <strong>continuous</strong> number (like price, temperature, or sales) based on input features.</p><p><strong>Real-world examples</strong>:</p><ul><li>Predict house price from size, location, and bedrooms</li><li>Forecast stock prices</li><li>Estimate exam scores from study hours</li></ul><h3>The Model Equation</h3><p>For one feature (simple linear regression):</p><p><strong>ŷ = β₀ + β₁x</strong></p><p>ŷ = predicted value<br> β₀ = y-intercept (bias)<br> β₁ = slope (weight for feature x)<br> x = input feature</p><p>For multiple features (multiple linear regression):</p><p><strong>ŷ = β₀ + β₁x₁ + β₂x₂ + … + βₙxₙ</strong></p><p>In vector form (much cleaner):</p><p><strong>ŷ = Xβ</strong></p><h3>How Do We Measure Error?</h3><p>We use <strong>Mean Squared Error (MSE)</strong> as the cost function:</p><p><strong>J(β) = (1/2m) Σᵢ (ŷᵢ — yᵢ)²</strong></p><ul><li>m = number of training examples</li><li>The square makes big errors hurt more</li><li>The 1/2​ is for mathematical convenience when taking derivatives</li></ul><p><strong>Intuition</strong>: Imagine the line trying to get as close as possible to all the blue dots on a scatter plot. The cost J Jtells us <strong>“how bad”</strong> the current line is.</p><h3>2. Logistic Regression: Classification, Not Regression</h3><h3>Why “Regression” in the name?</h3><p>Don’t be fooled! Logistic Regression is a <strong>classification</strong> algorithm. The name is historical — it still uses a linear combination of features, but then squashes the output into a probability.</p><h3>The Problem It Solves</h3><p>We want to predict <strong>categories</strong>:</p><ul><li>Spam or Not Spam (0 or 1)</li><li>Cancer or No Cancer</li><li>Buy or Not Buy</li></ul><p>Linear regression would output numbers outside [0,1] — useless for probabilities.</p><h3>The Magic: Sigmoid (Logistic) Function</h3><p>We wrap the linear output with the sigmoid:</p><p><strong>σ(z) = 1 / (1 + e⁻ᶻ)</strong></p><p>where <strong>z=Xβ</strong></p><p>The final prediction becomes:</p><p><strong>ŷ = σ(Xβ)</strong></p><p><strong>Sigmoid graph intuition</strong>: It takes any real number and squeezes it into (0,1). At z=0, probability = 0.5. Positive z → probability &gt; 0.5,<br>negative z →&lt;0.5</p><h3>Cost Function: Binary Cross-Entropy (Log Loss)</h3><p>We <strong>cannot</strong> use MSE here. Why?</p><ul><li>MSE creates a non-convex surface with gradient descent (many local minima)</li><li>Log loss is convex → only one global minimum</li></ul><p>The cost function for binary classification:</p><p><strong>J(β) = -(1/m) Σᵢ [ yᵢ log(ŷᵢ) + (1 — yᵢ) log(1 — ŷᵢ) ]</strong></p><p><strong>Intuition</strong>:</p><ul><li>If true label y=1and we predict 0.01, we get heavily penalized (log of small number is large negative)</li><li>If we predict correctly (0.99 when y=1), cost is almost zero</li></ul><h3>Decision Boundary</h3><p>We usually classify as 1 if <strong>ŷ≥0.5</strong> else 0. This creates a linear decision boundary in feature space.</p><p><strong>Pro tip</strong>: For non-linear boundaries, add polynomial features or use kernel tricks (but that’s beyond this post).</p><h3>3. Gradient Descent: The Engine That Makes Both Models Learn</h3><p>This is the <strong>star</strong> of the show. Both linear and logistic regression use the exact same optimization technique.</p><h3>The Hiking Analogy Everyone Loves</h3><p>Imagine you’re on a foggy mountain (the cost function surface) and you want to reach the lowest point (minimum error) without seeing the whole mountain.</p><ul><li>You feel the slope under your feet (gradient)</li><li>You take a step downhill in the steepest direction</li><li>Repeat until you can’t go lower</li></ul><p>That’s gradient descent.</p><h3>The Math (Don’t Panic)</h3><p>The gradient of the cost function tells us the direction of steepest increase. So we go the <strong>opposite</strong> direction:</p><p><strong>β := β — α ∇J(β)</strong></p><ul><li>α = learning rate (step size — super important!)</li><li>∇J(β) = gradient (vector of partial derivatives)</li></ul><p>For <strong>linear regression</strong>, the partial derivative w.r.t. each βj is:</p><p><strong>∂J/∂βⱼ = (1/m) Σᵢ₌₁ᵐ (ŷᵢ — yᵢ) xᵢⱼ</strong></p><p>For logistic regression, it becomes:</p><p><strong>∂J/∂βⱼ = (1/m) Σᵢ₌₁ᵐ (ŷᵢ — yᵢ) xᵢⱼ</strong></p><p>(Surprise! The update rule looks almost identical — only <strong>ŷᵢ </strong>changes because of the sigmoid.)</p><p><strong>Three Flavors of Gradient Descent</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WRa5rrrDnF1qenLGqqOUoA.png" /></figure><p><strong>Learning rate tips</strong>:</p><ul><li>Too big → overshoots, diverges</li><li>Too small → takes forever, gets stuck</li><li>Use learning rate decay or Adam optimizer in practice</li></ul><h3>Convergence Graph</h3><p>You’ll see the cost dropping sharply at first, then flattening. That’s your model learning!</p><h3>Putting It All Together: From Data to Prediction</h3><ol><li><strong>Prepare data</strong> (feature scaling is crucial for gradient descent!)</li><li><strong>Initialize</strong> β \boldsymbol{\beta} β randomly (or to zeros)</li><li><strong>Choose</strong> cost function (MSE or Log Loss)</li><li><strong>Loop</strong> until convergence:</li></ol><ul><li>Compute predictions</li><li>Compute gradient</li><li>Update parameters</li></ul><p>5. <strong>Evaluate</strong> on test set (R² for linear, accuracy/AUC for logistic)</p><p><strong>Quick Python From-Scratch Example (Linear Regression)</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/717/1*WtVYtuvf2oErCDzqO3aGaQ.png" /></figure><h3>Common Pitfalls &amp; Pro Tips</h3><ul><li><strong>Linear Regression assumptions</strong>: Linearity, independence, homoscedasticity, normality of residuals</li><li><strong>Logistic Regression</strong>: No multicollinearity, enough samples per class</li><li><strong>Overfitting</strong>: Use regularization (L1/Lasso or L2/Ridge)</li><li><strong>Feature scaling</strong>: Always standardize features before gradient descent</li><li><strong>When to choose which?</strong></li><li>Continuous target → Linear Regression</li><li>Binary / categorical target → Logistic Regression (or extend to Softmax for multi-class)</li></ul><h3>Conclusion: You Now Understand the Foundations of Modern ML</h3><p>Linear Regression gives you the straight line. Logistic Regression gives you the probability curve. Gradient Descent is the invisible hand that <strong>learns</strong> the best line or curve from data.</p><p>These three concepts power <strong>everything</strong> from simple dashboards to billion-parameter neural networks (they all use some form of gradient descent).</p><p><strong>Next steps for you</strong>:</p><ol><li>Implement both regressions from scratch on Kaggle datasets</li><li>Try scikit-learn version for speed</li><li>Move to neural networks — you’ll see the exact same gradient descent under the hood!</li></ol><p>You now have a solid, interview-ready understanding.</p><p><strong>Bookmark this post</strong>, share it with your study group, and start building!</p><p>#MachineLearning #LinearRegression #LogisticRegression #GradientDescent #DataScience #Python</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e5d09db2d423" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Business Models vs Database Entities in Backend Systems (Spring Boot & .NET Explained)]]></title>
            <link>https://medium.com/@lalanagurusinghe/business-models-vs-database-entities-in-backend-systems-spring-boot-net-explained-cd46043e1438?source=rss-4d3fe492a85f------2</link>
            <guid isPermaLink="false">https://medium.com/p/cd46043e1438</guid>
            <dc:creator><![CDATA[Lalana Gurusinghe]]></dc:creator>
            <pubDate>Thu, 05 Mar 2026 07:12:22 GMT</pubDate>
            <atom:updated>2026-03-05T07:13:33.609Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zCBs_0-kUT0gtkIgN_SM8w.png" /></figure><p>Modern backend applications are rarely built as a single block of code. Instead, they follow <strong>layered architectures</strong> that clearly separate responsibilities such as:</p><ul><li>Data storage</li><li>Business logic</li><li>API communication</li></ul><p>If you have worked with frameworks like <strong>Spring Boot</strong> or <strong>.NET</strong>, you may have noticed several types of classes such as:</p><ul><li><strong>Entities</strong></li><li><strong>DTOs</strong></li><li><strong>Business Models</strong></li><li><strong>Domain Models</strong></li></ul><p>Understanding the differences between these models is essential for designing <strong>scalable, maintainable, and clean backend systems</strong>.</p><p>In this article we will explore:</p><ul><li>What <strong>database entities</strong> are</li><li>What <strong>business models</strong> are</li><li>The <strong>two types of business models used in real systems</strong></li><li>Why frameworks like <strong>Spring Boot often use only DTOs and entities</strong></li><li>Whether <strong>business models can contain methods and behavior</strong></li></ul><h3>1. What Is a Database Entity?</h3><p>A <strong>database entity</strong> represents how data is stored in a <strong>database table</strong>.</p><p>In most backend frameworks, entities are mapped directly to tables using an <strong>ORM (Object Relational Mapper)</strong>.</p><p>Examples of ORMs:</p><ul><li>Spring Boot → <strong>Hibernate / JPA</strong></li><li>.NET → <strong>Entity Framework Core</strong></li></ul><h3>Example Entity (Spring Boot)</h3><pre>@Entity<br>public class RiskRatingLevel {<br>    @Id<br>    private Long id;<br>    private String name;<br>    private int min;<br>    private int max;<br>}</pre><p>This class directly maps to a database table:</p><h3>Characteristics of Entities</h3><p>Entities usually:</p><ul><li>Represent <strong>database tables</strong></li><li>Are managed by an <strong>ORM</strong></li><li>Map <strong>class fields → database columns</strong></li><li>Focus mainly on <strong>data persistence</strong></li><li>Avoid complex business logic</li></ul><p>Think of an <strong>entity</strong> as the system’s <strong>database representation of data</strong>.</p><h3>2. What Is a Business Model?</h3><p>A <strong>business model</strong> (often called a <strong>domain model</strong>) represents the <strong>real-world concept of the system</strong> rather than how the data is stored.</p><p>Example:</p><pre>public class RiskLevel {<br>    private int min;<br>    private int max;<br>    public boolean isWithinRange(int score){<br>        return score &gt;= min &amp;&amp; score &lt;= max;<br>    }<br>}</pre><p>Here the model does more than just hold data.</p><p>It also contains <strong>behavior related to the business rules</strong>.</p><h3>Where Business Models Are Used</h3><p>Business models are commonly used in:</p><ul><li><strong>Domain Driven Design (DDD)</strong></li><li><strong>Clean Architecture</strong></li><li><strong>Enterprise systems</strong></li><li><strong>Complex domain logic applications</strong></li></ul><p>They represent the <strong>core logic of the system</strong>.</p><h3>3. Two Types of Business Models</h3><p>Business models are generally designed in <strong>two different ways</strong>.</p><p>Understanding these two styles is very important for backend architecture.</p><h3>Type 1: Anemic Business Model</h3><p>An <strong>Anemic Model</strong> contains only <strong>data (variables)</strong> and no behavior.</p><p>Example:</p><pre>class User {<br>    private String name;<br>    private int age;<br>}</pre><p>In this design:</p><ul><li>The model only stores data</li><li>All business logic lives in <strong>service classes</strong></li></ul><p>Example service:</p><pre>public void updateUserAge(User user, int age){<br>    if(age &lt; 0){<br>        throw new RuntimeException(&quot;Invalid age&quot;);<br>    }<br>    user.setAge(age);<br>}</pre><h3>Characteristics</h3><ul><li>Models are <strong>simple data containers</strong></li><li>Business logic exists in <strong>services</strong></li><li>Very common in <strong>CRUD systems</strong></li><li>Often used in <strong>microservices</strong></li></ul><p>Many <strong>Spring Boot tutorials follow this style</strong>.</p><h3>Type 2: Rich Domain Model</h3><p>A <strong>Rich Domain Model</strong> contains both:</p><ul><li><strong>State (data)</strong></li><li><strong>Behavior (methods)</strong></li></ul><p>Example:</p><pre>class BankAccount {<br>    private double balance;<br>    public void withdraw(double amount){<br>        if(balance &lt; amount){<br>            throw new RuntimeException(&quot;Insufficient balance&quot;);<br>        }<br>        balance -= amount;<br>    }<br>}</pre><p>Usage:</p><pre>account.withdraw(100);</pre><p>Here the object <strong>knows how to manage itself</strong>.</p><h3>Characteristics</h3><ul><li>Business logic lives <strong>inside the model</strong></li><li>Services orchestrate operations</li><li>Promotes <strong>encapsulation</strong></li><li>Used in <strong>complex enterprise systems</strong></li></ul><p>This approach is heavily used in <strong>Domain Driven Design</strong>.</p><h3>4. DTOs vs Business Models</h3><p>Many developers ask:</p><blockquote><strong><em>If we already have entities and business models, why do we need DTOs?</em></strong></blockquote><p>A <strong>DTO (Data Transfer Object)</strong> is used to transfer data <strong>between layers or between the API and the client</strong>.</p><p>Example:</p><pre>public class RiskLevelDTO {<br>    private String name;<br>    private int min;<br>    private int max;<br>}</pre><h3>DTO Responsibilities</h3><p>DTOs are designed to:</p><ul><li>Expose <strong>only necessary data</strong></li><li>Hide internal database structures</li><li>Prevent <strong>over-fetching</strong></li><li>Improve <strong>API security</strong></li><li>Format responses for the client</li></ul><p>DTOs usually <strong>do not contain business logic</strong>.</p><h3>5. Why Many Spring Boot Projects Only Use DTOs and Entities</h3><p>If you have worked with Spring Boot, you may have noticed that many projects use only:</p><ul><li><strong>Entities</strong></li><li><strong>DTOs</strong></li></ul><p>Example architecture:</p><pre>Controller<br>   ↓<br>Service<br>   ↓<br>Repository<br>   ↓<br>Entity</pre><p>Why?</p><p>Because many backend systems are <strong>CRUD-focused applications</strong>, where:</p><ul><li>business logic is simple</li><li>services handle most logic</li><li>models only store data</li></ul><p>This simplifies the architecture.</p><p>However, when applications grow larger, developers often introduce <strong>domain models</strong>.</p><h3>6. Typical Layered Architecture</h3><p>A more structured backend architecture looks like this:</p><pre>Controller<br>   ↓<br>Service Layer<br>   ↓<br>Business Models / Domain Logic<br>   ↓<br>Repository<br>   ↓<br>Entities<br>   ↓<br>Database</pre><h3>Layer Responsibilities</h3><p>LayerResponsibilityControllerHandles HTTP requestsServiceOrchestrates business operationsBusiness ModelEncapsulates domain logicRepositoryCommunicates with databaseEntityMaps to database tablesDTOTransfers API data</p><p>Each layer has <strong>a single responsibility</strong>, which keeps systems maintainable.</p><h3>7. Can Business Models Contain Methods?</h3><p>Yes — <strong>business models can and often should contain methods</strong>.</p><p>This follows a core Object-Oriented principle called <strong>Encapsulation</strong>.</p><p>Encapsulation means an object contains both:</p><ul><li><strong>data</strong></li><li><strong>behavior</strong></li></ul><p>Example:</p><pre>public class BankAccount {<br>    private double balance;<br>    public void withdraw(double amount) {<br>        if(balance &lt; amount){<br>            throw new RuntimeException(&quot;Insufficient balance&quot;);<br>        }<br>        balance -= amount;<br>    }<br>}</pre><p>Instead of placing this logic inside a service, the object itself manages its own behavior.</p><h3>8. Benefits of Methods in Business Models</h3><h3>1. Better Encapsulation</h3><p>Objects understand their own rules.</p><pre>account.withdraw(100)</pre><p>instead of</p><pre>withdraw(account, 100)</pre><h3>2. Cleaner Service Layer</h3><p>Services coordinate actions instead of implementing every rule.</p><h3>3. Improved Reusability</h3><p>Domain logic can be reused across:</p><ul><li>APIs</li><li>background jobs</li><li>scheduled tasks</li><li>other services</li></ul><h3>9. Practical Example</h3><p>Consider a <strong>Risk Scoring System</strong>.</p><p>The system may include three different models.</p><h3>Entity</h3><pre>RiskRatingLevelEntity</pre><p>Represents a <strong>database table</strong>.</p><h3>DTO</h3><pre>RiskRatingLevelDTO</pre><p>Used for <strong>API responses</strong>.</p><h3>Business Model</h3><pre>RiskRatingLevel</pre><p>Contains business logic such as:</p><pre>isScoreValid()<br>calculateRisk()</pre><p>Each class serves a <strong>different responsibility</strong>.</p><h3>10. Quick Architecture Visualization</h3><p>A complete request flow may look like this:</p><pre>Client<br>   ↓<br>Controller<br>   ↓<br>DTO<br>   ↓<br>Service<br>   ↓<br>Business Model<br>   ↓<br>Repository<br>   ↓<br>Entity<br>   ↓<br>Database</pre><p>This separation keeps systems:</p><ul><li>maintainable</li><li>scalable</li><li>easy to extend</li></ul><h3>Key Takeaways</h3><p>Database entities and business models serve <strong>different purposes</strong> in backend architecture.</p><h3>Database Entities</h3><ul><li>represent database tables</li><li>managed by an ORM</li><li>focus on persistence</li></ul><h3>Business Models</h3><ul><li>represent domain concepts</li><li>may contain behavior</li><li>enforce business rules</li></ul><h3>DTOs</h3><ul><li>transfer data between layers</li><li>shape API responses</li><li>hide internal structures</li></ul><p>Using these models correctly improves:</p><ul><li><strong>maintainability</strong></li><li><strong>scalability</strong></li><li><strong>code clarity</strong></li></ul><h3>Final Thoughts</h3><p>Understanding the separation between <strong>entities, DTOs, and business models</strong> is a major step toward mastering <strong>backend architecture</strong>.</p><p>Smaller applications may work perfectly with only <strong>entities and DTOs</strong>. However, as systems grow larger and business logic becomes more complex, introducing <strong>domain models</strong> helps organize the code and prevent bloated service layers.</p><p>By applying these architectural patterns, developers can build backend systems that are <strong>clean, flexible, and easier to evolve over time</strong>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cd46043e1438" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>