<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Samith Chimminiyan on Medium]]></title>
        <description><![CDATA[Stories by Samith Chimminiyan on Medium]]></description>
        <link>https://medium.com/@samithc?source=rss-03b792d4c412------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*s7blqErbebA4kAUJh7wlNg.png</url>
            <title>Stories by Samith Chimminiyan on Medium</title>
            <link>https://medium.com/@samithc?source=rss-03b792d4c412------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Mon, 16 Nov 2026 00:39:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@samithc/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[History of BirdCLEF competitions in Kaggle]]></title>
            <link>https://medium.com/data-science-collective/history-of-birdclef-competitions-in-kaggle-32e101cf8bb1?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/32e101cf8bb1</guid>
            <category><![CDATA[image-classification]]></category>
            <category><![CDATA[kaggle-competition]]></category>
            <category><![CDATA[kaggle]]></category>
            <category><![CDATA[audio-processing]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Thu, 09 Apr 2026 14:01:02 GMT</pubDate>
            <atom:updated>2026-04-10T10:00:48.757Z</atom:updated>
            <content:encoded><![CDATA[<h3>What insights can we gain from BirdCLEF competitions in Kaggle over the years?</h3><blockquote>This story was written based on the research from claude.ai</blockquote><p>The BirdCLEF competition has been an annual competition for several years, and has helped several other conservation organizations. It is a challenge part of the LifeCLEF initiative, and focuses on creating machine learning models to automatically detect and identify bird species from soundscape recordings.</p><p>Bird sound recognition has become a transformative tool for biodiversity monitoring, with BirdNET standing out as a popular model driving this change.</p><p>The goal of the BirdCLEF+ 2026 competition is to develop machine learning frameworks capable of identifying understudied species within continuous audio data from Brazil’s Pantanal wetlands. Successful solutions will help advance biodiversity monitoring in the last wild places on Earth.</p><p>In this article, we are going through the previous year&#39;s competition and trying to get some insights, so we can understand what things can be reused from earlier competitions. I have used claude.ai for this research.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8CO_79ucbYlqmqiMeBtI5w.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@axholmephotography?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Chris Smith</a> on <a href="https://unsplash.com/photos/european-robin-perched-on-tree-branch-ZeIFgSO3MAU?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Below is the timeline for <a href="https://www.kaggle.com/competitions/birdclef-2026">BirdCLEF+ 2026</a>.</p><ul><li><strong>March 11, 2026</strong> — Start Date.</li><li><strong>May 27, 2026</strong> — Entry Deadline. You must accept the competition rules before this date to compete.</li><li><strong>May 27, 2026</strong> — Team Merger Deadline. This is the last day participants may join or merge teams.</li><li><strong>June 3, 2026</strong> — Final Submission Deadline.</li></ul><p>All deadlines are at 11:59 PM UTC on the corresponding day unless otherwise noted. The competition organizers reserve the right to update the contest timeline if they deem it necessary.</p><p>Let’s jump into the Quick Summary</p><p><strong>Quick Summary</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/847/1*GfDYhFNE3ucp64bmvERMAQ.png" /><figcaption>Quick Summary</figcaption></figure><p>Going through this Quick Summary, you might have been thinking why we need to refer to these old competitions. We should have just started implementing a new prediction model and submitted our submission for the competitions.</p><p>If you have been working on Kaggle, you might be aware of how difficult the task is. Usually, there are write-ups from the topper in the leatherboard. They will be describing there apporach and how we were able to score high on the leatherboard. A few of them share their entire code. So, for a beginner to learn and reach the top of a competition, they need to refer to the previous competitions and the best practices and techniques that can be reused.</p><p>Especially with this competition, since these are audio signals that need to be converted to a Mel Spectrogram, and they need to try out different models and save the models. For creating the models GPU can be used, but later these models are ensembled to create a submission file for which a GPU is not recommended or cannot be used. So it is very important to refer to all the resources available from previous competitions.</p><p>So let&#39;s see a table with some comprehensive details.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*K0gXRCc_vOYUcfwipZkwgw.png" /></figure><p>Finally, we can check what things can be reused for this year&#39;s competition.</p><p><strong>Cross-Year Patterns: What Has Stayed True Every Year</strong></p><figure><img alt="Cross-Year Patterns: What Has Stayed True Every Year" src="https://cdn-images-1.medium.com/max/532/1*lXCjfV0Qebk8sFa3aa35BA.png" /></figure><p><strong>Key Data Sources Reusable for 2026</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/687/1*KRPj8G8k_LCsVyuZLpIXrA.png" /></figure><p>In this discussion forum for BirdCLEF+ 2026 i have seen a <a href="https://www.kaggle.com/competitions/birdclef-2026/discussion/684207">discussion</a> regarding LLM tools (i.e., GPT, Gemini, Claude), and many are using the same. But for beginners, I think the best way to go is the traditional way by using the techniques from earlier competitions. Our only aim with competitions should not be winning. We also need to learn something new.</p><h3>That’s a Wrap!:</h3><p>I believe this article has given a general overview of the history of the BirdCLEF competitions. Even though this competition started in 2014, the details mentioned above are from 2021. From 2021 onward, competitions moved to Kaggle. I hope you have liked it.</p><h3>Reference:</h3><ol><li><a href="https://www.kaggle.com/competitions/birdclef-2026/overview">BirdCLEF+ 2026 | Kaggle</a></li></ol><h3>Connect With Me</h3><p>You can also connect with me on <a href="https://x.com/samithcsachi">Twitter</a>, <a href="https://www.kaggle.com/samithsachidanandan">Kaggle</a>, and <a href="http://www.linkedin.com/in/samithchimminiyan">LinkedIn</a>.</p><p><em>Feel free to </em><strong><em>hold down the clap button 👏</em></strong><em> (you can clap up to 50 times!) to help others find this article. What are your thoughts? Let me know in the responses!</em></p><p>Cheers,</p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=32e101cf8bb1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/history-of-birdclef-competitions-in-kaggle-32e101cf8bb1">History of BirdCLEF competitions in Kaggle</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A General Introduction to Numpy]]></title>
            <link>https://samithc.medium.com/a-general-introduction-to-numpy-030508fea207?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/030508fea207</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[numpy]]></category>
            <category><![CDATA[python-programming]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Tue, 31 Mar 2026 14:01:03 GMT</pubDate>
            <atom:updated>2026-03-31T14:01:03.025Z</atom:updated>
            <content:encoded><![CDATA[<p><strong>NumPy: the absolute basics for beginners</strong></p><p>NumPy(Numerical Python) is a fundamental library for Python numerical computing. It provides efficient multi-dimensional array objects and various mathematical functions for handling large datasets, making it a critical tool for professionals in fields that require heavy computation.</p><p>In this article, we are going to walk through the basics of Numpy, which is an important skill to be mastered for Data Science.</p><h3>Key Features of NumPy</h3><p>NumPy has various features that make it popular over lists.</p><ul><li><strong>N-Dimensional Arrays</strong>: NumPy’s core feature is ndarray, an N-dimensional array object that supports homogeneous data types.</li><li><strong>Arrays with High Performance</strong>: Arrays are stored in contiguous memory locations, enabling faster computations than Python lists<strong> </strong>(Please see Numpy Array vs Python List for details).</li><li><strong>Broadcasting: </strong>This allows element-wise computations between arrays of different shapes. It simplifies operations on arrays of various shapes by automatically aligning their dimensions without creating new data.</li><li><strong>Vectorization:</strong> Eliminates the need for explicit Python loops by applying operations directly on entire arrays.</li><li><strong>Linear algebra</strong>: NumPy contains routines for linear algebra operations, such as matrix multiplication, decompositions, and determinants.</li></ul><h3>Installing NumPy in Python</h3><p>To begin using NumPy, you need to install it first. This can be done using the following pip command:</p><pre>pip install numpy</pre><p>Once installed, import the library with the alias np</p><pre>import numpy as np</pre><p>To understand well, we can group Numpy into 15 levels. Let’s jump into each level.</p><h3>Level 1: Creating Numpy Arrays</h3><p>We shall check how to create a NumPy array from a Python list. Arrays are the core data structure in NumPy and are optimized for numerical operations.</p><pre>import numpy as np <br><br>numbers = np.array([1, 2, 3, 4, 5])<br><br>print(numbers) </pre><pre>[1 2 3 4 5]</pre><h3>Level 2: Arrays with Zeros and Ones</h3><p>NumPy allows you to initialize with zeros and ones quickly. This is useful for placeholders and matrix multiplication.</p><pre>import numpy as np <br><br>zero_array = np.zeros(4)<br>one_array = np.ones((2,3))<br><br>print(zero_array)<br>print(one_array) </pre><pre>[0. 0. 0. 0.]<br>[[1. 1. 1.]<br> [1. 1. 1.]]</pre><h3>Level 3: Random Number Arrays</h3><p>Random arrays are useful for simulations, testing, and experiments. NumPy can generate random values within a given shape.</p><pre>import numpy as np <br><br>random_values = np.random.rand(3, 2)<br><br>print(random_values)</pre><pre>[[0.93562587 0.17394292]<br> [0.71929706 0.62396122]<br> [0.2143406  0.08835186]]</pre><h3>Level 4: Reshaping Arrays</h3><p>Reshaping changes the structure of an array without modifying its data. This is helpful when working with multidimensional data.</p><pre>import numpy as np <br><br>data = np.array([2 , 4, 6, 8, 10, 12])<br>reshaped_data = data.reshape(3,2)<br><br>print(reshaped_data)</pre><pre>[[ 2  4]<br> [ 6  8]<br> [10 12]]</pre><h3>Level 5: Array Indexing and Slicing</h3><p>Indexing and slicing allow you to access specific elements or ranges from an array. This is essential for data selection.</p><pre>import numpy as np <br><br>values = np.array([5, 10, 15, 20, 25])<br><br>print(values[3])<br><br>print(values[-1:])</pre><pre>20<br>[25]</pre><h3>Level 6: Basic Mathematical Operations</h3><p>NumPy supports element-wise operations between arrays of the same or compatible shapes.</p><pre>import numpy as np <br><br>a = np.array([1, 2, 3])<br>b = np.array([4, 5, 6]) <br><br>result = a * b <br>print(result) <br><br>result = a + b <br>print(result)</pre><pre>[ 4 10 18]<br>[5 7 9]</pre><h3>Level 7: Statistical Operations</h3><p>NumPy provides built-in statistical functions such as mean, median, and standard deviation for data analysis.</p><pre>import numpy as np <br><br>scores = np.array([10, 20, 30, 40, 50])<br><br>print(np.mean(scores))<br><br>print(np.median(scores))<br><br>print(np.std(scores))</pre><pre>30.0<br>30.0<br>14.142135623730951</pre><h3>Level 8: Broadcasting</h3><p>Broadcasting allows NumPy to perform operations between arrays of different shapes with automatically expanding dimensions.</p><pre>import numpy as np <br><br>arr = np.array([1, 2, 3])<br>scaled = arr + 5 <br><br>print(scaled) </pre><pre>[6 7 8]</pre><h3>Level 9: Boolean Indexing</h3><p>Boolean indexing filters array elements based on conditions, making data selection simple and expressive.</p><pre>import numpy as np <br><br>data = np.array([12, 18, 25, 30, 42])<br>filtered = data[data&gt;20]<br><br>print(filtered) </pre><pre>[25 30 42]</pre><h3>Level 10: Vectorized Operations</h3><p>Vectorization allows operations on entire arrays without loops resulting in faster and cleaner code.</p><pre>import numpy as np <br><br>nums = np.array([1, 2, 3, 4])<br><br>result = np.sqrt(nums)<br><br>print(result) </pre><pre>[1.         1.41421356 1.73205081 2.        ]</pre><h3>Level 11: Linear Algebra Operations</h3><p>NumPy includes powerful linear algebra tools such as matrix multiplication and determinant calculation.</p><pre>import numpy as np <br><br>matrix_a = np.array([[1, 2], [3, 4]])<br>matrix_b = np.array([[2, 0], [1, 2]])<br><br>product = np.dot(matrix_a, matrix_b)<br>determinant = np.linalg.det(matrix_a)<br><br>print(product)<br>print(determinant) </pre><pre>[[ 4  4]<br> [10  8]]<br>-2.0000000000000004</pre><h3>Level 12: Aggregation Functions</h3><p>Aggregation functions summarize data using operations like sum, minimum, maximum, and cumulative sum.</p><pre>import numpy as np <br><br>values = np.array([3, 6, 9, 12])<br><br>print(np.sum(values))<br>print(np.min(values))<br>print(np.max(values))<br>print(np.cumsum(values))</pre><pre>30<br>3<br>12<br>[ 3  9 18 30]</pre><h3>Level 13: Sorting and Searching</h3><p>Sorting arranges data in order, while searching helps locate elements or their indices that match specific conditions.</p><pre>import numpy as np <br><br>arr = np.array([45, 10, 30, 20])<br><br>sorted_arr = np.sort(arr)<br>indexes = np.where(arr&gt; 25) <br><br>print(sorted_arr)<br>print(indexes)</pre><pre>[10 20 30 45]<br>(array([0, 2]),)</pre><h3>Level 14: Concatenation and Splitting</h3><p>Arrays can be combined or split into smaller parts, which is useful for data processing and restructuring.</p><pre>import numpy as np <br><br>x = np.array([1, 2, 3])<br>y = np.array([4, 5, 6]) <br><br>combined = np.concatenate((x, y))<br>split_arrays = np.split(combined, 2)<br><br><br>print(combined) <br><br>print(split_arrays) </pre><pre>[1 2 3 4 5 6]<br>[array([1, 2, 3]), array([4, 5, 6])]</pre><h3>Level 15: Data Type Conversion</h3><p>NumPy allows easy conversion of array data types, which is important for memory optimization and formatting.</p><pre>import numpy as np <br><br>numbers = np.array([2.5, 4.8, 6.1])<br>int_numbers = numbers.astype(int)<br>string_numbers = int_numbers.astype(str)<br><br>print(int_numbers)<br>print(string_numbers) <br></pre><pre><br>[2 4 6]<br>[&#39;2&#39; &#39;4&#39; &#39;6&#39;]</pre><h3>NumPy and Machine Learning</h3><p>NumPy is a core library used to work with numerical data in the machine learning domain. Datasets are represented by arrays, which make operations fast (based on arrays or vectors) and allow any operations that are performed in a typical linear algebra/statistics/data processing application.</p><p>Examples of operations accomplished through NumPy include normalization of datasets, calculation of loss functions, and updating gradient values.</p><p>The libraries used to build machine learning models, including scikit-learn, TensorFlow, and PyTorch, are modeled after many of the same principles as NumPy. For this reason, mastering NumPy is a critical first step for anyone entering machine learning, as it provides the performance and functionality necessary to work with data at scale.</p><h3>That’s a Wrap!:</h3><p>I believe this article has given a general overview of what NumPy is in Python. I have tried to write this article in a simple manner so that even a beginner can understand. I hope you have liked it.</p><h3>Reference:</h3><ol><li><a href="https://www.geeksforgeeks.org/python/introduction-to-numpy/">NumPy Introduction — GeeksforGeeks</a></li></ol><h3>Connect With Me</h3><p>You can also connect with me on <a href="https://x.com/samithcsachi">Twitter</a>, <a href="https://www.kaggle.com/samithsachidanandan">Kaggle</a>, and <a href="http://www.linkedin.com/in/samithchimminiyan">LinkedIn</a>.</p><p><em>Feel free to </em><strong><em>hold down the clap button 👏</em></strong><em> (you can clap up to 50 times!) to help others find this article. What are your thoughts? Let me know in the responses!</em></p><p>Cheers,</p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=030508fea207" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Python Classes Explained Simply]]></title>
            <link>https://medium.com/data-science-collective/python-classes-explained-simply-1f3cd2acbf2c?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/1f3cd2acbf2c</guid>
            <category><![CDATA[class]]></category>
            <category><![CDATA[python-programming]]></category>
            <category><![CDATA[objectorientedprogramming]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Tue, 24 Mar 2026 14:01:06 GMT</pubDate>
            <atom:updated>2026-03-25T18:12:37.759Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A beginner’s guide to OOP — from blueprint to running code</em></h4><p>When you start learning Python, you will easily follow through the basics like Variables and Data Types, Basic Data Structures, Control Flow, Functions, etc., without much difficulty. It&#39;s because it&#39;s easy to interrupt the usage of those as a beginner, and when we reach the term “Python Class,” most people feel lost in jargon, and might have to Google it to understand it better. When we first see <em>self</em>, __init__, constructors, instances, we might be thinking, What is it actually?</p><p>Most tutorials jump straight into the technical details without explaining the <strong>why</strong>.</p><p>In this article, we are going to walk through the Python Class step by step in layman’s language so that even a very biginer shall understand the concept.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9yvVwtnNfxdw3A9GdCKT1Q.png" /></figure><h3>Introduction</h3><p>Python is an object-oriented language, allowing you to structure your code using classes and objects for better organization and reusability. Object Oriented Programming is a way of organizing code that uses objects and classes to represent real-world entities and their behavior. In OOP, an object has attributes thing that have specific data and can perform certain actions using methods.</p><ul><li>Organizes code into classes and objects.</li><li>Supports encapsulation to group data and methods together.</li><li>Enables inheritance for reusability and hierarchy.</li><li>Allows polymorphism for flexible method implementation.</li><li>Improves modularity, scalability, and maintainability.</li></ul><p>Above is the theory behind the OOP concepts, but we need to understand that the core idea is to have reusability. As programmers, ideally, we keep our code DRY (Don’t Repeat Yourself).</p><h3><strong>What is a class?</strong></h3><p>When we hear the term &quot;class,&quot; we need to think of it as a blueprint. For example, a blueprint of a house is not a house. It&#39;s just a plan that describes what every house built from it will look like. Likewise, a Python class describes what every object created from it will contain.</p><p>To elaborate more, we can create a Book class. So Book class will describe what a Book is. Each book will have a title, an author, and some pages. So our class will be capturing it.</p><pre>class Book:<br>    # This is just the blueprint<br>    # No actual book exists yet</pre><p>The class itself does nothing. It’s just a description. The magic happens when you create an object from it.</p><h3>The __init__ method — the one-time setup</h3><p>The __init__ a method in Python is a special initializer method (often referred to as a &quot;constructor&quot; in other languages) that runs automatically when a new instance (object) of a class is created. __init__is the short form for initialize. Python calls it automatically the moment you create an object. You never call it yourself; Python does it for you. Its main purpose is to initialize the object’s attributes and set up its initial state. When an object is created, memory is allocated for it, and __init__ helps organize that memory by assigning values to attributes.</p><p>So what actully it do? Just think of it like the setup step when you first open a brand-new book. You stamp your name inside, write the date, and maybe dog-ear the first page. You’re setting it up before you start using it.</p><pre>class Book:<br>    def __init__(self, title, author, pages):<br>        self.title = title<br>        self.author = author<br>        self.pages = pages<br>        self.pages_read = 0   # always starts at 0</pre><h4>But wait — what is self?</h4><p>Most will be wondering why we are using the self. self is Python’s way of saying “this specific object”. Imagine you have three books on your shelf. When you call a method on book1, Python needs to know <em>which</em> book to act on. That’s what self does — it points to the specific object in memory.</p><pre>book1 = Book(&quot;Harry Potter&quot;, &quot;J.K. Rowling&quot;, 500)<br>book2 = Book(&quot;Dune&quot;, &quot;Frank Herbert&quot;, 900)<br>book3 = Book(&quot;1984&quot;, &quot;George Orwell&quot;, 300)<br><br># Each book has its OWN separate data<br># self.title for book1 = &quot;Harry Potter&quot;<br># self.title for book2 = &quot;Dune&quot;<br># self.title for book3 = &quot;1984&quot;</pre><h4>Why self.title and not just title?</h4><p>In everyone&#39;s mind, this question pops up.</p><p>When you write self.title = title, there are actually two different things called title:</p><ul><li>title (right side) — a temporary parameter. It only lives inside __init__ and disappears when the method finishes.</li><li>self.title (left side) — a permanent attribute saved on the object. It lives as long as the object lives.</li></ul><p>Without self.title, your data would vanish the moment __init__ finished running. With it, the data is saved permanently on the object for every other method to use.</p><p>So, in common words, we can say that it is used to save data permanently on the object so that every other method can use.</p><p>Every self.something you plan to use anywhere in your class MUST be defined in __init__ first. If you skip it, every other method will crash trying to access something that doesn’t exist.</p><h3>Methods — what your object can do</h3><p>After initializing, we need to write methods, and we can write as many methods as we want. So what is a method? By definition method is a function that is bound to an object or a class and operates on its data. So the method is very similar to functions orelse we can think of it as just functions with some changes. Below are difference between function and methods.</p><h4>Function</h4><ul><li>Can exist independently outside of classes (in procedural programming).</li><li>Accepts parameters and may return a value.</li><li>Can be called directly by its name.</li></ul><h4>Method</h4><ul><li>Belongs to a class (can be an object or static method).</li><li>Can access class variables and other methods.</li><li>Called using the object (or class name if static).</li></ul><p>Each method takes self as its first parameter so it can reach into the object’s stored data.</p><p>Think of the methods as the abilities of your Book object — things it can do or tell you about itself.</p><pre>class Book:<br>    def __init__(self, title, author, pages):<br>        self.title = title<br>        self.author = author<br>        self.pages = pages<br>        self.pages_read = 0<br><br>    def get_summary(self):<br>        return f&quot;{self.title} by {self.author} ({self.pages} pages)&quot;<br><br>    def is_long(self):<br>        if self.pages &gt; 400:<br>            return f&quot;{self.title} is a long book&quot;<br>        else:<br>            return f&quot;{self.title} is a short book&quot;<br><br>    def read(self, pages_read_today):<br>        self.pages_read += pages_read_today<br>        remaining = self.pages - self.pages_read<br>        return f&quot;Read {self.pages_read} pages. {remaining} remaining.&quot;<br><br>    def is_finished(self):<br>        if self.pages_read &gt;= self.pages:<br>            return f&quot;You finished &#39;{self.title}&#39;!&quot;<br>        else:<br>            return f&quot;{self.pages - self.pages_read} pages to go.&quot;</pre><p>Notice that every method carries self as its first parameter. This is how they share the same data, self is the shared memory locker that __init__ stocked up, and every method has the key.</p><h3>The main() function — where you use it all</h3><p>The main() function is the entry point of an executable program. When a program starts running, the operating system or runtime environment calls main() to begin execution. Here you create objects using __init__ and then call the methods you need.</p><pre>def main():<br>    # __init__ fires here automatically<br>    book1 = Book(&quot;Harry Potter&quot;, &quot;J.K. Rowling&quot;, 500)<br>    book2 = Book(&quot;Dune&quot;, &quot;Frank Herbert&quot;, 300)<br><br>    print(&quot;=== Book 1 ===&quot;)<br>    print(book1.get_summary())<br>    print(book1.is_long())<br>    print(book1.read(100))<br>    print(book1.read(50))<br>    print(book1.is_finished())<br><br>    print(&quot;=== Book 2 ===&quot;)<br>    print(book2.get_summary())<br>    print(book2.is_long())<br>    print(book2.read(150))<br>    print(book2.read(160))<br>    print(book2.is_finished())</pre><pre>=== Book 1 ===<br>Harry Potter by J.K. Rowling (500 pages)<br>Harry Potter is a long book<br>Read 100 pages. 400 remaining.<br>Read 150 pages. 350 remaining.<br>350 pages to go.<br><br>=== Book 2 ===<br>Dune by Frank Herbert (300 pages)<br>Dune is a short book<br>Read 150 pages. 150 remaining.<br>Read 310 pages. -10 remaining.<br>You finished &#39;Dune&#39;!</pre><h3><strong>if __name__ == “__main__” :— the import guard</strong></h3><p>We have already covered how to create a class and execute it. But without mentioning if __name__ == “__main__”: it won’t be complete, as it allows developers to separate executable code from reusable functions.</p><p>I am sure that most beginners have wondered why we are using this. What is the purpose?</p><p>Every Python file has a built-in variable called __name__. When you run a file directly, Python sets it to “__main__”. When another file imports it, Python sets it to the filename instead.</p><pre># Without the guard:<br>def main():<br>    ...<br>main()   # This runs even when imported <br><br># With the guard:<br>def main():<br>    ...<br>if __name__ == &quot;__main__&quot;:<br>    main()   # Only runs when you execute this file directly</pre><p>Every Python file has a built-in variable called __name__. When you run a file directly, Python sets it to “__main__”. When another file imports it, Python sets it to the filename instead.</p><p>Always use if __name__ == “__main__” at the bottom of any file that defines a class. It costs nothing and saves you headaches the moment your project grows beyond one file.</p><p>So, in simple terms, we can say that this is used to ensure that when a Python file is imported, only its functions/classes are used, and the main execution code does not run.</p><h3>The Complete Code</h3><p>If you would like to explore the complete working code of the Book class, you can refer to the code below.</p><pre>class Book:<br>    def __init__(self, title, author, pages):<br>        self.title = title<br>        self.author = author<br>        self.pages = pages<br>        self.pages_read = 0<br><br>    def get_summary(self):<br>        return f&quot;{self.title} by {self.author} ({self.pages} pages)&quot;<br><br>    def is_long(self):<br>        if self.pages &gt; 400:<br>            return f&quot;{self.title} is a long book&quot;<br>        else:<br>            return f&quot;{self.title} is a short book&quot;<br><br>    def read(self, pages_read_today):<br>        self.pages_read += pages_read_today<br>        remaining = self.pages - self.pages_read<br>        return f&quot;Read {self.pages_read} pages. {remaining} remaining.&quot;<br><br>    def is_finished(self):<br>        if self.pages_read &gt;= self.pages:<br>            return f&quot;You finished &#39;{self.title}&#39;!&quot;<br>        else:<br>            return f&quot;{self.pages - self.pages_read} pages to go.&quot;<br><br><br>def main():<br>    book1 = Book(&quot;Harry Potter&quot;, &quot;J.K. Rowling&quot;, 500)<br>    print(book1.get_summary())<br>    print(book1.read(100))<br>    print(book1.is_finished())<br><br><br>if __name__ == &quot;__main__&quot;:<br>    main()</pre><h3>Key Concepts Covered :</h3><p>• <strong>Class :</strong> The blueprint. It defines what the object will look like. No object exists yet.</p><p><strong>• __init__ : </strong>The one-time setup. Runs automatically when you create an object. Stores all the data using self.something, so every method can share it.</p><p><strong>• self</strong> : The object itself. Passed into every method, so Python knows which specific object to act on.</p><p><strong>• Methods : </strong>The abilities of the object. Each one takes self and uses the shared data set up by __init__.</p><ul><li><strong>main() + if __name__ == “__main__”:</strong> : where you create objects and use them. The guard ensures main() only runs when you execute the file directly, not when it is imported.</li></ul><p><em>__init__ builds the locker. self is the key. Every method uses that key to read and update what’s inside.</em></p><h3>That’s a Wrap!:</h3><p>I believe this article has given a general overview of what a class looks like in Python. I have tried to write this article as I would have approached this concept as a beginner, and what questions might come to my mind. I hope this has given an indepth over view about the fundamental concept of <strong>Object-Oriented Programming (OOP) — class.</strong></p><h3>Reference:</h3><ol><li><a href="https://www.w3schools.com/python/python_oop.asp">Python OOP (Object-Oriented Programming)</a></li><li><a href="https://www.geeksforgeeks.org/python/python-oops-concepts/">Python OOP Concepts — GeeksforGeeks</a></li></ol><h3><strong>Furthermore</strong>!</h3><p>Join my free newsletter where I share practical insights, real-world learnings, and weekly ideas from my journey as a Machine Learning Engineer and Data Scientist.</p><p>I write about Machine Learning, Data Science, and how to actually apply these concepts beyond theory, focusing on intuition, hands-on workflows, and building things that matter.</p><p>If you’re looking to better understand ML or see how it works in real-world scenarios, this is for you.</p><p><a href="https://datagreekinsights.substack.com/subscribe">Subscribe to Data Greek Insights</a></p><h3>Connect With Me</h3><p>You can also connect with me on <a href="https://x.com/samithcsachi">Twitter</a>, <a href="https://www.kaggle.com/samithsachidanandan">Kaggle</a>, and <a href="http://www.linkedin.com/in/samithchimminiyan">LinkedIn</a>.</p><p><em>Feel free to </em><strong><em>hold down the clap button 👏</em></strong><em> (you can clap up to 50 times!) to help others find this article. What are your thoughts? Let me know in the responses!</em></p><p>Cheers,</p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1f3cd2acbf2c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/python-classes-explained-simply-1f3cd2acbf2c">Python Classes Explained Simply</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[General Introduction to JAX]]></title>
            <link>https://pub.towardsai.net/general-introduction-to-jax-a01c8387a13b?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/a01c8387a13b</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[jax]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Sun, 22 Mar 2026 20:20:58 GMT</pubDate>
            <atom:updated>2026-03-22T20:23:35.682Z</atom:updated>
            <content:encoded><![CDATA[<p>Whenever we talk about Neural Networks, We usually ask whether we need to learn PyTorch or TensorFlow. <a href="https://pytorch.org/">PyTorch</a> and <a href="https://www.tensorflow.org/">TensorFlow</a> are the two most dominant open-source frameworks for building, training, and deploying neural networks. While both offer comprehensive tools and support for GPU acceleration, their core design philosophies and optimal use cases differ.</p><p>But is that the only option for Neural Networks? No, not at all.</p><p>There are many options, including Keras, JAX, MXNet, etc.</p><p>I wrote an article regarding the What’s Next After PyTorch and TensorFlow over <a href="https://medium.com/towards-artificial-intelligence/whats-next-after-pytorch-and-tensorflow-a-look-at-emerging-machine-learning-technologies-a74d7fe2bcc4">here</a></p><p>In this article, we are going to explore JAX — the machine learning technologies that every machine learning engineer needs to know beyond Pytorch and Tensorflow.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*bAuf34R94ybrbvYGpNhltg.png" /></figure><h3>Introduction</h3><p>JAX is a powerful library for <strong>high-performance numerical computing</strong>, combining NumPy-like APIs, automatic differentiation, and <strong>XLA (Accelerated Linear Algebra)</strong> compilation. Currently, Machine Learning is evolving, and apart from Model Accuracy, performance and scalability are becoming important, and that&#39;s where JAX comes into the picture.</p><p>NumPy (Numerical Python) is the fundamental open-source library for scientific computing in Python and Many think JAX is just another alternative but in reality it is a powerful tool designed for high-performance machine learning research and production workflows.</p><h3>What is JAX?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/250/0*bh_sV9NR45qWs8No.png" /></figure><p>JAX is a Python library for accelerator-oriented array computation and program transformation, designed for high-performance numerical computing and large-scale machine learning. It was developed by Google.</p><p>What makes JAX special :</p><ul><li>It gives the simplicity of NumPy</li><li>Automatic differentiation</li><li>GPU/TPU acceleration</li><li>Just-In-Time (JIT) compilation</li></ul><p>These features are important for building and optimizing powerful machine learning models.</p><h3>Key Features</h3><ol><li><strong>NumPy-like API: </strong>If someone uses NumPy and is familiar with Python scientific computing, they will feel JAX very similar.</li></ol><pre>import numpy as np <br><br>x_np =np.linspace(0,10,1000)<br>y_np = 2 * np.sin(x_np) * np.cos(x_np)<br>plt.plot(x_np, y_np);</pre><pre>import jax.numpy as jnp <br><br>x_jnp =jnp.linspace(0,10,1000)<br>y_jnp = 2 * jnp.sin(x_jnp) * jnp.cos(x_jnp)<br>plt.plot(x_jnp, y_jnp);</pre><p><strong>2. Automatic Differentiation: </strong>JAX’s autodiff makes it easy to compute higher-order derivatives, because the functions that compute derivatives are themselves differentiable. Thus, higher-order derivatives are as easy as stacking transformations. A set of composable function transformations includes jax.grad, jax.jvp, and jax.vjp.</p><pre>import jax<br>import jax.numpy as jnp<br><br>def f(x):<br>    return x**3 + 2*x**2 - 5*x + 1<br><br>dfdx = jax.grad(f)<br>print(dfdx(2.0))</pre><p><a href="https://docs.jax.dev/en/latest/_autosummary/jax.grad.html#jax.grad"><strong>jax.grad()</strong></a> takes a function and returns a function. If you have a Python function f that evaluates the mathematical function, then jax.grad(f) is a Python function that evaluates the mathematical function. That means grad(f)(x) represents the value .</p><p><strong>3. JIT Compilation : </strong>It is a technique that improves application performance by translating code (often intermediate bytecode) into native machine code at runtime, rather than before execution. It boosts performance by identifying frequently executed “hot spots” and compiling them, combining the flexibility of interpreted code with the speed of compiled code.</p><pre>import jax<br>import jax.numpy as jnp<br>import time<br><br>def slow_function(x):<br>    for _ in range(100):<br>        x = jnp.sin(x) + jnp.cos(x)<br>    return x<br><br><br>fast_function = jax.jit(slow_function)<br><br><br>x = jnp.array([1.0, 2.0, 3.0])<br><br><br>start = time.time()<br>for _ in range(100):<br>    result = slow_function(x).block_until_ready() <br>elapsed_no_jit = time.time() - start<br>print(f&quot;Without JIT: {elapsed_no_jit:.4f}s&quot;)<br><br><br>fast_function(x).block_until_ready()<br><br><br>start = time.time()<br>for _ in range(100):<br>    result = fast_function(x).block_until_ready()<br>elapsed_jit = time.time() - start<br>print(f&quot;With JIT:    {elapsed_jit:.4f}s&quot;)<br>print(f&quot;Speedup:     {elapsed_no_jit / elapsed_jit:.2f}x&quot;)</pre><pre>Without JIT: 0.4398s<br>With JIT:    0.0026s<br>Speedup:     171.09x</pre><p><strong>4. Hardware Acceleration: </strong>JAX provides hardware acceleration and it runs seamlessly on CPUs, GPUs, and TPUs. It achieves high performance by automatically fusing operations, optimizing memory access, and enabling massive parallelization with minimal code changes</p><p><strong>5. Functional Programming Style:</strong> It focuses on pure functions, immutable data structures, and composable function transformations, enabling automatic differentiation, JIT compilation (via XLA), and automatic parallelization.</p><h3><strong>What is the idea behind JAX?</strong></h3><p>If we want to push the boundaries of the discipline of Machine Learning, it will be great if we decouple the mathematics from everything else, and something that’s designed for accelerators, so that we can do the math really fast. This will give us the freedom to iterate, innovate, and make new discoveries; that&#39;s the idea behind JAX.</p><ul><li>J stands for <strong>JIT Compilation</strong></li><li>A stands for <strong>Autograd</strong></li><li>X stands for <strong>XLA Compiler</strong></li></ul><h3>Who Uses JAX?</h3><p>JAX is widely adopted for its speed, automatic differentiation, and ability to run on TPUs. So it is used both in research and in industries, performance and scalability matter.</p><ol><li><strong>Research Labs: </strong>In Organizations like DeepMind, JAX serves as a foundational tool for high-performance ML research. It powers major projects like Gemini and Gemma. Advanced work includes Reinforcement Learning, Scientific simulations, and Large-scale neural networks.</li><li><strong>Machine Learning Engineers &amp; Researchers: </strong>Many developers who are working on Custom deep learning architectures, Optimization-heavy problems, and Experimental ML systems, JAX is a popular choice, and they use JAX heavil<strong>y.</strong></li><li><strong>High-Performance Computing &amp; Scientific Research: </strong>Researchers use JAX to combine numerical computing (NumPy) with hardware acceleration for specialized tasks like physical simulations, climate modeling, computational biology, and graph neural networks, etc. It is ideal for these domains as it is able to scale across the accelerators.</li></ol><h3><strong>How to Use JAX?</strong></h3><p>It&#39;s very easy to get started with JAX.</p><p>Installation:</p><ul><li><strong>CPU-only (Linux/macOS/Windows)</strong></li></ul><pre>pip install -U jax</pre><ul><li><strong>GPU (NVIDIA, CUDA 13)</strong></li></ul><pre>pip install -U &quot;jax[cuda13]&quot;</pre><ul><li><strong>GPU (AMD, ROCm)</strong></li></ul><pre>pip install -U &quot;jax[rocm7-local]&quot;</pre><ul><li><strong>TPU (Google Cloud TPU VM)</strong></li></ul><pre>pip install -U &quot;jax[tpu]&quot;</pre><p>Refer to the codes i have provided earlier, which are some basic examples of JAX. Below is one more example, which covers Vectorization.</p><pre>from jax import vmap<br><br>def square(x):<br>    return x ** 2<br><br>vectorized_square = vmap(square)<br>print(vectorized_square(jnp.array([1, 2, 3])))</pre><h3><strong>Why Should You Learn JAX?</strong></h3><p>Anyone looking for a career in advanced machine learning or research-level engineering should consider JAX; it will definitely give them a competitive edge.</p><p><strong>JAX Is Ideal:</strong></p><ul><li>You seek the highest level of performance possible.</li><li>You work extensively with a broad array of big models.</li><li>You want precise and detailed control over your own training pipelines.</li><li>You are conducting research/ experimental ML</li></ul><p><strong>JAX Isn’t the Right Tool:</strong></p><ul><li>You need to prototype rapidly — use PyTorch or TensorFlow</li><li>You prefer to build models using object-oriented programming techniques versus “function-based”.</li></ul><h3>That’s a Wrap!:</h3><p>I believe this article has given a general overview of JAX. I have tried to answer the question like What is JAX? Who Uses It? And How to Get Started? so that it becomes a general introduction to JAX.</p><p>JAX is moving us closer to machine learning systems that are composable, high-performance, and large-scale. If you want to build state-of-the-art AI systems or support advanced research laboratories, learning JAX will become increasingly important, valuable &amp; necessary in the future.</p><h3>Reference:</h3><ol><li><a href="https://docs.jax.dev/en/latest/notebooks/thinking_in_jax.html">JAX Documentation</a></li><li><a href="https://youtube.com/playlist?list=PL95lT3XlM14SRqhuoA1fO9WehD_pfli3T&amp;si=dcE0H3VlPml4CU_R">JAX Foundations</a></li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a01c8387a13b" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/general-introduction-to-jax-a01c8387a13b">General Introduction to JAX</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Bias–Variance Tradeoff: The Real Reason Your Model Fails]]></title>
            <link>https://medium.com/towards-data-engineering/the-bias-variance-tradeoff-the-real-reason-your-model-fails-d42d0068d06c?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/d42d0068d06c</guid>
            <category><![CDATA[bias-variance-tradeoff]]></category>
            <category><![CDATA[bias]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ml-concept]]></category>
            <category><![CDATA[variance]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Tue, 10 Mar 2026 18:01:00 GMT</pubDate>
            <atom:updated>2026-03-24T02:59:16.787Z</atom:updated>
            <content:encoded><![CDATA[<p>An untold story of Bias–Variance Tradeoff</p><p>Most failures in machine learning are not due to the quality of machine learning algorithms.</p><p>The majority of these issues arise from the misinterpretation of one important idea:</p><p>Bias versus Variance.</p><p>If this Trade-Off is truly understood, you will see that it should be true:</p><ul><li>The performance of a model in training doesn’t compare to its performance in production</li><li>Having More features will sometimes be quite detrimental to the model’s performance</li><li>Deep Learning Models experience a very high level of overfitting</li><li>Regularization works</li></ul><p>We will discuss this in layman’s terms below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pWtOX2WUbdgzNObAKhp1Uw.jpeg" /></figure><h3>What Is Bias?</h3><p>Bias is an error that occurs when the learning algorithm’s assumptions are overly simplistic.</p><p>Examples of high-bias models</p><ul><li>They either oversimplify the problem.</li><li>They ignore key features.</li><li>They will underfit the data.</li></ul><p>Example:</p><p>In an example, you try to fit a highly nonlinear data set with a linear regression model that doesn’t fit the data. Even if you applied the model correctly, it would still create an underfitted model due to not capturing the curve.</p><p>Mathematically:<br>High Bias → High Training Error → High Test Error</p><h3>What Is Variance?</h3><p>High variance refers to a model being overly sensitive to its training data, meaning it:</p><ul><li>-Memorizes the training data,</li><li>Learns noise as if it were part of the real data,</li><li>Is significantly worse on new (testing) data.</li></ul><p>One way you can create a model with high variance:</p><ul><li>Create a very deep decision tree without pruning.</li></ul><p>In some cases, you may even end with a model that has low training error but high test error. This is called overfitting.</p><p>Mathematically:<br>High variance → Low Training Error → High Test Error</p><h3>The Core Tradeoff</h3><p>The important part is that:</p><p>As the complexity of the model increases, the Bias decreases and Variance increases, and as the complexity of the model decreases, the Bias increases and Variance decreases.</p><p>So, as a Machine Learning Engineer / Data Scientist its our job to find the <strong>optimal balance</strong> that minimizes total error.</p><h3>Visuals for Better Understanding</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*ETHQO_pipOqFOzTM96T4rg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/1*kBqFHWKxC_Nmchp9X3O8zg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/571/1*n2TcxU0O_MGlABbVBf1wqQ.jpeg" /></figure><h3>Why This Matters in Real ML Projects</h3><p>This concept explains:<br>• The effectiveness of regularization.<br>• The importance of cross-validation.<br>• Why having more data is better.<br>• Why a simple model can outperform one that just marginally reduces bias.</p><h3>How to Diagnose Bias vs Variance</h3><p>If we are having high training error and high validation error, then the problem is high bias, and if we are having low training error and high validation error, then the problem is low variance. The ideal situation to have is low training error and validation error.</p><h3>Modern ML Twist</h3><p>In deep learning, an interesting phenomenon occurs when a large neural network has a very low bias and a very low variance, which is known as “Double Descent”. This contradicts classical intuition on the trade-off between bias and variance, but there is still a trade-off in these cases. The research on Double Descent is ongoing.</p><h3>That’s a Wrap!:</h3><p>In this article, I have just gone through the general intuition on the most common topic in Machine Learning. It’s better to check for Bias and Variance before switching algorithms, tuning hyperparameters, and adding more features…</p><p>Most ML debugging becomes simple once you answer that.</p><p>Hoping that you have liked the article.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d42d0068d06c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/towards-data-engineering/the-bias-variance-tradeoff-the-real-reason-your-model-fails-d42d0068d06c">The Bias–Variance Tradeoff: The Real Reason Your Model Fails</a> was originally published in <a href="https://medium.com/towards-data-engineering">Towards Data Engineering</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is Time Series Forecasting?]]></title>
            <link>https://medium.com/data-science-collective/what-is-time-series-forecasting-4956fa5afd6e?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/4956fa5afd6e</guid>
            <category><![CDATA[time-series-forecasting]]></category>
            <category><![CDATA[arima]]></category>
            <category><![CDATA[statistical-modeling]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Fri, 13 Feb 2026 13:18:39 GMT</pubDate>
            <atom:updated>2026-02-14T11:13:02.665Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*znJKcFuoVFfpADmeobjSkg.png" /></figure><p>Time Series Forecasting is an important area that all Machine Learning Engineers heard of. It’s often said that Time Series Forecasting pays the bills, but many run behind other topics and tend to ignore Time Series Forecasting.</p><p>Let’s check Time Series Forecasting in detail. In short, we can tell that anytime your dataset has a timestamp, you’re working with “time-series” data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZmymPJwDZW4fMLGR.png" /></figure><h3>What is a time series?</h3><p>A time series is a set of observations taken sequentially in time. The focus is on the word time. If we keep taking the same observation over time, we will get a time series.</p><h3>Types of time series</h3><p>There are two types of time series data based on time intervals, as outlined here:</p><p>• <strong>Regular time series:</strong> This is the most common type of time series, where we have observations coming in at regular intervals of time, such as every hour or every month.</p><p>• <strong>Irregular time series:</strong> There are a few time series where we do not have observations at regular intervals of time.</p><h3>Main areas of application for time series analysis</h3><ul><li><strong>Time series forecasting:</strong> Predicting the future values of a time series, given the past values</li><li><strong>Time series classification:</strong> Sometimes, instead of predicting the future value of the time series, we may also want to predict an action based on past values.</li><li><strong>Outlier detection:</strong> There are some situations where we only want to detect if something is going wrong or if something is out of the ordinary.</li><li><strong>Interpretation and causality:</strong> You can use time-series analysis to understand the whats and whys of the time series based on past values, understand the relationships between several related time series, or derive causal inferences based on time series data.</li></ul><h3>Why? Time Series Forecasting?</h3><p>Here’s why Time Series Forecasting is extremely powerful in Python</p><p>• Demand forecasting<br>• Sales &amp; revenue prediction<br>• Energy load forecasting<br>• Stock &amp; crypto analysis<br>• Anomaly detection in logs &amp; sensors</p><p>It usually brings millions of savings with a small improvement in forecasting accuracy. Some of the companies these forecasting is not an option its a must to enable them to sustain and compete with their competitors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ves_fk6_nvJ3U26a.png" /></figure><h3>When Should You Use Time Series Forecasting?</h3><p>There are several criteria to be met in order to perform a Time Series Forecasting. It is usually done when we have handsome data that can be measured over a period of time.</p><p>We need to understand many things before we go for Time Series Forecasting.</p><p>Business Problem is Clearly Defined: When the business problem is defined up front, it allows you to refer to it at any time, so you can ensure you are working towards your objective.</p><p>Sufficient Data is there to capture: In order to begin answering the above question and understanding the business problem and possible solutions, you need to have the data necessary to perform the time series analysis. Generally speaking, the more data you have, the better. Businesses often capture loads of data only to later decide what they are interested in asking of said data.</p><p>Choosing the Level of Accuracy You Need: Generally speaking, the further out you predict, the less accurate your prediction will be. Typically, time series data are recorded in a timestamp format containing the year, month, day, hour, minute, second, and millisecond it occurred. You don’t need to worry about a lack of granularity; you simply have to choose one. From that starting point, your most accurate prediction will be one unit into the future. Whether that is one day, week, or month is up to the granularity you chose. In this web traffic example, your data should be weekly because you want to spend on ads if you predict a dip in the upcoming 1–2 weeks. The granularity you choose is informed by business needs and what actions need to be taken when.</p><p>Determining How Often Model Outputs Are Needed: Because the predictions are made at the weekly level and the action to be taken is to increase ad spend, it would make the most sense to run the model every 2 weeks, predicting 1–2 weeks out. You’d have to check out what the predictions are, and if web traffic is expected to drop below the threshold, you’d want to have the ad operations team increase ad spend to level out the dip in traffic. In a more advanced system, the outputs of the model could be run automatically to inform the inputs to ad spend models, once the performance of the model is determined to be high enough.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Gz3KR03nHOP1yR6XtLi0eQ.png" /></figure><h3>That’s a Wrap!:</h3><p>I believe this article has given a general overview of Time Series Forecasting. Have you noticed that Time Series is a blend of Statistics, Machine learning, Domain knowledge, and Engineering discipline?.</p><p>We need to master Time Series Forecasting if we need to build a Machine Learning system that directly impacts business decisions.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4956fa5afd6e" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science-collective/what-is-time-series-forecasting-4956fa5afd6e">What is Time Series Forecasting?</a> was originally published in <a href="https://medium.com/data-science-collective">Data Science Collective</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Importance of a Well-Structured README.md File in a Data Science Portfolio project]]></title>
            <link>https://pub.towardsai.net/importance-of-a-well-structured-readme-md-file-in-a-data-science-portfolio-project-ec8b98eb7fc6?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec8b98eb7fc6</guid>
            <category><![CDATA[projects]]></category>
            <category><![CDATA[readme-file]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Sun, 08 Feb 2026 16:47:48 GMT</pubDate>
            <atom:updated>2026-02-14T03:08:14.474Z</atom:updated>
            <content:encoded><![CDATA[<h4>Explains my structure for the README.md file in projects.</h4><h3>Why is it important to maintain a good README file?</h3><p>A well-structured README.md file in a data science portfolio project is important because it serves as the project&#39;s primary documentation and first impression for potential employers, collaborators, and users. Its structure is crucial for effectively communicating the project&#39;s value, methodology, and results.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yalKF7V_AawcJ3g0iykXAA.png" /></figure><h3>What is a README file? Who do you write the README file for?</h3><p>A README file is written for anyone who might interact with your project, acting as an instruction manual and the project’s “face” or “resume”. This broad audience includes:</p><ul><li><strong>Other developers and potential contributors:</strong> It provides a map for new collaborators, guiding them on how to set up the environment, run tests, and contribute effectively. It helps them understand the project’s architecture, tech stack, and contribution guidelines.</li><li><strong>Users (both technical and non-technical):</strong> It explains what the project does, why it is useful, and how to install and use it. A good README makes the project approachable and saves users time and frustration.</li><li><strong>Project managers and stakeholders:</strong> The introduction and high-level description help non-engineering staff understand the project’s scope, purpose, and key functionality.</li><li><strong>Potential employers:</strong> A well-crafted README can showcase your work, the quality of your code, and your documentation habits, helping your project stand out in a portfolio.</li><li><strong>Your future self:</strong> Even if you are the sole developer, a README serves as a permanent reference point that can quickly jog your memory about project specifics, dependencies, and setup instructions if you return to it after a long time.</li></ul><h3>How to write a well-structured README file in a Data Science Portfolio project?</h3><p>In this article i shall walk through my structure for writing a well-structured README.md file. As a Machine Learning Engineer, I believe it is equally important to build an end-to-end portfolio project to create and complete the documentation of the project. If the documentation is not perfect, it will give hard times to people who are going through the project.</p><p>I have recently completed my portfolio project — Customer Segmentation &amp; Retention Strategy using Transactional Data. I shall walk you through the README file of the project.</p><h4>Introduction</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/894/1*Pwqyul2vj5c-wbqkFnijOQ.png" /></figure><p>I generally start the introduction section of my project with an image. Usually, it will be the image of the project, or I might get some image from unsplash.com that is related to the project.</p><p>Just below that i keep the badges from <a href="https://shields.io/">Shields.io</a>, which shall cover important details of the repo. Followed by the acknowledgement for <a href="https://shields.io/">Shields.io</a>.</p><p>The last part of the introduction is a short description of the project, along with the important links to access the project. In this section, you can add the project deployment links, articles, project explaining video, etc.</p><h4>Author and Table of Contents</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/895/1*0U4SQhljKuIzDSfsV2nPdQ.png" /></figure><p>In this section, we can mention the author or collaborators of the project, and the Table of Contents is a navigational guide to direct, quick access to specific sections.</p><h4>Problem Statement and Tech Stack</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/892/1*hRkmce8c0kHgWGG4xQd7FA.png" /></figure><p>In this section, we can give a brief on the problem statement and the technical tools we have used to create the project. I usually have projects which are having some real-world use cases. So i use to mention the same as the problem statement. Tech Stack: I used to mention the main tools I have used.</p><h4>Data Source</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/891/1*-3xtV5xc-FcyP5OJaHXxUw.png" /></figure><p>In this section, the details of the dataset need to be mentioned. Usually, we can get these details from the source that we used to download the dataset. If the dataset was created, then we need to mention the same.</p><h3>Glance at the results</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A3tKzt9QKOMb1HjNxVS3_A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BKC5fEKMRibdBlybrb4FNg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1023/1*gx8d-JLc8vl4Ht9eoMRDdA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*25qIRl-jQL8ISotLjqbjmQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NzGC2Yf1KCasH7L2dusfDA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YUNbLd0fuQ5iq0tLTx-M3g.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*izHd_vBCpHAfaFx18OnAZg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iByvBZgIUdaAkIGTD0N3ZQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mBtZC7_a2NzjDDS6fLwBEQ.png" /></figure><p>This is one of the most important sections of the README file. In this, we need to include the main images we created while performing the Exploratory Data Analysis, followed by the insights, findings, and results. In this, explain the results and our thought process of the project. By reading this section, the reader should understand the project in detail. That is the purpose of this section. I used to include most of the important details over here.</p><h4>Limitations and Lessons Learned</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G4iHtS_kMPjSGQzneN5htA.png" /></figure><p>These sections are used to explain the limitations and lessons learned. It is like explaining how we see the future of the project. Its a brief overview we are giving the reader how we think the project can go over the years.</p><h4>Run Locally</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hpmt_PRKtaaSEkypSLY5Yw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sNHXYAyUgsTCXCISPp9whg.png" /></figure><p>This section is used to mention the comments that are used if the repo is forked to the reader&#39;s system, and then how they need to execute the application. I used to give very minute details in this section so that even a beginner can test this app in their system locally.</p><h4>Explore the notebook, Contribution, Repository structure, License, and Contact</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iAMmA6uM4od6brwtiFfj7A.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*C6HzQJhmyWwF7-Wat1IjoQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DNiVqCi-Rl0CF4BpzsmWKA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VmhtQN9CqBXmIk6KV4QlaQ.png" /></figure><ul><li>Explore the Notebook: Even though the file is in the same repo i use to mention the link so that the reader can go to the top of the page. Sometimes I used to mention the Kaggle Notebook link also over here.</li><li>Contribution: Since this is an open source project, i used mention this for anyone who is interested in contributions.</li><li>Repository Structure: This is the actual structure of the project. Mostly, we tend to ignore the artifacts folder and will not commit it to the repo, so this will give a clear understanding.</li><li>License: Since this is an open source project i use the MIT license.</li><li>Contact: Any questions, suggestions, or collaborations in data science, feel free to reach out:</li></ul><h3>That’s a Wrap!:</h3><p>The article was aimed to give a clear understanding of how we can structure a README file that gives all the information needed to the user to understand the project. These sections are not mandatory and can be altered as per your project. The purpose of the README file is to give an overview of the project before it is run locally. I used to follow this structure and believe this can be followed by any data science professional.</p><p>If you like this article, please feel free to like and share.</p><p><em>Machine learning isn’t just about better models — it’s about better questions, better judgment, and better decisions And that’s where meaningful machine learning begins.</em></p><h4>Download Link</h4><p>Github Repo: — <a href="https://github.com/samithcsachi/Customer-Segmentation-Retention-Strategy/tree/main">Link</a></p><p>README.md: — <a href="https://github.com/samithcsachi/Customer-Segmentation-Retention-Strategy/blob/main/README.md">link</a></p><p><strong>Cheers,</strong></p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec8b98eb7fc6" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/importance-of-a-well-structured-readme-md-file-in-a-data-science-portfolio-project-ec8b98eb7fc6">Importance of a Well-Structured README.md File in a Data Science Portfolio project</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Customer Segmentation and Retention Strategy using Transactional Data]]></title>
            <link>https://pub.towardsai.net/customer-segmentation-and-retention-strategy-using-transactional-data-9b74e0d27e35?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/9b74e0d27e35</guid>
            <category><![CDATA[customer-churn]]></category>
            <category><![CDATA[data-analysis]]></category>
            <category><![CDATA[churn-prediction]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Fri, 30 Jan 2026 19:29:45 GMT</pubDate>
            <atom:updated>2026-02-05T05:02:23.317Z</atom:updated>
            <content:encoded><![CDATA[<h4>Today, I want to walk you through my project — Customer Segmentation &amp; Retention Strategy using Transactional Data.</h4><p><strong>So what is it?</strong></p><p>You will be definelty heard of the term Churn Prediction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HidIKmpui7spxqO7Q3Y_-A.jpeg" /><figcaption>Photo by <a href="https://unsplash.com/@blakewisz?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Blake Wisz</a> on <a href="https://unsplash.com/photos/woman-holding-magnetic-card-q3o_8MteFM0?utm_source=unsplash&amp;utm_medium=referral&amp;utm_content=creditCopyText">Unsplash</a></figcaption></figure><p>Churn prediction uses data analysis and machine learning to identify customers likely to stop using a product or service, allowing businesses to proactively retain them by spotting early warning signs.</p><p>In other terms, we can define customer churn as the number of existing customers lost, for any reason at all, over a given period of time. It provides companies with an understanding of customer satisfaction and customer loyalty, and can identify potential changes in a company’s bottom line.</p><h3>Why is it important?</h3><p>Churn prediction is critical because it allows businesses to proactively identify at-risk customers, enabling targeted retention strategies that are far more cost-effective than acquiring new ones.</p><h3>What are we analysing in this project?</h3><p>In this projects analysis is structured as follows.</p><ul><li><strong>Data Understanding &amp; Quality — </strong>Can the business rely on this data?</li><li><strong>Customer Activity Over Time — </strong>Understand overall customer engagement.</li><li><strong>Purchase Behavior Analysis — </strong>Understand frequency &amp; value patterns.</li><li><strong>Monetary Value Distribution — </strong>Identify value concentration.</li><li><strong>Recency &amp; Inactivity Patterns — </strong>Understand inactivity before modelling churn.</li><li><strong>RFM Segmentation — </strong>Segment customers by transactional value.</li><li><strong>Behavioural Stability (RSS) — </strong>Identify unstable vs predictable customers.</li><li><strong>Churn Label Validation — </strong>Validate churn definition.</li><li><strong>Segment × Churn × Value Analysis — </strong>Tie everything together.</li><li><strong>Business Recommendations </strong>— Convert analysis into strategy.</li></ul><h3>Problem Statement</h3><p>The Online Retail Store wants to reduce customer churn and increase lifetime value by targeting the right customers with the right interventions. In order to setup customer churn prediction model, they needto analyse the sales data, and with that analysis, they can set up an churn prediction model which suilts there business needs.</p><h3>Dataset</h3><p>This is the Online Retail dataset, provided by the UCI, which contains transactions (including cancellations) carried out by an online retailer (mainly focused on gift items) between 01/12/2009 and 09/12/2011.</p><ul><li><strong><em>Invoice:</em></strong> Invoice number. A 6-digit integral number uniquely assigned to each transaction. If this code starts with the letter ‘c’, it indicates a cancellation.</li><li><strong><em>StockCode:</em></strong> Product (item) code. A 5-digit integral number uniquely assigned to each distinct product.</li><li><strong><em>Description:</em></strong> Product (item) name.</li><li><strong><em>Quantity:</em></strong> The quantities of each product (item) per transaction. Numeric.</li><li><strong><em>InvoiceDate:</em></strong> Invoice date and time. The day and time when a transaction was generated.</li><li><strong><em>Price:</em></strong> Unit price. Product price per unit in sterling (£).</li><li><strong><em>CustomerID:</em></strong> Customer number. A 5-digit integral number uniquely assigned to each customer.</li><li><strong><em>Country:</em></strong> Country name. The name of the country where a customer resides.</li></ul><p>Dataset Source: — <a href="https://archive-beta.ics.uci.edu/dataset/502/online+retail+ii">https://archive-beta.ics.uci.edu/dataset/502/online+retail+ii</a></p><h3>Analysis</h3><p><strong>Data Understanding &amp; Quality</strong></p><p><strong>Goal:</strong> Can the business rely on this data?</p><p>Questions to answer :</p><ul><li>What is the time range of the data?</li><li>How many unique customers, transactions, and products?</li><li>Are there missing or invalid Customer IDs?</li><li>Are there negative quantities or prices (returns, cancellations)?</li><li>How many transactions are refunds vs purchases?</li></ul><p>Most of the data is clean, but it has some null values, duplicates, and anomalies in the stock code and description which need to cleaned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*p7Jm6G2qOymcdAC4IFsFGQ.png" /></figure><p>Customer ID has almost 25% of missing values. It&#39;s not recommended to impute such a large percentage of missing values this might cause bias or noise in the analysis. Customer segmentation is purely based on customer behaviour and preference. So it is very important to have accurate data. Description is having very negligible amount of missing values. Since it is a low-impact categorical feature, we have two options: Can drop this as the impact is less, or we can impute as “Unknown” and use it as a categorical feature. In this case i am inputting as “Unknown” for description and dropping the Customer ID.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*7ptDxYrSQDQVfHWmOltmFQ.png" /></figure><p>The dataset has 4646 unique stock code that is showing that the there is a different range of products in the dataset. This will be used to understand the behavior of the customer. Exploring the top 10 frequent stock codes will help to determine customer purchase frequency. Let’s check if there is any anomaly in the stock code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eu4VsVMYi3__f5JX0EgXYQ.png" /></figure><p>Anomalies in the description columns were cleaned. Standardised the Description column. The final step of the cleaning process filtered out the rows where the price is 0.</p><h4>Customer Activity Over Time</h4><p><strong>Goal:</strong> Understand overall customer engagement.</p><p>Questions to answer :</p><ul><li>How many active customers per month?</li><li>Is customer activity increasing, stable, or declining?</li><li>Are there seasonal patterns in purchases?</li><li>Do we see spikes due to holidays or promotions?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HLYi4smeWhIa4zmJgHm6Pw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BZ_sQNB9f45bPRgfAlbt_Q.png" /></figure><p>We can see almost similar patterns in both plots. Customer activity and sales are increasing from September to December for both years. Major events like Halloween and Christmas are during that time. We are seeing the spikes due to holidays.</p><h4><strong>Purchase Behavior</strong></h4><p><strong>Goal:</strong> Understand frequency &amp; value patterns.</p><p>Questions to answer :</p><ul><li>How often do customers place orders?</li><li>What is the distribution of purchase frequency?</li><li>What is the average order value (AOV)?</li><li>Are there many one-time buyers?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GjZKxhiDP1VIA4IxjojEvQ.png" /></figure><p>The Recency plot suggests that there are two types of customer either dormant or active. Almost 1500 customers are purchasing within 50 days, and another movement is near 350–400 days, so this suggests that there are almost 1500 -2000 regular buyers. The next plot shows that the average days between purchases is 0 -5 days, but this is for the repeat customers. Need to see the percentage of repeat customers next.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/481/1*1lbdRe4IShYd8owZSBozjA.png" /></figure><p>The pie chart shows that there are almost 75.6 % repeated customers, but we need to check whether they are long-term customers or 2nd purchase / 3rd purchase.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/859/1*9nC6PKGLDSFoE2a5xDW1kA.png" /></figure><p>This chart gives us a clear picture. Among the repeated customer most have exactly 2 purchases, and a few have 3 purchases, and dramatically it is getting reduced. So Very few customers become highly loyal (10+ purchases). This indicates there is no long-term relationship with customers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/219/1*m779CpU8jO_4bNtZe5IGiA.png" /></figure><p>Almost 50 % of the customers are in 1 -3 purchases.</p><h4>Monetary Value Analysis</h4><p><strong>Goal:</strong> Identify value concentration.</p><p>Questions to answer :</p><ul><li>How is revenue distributed across customers?</li><li>What % of revenue comes from top 10% customers?</li><li>Are high-value customers frequent or infrequent?</li><li>Do a few customers dominate revenue?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aith2JjNJ9RJxQEzhOym1w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*N7d4pcjGi8nhD3yw9LjUAA.png" /></figure><p>Below is the breakdown of revenue.</p><p>- 0–80% customers: £9.3M</p><p>- 80–90% customers: £1.3M</p><p>- 90–100% customers: £10.4M</p><p>As per the plot, the revenue is gradually increasing till 90 %, and there is a sudden increase in the revenue. Top 10 % contributes almost 50 % of revenue. This is an indication of over-reliance on the top 10 %, and if the top 10% churn, then the 50% of the revenue is gone. But there lies an opportunity to understand what makes the top 10% different and replicate their characteristics in the 80–90% segment. This is having massive growth potential.</p><h4>Recency &amp; Inactivity Patterns</h4><p><strong>Goal:</strong> Understand inactivity before modeling churn.</p><p>Questions to answer :</p><ul><li>How long since customers last purchased?</li><li>What does the recency distribution look like?</li><li>Where do natural breakpoints appear (30, 60, 90 days)?</li><li>Does inactivity correlate with lower future spending?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZPpUcCltwkwclserjwpEbg.png" /></figure><p>This chart gives a larger picture. 59.2% of customers are at-risk or lost, and Only 40.8% are actively engaged. Different measures need to be introduced to recover the lost or at-risk customers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TdIqkD6sl7NegSXFJzOKSw.png" /></figure><p>This indicates that the top spenders are regularly active and they are buying regularly. We can confirm this only by re-verifying because there is a high chance that the top spenders may be one-time buyers. Let’s verify this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/345/1*0If39s9thSxL8xsuvkrMHQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4XiUUiZn8Gdyl8OkAPHfMQ.png" /></figure><p>These visuals confirm that the top 10 % are regular buyers, and this also suggests that there is massive growth potential if we are able to convert the bottom 50 % like our top 10%.</p><h4><strong>RFM Segmentation (Customer Value Lens)</strong></h4><p><strong>Goal:</strong> Segment customers by transactional value.</p><p>Questions to answer :</p><ul><li>How do customers distribute across R, F, and M?</li><li>Which RFM segments generate the most revenue?</li><li>Are there high-value but low-recency customers?</li><li>Which segments are shrinking over time?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/941/1*unrlr3YS3lkkx8X8b9fpLg.png" /></figure><p>The heatmap indicates that around 574 top buying customers are dormant now. Need to find out the reasons why they became inactive and if we are able to return them its a huge advantage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Ff8cQzzg30GMibMhWT91AA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YfzO9OKJ88mbmKLBH8eacg.png" /></figure><p>These charts indicate the same problem. Champions — Customer is running the show, and the business is very much dependent on them. Need to bring back the lost top buyers. Need to develop a different strategy for each customer type.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*g7i04CZz--nsqyvLLAT9Cw.png" /></figure><p>This plot indicates that we are losing customers and are facing a retention crisis.</p><h3>Behavioural Stability (RSS) Analysis</h3><p><strong>Goal:</strong> Identify unstable vs predictable customers.</p><p>Questions to answer :</p><ul><li>Do customers spend consistently or erratically?</li><li>Is spending increasing or decreasing over time?</li><li>How variable are purchase intervals?</li><li>Which segments show declining engagement trends?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vd3LhWxYe5sd5tduasArAg.png" /></figure><p>This plot is indicating best customers aren’t growing their spending. At-risk customers are spiking in spending that might be the big purchase before they churn.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*yDF2s5har7JNE4DUYOUh3g.png" /></figure><p>This plot shows that even though the champions are regular buyers and active, it is difficult to predict the exact timing of purchase. Champions are amazing customers, but cannot manage them blindly because they can’t predict their behavior.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/850/1*kliIHRNaeM8czVD4tQvD5g.png" /></figure><p>The churn problem is not much related to spending. The spending trend range is identical, and there is no correlation.</p><h3>Churn Label Exploration</h3><p><strong>Goal:</strong> Validate churn definition.</p><p>Questions to answer :</p><ul><li>What % of customers are labeled as churned?</li><li>How does churn vary across RFM segments?</li><li>Are high-value customers churning less or more?</li><li>Does churn correlate with recency gaps?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/1*W_BJozmO8x6CQMIqV7EtBA.png" /></figure><p>More that 50 % of customers have churned overall. Many low-engagement customers churn, and few Champions churn</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/779/1*my5iX3z3HrRRduZIPPlCOA.png" /></figure><p>As expected, the Champions havea perfect 0% retention. Need to work on Best, Good and Hibernating customers.</p><h3><strong>Segment × Churn × Value Matrix</strong></h3><p><strong>Goal:</strong> Tie everything together.</p><p>Questions to answer :</p><ul><li>Which segments have high churn <em>and</em> high value?</li><li>Which segments have low churn but low value?</li><li>Where should retention budget be spent?</li><li>Where should we avoid intervention?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/983/1*wHZStQTpugcB9uy3yAndLA.png" /></figure><p>This plot indicates that there is a conversation threshold, not a retention problem.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*AQK97T7ghfobQjJcO3FvFw.png" /></figure><h3>Actionable Business Decisions</h3><p><strong>Goal:</strong> Convert analysis into strategy.</p><p>Questions to answer :</p><ul><li>Who should get retention offers?</li><li>Who should get early access to new features?</li><li>Who is likely to churn regardless of intervention?</li><li>What is the expected business impact of acting?</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*8sC7Ia8Qdbu8kctVFQ4_2g.png" /></figure><p>The business is totally dependent on the top-most best customers, and it is failing to convert the 75% of other tier customers, and that is resulting in a more than 50 % churn rate. The actual problem is the conversion rate and not retention. In order to recover, below Four strategic initiatives are recommended.</p><p>- Make the top-most best customers happy with VIP programs.</p><p>- Recover the dormant customer who was previously the top-most best customers through win-back campaigns</p><p>- Fix conversion pipeline for Best/Good customers</p><p>- Implement a visibility system to predict churn</p><h3>Conclusion</h3><p>This analysis is part of my end-to-end project for customer segmentation and retention strategy built on transactional data, combining RFM-based analytics, churn modelling, deployment, and monitoring to support data-driven retention decisions. Still, the project is ongoing. Need to complete the model training and then convert the project fully withthe life cycle of Model which includes Training, Evaluation, Deployment, Monitoring and Maintenance.</p><p>Hope you have liked the analysis. For a more detailed analysis, kindly refer to my repo.</p><p>Repo : <a href="https://github.com/samithcsachi/Customer-Segmentation-Retention-Strategy/blob/main/research/01_exploratory_analysis_and_business_insights.ipynb">Customer-Segmentation-Retention-Strategy/research/01_exploratory_analysis_and_business_insights.ipynb at main · samithcsachi/Customer-Segmentation-Retention-Strategy</a></p><p>Kaggle Notebook: <a href="https://www.kaggle.com/code/samithsachidanandan/customer-segmentation-retention-analysis">Customer Segmentation &amp; Retention Analysis</a></p><p><strong>Cheers,</strong></p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9b74e0d27e35" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/customer-segmentation-and-retention-strategy-using-transactional-data-9b74e0d27e35">Customer Segmentation and Retention Strategy using Transactional Data</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Hidden Assumptions Behind Linear Regression]]></title>
            <link>https://pub.towardsai.net/the-hidden-assumptions-behind-linear-regression-7ee4de664f42?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ee4de664f42</guid>
            <category><![CDATA[linear-regression]]></category>
            <category><![CDATA[mathematics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[algorithms]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Fri, 23 Jan 2026 10:29:58 GMT</pubDate>
            <atom:updated>2026-01-26T04:42:27.542Z</atom:updated>
            <content:encoded><![CDATA[<h4>Why Assumptions Matter More Than the Equation</h4><p>Linear Regression is one of the first topics that we hear when we start learning Machine Learning. Because of that, many people treat it as something to quickly “get through” before moving on to more complex models.</p><p>Often, it is labeled as a basic model that we need to learn and then move beyond.</p><p>Is that a mistake?<br>Yes. Absolutely.</p><p>Because if you truly understand Linear Regression, you already understand the core of Machine Learning.</p><h3>Why So?</h3><p>To understand this properly, we need to step back and examine what Linear Regression really is, how it’s used in practice, and the intuition behind the mathematics.</p><h3>What Linear Regression Really Is?</h3><p>In theory, Linear Regression is a supervised machine learning algorithm used to predict a continuous numerical value (e.g., price, age, temperature) based on the linear relationship between a dependent variable and one or more independent variables.</p><p>That definition is correct.<br>But it doesn’t tell us much.</p><p>At its core, Linear Regression is not about prediction first.<br>It’s about <strong>describing a relationship</strong> between variables in the simplest possible way.</p><h3>Dependent and Independent Variables (Without the Jargon)</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9H_7Czx0Yt1CUGOo.png" /></figure><p>The variable we want to predict is called the <strong>dependent variable</strong>.<br>The variables we use to make that prediction are called <strong>independent variables</strong>.</p><p>In a house price prediction problem:</p><ul><li>The <strong>price</strong> of the house is the dependent variable</li><li>Features like <strong>size</strong>, <strong>number of bedrooms</strong>, and <strong>location</strong> are independent variables</li></ul><p>So far, nothing surprising.</p><p>But the real insight comes when we <strong>visualize the data</strong>.</p><h3>What the Model Is Actually Trying to Do</h3><p>If we plot the data points on a scatter plot, Linear Regression tries to draw a straight line that best represents the overall trend in the data.</p><p>Not a perfect line.<br>Not a line that passes through every point.</p><p>A line that is <strong>“wrong in the least possible way.”</strong></p><p>That immediately raises an important question:</p><blockquote><em>How do we decide which line is “best”?</em></blockquote><h3>Can We Fit Any Line We Want?</h3><p>No.</p><p>The line is not chosen by intuition or convenience.<br>It is chosen by <strong>error</strong>.</p><p>For every data point:</p><ul><li>The model makes a prediction</li><li>We measure how far that prediction is from the actual value</li></ul><p>This difference is called the <strong>error</strong>.</p><p>When we calculate this error for all data points and combine them, we get the <strong>total error</strong> of the model.</p><p>Linear Regression simply asks:</p><blockquote>Which line produces the smallest total error across all data points?</blockquote><p>That’s it.</p><p>No magic.<br>No intelligence.<br>Just optimization.</p><h3>Where Linear Regression Came From (And Why That Matters)</h3><p>Linear Regression is not a pure Machine Learning concept.</p><p>Even before Machine Learning was introduced, it was used in statistics.</p><p>It mattered a lot more in statistics than it sounds.</p><h3>Originally used to explain instead of predicting</h3><p>The main purpose of linear regression is to understand the relationship with data so that prediction accuracy can be improved.</p><p>Statisticians used to answer many questions related to the data by using Linear Regression.</p><p>Linear Regression makes sure that we think more about assumptions, uncertainty, and interpretability. The concepts that we ignore more in model machine learning.</p><h3>Designed for a Simpler World</h3><p>Linear Regression is a model that <strong>makes strong assumptions</strong> in exchange for clarity.</p><p>It was introduced as a simple, explainable, and mathematically tractable model for a small, noisy, and easy-to-collect dataset.</p><p>Because of this, the model has assumptions like relationships are stable, noise behaves nicely, and inputs don’t fight each other. Even though it’s a powerful model.</p><h3>Why Linear Regression Still Matters in Modern ML</h3><p>We are nowadays we are dealing with some powerful models, so the ultimate questions come to everyone’s mind.</p><blockquote>Why do we need Linear Regression? What does Linear Regression bring to the table that complex models can’t?</blockquote><p><em>Linear Regression is preferred because it’s an honest model.</em></p><p>A linear model:</p><ul><li>exposes data leakage quickly</li><li>highlights weak or irrelevant features</li><li>sets a performance baseline that keeps expectations realistic</li></ul><p>If a complex model performs only marginally better than a linear one, that’s a signal, not a success.</p><p>Linear Regression also remains valuable when:</p><ul><li>interpretability matters more than raw accuracy</li><li>The data is limited or noisy</li><li>decisions need to be explained, not just optimized</li></ul><p>In many real systems, a slightly less accurate but explainable model is the correct choice.</p><h3>The Math — Only What You Need to Think Clearly</h3><p>When we think about math behind algorithms, we are presented with big formulas and equations.</p><p>But in the case of Linear Regression its not required. Linear Regression makes predictions using a weighted sum of inputs, and those weights need to be adjusted to reduce the error.</p><p>Loss function matters the most in Linear Regression. For every prediction, the model measures how wrong it is. Those errors are combined into a single number that represents how well the model is performing.</p><p>Mathematically, it is represented as below.</p><h3>Linear Regression From Scratch</h3><p>Implementing Linear Regression from scratch helps you <strong>see what actually happens under the hood</strong>.</p><p>There are two main ways to do it:</p><h3>Closed-Form Solution</h3><p>The simplest way is the closed-form solution, which calculates the best-fit line directly using a formula from linear algebra.</p><pre>import numpy as np <br><br>class LinearRegressionClosed:<br>    <br>    def __init__(self):<br>        self.coef_ = None<br>        self.intercept_ = None<br>        <br>    def fit(self, X, y):<br>        X = np.asarray(X)<br>        y = np.asarray(y)<br>     <br>        Xb = np.c_[np.ones((X.shape[0], 1)), X]<br> <br>        A = np.linalg.inv(Xb.T @ Xb) @ Xb.T @ y<br>        self.intercept_ = A[0]<br>        self.coef_ = A[1:]<br>    def predict(self, X):<br>        X = np.asarray(X)<br>        return X @ self.coef_ + self.intercept_</pre><p>This method solves for the exact line that minimizes error.</p><p>It’s fast and elegant for small datasets, but it doesn’t scale well to very large data.</p><p>If you want to see this tested on real data, check my GitHub: l<a href="https://github.com/samithcsachi/Kaggle_Notebooks/blob/main/linear-regression-from-scratch-closed-form.ipynb">inear-regression from scratch closed form</a></p><h3>Gradient Descent</h3><p>The other approach is iterative using gradient descent to gradually find the best line.</p><pre>import pandas as pd<br><br>def gradient_descent(m_now, b_now, points, alpha):<br>    x = points[&#39;MedInc&#39;].values<br>    y = points[&#39;MedHouseVal&#39;].values<br>    n = len(x)<br>    <br>    y_pred = m_now * x + b_now<br>    m_gradient = -(2/n) * sum(x * (y - y_pred))<br>    b_gradient = -(2/n) * sum(y - y_pred)<br>    <br>    m = m_now - alpha * m_gradient<br>    b = b_now - alpha * b_gradient<br>    return m, b<br>m = 0<br>b = 0<br>alpha = 0.01  <br>epochs = 1000<br><br>for i in range(epochs):<br>    m, b = gradient_descent(m, b, df, alpha)<br>    if i % 100 == 0:<br>        print(f&quot;Epoch {i}: m={m:.4f}, b={b:.4f}&quot;)<br>print(&quot;\nFinal values:&quot;)<br>print(f&quot;m = {m}&quot;)<br>print(f&quot;b = {b}&quot;)</pre><p>Gradient descent teaches you the process of learning:</p><p>start with a guess → measure error → adjust gradually until it fits</p><p>This is exactly what all Machine Learning models do, even deep neural networks.</p><p>If you want to see this tested on real data, check my GitHub: <a href="https://github.com/samithcsachi/Kaggle_Notebooks/blob/main/linear-regression-from-scratch-in-python.ipynb">linear-regression-from-scratch-in-python</a></p><h3>That’s a Wrap!:</h3><p>Linear Regression is not important because it’s simple. It’s important because it teaches you how models think, how data behaves, and where assumptions quietly shape results.</p><p>In this newsletter, I’ve tried to explain Linear Regression in my own way, why it’s useful, why it’s interesting, and why understanding it well changes the way you approach Machine Learning.</p><p>Once you see that, Machine Learning stops feeling like a collection of algorithms<br>and starts feeling like a system you can reason about.</p><h3>Resources &amp; Further Reading</h3><p>If you want to explore Linear Regression in more depth, here are some helpful resources:</p><p><strong>Tutorials &amp; Courses:</strong></p><ul><li><a href="https://developers.google.com/machine-learning/crash-course/linear-regression">Google ML Crash Course — Linear Regression</a> — Great for intuition and interactive exercises.</li><li><a href="https://youtu.be/qxo8p8PtFeA?si=QCJc5KG8t2egb0Rn">YouTube: Linear Regression Explained</a> — Concise, visual explanation.</li><li><a href="https://en.wikipedia.org/wiki/Linear_regression">Linear Rregression — Wiki</a></li></ul><p><strong>Research &amp; Articles:</strong></p><ul><li>Fisher, R.A. (1922). <em>On the Interpretation of χ² from Contingency Tables, and the Calculation of P</em>. — Classic statistics paper introducing regression ideas.</li><li>Seber, G.A.F., &amp; Lee, A.J. (2012). <em>Linear Regression Analysis</em>. — Comprehensive reference for understanding assumptions and theory in regression.</li></ul><p><strong>Pro Tip:</strong></p><p>Even if you don’t read the papers in full, skimming the introduction and conclusions can give you insights into <strong>why Linear Regression works the way it does</strong> and why it’s still relevant in modern ML.</p><p>Cheers,</p><p><strong>Samith Chimminiyan</strong></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ee4de664f42" width="1" height="1" alt=""><hr><p><a href="https://pub.towardsai.net/the-hidden-assumptions-behind-linear-regression-7ee4de664f42">The Hidden Assumptions Behind Linear Regression</a> was originally published in <a href="https://pub.towardsai.net/">Towards AI</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Key Concepts from Unsupervised Learning, Recommenders, Reinforcement Learning]]></title>
            <link>https://samithc.medium.com/key-concepts-from-unsupervised-learning-recommenders-reinforcement-learning-f7c0f25a9d72?source=rss-03b792d4c412------2</link>
            <guid isPermaLink="false">https://medium.com/p/f7c0f25a9d72</guid>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Samith Chimminiyan]]></dc:creator>
            <pubDate>Mon, 19 Jan 2026 05:28:01 GMT</pubDate>
            <atom:updated>2026-01-19T05:28:01.085Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/853/0*Mjzx77rAHnZQu-oK" /></figure><p>I recently completed my Coursera certification in machine learning, focusing on advanced topics including unsupervised learning, recommender systems, and reinforcement learning. I’m sharing my structured notes to help fellow professionals understand these powerful techniques that are transforming various industries.</p><h3>Part 1: Unsupervised Learning</h3><h3>What is Clustering?</h3><p>Clustering algorithms look at data points and automatically identify related or similar points. This unsupervised learning technique groups data points into clusters where items within a cluster share more similarities than with those in other clusters. It’s a powerful way to discover hidden patterns and structures without predefined categories.</p><h3>K-means Algorithm</h3><p>The K-means algorithm begins by randomly selecting initial cluster centroids and then iteratively performs two steps:</p><ul><li>Assigning each data point to the nearest centroid</li><li>Updating the centroids to the mean of their assigned points</li></ul><p>This process continues until the assignments and centroids stabilize. If a cluster ends up with no assigned points, we can either eliminate or reinitialize that cluster. K-means can be useful even when clusters aren’t clearly separated, such as in sizing t-shirts based on customer measurements.</p><h3>Optimization Objective</h3><p>The distortion function is the average squared distance between each data point and its assigned cluster centroid. K-means aims to minimize this function.</p><h3>Choosing the Number of Clusters</h3><p>The “elbow method” helps automatically choose the optimal number of clusters by plotting the cost function against different values of K. However, choosing K based on downstream application needs is often more practical, such as balancing fit quality and manufacturing costs in t-shirt sizing.</p><h3>Anomaly Detection</h3><p>Anomaly detection identifies rare items, events, or observations that deviate significantly from the majority of the data. It’s used to spot unusual patterns, potential errors, or malicious activity in fields like finance, cybersecurity, and healthcare.</p><p>The method relies on density estimation to calculate the probability of a data point occurring; if the probability is below a threshold (epsilon), it’s considered an anomaly. Real-world applications include fraud detection, manufacturing quality control, and monitoring computer systems.</p><h3>Gaussian (Normal) Distribution</h3><p>A Gaussian or normal distribution is characterized by its bell-shaped curve, defined by two parameters: the mean (μ) and standard deviation (σ). This distribution is widely used due to its frequent occurrence in natural and social sciences.</p><h3>Anomaly Detection vs. Supervised Learning</h3><p>Unlike supervised learning, anomaly detection requires careful feature engineering to ensure meaningful results. It’s recommended to transform features to resemble Gaussian distributions using techniques like logarithmic or power transformations to improve model performance.</p><h3>Part 2: Recommender Systems</h3><h3>Making Recommendations</h3><p>The goal of a recommender system is to estimate how a user might rate an unseen item and recommend those likely to receive high ratings. Key variables include the number of users, number of items, whether a user has rated an item, and the actual rating.</p><h3>Using Per-Item Features</h3><p>By representing each item with feature vectors (e.g., romance and action scores for movies), the system builds a personalized linear regression model for each user. The cost function minimizes the squared error between predicted and actual ratings, incorporating regularization to prevent overfitting.</p><h3>Collaborative Filtering Algorithm</h3><p>Collaborative filtering learns item features directly from user ratings without predefined attributes. It begins by assuming known user parameters and demonstrates how to infer item features that best match observed ratings. The algorithm jointly optimizes user parameters and item features using gradient descent, enabling accurate predictions even for unrated items.</p><h3>Binary Labels: Favs, Likes, and Clicks</h3><p>Collaborative filtering can be adapted to work with binary labels — such as likes, clicks, or purchases — instead of traditional star ratings. To handle these binary outcomes, the algorithm transitions from a linear regression model to a logistic regression framework, using the logistic function to predict the probability of user engagement.</p><h3>Mean Normalization</h3><p>Mean normalization improves prediction accuracy, especially for new users who haven’t rated any items yet. By subtracting the average rating of each item from individual ratings, the algorithm avoids defaulting to zero-star predictions for unrated users.</p><h3>Finding Related Items</h3><p>Collaborative filtering allows us to find related items by calculating the squared distance between learned feature vectors, identifying those with the most similar profiles. This technique helps suggest alternatives that share underlying traits with the item a user is viewing.</p><h3>Collaborative Filtering vs. Content-Based Filtering</h3><p>Collaborative filtering relies on user-item interactions — recommending items based on ratings from users with similar preferences. In contrast, content-based filtering uses explicit features of users (like age, gender) and items (genre, release year) to find matches.</p><p>In summary, collaborative filtering leverages user ratings of different items, while content-based filtering uses features of users and items to find good matches.</p><h3>Deep Learning for Content-Based Filtering</h3><p>In deep learning approaches, a user network processes attributes like age and gender to produce a vector, while an item network transforms features like release year and cast into another vector. The predicted rating is computed as the dot product of these vectors, and the model is trained to minimize the error between predictions and actual outcomes.</p><h3>Recommending from a Large Catalog</h3><p>Large-scale recommender systems typically implement a two-step approach:</p><ul><li>Retrieval: Quickly generates a broad list of plausible candidates using precomputed similarities, user preferences, and popular items in the user’s region.</li><li>Ranking: Uses a neural network to score these candidates based on predicted user engagement, selecting the most promising ones.</li></ul><p>This two-step approach balances speed and accuracy, making real-time recommendations feasible even with enormous item inventories.</p><h3>Part 3: Reinforcement Learning</h3><h3>What is Reinforcement Learning?</h3><p>Reinforcement learning (RL) focuses on decision-making by autonomous agents — systems that can make decisions and act in response to their environment independent of direct human instruction.</p><h3>The Return in Reinforcement Learning</h3><p>The “return” represents the total reward an agent accumulates over time, adjusted by a discount factor. The discount factor (typically less than 1) prioritizes immediate rewards over distant ones, reflecting that sooner rewards are more valuable.</p><h3>Making Decisions: Policies in Reinforcement Learning</h3><p>A policy in reinforcement learning (denoted by π(s)) formalizes different decision strategies. The goal is to find an optimal policy that maximizes the expected return over time. While “policy” is the standard term, “controller” might be more intuitive in some contexts.</p><h3>Markov Decision Process (MDP)</h3><p>A Markov Decision Process is a mathematical framework for modeling sequential decision-making problems where outcomes are uncertain. The term “Markov” refers to the property that the future depends only on the current state and not on how you arrived there.</p><h3>State-Action Value Function</h3><p>The state-action value function Q(s,a) gives the expected return if you start in state s, take action a once, and then behave optimally afterward. The highest Q-value from a state indicates the best action to take, forming the basis for deriving an optimal policy.</p><h3>Continuous State Space Applications</h3><p>Unlike discrete models, continuous state spaces represent positions and movements using real-valued vectors. For instance, a car’s state might include coordinates, orientation, and velocity components, while a helicopter’s state expands to 12 variables including roll, pitch, yaw, and their respective angular velocities.</p><h3>Deep Q-Networks (DQN)</h3><p>In Deep Q-Networks, the agent’s state and actions are fed into a neural network trained to predict the expected return. The target value is computed using the Bellman equation: Q(s,a) = R(s) + γmaxQ(s′,a′), where s′ is the next state and a′ spans all possible actions.</p><p>To build the training set, the agent explores the environment, collecting experiences that are stored in a replay buffer. Batches are sampled to train the network, gradually improving its predictions.</p><h3>Algorithm Refinements</h3><ul><li>Improved Neural Network Architecture: Instead of separate inferences for each action, the refined model outputs Q-values for all actions simultaneously, reducing computational overhead.</li><li>ϵ-greedy Policy: The agent mostly chooses the action with the highest estimated reward but occasionally selects random actions to explore potentially better strategies.</li><li>Mini-batch and Soft Updates: Mini-batch training speeds up learning by using small, randomly selected subsets of data, while soft updates gradually blend new network parameters with old ones for improved stability.</li></ul><p>These advanced machine learning techniques are transforming how systems learn patterns, make recommendations, and take actions in complex environments. As I continue to apply these concepts in my professional work, I’m excited about the potential they hold for solving challenging real-world problems.</p><p>Notion Notes: — <a href="https://www.notion.so/Unsupervised-Learning-Recommenders-Reinforcement-Learning-2539ea19bfa7804a8754cda381aa7cd0?source=copy_link">https://www.notion.so/Unsupervised-Learning-Recommenders-Reinforcement-Learning-2539ea19bfa7804a8754cda381aa7cd0?source=copy_link</a></p><p>#MachineLearning #UnsupervisedLearning #RecommenderSystems #ReinforcementLearning #DataScience #AI #CourseraCertification #ProfessionalDevelopment</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f7c0f25a9d72" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>