<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Rajesh on Medium]]></title>
        <description><![CDATA[Stories by Rajesh on Medium]]></description>
        <link>https://medium.com/@sanjaykumar.kedu?source=rss-53969f4aa816------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*g3pRziwWSIXcvo4Jg7At0Q.jpeg</url>
            <title>Stories by Rajesh on Medium</title>
            <link>https://medium.com/@sanjaykumar.kedu?source=rss-53969f4aa816------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 23:19:58 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sanjaykumar.kedu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Understanding the Non-Divisible Subset Problem: A Deep Dive into the Modulo Approach]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/understanding-the-non-divisible-subset-problem-a-deep-dive-into-the-modulo-approach-d5db7be7d2e8?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/d5db7be7d2e8</guid>
            <category><![CDATA[leetcode-medium]]></category>
            <category><![CDATA[dsa-courses]]></category>
            <category><![CDATA[hackathons]]></category>
            <category><![CDATA[dsa-problem]]></category>
            <category><![CDATA[leetc]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Sun, 25 May 2025 18:27:57 GMT</pubDate>
            <atom:updated>2025-05-25T18:27:57.881Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><p>Algorithmic challenges involving divisibility often seem tricky at first glance. One classic example is the <strong>Non-Divisible Subset</strong> problem, which requires finding the largest subset of integers where the sum of <strong>any two elements is not divisible by a given integer </strong><strong>k</strong>.</p><p>In this post, we will:</p><ul><li>Understand the problem statement in detail.</li><li>Explore why brute force is not feasible.</li><li>Dive deep into the modulo approach to solve this problem efficiently.</li><li>Walk through a detailed example.</li><li>Provide a clean and efficient Java implementation.</li><li>Discuss broader applications of this approach.</li></ul><h3>Problem Statement</h3><p>You are given:</p><ul><li>A set S of <strong>distinct integers</strong>.</li><li>An integer k.</li></ul><p>You need to find the <strong>size of the largest subset</strong> S&#39; of S such that <strong>for any two numbers </strong><strong>a and </strong><strong>b in </strong><strong>S&#39;, </strong><strong>(a + b) is not divisible by </strong><strong>k</strong>.</p><h3>Example</h3><p>Suppose:</p><ul><li>S = {19, 10, 12, 24, 25, 22}</li><li>k = 4</li></ul><p>Possible subsets where the sum of any two numbers is <strong>not</strong> divisible by 4 are:</p><ul><li>{10, 12, 25}</li><li>{19, 22, 25}</li></ul><p>The largest such subset size is 3.</p><h3>Why Not Brute Force?</h3><p>You might think of:</p><p>Generating all possible subsets.</p><p>Checking for each subset if the sum of any two elements is divisible by k.</p><p>Selecting the largest valid subset.</p><p>However:</p><ul><li>Number of subsets = 2^n (exponential growth).</li><li>Checking all pairs inside each subset further increases complexity.</li><li>For even moderately large n, this approach becomes infeasible.</li></ul><h3>The Modulo Approach: The Key Insight</h3><h3>The Core Idea: Group by Remainders</h3><p>Every integer can be classified by its remainder when divided by k. For each number num in the set, compute:</p><blockquote>remainder = num % k</blockquote><p>Numbers with the same remainder share a special property related to sums modulo k.</p><h3>Important Observations:</h3><ul><li>Two numbers whose remainders sum up to k will produce a sum divisible by k.</li><li>For example, if one number is r, and another is k - r, then:</li></ul><pre>(r + (k - r)) % k = k % k = 0</pre><ul><li>Numbers with remainder 0 are divisible by k. Any two such numbers sum to a multiple of k.</li><li>If k is even, numbers with remainder k/2 also sum with themselves to a multiple of k.</li></ul><h3>How to Build the Largest Non-Divisible Subset?</h3><ol><li><strong>Group numbers by their remainders modulo </strong><strong>k.</strong></li><li>For each remainder i, count how many numbers have remainder i.</li><li><strong>Include at most one number from the </strong><strong>0 remainder group.</strong></li><li>Since any two numbers both divisible by k sum to a multiple of k, picking more than one would break the condition.</li><li><strong>For each pair of complementary remainders </strong><strong>(i, k - i), choose the group with more elements.</strong></li><li>We cannot pick elements from both remainder groups because pairs from both would sum to a multiple of k.</li><li><strong>If </strong><strong>k is even, include at most one element from the </strong><strong>k/2 remainder group.</strong></li></ol><h3>Why do we pick the larger group from (i, k - i)?</h3><p>Including elements from both groups would inevitably produce pairs summing to k. So to maximize size, pick the group with more elements.</p><p><strong>Step-by-Step Example</strong></p><p>Let’s use:</p><ul><li>S = {19, 10, 12, 24, 25, 22}</li><li>k = 4</li></ul><h3>Step 1: Calculate remainders</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/810/1*fKsCQlSbptRZXjkcrd4uCA.png" /></figure><h3>Step 2: Count frequency of each remainder</h3><ul><li>Remainder 0 → 2 numbers (12, 24)</li><li>Remainder 1 → 1 number (25)</li><li>Remainder 2 → 2 numbers (10, 22)</li><li>Remainder 3 → 1 number (19)</li></ul><h3>Step 3: Start building subset size</h3><ul><li>Include <strong>at most one</strong> from remainder 0: count = 1</li><li>For remainder pairs (1, 3):</li><li>Count for remainder 1: 1</li><li>Count for remainder 3: 1</li><li>Pick max → 1 → count = 1 + 1 = 2</li><li>For remainder pair (2, 2) (since k is even, special case):</li><li>Include <strong>at most one</strong> → count = 2 + 1 = 3</li></ul><h3>Final subset size: 3</h3><h3>Code Implementation in Java</h3><pre>import java.util.List;<br>public class NonDivisibleSubset {<br>    public static int nonDivisibleSubset(int k, List&lt;Integer&gt; s) {<br>        int[] modCounts = new int[k];<br>        <br>        // Count frequencies of each remainder<br>        for (int num : s) {<br>            modCounts[num % k]++;<br>        }<br>        <br>        int count = 0;<br>        <br>        // Include at most one element from remainder 0 group<br>        if (modCounts[0] &gt; 0) {<br>            count++;<br>        }<br>        <br>        // Loop from 1 to k/2 to handle pairs<br>        for (int i = 1; i &lt;= k / 2; i++) {<br>            if (i == k - i) {<br>                // Special case for even k, include only one element<br>                if (modCounts[i] &gt; 0) {<br>                    count++;<br>                }<br>            } else {<br>                // Pick the larger group between remainder i and k-i<br>                count += Math.max(modCounts[i], modCounts[k - i]);<br>            }<br>        }<br>        <br>        return count;<br>    }<br>}</pre><h3>When Can You Use This Pattern?</h3><p>The modulo grouping technique is widely useful in:</p><ul><li><strong>Sum divisibility problems:</strong> Where sums or pairs need to satisfy divisibility conditions.</li><li><strong>Subarray sum problems:</strong> Using prefix sums modulo k to find subarrays divisible by k.</li><li><strong>Number theory problems:</strong> Including Chinese Remainder Theorem applications.</li><li><strong>Cyclic problems:</strong> Clock arithmetic, periodic scheduling.</li><li><strong>Partitioning problems:</strong> Grouping elements based on remainder classes.</li></ul><h3>Conclusion</h3><p>The Non-Divisible Subset problem teaches a valuable lesson:</p><blockquote><strong><em>Transform complex pairwise constraints into simple frequency counts using modulo arithmetic!</em></strong></blockquote><p>This approach:</p><ul><li>Avoids costly brute force.</li><li>Works efficiently even for large input sizes.</li><li>Demonstrates the power of mathematical insight in algorithm design.</li></ul><p>If you want to practice, try applying this technique to other problems involving divisibility or sums!</p><h3>Happy coding! 🚀</h3><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d5db7be7d2e8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Data and Data Flow in System Design: A Beginner-Friendly Guide]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/data-and-data-flow-in-system-design-a-beginner-friendly-guide-c51da53d8e53?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/c51da53d8e53</guid>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Mon, 20 Jan 2025 18:04:15 GMT</pubDate>
            <atom:updated>2025-01-20T18:04:15.996Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VInbRiXFtUwObfua" /><figcaption>Photo by <a href="https://unsplash.com/@benchaccounting?utm_source=medium&amp;utm_medium=referral">Bench Accounting</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>When we interact with any digital system, whether it’s a mobile app, a website, or a software application, one thing underpins it all: <strong>data</strong>. Data is the beating heart of computing systems, and understanding how it flows and behaves is key to designing efficient and reliable systems. In this blog, we’ll explore data, its journey through a system, and the factors that influence how systems manage it. Let’s dive in!</p><h3>What is Data? Why is it Important?</h3><p>At its core, data is any piece of information stored, processed, and transmitted by a computing system. It could be a username you type into a login form, an image you upload, or the metrics collected by an IoT device. Data serves as the foundation for:</p><ul><li><strong>User interactions</strong> (e.g., signing in, creating a post, or searching for a product)</li><li><strong>Business processes</strong> (e.g., tracking inventory, processing payments, or generating reports)</li><li><strong>Decision-making</strong> (e.g., using customer data for personalized recommendations)</li></ul><p>Operations like <strong>create, read, update, delete, and write (CRUD)</strong> are fundamental to managing data in any system.</p><h3>Where Does Data Reside in a System?</h3><p>In a computing system, data travels across several layers. Each layer has its own purpose and role in handling data:</p><p><strong>User or Business Layer:</strong> This is where data originates.</p><p>For example:</p><ul><li>A user submits a form on a website (e.g., registering for an account).</li><li>A business process generates data (e.g., sales reports or analytics).</li></ul><p><strong>Application Layer:</strong> Here, data is processed and interpreted. It’s where the logic of the system resides, transforming raw input into meaningful outcomes.</p><p>For instance:</p><ul><li>Validating user credentials during login.</li><li>Calculating the total price in a shopping cart.</li></ul><p><strong>Data Centers or Data Stores:</strong> Data is stored and retrieved from databases.</p><p>For example:</p><ul><li>User profiles might be stored in a relational database.</li><li>Product search data might be indexed in a specialized search engine.</li></ul><p><strong>Network Layer:</strong> Data flows between different system components through networks. This involves protocols like HTTP or TCP/IP, ensuring data packets are sent and received correctly.</p><p><strong>Hardware (Storage and Memory):</strong> At the lowest level, data is stored in binary (0’s and 1’s) on physical hardware such as hard drives, SSDs, or memory chips.</p><h3>Factors That Influence Data in a System</h3><p>When designing a system, we need to consider several factors that impact how data is handled:</p><p><strong>Security:</strong></p><ul><li>Data must be protected from unauthorized access and breaches.</li><li>Techniques include encryption, access control, and secure authentication.</li></ul><p>Example: A banking app encrypting transaction data to prevent fraud.</p><p><strong>Consumption:</strong></p><ul><li>Different users and applications consume data in various ways.</li><li>Systems must support efficient data access, whether it’s a real-time application (like messaging apps) or an analytical tool (like business dashboards).</li></ul><p><strong>Types of Data:</strong></p><ul><li><strong>Structured Data:</strong> Organized into rows and columns (e.g., SQL databases).</li><li><strong>Unstructured Data:</strong> Text, images, videos, etc. (e.g., NoSQL databases).</li></ul><p><strong>Volume of Data:</strong></p><ul><li>Systems must scale to handle the growing volume of data, which can range from a few megabytes to petabytes.</li></ul><p>Example: Social media platforms store enormous amounts of user-generated content.</p><h3>Choosing the Right Database for Your System</h3><p>No one-size-fits-all database exists. The choice of database depends on the unique requirements of the system and its use case. Let’s explore the main types:</p><h4>1. SQL Databases</h4><p><strong>Description:</strong></p><ul><li>Store structured data in tables with predefined schemas.</li><li>Support powerful querying using SQL (Structured Query Language).</li></ul><p><strong>Pros:</strong></p><ul><li>Strong consistency and reliability.</li><li>Suitable for complex transactions.</li></ul><p><strong>Cons:</strong></p><ul><li>May struggle with horizontal scaling.</li></ul><p><strong>Example Use Case:</strong></p><ul><li>Banking systems where data consistency is critical.</li><li>Example database: MySQL, PostgreSQL.</li></ul><h4>2. NoSQL Databases</h4><p><strong>Description:</strong></p><ul><li>Handle unstructured or semi-structured data.</li><li>Flexible schemas and optimized for horizontal scaling.</li></ul><p><strong>Types:</strong></p><ul><li><strong>Key-Value Stores:</strong> Redis, DynamoDB (e.g., caching, session management).</li><li><strong>Document Stores:</strong> MongoDB (e.g., content management systems).</li><li><strong>Graph Databases:</strong> Neo4j (e.g., social networks, recommendation engines).</li></ul><p><strong>Pros:</strong></p><ul><li>High scalability and performance for large datasets.</li><li>Better suited for unstructured data.</li></ul><p><strong>Cons:</strong></p><ul><li>Limited support for complex transactions.</li></ul><p><strong>Example Use Case:</strong></p><ul><li>A social media platform storing user profiles and posts.</li></ul><h4>3. Columnar Databases</h4><p><strong>Description:</strong></p><ul><li>Store data by columns instead of rows.</li><li>Optimized for analytical queries and aggregations.</li></ul><p><strong>Pros:</strong></p><ul><li>Fast query performance for analytics.</li></ul><p><strong>Cons:</strong></p><ul><li>Not ideal for transactional workloads.</li></ul><p><strong>Example Use Case:</strong></p><ul><li>Business intelligence tools.</li><li>Example database: Apache Cassandra, Google BigQuery.</li></ul><h4>4. Search Databases</h4><p><strong>Description:</strong></p><ul><li>Specialized for indexing and searching large datasets efficiently.</li></ul><p><strong>Pros:</strong></p><ul><li>Extremely fast search performance.</li><li>Powerful full-text search capabilities.</li></ul><p><strong>Cons:</strong></p><ul><li>Not designed for general-purpose data storage.</li></ul><p><strong>Example Use Case:</strong></p><ul><li>E-commerce platforms with product search.</li><li>Example database: Elasticsearch.</li></ul><h3>Real-World Examples of Data Flow</h3><p>Let’s look at how data flows through a real-world system:</p><h4>1. Food Delivery App (e.g., Swiggy, Uber Eats):</h4><ul><li><strong>User Layer:</strong> A customer places an order through the app.</li><li><strong>Application Layer:</strong> The app processes the order, calculates delivery time, and assigns a delivery partner.</li><li><strong>Data Store:</strong> User profiles, restaurant menus, and order histories are stored in databases.</li><li><strong>Network Layer:</strong> Data about the order is transmitted between the customer, restaurant, and delivery partner.</li><li><strong>Hardware:</strong> Data is stored as binary at the hardware level.</li></ul><h4>2. Streaming Platform (e.g., Netflix):</h4><ul><li><strong>User Layer:</strong> A user selects a movie or series.</li><li><strong>Application Layer:</strong> The system fetches the relevant video file and user preferences.</li><li><strong>Data Store:</strong> Videos are stored in distributed file systems or object storage.</li><li><strong>Network Layer:</strong> Data is streamed over the internet to the user’s device.</li><li><strong>Hardware:</strong> Video files are stored as binary on servers.</li></ul><h3>Conclusion</h3><p>Data is the core entity of any system, flowing seamlessly across layers to deliver value to users. By understanding data’s journey, its storage, and the factors influencing it, you can design systems that are secure, scalable, and efficient. Remember, every system has unique needs, so choose the right tools and databases based on the problem you’re solving.</p><p>Whether you’re building the next social network, an e-commerce platform, or an analytics tool, keeping data at the center of your design will help you create robust systems. Happy designing!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c51da53d8e53" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Proxies: Your Guide to Forward and Backward Proxies with Real-World Examples]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/understanding-proxies-your-guide-to-forward-and-backward-proxies-with-real-world-examples-17015da5ed28?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/17015da5ed28</guid>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Sun, 19 Jan 2025 18:13:30 GMT</pubDate>
            <atom:updated>2025-01-19T18:13:30.702Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*G0rKydmjaPjoFUgO" /><figcaption>Photo by <a href="https://unsplash.com/@writecodenow?utm_source=medium&amp;utm_medium=referral">Boitumelo</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In the world of the internet, there’s a hidden player that you often interact with, but you might not even know it exists — <strong>the proxy</strong>. While the name may sound a bit technical, proxies are essential in enhancing our internet experiences. Whether you’re browsing your favorite website, streaming content, or even securing your online privacy, proxies play a significant role.</p><p>In this blog, we’ll break down what proxies are, explain the different types like <strong>forward proxies</strong> and <strong>backward proxies</strong>, and dive into some practical use cases. Don’t worry, we’ll keep it simple, engaging, and full of real-world examples!</p><h3>What is a Proxy?</h3><p>Imagine you’re in a crowded coffee shop, trying to talk to someone sitting across the room. It’s loud, and the conversation is difficult. But then, someone steps in as a <strong>middleman</strong>, passing notes back and forth between you and the person you’re trying to talk to. This middleman makes sure the message gets delivered correctly and quickly, keeping the noise out.</p><p>In the internet world, a <strong>proxy server</strong> acts as that middleman. It’s a computer or system that sits between your device (like your laptop or phone) and the internet, forwarding requests for information on your behalf.</p><h3>How Does a Proxy Work?</h3><p>When you want to access a website, your device sends a request to the proxy server instead of directly reaching out to the website. The proxy server then makes the request to the website for you and passes the information back to you.</p><p>Let’s break it down with an example:</p><ol><li><strong>You want to access a website.</strong></li><li><strong>Your device</strong> sends the request to a proxy server.</li><li><strong>The proxy server</strong> fetches the website data on your behalf.</li><li><strong>The proxy server</strong> sends the website data back to your device.</li></ol><p>It’s like having a personal assistant handle the grunt work of fetching information for you, while you focus on your work!</p><h3>Types of Proxies</h3><p>There are several types of proxies, each serving different purposes. We’ll focus on two common types: <strong>forward proxies</strong> and <strong>backward proxies</strong>.</p><h3>1. Forward Proxy: Your Internet Gatekeeper</h3><p>A <strong>forward proxy</strong> is like a <strong>traffic controller</strong> for your internet requests. It sits between your device and the web, forwarding your requests to the appropriate server. In simple terms, it makes requests on your behalf and hides your identity from the websites you visit.</p><h4>How Does a Forward Proxy Work?</h4><p>Let’s consider an example of using a forward proxy at a corporate office:</p><ul><li>Your company has set up a forward proxy to manage internet access. When you try to visit a website, your request goes to the <strong>company’s proxy server</strong> first.</li><li>The proxy then forwards your request to the target website.</li><li>The website sends the information back to the proxy, and then the proxy forwards the data to you.</li></ul><h4>Use Cases of Forward Proxy</h4><p><strong>Content Filtering</strong>: Many schools and workplaces use forward proxies to block access to certain websites like social media, gaming sites, or any content that’s inappropriate or distracting.</p><p><strong>Example</strong>: If you try to access Facebook at work, the proxy server will block that request, ensuring productivity is maintained.</p><p><strong>Bypassing Geographical Restrictions</strong>: Forward proxies are commonly used to access content restricted to certain regions.</p><p><strong>Example</strong>: If you’re in India and want to access a U.S.-based website like Netflix, a forward proxy can make it look like you’re in the U.S., allowing you to access restricted content.</p><p><strong>Improving Performance</strong>: Forward proxies can cache content, so if you visit the same website often, the proxy may store a copy of the page. This reduces load times and saves bandwidth.</p><h4>Drawbacks of Forward Proxy</h4><ul><li><strong>Performance Issues</strong>: If the proxy server is not well-maintained or overloaded, it can slow down your internet connection.</li><li><strong>Privacy Concerns</strong>: Since all your requests go through the proxy, the proxy server can see your online activity.</li></ul><h3>2. Backward Proxy (Reverse Proxy): Protecting the Servers</h3><p>Now let’s shift to <strong>reverse proxies</strong> or <strong>backward proxies</strong>. Unlike forward proxies, which focus on protecting users, a reverse proxy is like a <strong>bouncer</strong> at a club, standing between the user and the server, protecting the backend.</p><h4>How Does a Reverse Proxy Work?</h4><p>Imagine you’re visiting a popular website, and thousands of users are making requests at the same time. Instead of directly connecting to the website’s server, your request goes to a reverse proxy first.</p><p>The reverse proxy then decides which of the backend servers should handle the request based on factors like load, security, and speed. It forwards the request to the right server and returns the data to you.</p><h4>Use Cases of Reverse Proxy</h4><p><strong>Load Balancing</strong>: When a website has multiple servers, a reverse proxy can distribute the incoming traffic evenly across these servers, ensuring no single server is overwhelmed.</p><p><strong>Example</strong>: Think of a busy e-commerce website during Black Friday sales. A reverse proxy will spread the user requests across different servers, preventing any server from crashing.</p><p><strong>Security</strong>: A reverse proxy can protect backend servers from direct exposure to the internet. It acts as a shield, blocking malicious requests and ensuring that only legitimate traffic gets through.</p><p><strong>Example</strong>: If you’re running a banking website, a reverse proxy can help protect sensitive information by filtering out harmful requests before they reach your server.</p><p><strong>SSL Termination</strong>: A reverse proxy can handle SSL encryption and decryption for your servers, ensuring that secure connections (HTTPS) are maintained without overloading backend servers.</p><h4>Drawbacks of Reverse Proxy</h4><ul><li><strong>Single Point of Failure</strong>: If the reverse proxy server goes down, it can affect the entire system. This is why it’s essential to have redundant proxy servers.</li><li><strong>Complexity</strong>: Setting up and maintaining reverse proxies can be complex, especially for large-scale applications.</li></ul><h3>Real-World Examples of Proxies</h3><p>Now, let’s look at a couple of real-world examples where proxies are actively used:</p><p><strong>Online Privacy and Anonymity</strong>: When you browse the web using a proxy, websites see the proxy’s IP address instead of yours. This can help you <strong>anonymize</strong> your browsing activity, ensuring that your personal information and location remain private.</p><p><strong>Example</strong>: If you’re using a public Wi-Fi network and don’t want others to know your browsing history, you might use a forward proxy to hide your true IP address.</p><p><strong>Streaming Services (e.g., Netflix, Hulu)</strong>: If you’ve ever used a VPN or proxy to access content from a different country, you’ve likely used a <strong>forward proxy</strong>. This helps you bypass regional restrictions and access a wider range of shows and movies.</p><p><strong>Example</strong>: You’re in India, but you want to watch Netflix U.S. content. Using a proxy, you can make it appear like you’re browsing from the U.S.</p><p><strong>Web Scraping</strong>: If you’re gathering information from websites (e.g., competitor analysis), proxies help by rotating IP addresses to avoid getting blocked. Web scrapers use proxies to fetch data from various sites without revealing their real identity.</p><p><strong>Example</strong>: A company might use a reverse proxy to protect the data they’re collecting while scraping, ensuring that malicious actors don’t access the same data.</p><h3>Conclusion:</h3><p>Proxies are versatile tools that can serve a wide variety of purposes — whether you’re protecting your identity, improving website performance, or managing complex server traffic. From forward proxies to reverse proxies, understanding how they work helps you use them more effectively.</p><p>In the end, proxies might be invisible to the average user, but their influence is far-reaching. So the next time you access a website, remember: a proxy could be silently working behind the scenes to ensure your data flows smoothly and securely.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=17015da5ed28" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding the Client-Server Model: A Beginner’s Guide to System Design]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/understanding-the-client-server-model-a-beginners-guide-to-system-design-e212d4a97e07?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/e212d4a97e07</guid>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[beginners-guide]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[clien]]></category>
            <category><![CDATA[servers]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Sat, 18 Jan 2025 18:17:08 GMT</pubDate>
            <atom:updated>2025-01-18T18:17:08.065Z</atom:updated>
            <content:encoded><![CDATA[<p>Imagine you walk into a coffee shop and order a cappuccino. You’re the client, making a request, and the barista is the server, fulfilling that request. This interaction, where one entity (the client) requests a service and another entity (the server) provides it, mirrors the fundamental concept of the <strong>client-server model</strong> in system design.</p><p>The client-server model is a cornerstone of how most computer networks and applications work today, including the websites you browse, the apps you use, and even the games you play online. Let’s break it down step by step, using real-world analogies and examples to make it easy to understand.</p><h3>What is the Client-Server Model?</h3><p>In simple terms, the client-server model is a way to structure communication between two entities:</p><ol><li><strong>Client</strong>: This is the entity that makes a request for a resource or service. Examples include your web browser, a mobile app, or even your computer running a specific program.</li><li><strong>Server</strong>: This is the entity that processes the request and provides the requested resource or service. Examples include web servers, database servers, or application servers.</li></ol><h3>Key Characteristics of the Client-Server Model:</h3><ul><li><strong>Centralization</strong>: The server acts as a central hub that manages resources or services.</li><li><strong>Request-Response Cycle</strong>: The client sends a request, and the server responds.</li><li><strong>Separation of Concerns</strong>: Clients focus on displaying information or interacting with users, while servers handle data processing and storage.</li></ul><h3>How Does It Work?</h3><p>Let’s look at a common example: <strong>browsing the web.</strong></p><ol><li><strong>Client Request</strong>: You type www.example.com into your browser. Your browser acts as the client.</li><li><strong>Server Response</strong>: The browser sends a request to a web server, asking for the website’s content.</li><li><strong>Processing and Delivery</strong>: The server processes the request, retrieves the necessary HTML, CSS, and JavaScript files, and sends them back to your browser.</li><li><strong>Rendering</strong>: Your browser renders the files and displays the website on your screen.</li></ol><p>This interaction happens so quickly that you hardly notice the complexity behind it!</p><h3>Real-World Examples of the Client-Server Model</h3><h4>Example 1: Online Shopping (e.g., Amazon)</h4><ul><li><strong>Client</strong>: The Amazon app or website on your device.</li><li><strong>Server</strong>: Amazon’s backend systems, which include servers that manage product catalogs, user accounts, and order processing.</li><li><strong>How it works</strong>: When you search for a product, the client (your app or browser) sends a request to Amazon’s server. The server processes the query, fetches the relevant products, and sends the data back to your device for display.</li></ul><h4>Example 2: Food Delivery Apps (e.g., Zomato, Uber Eats)</h4><ul><li><strong>Client</strong>: The food delivery app on your phone.</li><li><strong>Server</strong>: The servers managing restaurant menus, user orders, and delivery logistics.</li><li><strong>How it works</strong>: When you place an order, the app sends your request to the server. The server communicates with restaurants and delivery personnel to process your order, then sends updates back to the app.</li></ul><h4>Example 3: Multiplayer Online Games (e.g., Fortnite, PUBG)</h4><ul><li><strong>Client</strong>: The game software running on your console or PC.</li><li><strong>Server</strong>: The game server that manages player interactions, game state, and real-time updates.</li><li><strong>How it works</strong>: When you make a move, the client sends your actions to the server. The server updates the game’s state and informs other players’ clients of your actions.</li></ul><h3>Why Use the Client-Server Model?</h3><p>The client-server model offers several advantages:</p><ol><li><strong>Centralized Control</strong>: Servers act as a single source of truth, ensuring consistency.</li><li><strong>Scalability</strong>: Servers can handle multiple clients, and additional servers can be added as needed.</li><li><strong>Security</strong>: Sensitive data can be stored and managed centrally on the server.</li><li><strong>Resource Management</strong>: Clients can be lightweight, relying on servers for heavy lifting (e.g., computation or data storage).</li></ol><h3>Challenges of the Client-Server Model</h3><p>Despite its advantages, the client-server model has its challenges:</p><ol><li><strong>Single Point of Failure</strong>: If the server goes down, clients can’t access the service.</li><li><strong>Scalability Issues</strong>: A sudden surge in client requests (e.g., during a Black Friday sale) can overwhelm the server.</li><li><strong>Network Dependency</strong>: The model relies on a stable network connection.</li><li><strong>Latency</strong>: The request-response cycle can introduce delays, especially for geographically distant clients.</li></ol><h3>Enhancements to the Client-Server Model</h3><p>To overcome these challenges, modern systems often incorporate enhancements:</p><ol><li><strong>Load Balancing</strong>: Distributes client requests across multiple servers to prevent overloading.</li><li><strong>Caching</strong>: Stores frequently accessed data closer to clients (e.g., in Content Delivery Networks or local memory).</li><li><strong>Replication</strong>: Creates multiple copies of servers or databases to improve availability and reliability.</li><li><strong>Cloud Computing</strong>: Hosts servers in the cloud to scale resources dynamically based on demand.</li></ol><h3>Visualizing the Client-Server Model</h3><p>Here’s a simplified analogy:</p><ul><li><strong>Client</strong>: A customer dining at a restaurant, placing an order.</li><li><strong>Server</strong>: The kitchen staff preparing the meal.</li><li><strong>Request</strong>: The customer’s order.</li><li><strong>Response</strong>: The meal served to the customer.</li></ul><p>The restaurant ensures that multiple customers can dine simultaneously by having enough kitchen staff (servers) and efficient processes (load balancing and caching).</p><h3>Conclusion</h3><p>The client-server model is everywhere in our digital lives, from streaming your favorite Netflix show to checking your bank balance online. By understanding how it works, you’ll gain insights into how modern applications and systems are designed and operated.</p><p>Remember, the key to mastering system design is to start small, understand the basics, and then explore advanced concepts like distributed systems and microservices. For now, appreciate the elegance of the client-server model and its role in powering the digital world around us.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e212d4a97e07" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Introduction to System Design: Building Blocks of Effective Systems]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/introduction-to-system-design-building-blocks-of-effective-systems-5bf0ac96d8c0?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/5bf0ac96d8c0</guid>
            <category><![CDATA[beginner]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Fri, 17 Jan 2025 18:22:36 GMT</pubDate>
            <atom:updated>2025-01-17T18:22:36.085Z</atom:updated>
            <content:encoded><![CDATA[<p>System design is an integral part of creating software and hardware solutions that solve real-world problems. Whether you’re developing a small application or architecting a large-scale distributed system, understanding the fundamentals of system design is essential.</p><p>Let’s dive into the basics: what a system is, its key elements, and how these concepts manifest in real-life examples.</p><h3>What is a System?</h3><p>In its simplest terms, a system is a collection of interconnected components that work together to achieve a specific goal. Systems can be found everywhere — in nature, technology, and even in our daily lives. The core idea is that a system integrates different parts to function as a cohesive whole.</p><h3>Defining a System: Key Characteristics</h3><p>A system can be defined and understood through the following elements:</p><p><strong>Users</strong><br> Every system is designed to serve specific users. These users can be individuals, businesses, or even other systems. For instance, the users of a food delivery app include customers, delivery agents, and restaurant owners.</p><p><strong>User Requirements</strong><br> Systems exist to fulfill specific needs or requirements. For example:</p><ul><li>A messaging app needs to provide instant communication.</li><li>An e-commerce platform must enable secure transactions and fast delivery.</li></ul><p><strong>Components</strong><br> A system comprises multiple components working together. These components can be broadly categorized into:</p><ul><li><strong>Tangible Components:</strong> Physical elements such as servers, devices, and networking hardware.</li><li><strong>Logical Components:</strong> Software elements like databases, APIs, and algorithms.</li></ul><h3>Real-Life Example: Food Delivery System</h3><p>To better understand these concepts, let’s consider the example of a food delivery system like Swiggy or Uber Eats.</p><p><strong>Users:</strong><br> Customers (to place orders), delivery agents (to deliver food), and restaurants (to prepare food).</p><p><strong>User Requirements:</strong></p><ul><li>Customers: Easy browsing of menus, secure payment options, and real-time order tracking.</li><li>Delivery agents: Efficient route navigation and order management.</li><li>Restaurants: Seamless order notifications and customer feedback.</li></ul><p><strong>Components:</strong></p><ul><li><strong>Tangible Components:</strong> Mobile phones (users interact via apps), delivery vehicles, and cloud servers.</li><li><strong>Logical Components:</strong> The app itself, the recommendation algorithm for food suggestions, and the payment gateway.</li></ul><h3>System Components: Tangible vs. Logical</h3><p>Every system is composed of two types of components: tangible and logical. Let’s explore these further with real-world examples.</p><p><strong>Tangible Components</strong></p><p>These are the physical, hardware-based parts of the system. They include:</p><ul><li><strong>Servers:</strong> Used to store and manage the application’s data. For example, Amazon Web Services (AWS) provides server infrastructure for scalable systems.</li><li><strong>Networking Devices:</strong> Routers and switches that enable communication between different parts of the system.</li><li><strong>User Devices:</strong> Smartphones, laptops, or any hardware used to access the system.</li></ul><p><strong>Logical Components</strong></p><p>These are the software-based or abstract parts of the system. Examples include:</p><ul><li><strong>Database Management Systems (DBMS):</strong> MySQL or MongoDB, which store and manage data.</li><li><strong>Application Programming Interfaces (APIs):</strong> Used to allow different parts of the system to communicate. For instance, a weather app might use a public API to fetch real-time weather updates.</li><li><strong>Business Logic:</strong> Algorithms that define how the system processes data to fulfill user requirements. For example, a recommendation algorithm in Netflix suggests content based on viewing history.</li></ul><h3>Why Understanding System Design Matters</h3><p>System design is more than just understanding components — it’s about bringing them together efficiently. A well-designed system can scale to serve millions of users, handle complex requirements, and remain reliable under high demand.</p><p>By grasping the basic principles of system design, you’ll be equipped to build solutions that are not only functional but also sustainable and user-friendly.</p><h3>Conclusion</h3><p>System design forms the backbone of every successful application or service. By understanding what a system is, identifying its users and requirements, and distinguishing its tangible and logical components, you’re one step closer to designing robust systems that make a real-world impact.</p><p>Whether you’re building a small web app or architecting a global service, the principles of system design remain universal — and mastering them will set you apart in your tech journey.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5bf0ac96d8c0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Journey Through My First Big Interview: A Lesson in Growth]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/a-journey-through-my-first-big-interview-a-lesson-in-growth-a855c940d225?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/a855c940d225</guid>
            <category><![CDATA[success]]></category>
            <category><![CDATA[work]]></category>
            <category><![CDATA[experience]]></category>
            <category><![CDATA[interview]]></category>
            <category><![CDATA[intern]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Wed, 25 Sep 2024 11:38:32 GMT</pubDate>
            <atom:updated>2024-09-25T11:38:32.527Z</atom:updated>
            <content:encoded><![CDATA[<p>It’s often said that the journey is just as important as the destination, and nothing could be more true of my experience with my first interview at a renowned service-based company. I walked in with excitement, optimism, and a firm belief in my skills. While the interview didn’t go exactly as I had imagined, it was a blend of successes, challenges, and valuable lessons that will stay with me forever.</p><p>This blog is a reflection of that day — a mixed bag of experiences that I’ll carry forward to shape my future interviews and professional journey.</p><h3>The Beginning: Confidence and Clarity</h3><p>Like any interview, it started with the usual “Tell me about yourself.” I was prepared for this. I introduced myself, highlighting my educational background and my experience as a blog writer on Medium, particularly for Donald DevOps. I talked about my 8+ months as a Machine Learning Engineer (MLE) and a major project called <strong>Curious Bees</strong>, where I had the opportunity to implement exciting AI features. I felt good as I shared the details, confident in my story and experiences.</p><p>One of the most meaningful parts of the interview was when I explained why I switched to data science. I shared how my love for mathematics drove me to apply theoretical solutions to real-world problems, but I needed a tool to do that — hence, my dive into data science.</p><p>I backed up this passion with an example of a project where my team and I automated the evaluation of student introduction videos for corporate ambassador positions. We isolated audio, transcribed it, and analyzed fluency, grammar, sentiment, professionalism, and relevance — all to create a scorecard for over 150 students. To our surprise, our model’s scores aligned closely with the human evaluators’ assessments, and I proudly shared this achievement with the interviewer.</p><h3>The Shift: Into the World of DSA</h3><p>From there, the interview shifted to data structures and algorithms (DSA) — a territory I was comfortable navigating. We discussed various array-related problems, including merging two unsorted arrays into a sorted one. I presented both brute force and optimal solutions, hoping to showcase my problem-solving abilities.</p><p>However, things started to take an unexpected turn here. The tone of the interview shifted. The interviewer, who had initially seemed impressed by my technical abilities and project experiences, now seemed irritated. His questions became sharper, and the atmosphere more tense. I remained calm and polite, trying to explain my thought process without being defensive, but I could sense that something was off. It was as if I had unknowingly stepped on his toes.</p><h3>The Unexpected Feedback: A Harsh Reality Check</h3><p>Toward the end of the interview, the HR asked if I had any questions. I used this opportunity to inquire about the growth opportunities and upskilling options in their organization, and he responded professionally. Then, I asked a question that has become a crucial part of my self-improvement journey: “Could you give me feedback on areas where I can improve?”</p><p>His response was direct and unexpected. He told me, <strong>“Don’t say unwanted words and answers. You’re complicating even simple things. You don’t know the experience of the person sitting in front of you. Just keep things simple and don’t overshow yourself.”</strong></p><p>I have to admit, these words stung. I had been polite, careful not to oversell myself, and I thought I had explained things clearly. Hearing this feedback felt like a punch to the gut. In that moment, I questioned my approach, my communication, and whether I had misread the whole situation. Had I really overcomplicated things? Had I come across as arrogant?</p><h3>Reflection: The Takeaways From a Mixed Experience</h3><p>As I walked out of that interview, I couldn’t shake the feeling of disappointment. But after taking time to reflect, I realized that this experience, while uncomfortable, was an important step in my growth.</p><ol><li><strong>Less is More</strong>: While I was trying to provide detailed answers, the interviewer was looking for simplicity. In future interviews, I’ll focus on delivering concise, to-the-point answers unless more information is requested.</li><li><strong>Read the Room</strong>: Not every interviewer wants the same level of detail or explanation. Paying attention to subtle cues — body language, tone of voice, or pacing — can help me adjust my approach in real-time to match what the interviewer is looking for.</li><li><strong>Feedback is a Gift, Even When It Hurts</strong>: The feedback I received wasn’t easy to hear, but it taught me that even in situations where I think I’m doing everything right, there’s always room to improve. Growth doesn’t come from staying in your comfort zone; it comes from facing tough situations and learning from them.</li><li><strong>Confidence Without Ego</strong>: I’m proud of my accomplishments, but I’ve learned the importance of balancing confidence with humility. Just because I know my stuff doesn’t mean I need to explain everything in depth — sometimes a simpler answer is the best answer.</li></ol><h3>Moving Forward: A Mindset of Growth</h3><p>Though I walked away from the interview feeling unsure, I now see it as a valuable lesson. It taught me about the fine line between clarity and over-explanation, about the importance of reading the room, and about handling critical feedback with grace.</p><p>Regardless of whether I get the job, this interview has shaped me in ways that will benefit me in future interviews and professional endeavors. It’s not just about getting everything right — it’s about learning, growing, and becoming better every step of the way.</p><p>For anyone reading this who may be in the same boat — feeling unsure after a tough interview or grappling with unexpected feedback — remember that every experience, good or bad, is part of your journey. Use it to reflect, improve, and keep moving forward.</p><h3>A Reminder to Myself: The Road Ahead</h3><p>This experience will stay with me not as a failure, but as a stepping stone toward becoming better. I’ll take the lessons I’ve learned, keep honing my skills, and always remain open to growth. The next time I step into an interview, I’ll carry this memory with me — not as a burden, but as a reminder of how far I’ve come and how much further I’m capable of going.</p><p><strong>To my future self: Keep pushing, keep growing, and remember — every experience, no matter how mixed, is a part of your success.</strong></p><p>This was my first big interview, and while it wasn’t perfect, it was perfectly necessary for my growth.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a855c940d225" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Understanding Hugging Face Model File Formats, GGML, and GGUF!]]></title>
            <link>https://blog.devops.dev/understanding-hugging-face-model-file-formats-ggml-and-gguf-914b0ebd1131?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/914b0ebd1131</guid>
            <category><![CDATA[devops]]></category>
            <category><![CDATA[gpt]]></category>
            <category><![CDATA[llm]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[llama-2]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Mon, 16 Sep 2024 11:26:20 GMT</pubDate>
            <atom:updated>2024-09-27T09:35:24.464Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/740/1*dQAz3uY69ZxA6NPMa3wrbQ.png" /></figure><h3>Introduction:</h3><p>If you’re like me, navigating the world of machine learning can sometimes feel like learning a new language — there are tons of acronyms, file formats, and tools that can get overwhelming pretty quickly! When I first started using large language models (like GPT) from Hugging Face, I was confused by all the different files: model.bin, config.json, .onnx—what do they all do? And how do GGML and GGUF fit into the picture?</p><p>In this blog, I’m going to break everything down so that you don’t have to struggle like I did. We’ll take a look at the different file formats Hugging Face uses, talk about newer formats like GGML and GGUF, and figure out their pros and cons. Whether you’re just starting out or you’ve been working with these models for a while, this guide should help you get a better handle on the basics.</p><p>Ready? Let’s dive in!</p><h3>1. Hugging Face Model File Formats: What Are They?</h3><p>Hugging Face provides pretrained models in multiple file formats that help developers easily load, fine-tune, and deploy models. Understanding these files is key to using Hugging Face models effectively.</p><h4>Key Hugging Face File Formats:</h4><ol><li><strong>model.bin</strong>:</li></ol><ul><li>This is the <strong>primary file</strong> that contains the model’s <strong>weights</strong>. These weights are learned parameters of the model that help it make predictions or perform tasks.</li><li><strong>What are weights?</strong> Weights are the numerical values that get adjusted during training, enabling the model to learn.</li></ul><p>2. <strong>config.json</strong>:</p><ul><li>The config.json file contains the <strong>model’s architecture</strong> details such as the number of layers, hidden units, attention heads, etc. This file is crucial for correctly loading the model’s structure.</li><li><strong>What does it do?</strong> It tells the framework (like PyTorch or TensorFlow) how to assemble the model before loading the weights from model.bin.</li></ul><p>3. <strong>tokenizer.json</strong>:</p><ul><li>This file contains the <strong>tokenizer</strong> information, which is responsible for breaking down input text into smaller pieces (tokens) that the model can understand.</li><li><strong>Why is this important?</strong> Without proper tokenization, the model wouldn’t be able to process textual input correctly.</li></ul><p>4. <strong>vocab.txt</strong>:</p><ul><li>For some models, especially those based on transformers like BERT, vocab.txt holds the vocabulary list of all possible tokens the model can recognize.</li><li><strong>What’s inside?</strong> A list of words or subwords that the model can break text into, helping it convert input text into a form the model can work with.</li></ul><p>5. <strong>merges.txt</strong>:</p><ul><li>This file is used in <strong>Byte-Pair Encoding (BPE)</strong> tokenization, which merges smaller units of text into larger ones. It’s common in models like GPT-2.</li><li><strong>Why does it matter?</strong> It helps the tokenizer decide how to efficiently break down and merge text tokens.</li></ul><h4>Advanced Hugging Face File Formats:</h4><p>In addition to the basic files, Hugging Face supports more advanced formats for optimized performance, model sharing, and deployment.</p><p>6. <strong>model.onnx</strong>:</p><ul><li><strong>ONNX</strong> stands for <strong>Open Neural Network Exchange</strong>, a format designed to be interoperable between different frameworks like PyTorch and TensorFlow.</li><li><strong>Why use ONNX?</strong> It allows models trained in one framework (e.g., PyTorch) to be used in another (e.g., TensorFlow), enabling flexibility in model deployment.</li><li><strong>Pros</strong>: Lightweight, cross-platform compatibility, and faster inference on CPUs and GPUs.</li><li><strong>Cons</strong>: Converting models into ONNX may sometimes lead to performance differences or incompatibilities.</li></ul><p>7. <strong>model.safetensors</strong>:</p><ul><li><strong>Safetensors</strong> is a new, efficient file format designed to store model weights securely and in a smaller size compared to traditional .bin files.</li><li><strong>Why use Safetensors?</strong> It’s faster to load, reduces the risk of malicious code (since it doesn’t store code, only data), and is easier to share across systems.</li><li><strong>Pros</strong>: Secure, lightweight, and faster loading compared to .bin.</li><li><strong>Cons</strong>: Less widely adopted so far, though growing in popularity.</li></ul><p>8. <strong>generation_config.json</strong>:</p><ul><li>This file holds the <strong>generation parameters</strong> (like temperature, top-k sampling, max tokens) that control how the model generates text.</li><li><strong>Why is it useful?</strong> It allows you to customize how the model generates text, affecting creativity, diversity, or predictability of the generated responses.</li></ul><h3>2. GGML: GPT-Generated Model Language</h3><p><strong>GGML</strong> is a format developed to simplify the use of large language models like GPT, especially for running them on CPUs. It bundles everything into one file for easy sharing and loading.</p><h4>Key Features of GGML:</h4><ul><li><strong>Single File Format</strong>: GGML consolidates the model and configuration into a single file, reducing complexity for sharing.</li><li><strong>CPU-Compatible</strong>: GGML is designed to run efficiently on CPUs, making it accessible for those without high-end GPUs.</li></ul><h4>Pros of GGML:</h4><ul><li><strong>Convenience</strong>: No need to manage multiple files like in Hugging Face formats.</li><li><strong>CPU-Friendly</strong>: You can run models on standard hardware without GPUs.</li></ul><h4>Cons of GGML:</h4><ul><li><strong>Limited Metadata</strong>: GGML lacks support for storing extra information like model version or configuration, making it less flexible.</li><li><strong>Compatibility Issues</strong>: GGML struggled with introducing new features, requiring manual adjustments for older models.</li></ul><h3>3. GGUF: GPT-Generated Unified Format</h3><p><strong>GGUF</strong> is the evolution of GGML, solving many of its limitations. Introduced in 2023, GGUF adds more functionality, better metadata support, and future-proofing for large language models.</p><h4>Key Features of GGUF:</h4><ul><li><strong>Unified Format</strong>: GGUF retains the single-file approach of GGML but introduces more flexibility.</li><li><strong>Metadata Support</strong>: Unlike GGML, GGUF allows for storing extra information like model version, architecture, and configuration.</li><li><strong>Backward Compatibility</strong>: GGUF is compatible with older GGML models but can handle newer features with ease.</li></ul><h4>Pros of GGUF:</h4><ul><li><strong>Flexibility</strong>: Supports new features and stores metadata.</li><li><strong>Backward Compatibility</strong>: Works with older GGML models without breaking them.</li><li><strong>Easier to Use</strong>: Less need for manual parameter adjustments, improving the user experience.</li></ul><h4>Cons of GGUF:</h4><ul><li><strong>Transition Period</strong>: Moving from GGML to GGUF may take time for users who already have GGML models.</li><li><strong>Learning Curve</strong>: Though easier to use, GGUF introduces new concepts that users need to learn.</li></ul><h3>Comparing Hugging Face Models, GGML, and GGUF</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/829/1*cFCu4DF6aXB42aFII0mFyQ.png" /><figcaption>comparison table</figcaption></figure><h3>Conclusion</h3><p>Hugging Face, GGML, and GGUF are all powerful formats with different use cases depending on your needs. Here’s a quick takeaway:</p><ul><li><strong>Hugging Face</strong> models offer flexibility with separate files for weights, configuration, and tokenization, making them ideal for customization and compatibility across platforms like PyTorch and TensorFlow.</li><li><strong>GGML</strong> provided a simple single-file solution but lacked flexibility, especially as newer features were introduced.</li><li><strong>GGUF</strong> is the latest evolution, offering the best of both worlds with a unified file, backward compatibility, and added metadata for future-proofing.</li></ul><p>Whether you’re a beginner trying to load your first model or an advanced user looking for optimization, understanding these formats will help you choose the right tool for your workflow.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=914b0ebd1131" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/understanding-hugging-face-model-file-formats-ggml-and-gguf-914b0ebd1131">Understanding Hugging Face Model File Formats, GGML, and GGUF!</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering GitHub: A Beginner’s Guide to Version Control and Collaboration]]></title>
            <link>https://blog.devops.dev/mastering-github-a-beginners-guide-to-version-control-and-collaboration-626f8cc4ba23?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/626f8cc4ba23</guid>
            <category><![CDATA[version-control-system]]></category>
            <category><![CDATA[github-actions]]></category>
            <category><![CDATA[genai]]></category>
            <category><![CDATA[conventional-commits]]></category>
            <category><![CDATA[github]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Fri, 19 Jul 2024 18:43:59 GMT</pubDate>
            <atom:updated>2024-09-17T18:48:31.781Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vfpzUX_F_54uaUi_" /><figcaption>Photo by <a href="https://unsplash.com/@rubaitulazad?utm_source=medium&amp;utm_medium=referral">Rubaitul Azad</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3><strong>Introduction:</strong></h3><p>In today’s digital world, version control and collaboration are essential for software development. GitHub is a powerful platform that enables developers to manage their code, collaborate with others, and track changes effectively. This guide is tailored for beginners who want to get started with GitHub, offering explanations, code examples, and real-life scenarios.</p><h3><strong>What is GitHub?</strong></h3><p>GitHub is a web-based platform that uses Git, a distributed version control system, to help developers manage and share their code. Git, created by Linus Torvalds, allows multiple people to work on the same project simultaneously, track changes, and revert to previous versions if needed. GitHub builds on Git by providing a user-friendly interface, hosting repositories in the cloud, and offering additional collaboration features.</p><h3><strong>Key Concepts:</strong></h3><ul><li><strong>Repository (Repo):</strong> A project space that contains all your project’s files and the revision history.</li><li><strong>Branch:</strong> A parallel version of your repository. It’s used to develop features separately from the main codebase.</li><li><strong>Commit:</strong> A snapshot of your project at a point in time. It represents a single change to the codebase.</li><li><strong>Pull Request:</strong> A request to merge changes from one branch to another. It’s a key part of collaboration on GitHub.</li><li><strong>Fork:</strong> A copy of a repository that allows you to freely experiment with changes without affecting the original project.</li></ul><h3><strong>Why Use GitHub?</strong></h3><p>GitHub offers several advantages that make it a vital tool for developers:</p><ul><li><strong>Version Control:</strong> Keep a detailed history of changes to your code, allowing you to revert to previous versions if necessary.</li><li><strong>Collaboration:</strong> Work with other developers seamlessly. GitHub’s interface makes it easy to review code, manage contributions, and discuss changes.</li><li><strong>Backup:</strong> Store your code securely in the cloud. This ensures that your work is safe even if something happens to your local machine.</li><li><strong>Project Management:</strong> Use issues and project boards to track bugs, feature requests, and tasks.</li><li><strong>Showcase Work:</strong> Create a portfolio of projects to share with potential employers or collaborators. GitHub profiles are often reviewed by recruiters to gauge a candidate’s skills and experience.</li></ul><h3><strong>Getting Started with GitHub</strong></h3><h4>Creating a Repository</h4><p>A repository (or repo) is where your project files are stored. Let’s create a new repository on GitHub:</p><ol><li><strong>Sign in to GitHub:</strong> If you don’t have an account, sign up at <a href="https://github.com/">github.com</a>.</li><li><strong>Create a New Repository:</strong></li></ol><ul><li>Click the + icon in the top right corner and select &quot;New repository.&quot;</li><li>Name your repository (e.g., my-first-repo).</li><li>Add a description (optional).</li><li>Choose to make the repository public or private.</li><li>Initialize the repository with a README file (this is a good practice as it provides an overview of your project).</li><li>Click “Create repository.”</li></ul><h4>Cloning a Repository</h4><p>Cloning a repository means creating a local copy on your machine. To clone your newly created repository:</p><pre>git clone https://github.com/your-username/my-first-repo.git<br>cd my-first-repo</pre><h4>Basic Git Commands</h4><p>Here are some basic Git commands to get you started:</p><p><strong>git status:</strong> Check the status of your files in the working directory.</p><pre>git status</pre><p><strong>git add:</strong> Add files to the staging area.</p><pre>git add filename</pre><p><strong>git commit:</strong> Commit changes to the repository with a descriptive message.</p><pre>git commit -m &quot;Initial commit&quot;</pre><p><strong>git push:</strong> Push changes from your local repository to GitHub.</p><pre>git push origin main</pre><h4>Making Changes and Committing</h4><p>Let’s make a change to your repository and commit it:</p><p>Create a new file called hello_world.py and add the following code:</p><pre>print(&quot;Hello, GitHub!&quot;)</pre><p>Save the file and add it to the staging area:</p><pre>git add hello_world.py</pre><p>Commit the changes:</p><pre>git commit -m &quot;Add hello_world.py script&quot;</pre><p>Push the changes to GitHub:</p><pre>git push origin main</pre><h4>Branching and Pull Requests</h4><p>Branches allow you to work on features independently from the main codebase. Let’s create a new branch:</p><p>Create and switch to a new branch:</p><pre>git checkout -b new-feature</pre><p>Make changes and commit them:</p><pre>echo &quot;print(&#39;New Feature&#39;)&quot; &gt;&gt; new_feature.py <br>git add new_feature.py git commit -m &quot;Add new feature script&quot;</pre><p>Push the branch to GitHub:</p><pre>git push origin new-feature</pre><p>Create a Pull Request on GitHub to merge your changes into the main branch. This is done via the GitHub web interface.</p><h3><strong>Real-Life Example:</strong></h3><p>Imagine you’re working on a collaborative project, building a website. You and your team members each create branches to work on different features — one works on the homepage, another on the contact form, and another on the backend API. Using GitHub, you can manage these branches, review each other’s code, and merge the changes back into the main branch without conflicts.</p><p>Here’s a simple example:</p><h4><strong>Project Setup:</strong></h4><ul><li>Create a repository for the project.</li><li>Each team member clones the repository to their local machine.</li></ul><h4><strong>Feature Development:</strong></h4><ul><li>Each member creates a branch for their feature.</li><li>They work on their features independently, committing changes to their branches.</li></ul><h4><strong>Pull Requests:</strong></h4><ul><li>When a feature is complete, a pull request is created to merge the branch into the main branch.</li><li>Team members review the pull request, suggest changes if necessary, and approve it.</li></ul><h4><strong>Merging:</strong></h4><ul><li>Once approved, the branch is merged into the main branch.</li><li>All members pull the latest changes to keep their local repositories up-to-date.</li></ul><h3><strong>Conclusion:</strong></h3><p>Congratulations! You’ve taken your first steps in understanding and using GitHub. From creating repositories to managing branches and collaborating with others, GitHub is an invaluable tool in modern software development. Keep experimenting, explore more advanced features, and soon you’ll be contributing to projects like a pro.</p><p>Feel free to leave your questions and feedback in the comments below. Happy coding!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=626f8cc4ba23" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/mastering-github-a-beginners-guide-to-version-control-and-collaboration-626f8cc4ba23">Mastering GitHub: A Beginner’s Guide to Version Control and Collaboration</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Set Matrix Zeros: A Beginner’s Guide]]></title>
            <link>https://medium.com/@sanjaykumar.kedu/how-to-set-matrix-zeros-a-beginners-guide-ece4b3d350ee?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/ece4b3d350ee</guid>
            <category><![CDATA[data-structure-algorithm]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[dsa-courses]]></category>
            <category><![CDATA[coding]]></category>
            <category><![CDATA[matrix]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Thu, 18 Jul 2024 19:37:19 GMT</pubDate>
            <atom:updated>2024-07-18T19:37:19.926Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_ySQ4tzxdkKlXP8l" /><figcaption>Photo by <a href="https://unsplash.com/@cdr6934?utm_source=medium&amp;utm_medium=referral">Chris Ried</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>In this post, we’ll walk through an interesting problem in computer science: setting matrix rows and columns to zero based on specific conditions. This guide is beginner-friendly and provides a step-by-step explanation along with two different approaches to solve the problem. By the end of this tutorial, you’ll understand the problem, the intuition behind it, and be able to implement the solution in Python.</p><h3>Problem Statement</h3><p>Given a matrix, if any element in the matrix is 0, set its entire row and column to 0. Finally, return the modified matrix.</p><h3>Example</h3><p>Let’s say we have the following matrix:</p><pre>1 1 1<br>1 0 1<br>1 1 1</pre><p>The output should be:</p><pre>1 0 1<br>0 0 0<br>1 0 1</pre><h3>Brute Force Approach</h3><p>The brute force approach is straightforward. We’ll traverse the entire matrix to find the zero elements, and then mark their corresponding rows and columns.</p><h3>Algorithm / Intuition</h3><ol><li>Use two nested loops to traverse all cells of the matrix.</li><li>If a cell contains 0, mark all cells in the corresponding row and column with -1, except those which are already 0.</li><li>After marking, convert all -1s to 0.</li></ol><p><strong>Note:</strong> We assume the matrix does not contain negative numbers. If it does, we need a different marker instead of -1.</p><h3>Code Implementation</h3><pre>def markRow(matrix, n, m, i):<br>    for j in range(m):<br>        if matrix[i][j] != 0:<br>            matrix[i][j] = -1<br>def markCol(matrix, n, m, j):<br>    for i in range(n):<br>        if matrix[i][j] != 0:<br>            matrix[i][j] = -1<br>def zeroMatrix(matrix, n, m):<br>    for i in range(n):<br>        for j in range(m):<br>            if matrix[i][j] == 0:<br>                markRow(matrix, n, m, i)<br>                markCol(matrix, n, m, j)<br>    <br>    for i in range(n):<br>        for j in range(m):<br>            if matrix[i][j] == -1:<br>                matrix[i][j] = 0<br>    <br>    return matrix<br>if __name__ == &quot;__main__&quot;:<br>    matrix = [[1, 1, 1], [1, 0, 1], [1, 1, 1]]<br>    n = len(matrix)<br>    m = len(matrix[0])<br>    ans = zeroMatrix(matrix, n, m)<br>    print(&quot;The Final matrix is:&quot;)<br>    for row in ans:<br>        print(row)</pre><h3>Complexity Analysis</h3><ul><li><strong>Time Complexity:</strong> O((N×M)×(N+M))+O(N×M), where N is the number of rows and M is the number of columns.</li><li><strong>Space Complexity:</strong> O(1), as we are not using any extra space.</li></ul><h3>Improved Approach</h3><p>We can optimize the brute force approach by reducing the number of traversals. Instead of marking cells during traversal, we mark rows and columns after traversal using two extra arrays.</p><h3>Algorithm / Intuition</h3><ol><li>Create two arrays: row of size NNN and col of size MMM, both initialized with 0.</li><li>Traverse the matrix. If a cell (i,j)(i, j)(i,j) is 0, mark row[i] and col[j] as 1.</li><li>Traverse the matrix again and set cell (i,j)(i, j)(i,j) to 0 if either row[i] or col[j] is 1.</li></ol><h3>Code Implementation</h3><pre>def zeroMatrix(matrix, n, m):<br>    row = [0] * n<br>    col = [0] * m<br>    for i in range(n):<br>        for j in range(m):<br>            if matrix[i][j] == 0:<br>                row[i] = 1<br>                col[j] = 1<br>    for i in range(n):<br>        for j in range(m):<br>            if row[i] or col[j]:<br>                matrix[i][j] = 0<br>    return matrix<br>if __name__ == &quot;__main__&quot;:<br>    matrix = [[1, 1, 1], [1, 0, 1], [1, 1, 1]]<br>    n = len(matrix)<br>    m = len(matrix[0])<br>    ans = zeroMatrix(matrix, n, m)<br>    print(&quot;The Final matrix is:&quot;)<br>    for row in ans:<br>        print(row)</pre><h3>Complexity Analysis</h3><ul><li><strong>Time Complexity:</strong> O(2×(N×M))</li><li><strong>Space Complexity:</strong> O(N)+O(M)</li></ul><h3>Conclusion</h3><p>We’ve covered two approaches to solve the matrix zero-setting problem. The brute force approach is simple but less efficient, while the improved approach uses extra space to achieve better time complexity. By understanding and implementing these methods, you can handle similar matrix manipulation problems more efficiently.</p><p>Feel free to try these solutions and adapt them to your specific needs. Happy coding</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ece4b3d350ee" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Optimizing Model Deployment: A Guide to Quantization with llama-cpp Python]]></title>
            <link>https://blog.devops.dev/optimizing-model-deployment-a-guide-to-quantization-with-llama-cpp-python-1c45b4664189?source=rss-53969f4aa816------2</link>
            <guid isPermaLink="false">https://medium.com/p/1c45b4664189</guid>
            <category><![CDATA[ollama]]></category>
            <category><![CDATA[mlops]]></category>
            <category><![CDATA[llama-cpp]]></category>
            <category><![CDATA[llama-2]]></category>
            <category><![CDATA[llm]]></category>
            <dc:creator><![CDATA[Rajesh]]></dc:creator>
            <pubDate>Wed, 17 Jul 2024 19:03:31 GMT</pubDate>
            <atom:updated>2024-08-12T18:24:03.285Z</atom:updated>
            <content:encoded><![CDATA[<p>In the realm of artificial intelligence (AI), the efficiency of model deployment plays a critical role in real-world applications. One powerful technique to enhance deployment efficiency is quantization. This guide explores how to leverage the llama-cpp Python library to quantize a pre-trained model, reducing its size and computational requirements while preserving performance.</p><h3>Introduction to Quantization</h3><p>Quantization is the process of reducing the precision of a neural network’s parameters and activations. By converting high-precision data (e.g., 32-bit floating point numbers) into lower-precision representations (e.g., 16-bit or even 8-bit integers), quantization makes models more efficient for deployment on various platforms, including mobile devices and embedded systems.</p><h3>Quantization Formats Supported by llama-cpp</h3><p>llama-cpp supports several common quantization formats:</p><ol><li><strong>GGUF (General Graph Update Format)</strong>:</li></ol><p><strong>Use Case</strong>: GGUF is a format designed for efficient representation and deployment of quantized models. It supports various data types and is suitable for scenarios requiring compact model storage and fast inference.</p><p><strong>2. BF16 (Bfloat16)</strong>:</p><p><strong>Use Case</strong>: Bfloat16 is a 16-bit floating-point format used in machine learning frameworks like TensorFlow for model training and inference. It strikes a balance between range and precision, making it suitable for AI applications where reduced precision can accelerate computations without significant loss of accuracy.</p><p><strong>3. INT8 (8-bit Integer)</strong>:</p><p><strong>Use Case</strong>: INT8 quantization reduces model size and computational complexity further compared to floating-point formats. It’s widely used for deploying models on edge devices and scenarios demanding low latency and power efficiency.</p><p><strong>4. FP16 (Half Precision Floating Point)</strong>:</p><p><strong>Use Case</strong>: FP16 is a 16-bit floating-point format that provides better precision than INT8 while still offering computational benefits compared to higher-precision formats like FP32. It’s commonly used in GPU-accelerated deep learning models for faster training and inference.</p><h3>Step-by-Step Guide</h3><h4>1. Setup and Installation</h4><p>Before diving into quantization, ensure you have the necessary tools and libraries installed. Here’s how you can set up your environment:</p><pre># Install necessary packages<br>!apt-get update<br>!apt-get install -y build-essential cmake<br><br># Install llama-cpp-python with CUDA support<br>!CMAKE_ARGS=&quot;-DGGML_CUDA=ON&quot; pip install llama-cpp-python --no-cache-dir<br># Verify CUDA installation<br>!nvcc --version<br>!nvidia-smi</pre><h4>2. Download and Prepare the Model</h4><p>Begin by fetching a pre-trained model from the Hugging Face Hub. For demonstration, let’s use a model named bge-large-en-v1.5:</p><pre>import os<br>from huggingface_hub import snapshot_download<br><br>model_name = &quot;BAAI/bge-large-en-v1.5&quot;<br>base_model_dir = &quot;./original_model/&quot;<br># Download the model<br>snapshot_download(repo_id=model_name, local_dir=base_model_dir, local_dir_use_symlinks=False)</pre><h4>3. Convert the Model to Quantized Format</h4><h4>Create a Directory for the Quantized Model</h4><pre>!mkdir ./quantized_model/</pre><h4>Convert the Model to GGUF Format</h4><p>Use llama-cpp to convert the model to a quantized format. Here, we’ll convert it to bf16 format:</p><pre>!python llama.cpp/convert_hf_to_gguf.py ./original_model/ --outtype bf16 --outfile ./quantized_model/bge-large-en-1.5.gguf</pre><h3>4. Verification and Configuration</h3><p>Ensure the conversion was successful by checking the file sizes of the original and quantized models:</p><pre>import os<br># Quantized model file size<br>file_path_quantized = &#39;./quantized_model/bge-large-en-1.5.gguf&#39;<br>if os.path.exists(file_path_quantized):<br>    file_size_quantized = os.stat(file_path_quantized).st_size<br>    file_size_gb_quantized = file_size_quantized / (1024 * 1024 * 1024)<br>    print(f&quot;Size of &#39;{file_path_quantized}&#39;: {file_size_quantized} bytes ({file_size_gb_quantized:.2f} GB)&quot;)<br>else:<br>    print(f&quot;File &#39;{file_path_quantized}&#39; not found.&quot;)<br># Original model file size<br>file_path_original = &#39;./original_model/onnx/model.onnx&#39;<br>if os.path.exists(file_path_original):<br>    file_size_original = os.stat(file_path_original).st_size<br>    file_size_gb_original = file_size_original / (1024 * 1024 * 1024)<br>    print(f&quot;Size of &#39;{file_path_original}&#39;: {file_size_original} bytes ({file_size_gb_original:.2f} GB)&quot;)<br>else:<br>    print(f&quot;File &#39;{file_path_original}&#39; not found.&quot;)<br><br></pre><h3>Technologies Used</h3><ul><li><strong>Python</strong>: Programming language for scripting and model manipulation.</li><li><strong>CUDA</strong>: GPU acceleration for faster computations.</li><li><strong>Hugging Face Hub</strong>: Repository for accessing pre-trained models.</li><li><strong>llama-cpp</strong>: Library for model conversion and embedding generation.</li></ul><h3>Conclusion</h3><p>Quantization with llama-cpp Python offers a robust method to optimize AI models for efficient deployment. By reducing model size and computational complexity, quantization enhances performance on resource-constrained devices without sacrificing accuracy. Integrating these techniques into AI workflows ensures scalable and cost-effective solutions for diverse applications.</p><p>Begin experimenting with quantization today to unlock the full potential of your AI models in real-world scenarios</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1c45b4664189" width="1" height="1" alt=""><hr><p><a href="https://blog.devops.dev/optimizing-model-deployment-a-guide-to-quantization-with-llama-cpp-python-1c45b4664189">Optimizing Model Deployment: A Guide to Quantization with llama-cpp Python</a> was originally published in <a href="https://blog.devops.dev/">DevOps.dev</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>