<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Shuhan Mirza on Medium]]></title>
        <description><![CDATA[Stories by Shuhan Mirza on Medium]]></description>
        <link>https://medium.com/@shuhanmirza?source=rss-ddac69adc4d9------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*Xqa5EBs0rXe4qSRDBWKVUQ.jpeg</url>
            <title>Stories by Shuhan Mirza on Medium</title>
            <link>https://medium.com/@shuhanmirza?source=rss-ddac69adc4d9------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 11:16:04 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@shuhanmirza/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Kickstarting Web Scraping with Residential Proxies: A Python Guide]]></title>
            <link>https://shuhanmirza.medium.com/kickstarting-web-scraping-with-residential-proxies-a-python-guide-f1e6bebc3aa3?source=rss-ddac69adc4d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/f1e6bebc3aa3</guid>
            <category><![CDATA[python]]></category>
            <category><![CDATA[proxy]]></category>
            <category><![CDATA[web-scraping]]></category>
            <dc:creator><![CDATA[Shuhan Mirza]]></dc:creator>
            <pubDate>Tue, 05 Dec 2023 10:04:33 GMT</pubDate>
            <atom:updated>2025-01-18T02:09:32.462Z</atom:updated>
            <content:encoded><![CDATA[<p>Web scraping is a powerful tool for data collection, but it’s often hindered by challenges like IP bans and geo-restrictions. This is where residential proxies come into play. In this blog, we’ll explore how to kickstart web scraping using residential proxies, with practical Python examples to get you started.</p><h3>Understanding the Basics</h3><h4><strong>Why Residential Proxies?</strong></h4><p>Residential proxies offer IP addresses tied to a physical location, making your scraping activities appear as regular user behavior. This reduces the risk of being detected and blocked by target websites.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*QMnFmT6uFHjgqR35OIhidg.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><h4><strong>Setting Up the Environment</strong></h4><p>Before diving into coding, ensure you have Python installed along with necessary libraries like requests and beautifulsoup4. If not, you can install them using pip:</p><pre>pip install requests beautifulsoup4</pre><h3>Example: Simple Web Scraper Using Residential Proxies</h3><h4><strong>Setting Up the Proxy</strong></h4><p>First, you need access to residential proxy services. There are several providers available in the market.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*dYLXAz47pDPgdLQ9009ZOQ.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Once you have your proxy details (IP address and port), you can set it up in Python:</p><pre>import requests<br><br># Replace with your residential proxy details<br>proxies = {<br>    &#39;http&#39;: &#39;http://user:pass@gate.nodemaven.com:8080&#39;,<br>    &#39;https&#39;: &#39;http://user:pass@gate.nodemaven.com:8080&#39;<br>}</pre><h4>Writing a Basic Scraper</h4><p>Let’s scrape a simple website for demonstration. We’ll use requests to make requests and BeautifulSoup from beautifulsoup4 to parse HTML.</p><pre>from bs4 import BeautifulSoup<br><br>url = &#39;https://example.com&#39;<br><br>response = requests.get(url, proxies=proxies)<br>soup = BeautifulSoup(response.content, &#39;html.parser&#39;)<br><br># Example: Extracting all paragraph texts<br>paragraphs = soup.find_all(&#39;p&#39;)<br>for para in paragraphs:<br>    print(para.get_text())</pre><h4>Handling Exceptions</h4><p>It’s important to handle exceptions that might occur due to network issues or bad proxy connections.</p><pre>try:<br>    response = requests.get(url, proxies=proxies)<br>    # rest of your scraping logic<br>except requests.exceptions.RequestException as e:<br>    print(f&#39;Error: {e}&#39;)</pre><h3>Advanced Tips</h3><h4>Rotating Proxies</h4><p>To avoid detection, it’s wise to rotate through multiple proxies. You can create a list of proxies and select one randomly for each request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/1*gfwHaeLqf43AcSYNmGoB9A.png" /><figcaption>downloaded from <a href="https://knowyourmeme.com/memes/good-luck-im-behind-7-proxies">knowyourmeme.com</a></figcaption></figure><pre>import random<br><br># List of proxies<br>proxy_list = [&#39;http://user:pass@gate.nodemaven.com:8080&#39;, &#39;http://user1:pass1@gate.nodemaven.com:8080&#39;, &#39;http://user2:pass2@gate.nodemaven.com:8080&#39;, ...]<br><br># Function to get a random proxy<br>def get_random_proxy():<br>    proxy = random.choice(proxy_list)<br><br>    return {<br>        &quot;http&quot;: proxy,<br>        &quot;https&quot;: proxy<br>    }<br><br># Use the random proxy for your requests<br>proxies = get_random_proxy()</pre><h4>Adding Headers</h4><p>Some websites may check for the presence of headers to identify bot traffic. Adding headers can make your requests look more like a regular browser.</p><pre>headers = {<br>    &#39;User-Agent&#39;: &#39;Your User Agent String&#39;<br>}<br>response = requests.get(url, headers=headers, proxies=proxies)</pre><h4>Dealing with JavaScript-Heavy Sites</h4><p>For sites heavily reliant on JavaScript, tools like Selenium or Puppeteer (with a Python wrapper) are more suitable. Setting up residential proxies in these environments is similarly straightforward. I will cover that in a future article.</p><h3>Conclusion</h3><p>Residential proxies are invaluable in the realm of web scraping, offering the ability to bypass restrictions while maintaining a high level of anonymity. This guide provides a starting point for Python enthusiasts looking to integrate residential proxies into their scraping projects. As always, remember to scrape responsibly and adhere to legal and ethical standards.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/630/1*2g2ieGx4G6okP0p2P5T96A.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>If you found this helpful please show support by subscribing and posting comments. If you have any feedback, please ping me on my LinkedIn: <a href="https://linkedin.com/in/shuhanmirza/">https://linkedin.com/in/shuhanmirza/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f1e6bebc3aa3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[S.O.L.I.D Principle: The Superhero’s Guide to Writing Clean Code!]]></title>
            <link>https://shuhanmirza.medium.com/s-o-l-i-d-principle-the-superheros-guide-to-writing-clean-code-d67324506ecf?source=rss-ddac69adc4d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/d67324506ecf</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[clean-code]]></category>
            <dc:creator><![CDATA[Shuhan Mirza]]></dc:creator>
            <pubDate>Fri, 17 Feb 2023 12:13:37 GMT</pubDate>
            <atom:updated>2023-02-17T15:12:18.356Z</atom:updated>
            <content:encoded><![CDATA[<h4>Want to Write Better Code? Here’s How to Do It! Explained with memes 😏</h4><h4>Save Your Code from the Dark Abyss of Spaghetti Code with These Simple Principles!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/490/1*O9kYWfd9Jsvt2xxDgdRFkg.gif" /><figcaption>Scene from Bruce Almighty</figcaption></figure><p>If you’re a software developer, you may have heard about S.O.L.I.D. It’s an acronym that stands for:</p><ul><li>Single Responsibility Principle (SRP)</li><li>Open/Closed Principle (OCP)</li><li>Liskov Substitution Principle (LSP)</li><li>Interface Segregation Principle (ISP)</li><li>Dependency Inversion Principle (DIP)</li></ul><p>These principles were first introduced by <a href="https://en.wikipedia.org/wiki/Robert_C._Martin">Robert C. Martin (a.k.a Uncle Bob)</a>, and they aim to help developers build software that is easier to maintain, extend and understand.</p><p>Let’s dive into each principle in more detail, with some example Java code to help illustrate the concepts.</p><h4>1. Single Responsibility Principle (SRP)</h4><p>The SRP states that a class should have only one sole purpose. In other words, a class should have only one responsibility. This principle helps to ensure that your code is focused, readable, and easy to maintain. We should not burden a class with excessive functionality.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*1qQ8gjHzQZQWGnt-QHB23w.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Let’s say you have a User class that handles user authentication and email notifications. Here&#39;s an example of how you can refactor this class to follow the SRP:</p><pre>// Before refactoring<br><br>public class User {<br>  public void login(String username, String password) {<br>    // Authenticate user<br>  }<br>  <br>  public void sendEmail(String to, String subject, String body) {<br>    // Send email<br>  }<br>}</pre><pre>// After refactoring<br><br>public class User {<br>  public void login(String username, String password) {<br>    // Authenticate user<br>  }<br>}<br><br>public class EmailService {<br>  public void sendEmail(String to, String subject, String body) {<br>    // Send email<br>  }<br>}</pre><h4>2. Open/Closed Principle (OCP)</h4><p>The OCP states that software entities (classes, modules, functions, etc.) should be open for extension but closed for modification. In other words, you should be able to add new functionality to your code without changing the existing code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*rXPA6qZ0Q5JBer9YhZZcbw.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Let’s say you have a PaymentProcessor class that handles payment processing for different payment methods. Here&#39;s an example of how you can refactor this class to follow the OCP:</p><pre>// Before refactoring<br><br>public class PaymentProcessor {<br>  public void processPayment(Payment payment) {<br>    if (payment.getMethod() == PaymentMethod.CREDIT_CARD) {<br>      // Process credit card payment<br>    } else if (payment.getMethod() == PaymentMethod.PAYPAL) {<br>      // Process PayPal payment<br>    } else if (payment.getMethod() == PaymentMethod.BITCOIN) {<br>      // Process Bitcoin payment<br>    }<br>  }<br>}</pre><pre>// After refactoring<br><br>public interface PaymentMethodProcessor {<br>  void processPayment(Payment payment);<br>}<br><br>public class CreditCardProcessor implements PaymentMethodProcessor {<br>  public void processPayment(Payment payment) {<br>    // Process credit card payment<br>  }<br>}<br><br>public class PayPalProcessor implements PaymentMethodProcessor {<br>  public void processPayment(Payment payment) {<br>    // Process PayPal payment<br>  }<br>}<br><br>public class BitcoinProcessor implements PaymentMethodProcessor {<br>  public void processPayment(Payment payment) {<br>    // Process Bitcoin payment<br>  }<br>}<br><br>public class PaymentProcessor {<br>  private Map&lt;PaymentMethod, PaymentMethodProcessor&gt; processors = new HashMap&lt;&gt;();<br>  <br>  public PaymentProcessor() {<br>    processors.put(PaymentMethod.CREDIT_CARD, new CreditCardProcessor());<br>    processors.put(PaymentMethod.PAYPAL, new PayPalProcessor());<br>    processors.put(PaymentMethod.BITCOIN, new BitcoinProcessor());<br>  }<br>  <br>  public void processPayment(Payment payment) {<br>    PaymentMethodProcessor processor = processors.get(payment.getMethod());<br>    if (processor != null) {<br>      processor.processPayment(payment);<br>    }<br>  }<br>}</pre><p>Now, we can add a ‘Stripe Payment Processor’ class without changing existing codes.</p><h4>3. Liskov Substitution Principle (LSP)</h4><p>The Liskov Substitution Principle (LSP) states that objects of a superclass should be replaceable with objects of a subclass without breaking the application. In other words, a subclass should be able to be used in place of its superclass without causing unexpected behavior or errors.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/625/1*R2ncJDk1JGaytJUvHV8o6w.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Let’s say you have a Rectangle class that calculates the area of a rectangle. Here&#39;s an example of how you can refactor this class to follow the LSP:</p><pre>// Before refactoring<br><br>public class Rectangle {<br>  private int width;<br>  private int height;<br>  <br>  public int getWidth() {<br>    return width;<br>  }<br>  <br>  public void setWidth(int width) {<br>    this.width = width;<br>  }<br>  <br>  public int getHeight() {<br>    return height;<br>  }<br>  <br>  public void setHeight(int height) {<br>    this.height = height;<br>  }<br>  <br>  public int getArea() {<br>    return width * height;<br>  }<br>}<br><br>public class Square extends Rectangle {<br>  public void setWidth(int width) {<br>    super.setWidth(width);<br>    super.setHeight(width);<br>  }<br>  <br>  public void setHeight(int height) {<br>    super.setWidth(height);<br>    super.setHeight(height);<br>  }<br>}<br><br>// Test code<br>Rectangle rect = new Square();<br>rect.setWidth(5);<br>rect.setHeight(10);<br>System.out.println(rect.getArea()); // Expected output: 50</pre><p>In this example, the Square class inherits from the Rectangle class, but it violates the LSP because it changes the behavior of the superclass. The Square class sets both the width and height to the same value, which is not what you would expect from a rectangle. To fix this, you can change the design to use separate Square and Rectangle classes.</p><pre>// After refactoring<br><br>public class Rectangle {<br>  protected int width;<br>  protected int height;<br>  <br>  public Rectangle(int width, int height) {<br>    this.width = width;<br>    this.height = height;<br>  }<br>  <br>  public int getWidth() {<br>    return width;<br>  }<br>  <br>  public void setWidth(int width) {<br>    this.width = width;<br>  }<br>  <br>  public int getHeight() {<br>    return height;<br>  }<br>  <br>  public void setHeight(int height) {<br>    this.height = height;<br>  }<br>  <br>  public int getArea() {<br>    return width * height;<br>  }<br>}<br><br>public class Square extends Rectangle {<br>  public Square(int size) {<br>    super(size, size);<br>  }<br>}<br><br>// Test code<br>Rectangle rect = new Rectangle(5, 10);<br>System.out.println(rect.getArea()); // Expected output: 50<br><br>Square square = new Square(5);<br>System.out.println(square.getArea()); // Expected output: 25</pre><h4>4. Interface Segregation Principle (ISP)</h4><p>The Interface Segregation Principle (ISP) states that a client should not be forced to implement interfaces they do not use. In other words, you should separate interfaces into smaller, more focused ones that are easier to implement and change.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/888/1*tjhn9llun3GAPNQCJabbow.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Let’s say you have a Printer interface that has two methods: print() and scan(). Here&#39;s an example of how you can refactor this interface to follow the ISP:</p><pre>// Before refactoring<br><br>public interface Printer {<br>  void print();<br>  void scan();<br>}<br><br>public class LaserPrinter implements Printer {<br>  public void print() {<br>    // Print document<br>  }<br>  <br>  public void scan() {<br>    // Scan document<br>  }<br>}<br><br>public class InkjetPrinter implements Printer {<br>  public void print() {<br>    // Print document<br>  }<br>  <br>  public void scan() {<br>    // Do nothing<br>  }<br>}</pre><pre>// After refactoring<br><br>public interface Printable {<br>  void print();<br>}<br><br>public interface Scanable {<br>  void scan();<br>}<br><br>public class LaserPrinter implements Printable, Scanable {<br>  public void print() {<br>    // Print document<br>  }<br><br>public void scan() {<br>  // Scan document<br>  }<br>}<br><br>public class InkjetPrinter implements Printable {<br>  public void print() {<br>    // Print document<br>  }<br>}<br><br>// Test code<br>Printable laserPrinter = new LaserPrinter();<br>laserPrinter.print();<br>((Scanable)laserPrinter).scan();<br><br>Printable inkjetPrinter = new InkjetPrinter();<br>inkjetPrinter.print();</pre><p>In this example, the Printer interface has two methods, print() and scan(). The LaserPrinter and InkjetPrinter classes both implement the Printer interface, but the InkjetPrinter class doesn’t use the scan() method, which violates the ISP. To fix this, you can create separate Printable and Scanable interfaces that are easier to implement and use.</p><h4>5. Dependency Inversion Principle (DIP)</h4><p>The Dependency Inversion Principle (DIP) states that high-level modules should not depend on low-level modules. Instead, both should depend on abstractions. Abstractions should not depend on details. Details should depend on abstractions. In other words, your code should depend on interfaces or abstract classes, rather than concrete implementations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/500/1*1USExcQUW79dH4LY1EPp4w.jpeg" /><figcaption>Made by the author using imgflip.com</figcaption></figure><p>Let’s say you have a Book class that needs to retrieve data from a Database class. Here’s an example of how you can refactor this code to follow the DIP:</p><pre>// Before refactoring<br><br>public class Book {<br> private Database database;<br> <br> public Book() {<br> this.database = new Database();<br> }<br> <br> public String getTitle(int id) {<br> return database.getTitle(id);<br> }<br>}<br>public class Database {<br> public String getTitle(int id) {<br> // Retrieve title from database<br> return &quot;Title&quot;;<br> }<br>}</pre><pre>// After refactoring<br><br>public interface Database {<br> String getTitle(int id);<br>}<br>public class Book {<br> private Database database;<br> <br> public Book(Database database) {<br> this.database = database;<br> }<br> <br> public String getTitle(int id) {<br> return database.getTitle(id);<br> }<br>}<br>public class DatabaseImpl implements Database {<br> public String getTitle(int id) {<br> // Retrieve title from database<br> return &quot;Title&quot;;<br> }<br>}<br>// Test code<br>Database database = new DatabaseImpl();<br>Book book = new Book(database);<br>System.out.println(book.getTitle(123)); // Expected output: &quot;Title&quot;</pre><p>In this example, the Book class depends on the Database class, which violates the DIP. To fix this, you can create a Database interface that the Book class depends on, and create a separate DatabaseImpl class that implements the interface. This way, the Book class depends on an abstraction, rather than a concrete implementation. Following DIP makes the code pluggable. How? In the above example, we can switch to a different implementation of the database ( e.g. migrating to PgSQL from MySql) and the system won’t break.</p><h4>Conclusion</h4><p>The SOLID principles are a set of guidelines that can help you write better, more maintainable code. By following these principles, you can make your code more modular, flexible, and reusable.</p><p>Remember that these principles are not hard and fast rules, but rather guidelines to help you design better software. As with any guideline, there may be cases where it makes sense to deviate from the principles in order to achieve a specific goal. However, in general, following the SOLID principles will help you write cleaner, more maintainable, and more extensible code.</p><p>To summarize, the SOLID principles can be summed up as:</p><ul><li>SRP: A class should have only one sole purpose.</li><li>OCP: Software entities should be open for extension but closed for modification.</li><li>LSP: Subtypes should be substitutable for their base types.</li><li>ISP: Clients should not be forced to depend on interfaces they do not use.</li><li>DIP: High-level modules should not depend on low-level modules. Both should depend on abstractions.</li></ul><p>By applying these principles, you can improve the quality of your code and make it easier to maintain and extend over time. Remember that good design is an ongoing process, and it’s important to continually evaluate and refactor your code as necessary.</p><p>As a final note, it’s worth mentioning that the SOLID principles are just one aspect of good software design. There are many other design principles and best practices that you can apply to your code to make it more effective and maintainable. However, by following the SOLID principles, you can establish a solid foundation for your software and set yourself up for success in the long run.</p><p>If you want to learn more about the SOLID principles, there are many great resources available online, including books, articles, and videos. You can also explore other design patterns and best practices that can help you write better code.</p><p>I encourage you to continue learning and exploring new ideas and techniques for improving your software design. By investing in your skills and knowledge, you can become a better, more effective software developer and contribute to the success of your team and organization.</p><p>Thank you for reading, and happy coding!</p><pre>Want to Connect? If you have any feedback, <br>please ping me on my LinkedIn: https://linkedin.com/in/shuhanmirza/</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Zq8bhbPpS7XsTng2L8QF1A.png" /><figcaption><a href="https://levelup.gitconnected.com/solid-principles-simplified-with-illustrations-fe5265f68ec6">Source</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d67324506ecf" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Two simple methods to refactor deeply nested code]]></title>
            <link>https://shuhanmirza.medium.com/two-simple-methods-to-refactor-deeply-nested-code-78eb302bb0b4?source=rss-ddac69adc4d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/78eb302bb0b4</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[refactoring]]></category>
            <category><![CDATA[code-review]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[clean-code]]></category>
            <dc:creator><![CDATA[Shuhan Mirza]]></dc:creator>
            <pubDate>Thu, 05 Jan 2023 05:13:41 GMT</pubDate>
            <atom:updated>2023-01-09T05:27:18.618Z</atom:updated>
            <content:encoded><![CDATA[<h4>If you need more than 3 levels of indentation, you’re screwed!</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/498/1*9wSgwrZipROX1qaoBY2Iow.gif" /><figcaption>Gif by <a href="https://tenor.com/en-GB/users/dkobayahi">dkobayahi</a></figcaption></figure><p>“If you need more than 3 levels of indentation, you’re screwed anyway, and should fix your program.” You can find this line in the <a href="https://www.kernel.org/doc/html/v4.10/process/coding-style.html">official linux kernel coding style</a> by Linus Torvalds.</p><p>Dealing with deeply nested code is like trying to unravel an infinite set of <a href="https://en.wikipedia.org/wiki/Matryoshka_doll">Matryoshka Dolls</a> — it’s a never-ending task. As software engineers, it’s our responsibility to write code that’s easily readable by our teammates. But sometimes, we come across code that’s so deeply nested, it feels like a monster that needs to be defeated. That’s when it’s time to roll up our sleeves and slay the demon of deep nesting by refactoring the code. Trust me, your peers will thank you for making their lives easier.</p><p><strong>How? </strong>I will discuss two techniques for refactoring deeply nested code;</p><ol><li><strong>Extraction</strong></li><li><strong>Inversion</strong></li></ol><h3>Extraction</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qCwGKi4TL57464nzjn4zaQ.png" /><figcaption>Visual representation of code block extraction (Image licensed to author)</figcaption></figure><p>Deeply nested code can often be a sign that a block of code is violating the <em>Single Responsibility </em>principle from <em>SOLID </em>— it’s doing too many things, and it’s tough to understand what’s going on. In these cases, it can be helpful to extract code blocks and turn them into their own functions. This will not only make your code more organized and readable, but it will also make it easier to reuse and maintain in the long run.</p><p><strong>Let’s see an example of extraction:</strong></p><pre>#FIXME: extract the process of finding sum<br>def compute(x: int, y: int) -&gt; int:<br>    if x &gt; y:<br>        sum = 0<br>        for num in range(y, x + 1):<br>            sum += num<br>        return sum<br>    else:<br>        return 0</pre><pre>def process(x: int, y: int) -&gt; int:<br>    sum = 0<br>    for num in range(y, x + 1):<br>        sum += num<br>    return sum<br><br><br>def compute(x: int, y: int) -&gt; int:<br>    if x &gt; y:<br>        return process(x, y) #extracted function<br>    else:<br>        return 0</pre><h3>Inversion</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-LOdtb_iA-vCDB2p5WlCUw.png" /><figcaption>Visual representation of code block inversion (Image licensed to author)</figcaption></figure><p>When we’re coding, it’s easy to get caught up in the “happy path” mentality — we focus on the successful outcome and structure our logic accordingly. But this can lead to deeply nested code blocks that are tough to read and understand. That’s where the <strong>Guard Clause Pattern</strong> (aka the <strong>Bouncer Pattern</strong>) comes in.</p><p>By flipping the progression of our logic and looking for cases where we can <strong>return early</strong>, we can make our code more readable and maintainable. In other words, we’re bouncing out any cases that don’t meet our success criteria and keeping the party (aka our code) moving along smoothly.</p><p><strong>Let’s see an example of code inversion:</strong></p><pre>#FIXME: invert and add a guard clause<br>def compute(x: int, y: int) -&gt; int:<br>    if x &gt; y:<br>        sum = 0<br>        for num in range(y, x + 1):<br>            sum += num<br>        return sum<br>    else:<br>        return 0</pre><pre>def compute(x: int, y: int) -&gt; int:<br>    if x &lt;= y: #guard clause<br>        return 0<br><br>    sum = 0<br>    for num in range(y, x + 1):<br>        sum += num<br>    return sum</pre><p>Now let’s see what happens if we do both the extraction and inversion,</p><pre>def process(x: int, y: int) -&gt; int:<br>    sum = 0<br>    for num in range(y, x + 1):<br>        sum += num<br>    return sum<br><br><br>def compute(x: int, y: int) -&gt; int:<br>    if x &lt;= y:<br>        return 0<br><br>    return process(x, y)</pre><p>Doesn’t it look better? The refactoring process has improved the code’s readability and reduced cyclomatic complexity.</p><h3>Another Example</h3><p>Let’s see another example of deep-nested code and refactor it using above mentioned procedures. I choose <a href="https://onlinejudge.org/external/100/10070.pdf">UVa-10070</a> for illustrating the example, which is a fairly easy problem from <a href="https://onlinejudge.org/">UVa Online Judge</a>.</p><h4>Problem</h4><p>10070 Leap Year or Not Leap Year and …</p><blockquote>The ancient race of Gulamatu is very advanced in their year calculation scheme. They understand what leap year is (A year that is divisible by 4 and not divisible by 100 with the exception that years that are divisible by 400 are also leap year.) and they have also similar festival years. One is the Huluculu festival (happens on years divisible by 15) and the Bulukulu festival (Happens on years divisible by 55 provided that is also a leap year). Given an year you will have to state what properties these years have. If the year is not leap year nor festival year, then print the line ‘This is an ordinary year.’ The order of printing (if present) the properties is: leap year → huluculu → bulukulu.</blockquote><p><strong>Sample Input</strong></p><pre>2000<br>3600<br>4515<br>2001</pre><p><strong>Sample Output</strong></p><pre>This is leap year.<br><br>This is leap year.<br>This is huluculu festival year.<br><br>This is huluculu festival year.<br><br>This is an ordinary year.</pre><h4>Solution #1 [ deep-nested code ]</h4><pre>import sys<br><br>if __name__ == &quot;__main__&quot;:<br><br>    first_input = True<br><br>    for line in sys.stdin:<br>        if not first_input: #FIXME<br>            print()<br>        else:<br>            first_input = False<br><br>        input_year = int(line)<br><br>        is_leap_year = False<br>        is_huluculu_year = False<br>        is_bulukulu_year = False<br><br>        if input_year % 4 == 0: #FIXME<br>            if input_year % 100 == 0:<br>                if input_year % 400 == 0:<br>                    is_leap_year = True<br><br>                    if input_year % 55 == 0: #FIXME<br>                        is_bulukulu_year = True<br>            else:<br>                is_leap_year = True<br><br>                if input_year % 55 == 0: #FIXME<br>                    is_bulukulu_year = True<br><br>        if input_year % 15 == 0: #FIXME<br>            is_huluculu_year = True<br><br>        if is_leap_year:<br>            print(&quot;This is leap year.&quot;)<br>        if is_huluculu_year:<br>            print(&quot;This is huluculu festival year.&quot;)<br>        if is_bulukulu_year:<br>            print(&quot;This is bulukulu festival year.&quot;)<br><br>        if not is_leap_year and not is_huluculu_year and not is_bulukulu_year:<br>            print(&quot;This is an ordinary year.&quot;)</pre><p>The above code has been written dangerously deep-nested deliberately.</p><h4>Solution #2 [ Refactored ]</h4><pre>import sys<br><br>FLAG_FIRST_INPUT = True<br><br><br>def print_blank_line_if_not_first_input():<br>    global FLAG_FIRST_INPUT<br><br>    if FLAG_FIRST_INPUT:<br>        FLAG_FIRST_INPUT = False<br>        return<br><br>    print()<br><br><br>def check_leap_year(year: int) -&gt; bool:<br>    if year % 400 == 0:<br>        return True<br>    if year % 100 == 0:<br>        return False<br>    if year % 4 == 0:<br>        return True<br><br>    return False<br><br><br>def check_huluculu_year(year: int) -&gt; bool:<br>    if year % 15 == 0:<br>        return True<br><br>    return False<br><br><br>def check_bulukulu_year(is_leap_year: bool, year: int) -&gt; bool:<br>    if is_leap_year and year % 55 == 0:<br>        return True<br><br>    return False<br><br><br>def process_input(year: int):<br>    is_leap_year = check_leap_year(year)<br>    is_huluculu_year = check_huluculu_year(year)<br>    is_bulukulu_year = check_bulukulu_year(is_leap_year, year)<br><br>    if is_leap_year:<br>        print(&quot;This is leap year.&quot;)<br>    if is_huluculu_year:<br>        print(&quot;This is huluculu festival year.&quot;)<br>    if is_bulukulu_year:<br>        print(&quot;This is bulukulu festival year.&quot;)<br><br>    if not is_leap_year and not is_huluculu_year and not is_bulukulu_year:<br>        print(&quot;This is an ordinary year.&quot;)<br><br><br>if __name__ == &quot;__main__&quot;:<br><br>    for line in sys.stdin:<br>        print_blank_line_if_not_first_input()<br>        process_input(int(line))</pre><p>I refactored <em>solution#1</em> by code extracting and code inversion. If you think <em>solution#2 </em>can be written in a more readable way, please feel free to comment.</p><pre>Want to Connect?<br>If you have any feedback, please ping me on my <br><br>LinkedIn: https://linkedin.com/in/shuhanmirza/</pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=78eb302bb0b4" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How have we made our campus online radio application system (SUSTCast) horizontally scalable?]]></title>
            <link>https://shuhanmirza.medium.com/how-have-we-made-our-campus-online-radio-application-system-sustcast-horizontally-scalable-5d236d73cb48?source=rss-ddac69adc4d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/5d236d73cb48</guid>
            <category><![CDATA[system-architecture]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[horizontal-scaling]]></category>
            <category><![CDATA[media-streaming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Shuhan Mirza]]></dc:creator>
            <pubDate>Sun, 18 Sep 2022 05:44:00 GMT</pubDate>
            <atom:updated>2022-10-11T06:59:10.621Z</atom:updated>
            <content:encoded><![CDATA[<h4>A brief system design case study</h4><h3>How we scaled our campus online radio application system (SUSTCast)?</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/607/1*T7ysnnVkQ8yX95uFXbn3IQ.gif" /><figcaption>Image Source: <a href="https://www.esds.co.in/blog/wp-content/uploads/2015/11/Horizontal-scaling.gif">https://www.esds.co.in/</a></figcaption></figure><p>Creating a horizontally scalable system is probably the most satisfying thing for a system architect.</p><h3>Preface</h3><p><a href="https://github.com/sustcast">SUSTCast</a> is the online radio station of <a href="https://sust.edu/">Shahjalal University of Science and Technology</a>. It has a mobile application where users can log in and listen to the broadcasted programs. Pretty simple, right?</p><h4>Requirements</h4><ol><li>A Mobile Application<br>- Sign Up, Login<br>- Playing radio stream<br>- Showing notification (About upcoming featured program)</li></ol><p>2. 10,000 listeners in peak</p><h4>Constraints</h4><ol><li>No Funding, the system must be cost efficient</li></ol><h3>System Design</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Nc9bv_KvTUpHhc_6aERf_A.png" /><figcaption>System Architecture of SUSTCast</figcaption></figure><h4>Mobile Application</h4><p>The application was developed using a native android java framework. We incorporated the MVVM (Model-View-ViewModel) Design Pattern.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m8aHqXIOOtEu2iofNu7tYg.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y4z7nabjUuesQ3cYgJUxbQ.jpeg" /><figcaption>Mobile Application Layout</figcaption></figure><h4>Cloud Service</h4><p>Firebase was used for authenticating users, sending push notifications, and storing global configuration data for mobile applications.</p><h4>Streaming Service</h4><p><a href="https://icecast.org/">IceCast</a> is an excellent streaming service for audio streaming. We had an array of Icecast servers running on low-resource computers (both desktop PC and cloud) which were accessible on the public network. We deployed these servers forecasting bandwidth loads.</p><h4>Core Service</h4><p>The core service was a set of lightweight modules. It had a database, a script, and an ices2 (IceCast client) running in a docker container. The script’s responsibility was to stream the program to the IceCast servers using ices2. Furthermore, the core script updated the firebase real-time database with the metadata of the program being streamed.</p><h3>Why this design is horizontally scalable?</h3><p>Because, when the number of listeners increases it puts the load on two resources, cloud service (notification/database/auth) and streaming service. We have decoupled these services from the core service. Furthermore, the core service could be run from anywhere with an internet connection. Whenever we faced a spike in users, we just had to spawn some Icecast servers (AWS/Hetzner). Our mobile application would get the address of these servers from the firebase database and play the programs accordingly.</p><p>Special thanks to my teammate <a href="https://medium.com/@shihabistiak">Istiak Shihab</a> and <a href="https://medium.com/@abrar.fahim">Abrar Fahim</a> for helping me with this article. Also, kudos to all current members working for <a href="https://www.facebook.com/sustcast">SUSTCast</a>.</p><pre>Want to Connect?<br>If you have any feedback, please ping me on my LinkedIn: <a href="https://linkedin.com/in/shuhan-mirza/">https://linkedin.com/in/shuhan-mirza/</a></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5d236d73cb48" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Why Token-based Pagination Performs Better Than Offset Based?]]></title>
            <link>https://medium.com/better-programming/why-token-based-pagination-performs-better-than-offset-based-465e1139bb33?source=rss-ddac69adc4d9------2</link>
            <guid isPermaLink="false">https://medium.com/p/465e1139bb33</guid>
            <category><![CDATA[api]]></category>
            <category><![CDATA[database]]></category>
            <category><![CDATA[web-development]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-engineering]]></category>
            <dc:creator><![CDATA[Shuhan Mirza]]></dc:creator>
            <pubDate>Wed, 07 Sep 2022 12:06:37 GMT</pubDate>
            <atom:updated>2023-01-05T05:15:07.571Z</atom:updated>
            <content:encoded><![CDATA[<h4>It’s almost 1.5x faster</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xNZHrbeCeI2rTPWF0vau3A.gif" /><figcaption>Image licensed by author. From Canva</figcaption></figure><p>Almost every modern digital product has to incorporate some sort of pagination API. The main role of that API is to retrieve data from a large table chunk by chunk so that your Frontend application does not need to make an API call that has a huge payload. Smaller payload results in less latency.</p><h3>Offset based pagination</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tjqTIRK-a8tzMbeayS0P5w.png" /></figure><p>A commonly found approach for developing a pagination API is offset-based. Here, the Frontend requested <em>10 rows</em> after the <em>100th</em> row of the <em>products</em> table. This is harmless when the size of the table is small. However, latency increases with the increasing value of the<em> offset.</em></p><p><strong>Why?</strong> Because the database (not all kinds of database) actually prepares offset+limit numbers of rows for this query. So, when the offset value becomes larger, the database has to go through offset+limit numbers of rows.</p><p><strong>Let’s do a quick experiment.</strong> I have prepared a table named <em>products</em> with 110001 rows in a PostgreSQL database. It has 3 columns; namely id , name , andcreated_at .</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/67c15e1cc300491b8d5c2536880ac77f/href">https://medium.com/media/67c15e1cc300491b8d5c2536880ac77f/href</a></iframe><p>Now let’s run this query,</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4e0deafc78367956b85d91cb81d90e1b/href">https://medium.com/media/4e0deafc78367956b85d91cb81d90e1b/href</a></iframe><p>Database Response:</p><pre>Limit  (cost=1.74..1.91 rows=10 width=27) (actual time=0.552..0.554 rows=10 loops=1)<br>  -&gt;  Seq Scan on products  (cost=0.00..1909.01 rows=110001 width=27) (actual time=0.541..0.548 rows=110 loops=1)<br>Planning Time: 1.231 ms<br>Execution Time: 0.580 ms</pre><p>Here, we can see the database has processed <em>110 rows</em> and then returned <em>10 rows</em>. Now, let’s throttle this by increasing the offset number to <em>90000</em>.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/240b7a51f7e8584953f92b9cd6ffbdda/href">https://medium.com/media/240b7a51f7e8584953f92b9cd6ffbdda/href</a></iframe><pre>Limit  (cost=1561.90..1562.08 rows=10 width=27) (actual time=10.275..10.277 rows=10 loops=1)<br>  -&gt;  Seq Scan on products  (cost=0.00..1909.01 rows=110001 width=27) (actual time=0.425..6.666 rows=90010 loops=1)<br>Planning Time: 1.070 ms<br>Execution Time: 10.302 ms</pre><p>Now, you can see it has read <em>90010 rows</em> for returning only <em>10 rows</em>! Execution time also takes a significant amount of 10.302ms .</p><h3>Token based pagination</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nPb6lQeTBrvBbCDDooQjFA.png" /></figure><p>In this type of pagination, the API sends the consumer an next_token on every page. This token is used for the next API call and so on. A column or property is used to pivot through the data in the table. You can choose any column that fits the requirement. The main idea is that you have to come up with <em>conditions and parameters</em> that will help you navigate through the table. A common practice is to choose row creation time as a token if the creation time is unique. In the above diagram, created_at column of products table has been used as the token. You can see the API has sent the created_at of the <em>111th </em>row as the next_token along with 10 products as result. However, token-based pagination requires much less latency if the table gets bigger.</p><p><strong>Why?</strong> Because this query filters through the table using the condition in the query and starts scanning from the row where the condition is matched. In short, it only reads limit numbers of rows.</p><p><strong>Let’s do some experiments</strong> on the products table created earlier. We will query for the same results that were done in earlier experiments. Note that, proper indexing is will result in even better performance. I kept things simple.</p><p>let’s run this query for the <em>100th</em> to <em>110th</em> row,</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/aab3167159696b3b7f5776b2f9752a88/href">https://medium.com/media/aab3167159696b3b7f5776b2f9752a88/href</a></iframe><p>Here, we can already see it has less execution time than before. The database has filtered out and only calculated <em>10 rows </em>instead of <em>110 rows</em>.</p><p>Now, let’s throttle the query with a larger value of the token. This query targets <em>90000 to 90010th</em> rows.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/d8e3a1b15d097f7f6c531588845a62ba/href">https://medium.com/media/d8e3a1b15d097f7f6c531588845a62ba/href</a></iframe><pre>Limit  (cost=0.00..1.11 rows=10 width=27) (actual time=6.587..6.589 rows=10 loops=1)<br>  -&gt;  Seq Scan on products  (cost=0.00..2184.01 rows=19735 width=27) (actual time=6.586..6.587 rows=10 loops=1)<br>        &quot;Filter: (created_at &gt;= &#39;1662535267497&#39;::bigint)&quot;<br>        Rows Removed by Filter: 90000<br>Planning Time: 1.314 ms<br>Execution Time: 6.617 ms</pre><p>Did you notice the big difference? The query took only <em>6.617ms</em> for execution. The database did not have to go through <em>90000</em> rows, instead, it filtered through the table and only read the <em>10</em> required rows.</p><h3>Summary</h3><p>Database query for Offset based pagination has to read offset+limit number of rows, on the other hand, token-based query only requires to read limit numbers of rows with additional processing for the condition.</p><pre>Want to Connect?</pre><pre>If you have any feedback, please ping me on my LinkedIn: <a href="https://linkedin.com/in/shuhan-mirza/">https://linkedin.com/in/shuhanmirza/</a></pre><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=465e1139bb33" width="1" height="1" alt=""><hr><p><a href="https://medium.com/better-programming/why-token-based-pagination-performs-better-than-offset-based-465e1139bb33">Why Token-based Pagination Performs Better Than Offset Based?</a> was originally published in <a href="https://betterprogramming.pub/">Better Programming</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>