<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Anoop Nayak on Medium]]></title>
        <description><![CDATA[Stories by Anoop Nayak on Medium]]></description>
        <link>https://medium.com/@anoopnayak1?source=rss-1c919ece5cb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*gmqRYviHg1ySFn9dLMvx0Q.jpeg</url>
            <title>Stories by Anoop Nayak on Medium</title>
            <link>https://medium.com/@anoopnayak1?source=rss-1c919ece5cb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Thu, 19 Nov 2026 06:17:02 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@anoopnayak1/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[From Zero to Backend Hero — Part 1]]></title>
            <link>https://blog.stackademic.com/from-zero-to-backend-hero-part-1-c615c2f27a70?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/c615c2f27a70</guid>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[nodejs]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Fri, 06 Mar 2026 07:05:59 GMT</pubDate>
            <atom:updated>2026-03-06T08:48:11.430Z</atom:updated>
            <content:encoded><![CDATA[<p>From “What is an API?” to “Don’t Touch My Production Server”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gmFxUJmGh4gD1-9jRKX7Aw.png" /></figure><h3>Introduction</h3><p>Backend development is the <strong>invisible brain of a website or app</strong>.<br>You don’t see it, but without it <strong>nothing works</strong>.</p><p>It runs on the server and takes care of things like:</p><ul><li>saving data</li><li>processing requests</li><li>keeping things secure</li><li>talking to the database</li></ul><p>Think of it like a <strong>restaurant kitchen</strong> 🍔.</p><p>You only see the <strong>waiter and the menu (frontend)</strong>.<br> But the <strong>real magic happens in the kitchen (backend)</strong>.</p><p>You order a burger → the waiter sends the order to the kitchen → the chef prepares it → the waiter brings it back to you.</p><p>That <strong>kitchen system</strong> is the backend.</p><h3>Example</h3><p>Let’s say you log in to <strong>Instagram</strong>.</p><ol><li>You enter your <strong>username and password</strong>.</li><li>The <strong>backend checks the database</strong> to see if your account exists.</li><li>If it matches → it logs you in.</li><li>If not → it says <strong>“Wrong password.”</strong></li></ol><p>So basically:</p><p><strong>Frontend:</strong> What users see 👀<br><strong>Backend:</strong> The guy behind the curtain doing all the hard work 🧠</p><h3>Learn a Language</h3><p>Before becoming a backend developer, you need to <strong>pick a programming language</strong>.<br> Think of it like choosing the <strong>language you’ll talk to the computer in</strong>.</p><p>In web development there are two sides:</p><ul><li><strong>Frontend</strong> → What users see (HTML, CSS, JavaScript)</li><li><strong>Backend</strong> → The logic running on the server</li></ul><p>Backend developers usually work with languages like <strong>Python, Java, Node.js (JavaScript), Go, PHP, Rust</strong>.</p><p>These languages help you:</p><ul><li>create APIs</li><li>process user requests</li><li>talk to databases</li><li>handle authentication and security</li></ul><p>But don’t worry, you <strong>don’t need to learn all languages</strong>.<br>Just <strong>pick one and get really comfortable with it</strong>.</p><h3>Example</h3><p>Imagine you’re building a <strong>food ordering app</strong> 🍕</p><p>A user clicks <strong>“Order Pizza”</strong> on the app.</p><p>Your backend language will:</p><ol><li>Receive the request</li><li>Check the database for the pizza details</li><li>Calculate the price</li><li>Send the response back to the user</li></ol><p>So basically:</p><p><strong>Frontend:</strong> “I want pizza.”<br><strong>Backend language:</strong> “Alright, let me check the kitchen and get it for you.” 🍕</p><blockquote>Start with <strong>one language</strong>, build small projects, and slowly learn the tools around it like frameworks and databases.</blockquote><h3>How the Internet Works?</h3><p>The internet is basically a <strong>huge network of computers connected to each other</strong> 🌍.<br>All these computers talk to each other using common rules called <strong>protocols (like TCP/IP)</strong>.</p><p>When you open a website, a lot of things happen <strong>in a few milliseconds</strong>.</p><h3>Step by Step</h3><ol><li>You type a website in your browser<br>Example: <strong>google.com</strong></li><li>Your computer asks <strong>DNS</strong><br>DNS is like the <strong>internet’s phonebook</strong> 📖.<br>It converts the name <strong>google.com</strong> into an <strong>IP address</strong> (something like 142.250.x.x).</li><li>The request travels through your <strong>ISP (Internet Service Provider)</strong><br>Companies like Airtel, Jio, or ACT send your request to the internet.</li><li>The request passes through <strong>multiple routers</strong><br>Routers act like <strong>traffic managers</strong> that decide the best path for your request.</li><li>Finally, the request reaches the <strong>server</strong> where the website lives.</li><li>The server sends the response back (HTML, images, data, etc.), and your <strong>browser shows the website</strong>.</li></ol><h3>Example</h3><p>Imagine ordering food online 🍔</p><ol><li>You open <strong>Swiggy</strong> and order a burger.</li><li>The app finds the <strong>restaurant location</strong> (like DNS finding the IP).</li><li>The order travels through delivery routes (routers).</li><li>The restaurant prepares the burger (server processing).</li><li>The delivery partner brings it to you (response back to your browser).</li></ol><p>So in short:</p><p><strong>You request a website → the internet finds the server → the server sends the data → you see the website.</strong></p><p>All of this happens <strong>in less than a second</strong> ⚡.</p><h3>What is HTTP?</h3><p><strong>HTTP (Hypertext Transfer Protocol)</strong> is the rule that allows your <strong>browser and a server to talk to each other</strong> on the internet.</p><p>It works using a <strong>request → response model</strong>.</p><ul><li><strong>Request:</strong> Your browser asks for something</li><li><strong>Response:</strong> The server sends it back</li></ul><p>Every time you open a website, your browser sends an <strong>HTTP request</strong>, and the server replies with an <strong>HTTP response</strong>.</p><p>Another important thing: <strong>HTTP is stateless</strong>.<br>This simply means the server <strong>does not remember previous requests</strong>. Every request is treated like a <strong>brand-new conversation</strong>.</p><h3>Example</h3><p>Imagine you go to a <strong>coffee shop</strong> ☕</p><ol><li>You say: <strong>“Can I get a cappuccino?”</strong> → This is the <strong>HTTP request</strong></li><li>The barista makes it and gives it to you → This is the <strong>HTTP response</strong></li></ol><p>If you want another coffee, you have to <strong>ask again</strong>, because the barista doesn’t remember your previous order.</p><p>That’s exactly how HTTP works.</p><h3>HTTP vs HTTPS</h3><p>Sometimes you see <strong>HTTPS</strong> instead of HTTP.</p><p>The <strong>S stands for Secure</strong> 🔒.<br>It means the data between your browser and the server is <strong>encrypted</strong>, so hackers can’t easily read it.</p><p>So basically:</p><p><strong>HTTP:</strong> Browser and server talking<br><strong>HTTPS:</strong> Browser and server talking <strong>securely</strong>.</p><h3>What is a Domain Name?</h3><p>A <strong>domain name</strong> is the <strong>easy-to-read address of a website</strong> on the internet.</p><p>Computers actually find websites using <strong>IP addresses</strong> (numbers like 142.250.190.78).<br>But remembering numbers for every website would be a nightmare 😅.</p><p>So instead, we use <strong>domain names</strong> like:</p><ul><li>google.com</li><li>youtube.com</li><li>amazon.in</li></ul><p>These names are easier for humans to remember.</p><p>A domain name usually has two main parts:</p><ul><li><strong>Second-level domain</strong> → the main name (example: google)</li><li><strong>Top-level domain (TLD)</strong> → the ending like .com, .org, .in</li></ul><p>These domains are registered through <strong>domain registrars</strong> like GoDaddy, Namecheap, etc.</p><h3>Example</h3><p>Think of it like <strong>saving contacts in your phone</strong> 📱.</p><p>Instead of remembering your friend’s phone number like 9876543210, you save it as <strong>“Anoop”</strong>.</p><p>So when you call <strong>Anoop</strong>, your phone automatically dials the number.</p><p>Same thing happens on the internet:</p><p><strong>Domain name:</strong> google.com<br><strong>Actual address:</strong> 142.250.x.x</p><p>Your browser asks DNS to convert the <strong>domain name → IP address</strong>, and then connects to the server.</p><p>So basically:</p><blockquote><strong>Domain name = easy nickname for a website’s IP address.</strong> 🌐</blockquote><h3>What is Hosting?</h3><p><strong>Hosting</strong> is the place where your <strong>website lives on the internet</strong>.</p><p>A website has files like <strong>HTML, images, videos, and code</strong>. These files need to be stored on a <strong>server (a powerful computer)</strong> so people can access your website anytime.</p><p>Hosting companies provide this <strong>server space and resources</strong> so your website stays <strong>online 24/7</strong>.</p><p>There are different types of hosting:</p><ul><li><strong>Shared Hosting</strong> → Many websites share the same server (cheapest)</li><li><strong>VPS Hosting</strong> → You get a dedicated portion of a server</li><li><strong>Dedicated Hosting</strong> → One entire server only for your website</li><li><strong>Cloud Hosting</strong> → Your website runs on multiple servers for better scalability</li></ul><p>Hosting services usually also provide things like:</p><ul><li>security</li><li>backups</li><li>domain connection</li><li>technical support</li></ul><h3>Example</h3><p>Imagine you open a <strong>restaurant</strong> 🍽️.</p><ul><li>Your <strong>website code</strong> = the food recipes</li><li><strong>Domain name</strong> = the restaurant name</li><li><strong>Hosting</strong> = the building where the restaurant runs</li></ul><p>Without the building, customers <strong>have nowhere to go</strong>.</p><p>Similarly, without hosting, your website <strong>cannot be accessed on the internet</strong>.</p><p>So in simple words:</p><p><strong>Hosting = the home where your website lives on the internet.</strong> 🌐</p><h3>What is DNS?</h3><p><strong>DNS (Domain Name System)</strong> is the system that converts a <strong>website name into an IP address</strong> so computers can find the correct server.</p><p>Humans prefer names like <strong>google.com</strong>, but computers understand <strong>numbers (IP addresses)</strong>.<br>DNS acts like a <strong>translator between humans and computers</strong>.</p><p>When you type a website in your browser, DNS helps your computer find <strong>where that website actually lives on the internet</strong>.</p><h3>How it Works (Simple Steps)</h3><ol><li>You type a domain like <strong>google.com</strong> in your browser.</li><li>Your computer asks a <strong>DNS server</strong>: “What is the IP address for this website?”</li><li>DNS finds the correct <strong>IP address</strong>.</li><li>Your browser connects to that server and loads the website.</li></ol><p>Behind the scenes, DNS works in levels like:</p><ul><li><strong>Root servers</strong></li><li><strong>TLD servers</strong> (.com, .org, .in)</li><li><strong>Authoritative servers</strong> (final server that knows the real IP)</li></ul><p>But usually all this happens <strong>in milliseconds</strong>.</p><h3>Example</h3><p>Think of DNS like a <strong>contact list on your phone</strong> 📱.</p><p>You don’t remember your friend’s phone number.<br>You just tap <strong>“Anoop”</strong> in your contacts.</p><p>Your phone automatically finds the <strong>actual number</strong> and calls it.</p><p>Same thing happens on the internet:</p><p><strong>google.com → DNS → IP address → website loads</strong></p><p>So simply put:</p><blockquote><strong>DNS = the internet’s phonebook that helps find websites.</strong> 🌐</blockquote><h3>What is a Browser?</h3><p>A <strong>web browser</strong> is the application you use to <strong>open and view websites on the internet</strong>.</p><p>Some popular browsers are:</p><ul><li><strong>Google Chrome</strong></li><li><strong>Safari</strong></li><li><strong>Firefox</strong></li><li><strong>Microsoft Edge</strong></li></ul><p>When you open a website, the browser <strong>downloads the website files</strong> (HTML, CSS, JavaScript) and <strong>converts them into the page you see on your screen</strong>.</p><p>Behind the scenes, browsers use special components like:</p><ul><li><strong>Rendering engines</strong> (Blink, Gecko, WebKit) → to display the webpage layout</li><li><strong>JavaScript engines</strong> → to run JavaScript code and make websites interactive</li></ul><p>Modern browsers also include features like:</p><ul><li><strong>Tabs</strong> (open multiple websites at once)</li><li><strong>Bookmarks</strong> (save favorite websites)</li><li><strong>Extensions</strong> (extra tools like ad blockers)</li><li><strong>Security features</strong> like HTTPS and sandboxing</li></ul><h3>Example</h3><p>Imagine a website is like a <strong>recipe</strong> 🧑‍🍳.</p><ul><li><strong>HTML</strong> = ingredients</li><li><strong>CSS</strong> = presentation of the dish</li><li><strong>JavaScript</strong> = special actions (like animations or interactions)</li></ul><p>The <strong>browser is the chef</strong> that reads the recipe and prepares the final dish for you.</p><p>Without the browser, all those website files would just look like <strong>random code</strong>.</p><p>So in simple terms:</p><p><strong>Browser = the tool that reads website code and shows it as a beautiful webpage.</strong> 🌐</p><h3>Version Control Systems (VCS)</h3><p>A <strong>Version Control System (VCS)</strong> is a tool that helps developers <strong>track and manage changes in code over time</strong>.</p><p>When building software, code changes frequently. VCS keeps a <strong>history of all those changes</strong>, so you can:</p><ul><li>see what changed</li><li>go back to an older version</li><li>work with other developers without messing up the code</li></ul><p>It is extremely useful for <strong>team collaboration</strong>, because multiple developers can work on the same project safely.</p><p>There are two common types of version control:</p><ul><li><strong>Centralized VCS</strong> → One main server stores the code (example: <strong>Subversion</strong>)</li><li><strong>Distributed VCS</strong> → Every developer has a full copy of the project (example: <strong>Git</strong>, <strong>Mercurial</strong>)</li></ul><p>Today, <strong>Git</strong> is the most widely used version control system.</p><h3>Example</h3><p>Imagine you are writing a <strong>college assignment in a Word document</strong> 📄.</p><p>You keep saving versions like:</p><ul><li>assignment_v1</li><li>assignment_v2</li><li>assignment_final</li><li>assignment_final_realfinal</li></ul><p>It quickly becomes confusing 😅.</p><p>A Version Control System solves this problem by:</p><ul><li>automatically saving every change</li><li>keeping a proper history</li><li>allowing you to go back to any previous version</li></ul><p>So instead of messy file names, you simply <strong>commit changes</strong> and VCS tracks everything.</p><p>In simple words:</p><blockquote><strong>Version Control = a time machine for your code.</strong> ⏳💻</blockquote><h3>Repo Hosting Services</h3><p><strong>Repo hosting services</strong> are platforms where developers <strong>store and manage their code repositories online</strong>.</p><p>When you use version control tools like <strong>Git</strong>, your code is stored in something called a <strong>repository (repo)</strong>. Repo hosting services keep these repositories on the internet so developers can <strong>access them from anywhere and collaborate with others</strong>.</p><p>These platforms support version control systems like:</p><ul><li><strong>Git</strong></li><li><strong>Mercurial</strong></li><li><strong>Subversion</strong></li></ul><p>They also provide helpful features like:</p><ul><li><strong>Branching</strong> → work on new features without affecting the main code</li><li><strong>Pull Requests</strong> → review and merge code changes</li><li><strong>Issue Tracking</strong> → track bugs and tasks</li><li><strong>Code Reviews</strong> → teammates can check code before merging</li><li><strong>CI/CD Integration</strong> → automatically build and deploy applications</li></ul><p>Some popular repo hosting services are:</p><ul><li><strong>GitHub</strong></li><li><strong>GitLab</strong></li><li><strong>Bitbucket</strong></li></ul><h3>Example</h3><p>Imagine you and your friends are working on a <strong>group project</strong> 📚.</p><p>Instead of sending files like:</p><ul><li>project_v1.zip</li><li>project_updated_final.zip</li><li>project_final_real_final.zip 😅</li></ul><p>You upload everything to a <strong>shared online workspace</strong> where everyone can:</p><ul><li>see the latest version</li><li>add changes</li><li>review updates</li></ul><p>That shared workspace is what <strong>repo hosting services</strong> provide for developers.</p><p>So in simple terms:</p><p><strong>Repo hosting = an online home where developers store, share, and collaborate on code.</strong> 💻🌐</p><h3>Conclusion (Part 1)</h3><p>And that’s it for <strong>Part 1</strong> 🎉</p><p>We covered the <strong>basic building blocks of the internet and backend development</strong> — things like how the internet works, HTTP, DNS, browsers, version control, hosting, and more. These concepts form the <strong>foundation</strong> for becoming a backend developer.</p><p>If these basics are clear, learning the advanced backend topics becomes <strong>much easier</strong>.</p><p>Also… after giving <strong>so many food examples (pizza, burgers, coffee, restaurants)</strong> while explaining everything… <strong>I’m honestly getting hungry now</strong> 🍕🍔😄.</p><p>In <strong>Part 2</strong>, we’ll dive into the real backend stuff:</p><ul><li><strong>Databases</strong> — where your app stores data</li><li><strong>APIs</strong> — how applications talk to each other</li><li><strong>Authentication &amp; Authorization</strong> — how apps manage users and security</li></ul><p>So grab some food if you’re hungry too 😅… and in the <strong>next part we’ll start cooking the real backend magic</strong> 🚀.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c615c2f27a70" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/from-zero-to-backend-hero-part-1-c615c2f27a70">From Zero to Backend Hero — Part 1</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Boosting React/NextJs Performance with useFreshRef and useFreshTick Custom Hooks]]></title>
            <link>https://blog.stackademic.com/boosting-react-nextjs-performance-with-usefreshref-and-usefreshtick-custom-hooks-d82fe103631f?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/d82fe103631f</guid>
            <category><![CDATA[react]]></category>
            <category><![CDATA[frontend]]></category>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[hooks]]></category>
            <category><![CDATA[reactjs]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Mon, 08 Apr 2024 09:00:01 GMT</pubDate>
            <atom:updated>2024-04-10T09:57:28.463Z</atom:updated>
            <content:encoded><![CDATA[<p>In the world of React development, performance optimization is a crucial aspect. As applications grow in complexity, developers often find themselves battling against rendering inefficiencies and unnecessary re-renders. Fortunately, React offers various tools and techniques to address these challenges. Among them, custom hooks like useFreshRef and useFreshTick emerge as powerful allies, helping developers streamline their code and enhance performance.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MxMQBZaemRZLFIFKsESGBg.png" /></figure><h3>Understanding useFreshRef and useFreshTick</h3><p>Before delving into their utility, let’s grasp the essence of these custom hooks:</p><h3>useFreshRef</h3><p>— This hook facilitates the creation of a mutable reference that persists across re-renders without causing component updates.<br> — By using useRef under the hood, it ensures that the reference remains unchanged unless explicitly modified, preventing unnecessary renders triggered by changes in its value.</p><h3>useFreshTick</h3><p>— Unlike useEffect, which depends on its dependency array to execute, useFreshTick triggers an effect on every render, ensuring the execution of the effect without any dependencies.<br> — It’s particularly useful when developers need to perform actions on each render cycle, such as imperative DOM manipulations or state updates.</p><h3>Custom hook Implementation:</h3><pre>import { useRef, useEffect } from &#39;react&#39;;<br><br>// Custom hook: useFreshRef<br>function useFreshRef(initialValue) {<br>  const ref = useRef(initialValue);<br>  return ref;<br>}<br><br>// Custom hook: useFreshTick<br>function useFreshTick(callback) {<br>  useEffect(() =&gt; {<br>    // Execute the callback on every render<br>    callback();<br>  });<br>}</pre><h3>Example Usage of useFreshRef:</h3><pre>import React, { useRef } from &#39;react&#39;;<br><br>function MyComponent() {<br>  // Using useFreshRef to create a mutable reference<br>  const dataRef = useFreshRef({ value: 0 });<br><br>  const handleClick = () =&gt; {<br>    // Modifying the value without triggering re-renders<br>    dataRef.current.value += 1;<br>    console.log(&#39;Updated value:&#39;, dataRef.current.value);<br>  };<br><br>  return (<br>    &lt;div&gt;<br>      &lt;h2&gt;Value: {dataRef.current.value}&lt;/h2&gt;<br>      &lt;button onClick={handleClick}&gt;Increment Value&lt;/button&gt;<br>    &lt;/div&gt;<br>  );<br>}</pre><h3>Example Usage of useFreshTick:</h3><pre>import React, { useEffect } from &#39;react&#39;;<br><br>function MyComponent() {<br>  // Using useFreshTick to trigger an effect on every render<br>  useFreshTick(() =&gt; {<br>    // Perform side effects or state updates<br>    console.log(&#39;Component rendered!&#39;);<br>  });<br><br>  return (<br>    &lt;div&gt;<br>      &lt;h2&gt;Hello, World!&lt;/h2&gt;<br>    &lt;/div&gt;<br>  );<br>}</pre><h3>You may have a question now, Why Not Use useRef Directly?</h3><h4><strong>Stale Closures Risk</strong></h4><p>Directly using useRef within event handlers or effects can lead to capturing outdated values in closures, causing unexpected behavior.</p><h4><strong>Avoid Unnecessary Rerenders</strong></h4><p>Changes to a useRef object can trigger unnecessary component rerenders, even if those changes don&#39;t affect the UI directly.</p><h4><strong>Clarity and Intent</strong></h4><p>Opting for useFreshRef communicates clearly the intention to maintain a fresh reference across renders, enhancing code readability.</p><h4><strong>Consistent Behavior</strong></h4><p>useFreshRef abstracts away complexities, ensuring consistent behavior across components and reducing the risk of bugs.</p><h3>Why and When to Use Them?</h3><ol><li><strong>Preventing Unnecessary Rerenders:</strong><br> — React components often re-render even when the data they depend on hasn’t changed. By using useFreshRef, developers can store values that don’t trigger re-renders unnecessarily, thus optimizing performance.</li><li><strong>Fine-grained Control Over Side Effects:</strong><br> — Traditional useEffect hooks might trigger side effects on every render cycle or based on specific dependencies. However, useFreshTick offers more granular control, executing side effects consistently on each render, regardless of dependencies.</li><li><strong>Managing Performance-Critical Operations:</strong><br> — In scenarios where performance is paramount, such as animations or real-time data updates, these hooks provide a means to optimize rendering and ensure smoother user experiences.</li></ol><h3>How They Enhance Performance</h3><p><strong>1. Minimizing Re-renders:</strong><br> — By utilizing useFreshRef, components can avoid unnecessary re-renders triggered by changes in values that don’t affect the UI. This optimization leads to a leaner, more efficient rendering process.</p><p><strong>2. Efficient Side Effect Management:</strong><br> — useFreshTick enables developers to manage side effects more efficiently by executing them consistently on each render. This approach eliminates the need for complex dependency arrays, resulting in cleaner and more predictable code.</p><p><strong>3. Optimizing Resource Utilization:</strong><br> — With better control over when and how components update, developers can optimize the utilization of system resources, leading to faster rendering and improved overall performance.</p><h3>Conclusion</h3><p>In the quest for optimal React performance, custom hooks like useFreshRef and useFreshTick emerge as indispensable tools. By offering granular control over rendering and side effects, they empower developers to build highly efficient and responsive applications. Whether it’s minimizing unnecessary re-renders or fine-tuning performance-critical operations, these hooks pave the way for smoother user experiences and more maintainable codebases. Embracing these performance-enhancing techniques not only elevates the quality of React applications but also underscores the importance of continuous optimization in modern web development.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d82fe103631f" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/boosting-react-nextjs-performance-with-usefreshref-and-usefreshtick-custom-hooks-d82fe103631f">Boosting React/NextJs Performance with useFreshRef and useFreshTick Custom Hooks</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering System Design: Caching (Part 5)]]></title>
            <link>https://blog.stackademic.com/mastering-system-design-caching-part-5-7ea1b4776eeb?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/7ea1b4776eeb</guid>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[performance]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Tue, 02 Apr 2024 06:09:16 GMT</pubDate>
            <atom:updated>2024-04-12T00:46:54.984Z</atom:updated>
            <content:encoded><![CDATA[<p>Caching is the process of storing frequently accessed data in a temporary storage location, called a cache, in order to quickly retrieve it without the need to query the original data source. This can improve the performance of an application by reducing the number of times a data source must be accessed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tYH0Rx8D7SYOI4LA1MzJeA.png" /></figure><blockquote>Caching is like having a handy storage spot where you keep stuff you use often, so you don’t have to go far to get it when you need it</blockquote><h4>There are several caching strategies:</h4><ul><li>Refresh Ahead</li><li>Write-Behind</li><li>Write-through</li><li>Cache Aside</li></ul><h4>Additionally, the cache can be found in a number of locations. Some examples are:</h4><ul><li>Client Caching</li><li>CDN Caching</li><li>Web Server Caching</li><li>Database Caching</li><li>Application Caching</li></ul><p>Let&#39;s first discuss about the <strong>strategies</strong></p><h3>Refresh-ahead</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*142bNgW7M9Ga78ZU7aeESQ.png" /></figure><p>Refresh-ahead allows configuring the cache to update recently accessed data before it expires.</p><p>This can cut down on delays compared to reading through every time, as long as the cache accurately guesses what data will be needed next.</p><p>Refresh-ahead can result in reduced latency vs read-through if the cache can accurately predict which items are likely to be needed in the future.</p><h4>Disadvantage of refresh-ahead:</h4><ul><li>Not accurately predicting which items are likely to be needed in the future can result in reduced performance than without refresh-ahead.</li></ul><h3>Write-behind</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6ZXBRWnhzZ465BhUOpJctA.png" /></figure><p>In write-behind, the application does the following:</p><ul><li>Add/update entry in cache</li><li>Asynchronously write entries to the data store, improving write performance</li></ul><h4><strong>Disadvantages of write-behind:</strong></h4><ul><li>If the cache fails before its contents reach the data store, data loss may occur.</li><li>Compared to cache-aside or write-through, write-behind implementation is more complicated.</li></ul><h3>Write-through</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OBQjmiclnsfJjnsurw4g8A.png" /></figure><p>The application uses the cache as the main data store, reading and writing data to it, while the cache is responsible for reading and writing to the database:</p><ul><li>The application adds/updates entries in the cache</li><li>Cache synchronously writes entries to the data store</li><li>Return</li></ul><p>Because writing takes time, write-through is a sluggish process overall, while reads of the written data alone are quick in the future. If users are changing data rather than reading it, they are typically more patient with latency. The cache contains current data.</p><h4>Disadvantages</h4><ul><li>When a new node is created due to failure or scaling, the new node will not cache entries until the entry is updated in the database. Cache-aside in conjunction with write-through can mitigate this issue.</li><li>Most data written might never be read, which can be minimized with a TTL.</li></ul><h3>Cache-aside</h3><p>Reading and writing data from storage is handled by the application. No direct communication occurs between the cache and storage. This is what the application does:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/421/1*HNA5Q7AHy0ykqYb7zC7Yqg.png" /><figcaption>Image Source: Dev community</figcaption></figure><ul><li>Look for an entry in the cache, resulting in a cache miss.</li><li>Load entry from the database.</li><li>Include a new entry in the cache.</li><li>Return to the original entry.</li></ul><p><a href="https://memcached.org/">Memcached</a> is generally used in this manner. Subsequent reads of data added to the cache are fast. Cache-aside is also referred to as lazy loading. Only requested data is cached, which avoids filling up the cache with data that isn’t requested.</p><p><strong>Now lets discuss about the caching by location:</strong></p><h3>Client Caching</h3><p>Client-side caching involves storing frequently accessed data on the user’s device rather than the server, enhancing application performance by minimizing server requests.</p><p>For instance, web browsers cache web pages and resources like images and scripts, speeding up subsequent visits by retrieving stored content instead of requesting it again. Similarly, mobile apps can cache data locally to boost performance and decrease network usage. While client-side caching reduces server load, accelerates page load times, and cuts down on network traffic, it also carries risks, such as outdated data if not managed correctly, or consuming device memory or disk space.</p><h3>CDN Caching</h3><p>A distributed network of servers positioned strategically across the globe is known as a content delivery network (CDN). A content delivery network (CDN) is primarily used to provide high-performance and high-availability material to end users by caching frequently requested content on servers located closer to the customers.</p><p>When a user requests content from a CDN-enabled website, the CDN initially determines whether the requested content is already stored in a nearby server’s cache. The material is sent to the user from the closest server if it is discovered in the cache. If the content cannot be found in the cache, it is retrieved from the content’s original source, the origin server, and cached on a nearby server for later use.</p><p>Because CDN caching shortens the distance data must travel, lightens the burden on the origin server, and enables faster content delivery to end users, it can dramatically increase a website’s performance and availability.</p><h3>Web Server Caching</h3><p>Web server caching is like having a fast memory for a website. When you visit a website, the server can save certain parts of the site that are used a lot, like images or pages. Next time someone wants those parts, the server can quickly give them because it already has them ready to go. This saves time because the server doesn’t have to fetch everything again. Web server caching makes websites load faster and puts less strain on the server. It’s like having a snack ready instead of making it from scratch every time someone comes over.</p><h3>Database Caching</h3><p>Database caching is like keeping a cheat sheet handy for quick reference. When you ask a database for information, it can store the answers to common questions in a cache. So, if someone else asks the same question later, the database can quickly give them the answer from the cache instead of looking it up again. This speeds things up because it saves time searching through tons of data. Database caching makes retrieving information faster and reduces the workload on the database server. It’s like having key facts right at your fingertips, ready to share without delay.</p><h3>Application Caching</h3><p>Key-value stores that sit between your application and your data storage are in-memory caches like Memcached and Redis. It operates substantially faster than most databases since the data is kept in RAM rather than on disc. Since RAM has more capacity than disc, cache invalidation techniques like least recently used (LRU) can help maintain “hot” data in RAM by invalidating “cold” entries.Redis has the following additional features:</p><ul><li>Persistence option</li><li>Built-in data structures such as sorted sets and lists</li></ul><p>In general, file-based caching should be avoided since it complicates cloning and auto-scaling.</p><h3>Conclusion</h3><p>Caching is like having a secret stash of your favorite things nearby, making life easier and faster. Different strategies like refreshing ahead, writing behind, writing through, and cache aside help in managing this stash efficiently.</p><p>You can find caches in various places like on your device, across the globe in content delivery networks (CDNs), on web servers, in databases, and even within applications themselves. Each location serves a unique purpose, improving speed and reducing the load on servers.</p><p>Whether it’s storing website elements, database queries, or application data, caching boosts performance and makes things run smoother for everyone. So, next time you load a webpage or use an app, remember that caching is working behind the scenes to make your experience better and faster.</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7ea1b4776eeb" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/mastering-system-design-caching-part-5-7ea1b4776eeb">Mastering System Design: Caching (Part 5)</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Asymmetric Encryption— Whatsapp mechanism]]></title>
            <link>https://medium.com/@anoopnayak1/asymmetric-encryption-whatsapp-mechanism-79d1821e765c?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/79d1821e765c</guid>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[whatsapp]]></category>
            <category><![CDATA[asymmetric-encryption]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Thu, 08 Feb 2024 07:26:44 GMT</pubDate>
            <atom:updated>2024-02-16T18:53:48.095Z</atom:updated>
            <content:encoded><![CDATA[<h3>Asymmetric Encryption — Whatsapp mechanism</h3><p>In today’s world of computers and the internet, keeping our information safe is really important. Asymmetric encryption is a big part of that. But what is it, and how does it work? Let’s break it down using easy words and examples.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*iKQIC6B8QJe6BlgUl25uQQ.png" /></figure><h3>What is Asymmetric encryption?</h3><p>Asymmetric encryption is like a secret code that protects our digital stuff. It’s a way of mixing up information so only the right person can unmix it and understand it. Picture it as a lock and key system, but with a twist.</p><h3>The Lock and Key Example: A Simple Story</h3><p>Imagine you have a special box for your favorite things. To keep them safe, you have two keys: a public key and a private key.</p><ul><li><strong>Public Key</strong>: This key is like a magic lock. You can give copies to anyone who wants to send you something.</li><li><strong>Private Key</strong>: This key is super secret. It’s like a magic key that only you have. You don’t share this key with anyone.</li></ul><blockquote>Now, let’s say your friend wants to send you a message. They put their message in the box and lock it with the magic lock (public key) you gave them. Once locked, only you, with your magic key (private key), can open the box and read the message.</blockquote><blockquote>When you get the locked box, you use your magic key (private key) to open it and see the message inside. Since only you have the magic key, you’re the only one who can open the box and read the message. That’s how asymmetric encryption works in a simple way!</blockquote><h3>Why Do We Need Asymmetric encryption?</h3><p>Asymmetric encryption helps keep our digital messages safe in a few important ways:</p><ul><li><strong>Data Safety</strong>: It makes sure the messages we send stay the same and haven’t been changed by anyone.</li><li><strong>Checking Who Sent What</strong>: It makes sure the person sending the message is who they say they are.</li><li><strong>Proving Messages:</strong> It stops people from saying they didn’t send a message when they really did. It helps show that the messages are real.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5E0lItRE7ZtzwINQMDti1w.png" /></figure><h3>Real-Life Example: WhatsApp</h3><p>Let’s talk about WhatsApp, a popular messaging app used by lots of people. WhatsApp uses asymmetric encryption to keep your messages private and safe.</p><p>When you send a message on WhatsApp, it gets locked up using asymmetric encryption. Your message is mixed up using the recipient’s magic lock (public key) before it’s sent over the internet. Only the person getting the message, with their magic key (private key), can unlock and read it.</p><p>This makes sure that your chats stay just between you and the person you’re talking to, away from any prying eyes.</p><h3>Achieving Asymmetric encryption:</h3><p>This process achieves asymmetric encryption because:</p><ul><li>The encryption process uses mathematical algorithms to mix up the message in a unique way.</li><li>The public key is used to encrypt the message, making it secure for transmission.</li><li>The private key is kept secret and is used by the recipient to decrypt and read the message.</li><li>Asymmetric encryption ensures that even if someone intercepts the encrypted message, they can’t read it without the private key.</li></ul><h3><strong>Creating Asymmetric </strong>encryption<strong>: Step-by-Step</strong></h3><p>Imagine you want to send a secret message to your friend using asymmetric encryption. Here’s how you would do it:</p><h4>Generate Key Pairs:</h4><ul><li>First, you need to create a pair of special keys: a public key and a private key.</li><li>Think of the public key as a lock and the private key as the key that can unlock it.</li><li>You keep your private key safe and give the public key to your friend.</li></ul><h4>Encrypting the Message:</h4><ul><li>Now, you have a message you want to send to your friend.</li><li>You use your friend’s public key (the lock) to encrypt the message. This mixes up the message in a special way that only your friend’s private key (the key) can unmix.</li></ul><h4>Sending the Encrypted Message:</h4><ul><li>With the message all mixed up, you can send it over the internet or through any communication channel.</li><li>Even if someone intercepts the message, they won’t be able to read it because it’s encrypted with your friend’s public key.</li></ul><h4>Decrypting the Message:</h4><ul><li>When your friend receives the encrypted message, they use their private key (the key) to unlock it.</li><li>The private key unmixes the message, revealing the original content that only your friend can read.</li></ul><h3><strong>Will these keys ever expire?</strong></h3><h4>Public Key:</h4><ul><li>The public key is widely shared and used by anyone who wants to send encrypted messages to the owner of the corresponding private key.</li><li>It doesn’t typically expire unless there’s a specific reason to revoke it, such as a security breach or the owner’s decision to change keys.</li><li>However, some systems may have mechanisms in place for key rotation or renewal for added security. In such cases, the public key might be updated periodically, but this isn’t a universal practice.</li></ul><h4>Private Key:</h4><ul><li>The private key is kept secret and known only to the owner. It’s used for decrypting messages encrypted with the corresponding public key.</li><li>Generally, private keys don’t expire on their own. However, it’s essential to keep them secure and confidential to prevent unauthorized access.</li><li>If a private key is compromised or suspected to be compromised, it’s advisable to generate a new private key and corresponding public key pair to maintain security.</li><li>In certain security protocols or systems, private keys may have associated expiration dates or validity periods for added protection. After the expiration date, the private key becomes invalid, and a new key pair needs to be generated.</li></ul><h3>Conclusion</h3><p>In a world where our digital privacy is really important, understanding asymmetric encryption is key. It’s like an invisible shield that keeps our information safe and secure, letting us chat and share stuff online without worry.</p><p>Next time you send a message or share something online, remember the magic of asymmetric encryption keeping your info safe and sound.</p><p>And also don&#39;t forget to mention this during your next system design interviews about chatapp.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=79d1821e765c" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Keeping the Bad Bots away: Easy Bot Detection in Next.js using BotD]]></title>
            <link>https://medium.com/@anoopnayak1/keeping-the-bad-bots-away-easy-bot-detection-in-next-js-using-botd-8bb55d919c42?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/8bb55d919c42</guid>
            <category><![CDATA[nextjs]]></category>
            <category><![CDATA[middleware]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[bots]]></category>
            <category><![CDATA[typescript]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Sun, 04 Feb 2024 19:52:01 GMT</pubDate>
            <atom:updated>2024-02-16T18:58:02.425Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey there, web warriors! Ever felt like your website is a party, and some uninvited bots are trying to crash it? Well, fear not! We’ve got the perfect bouncer in town — it’s called Bot Detection with BotD in Next.js. In this blog, we’ll chat about why you need this digital superhero, how it works, and why adding it to your Next.js project is like giving your site a cool, secure cape!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kGcVp_NqY6qNKnw81gQbMw.png" /></figure><h3>Why We Need Bot Detection</h3><p>Okay, imagine you’re throwing a biryani party, and suddenly, a swarm of robot biryani lovers shows up. Weird, right? Well, that’s what happens on websites too! Some sneaky bots can mess up your stats, steal your goodies, and just be plain annoying. Bot Detection is like having a superhero squad that kicks these robot troublemakers out, making your online party much better.</p><h4>Benefits of Using Bot Detection in Next.js:</h4><ol><li>Enhanced Security: Imagine your website as a fortress, and bot detection as the vigilant guards at the gate. With BotD in Next.js, you can fortify your site against malicious bots, protecting sensitive information and maintaining the integrity of your application.</li><li>Accurate Analytics: Bots can distort your website statistics, making it challenging to understand your real user engagement. Bot detection helps filter out these artificial interactions, providing accurate insights into user behavior and preferences.</li><li>Preventing Scraping Attacks: Some bots are designed to scrape your content, stealing valuable information for various purposes. BotD identifies and thwarts these scraping attempts, safeguarding your intellectual property and sensitive data.</li><li>Ensuring Fair Resource Usage: Bots can consume server resources, causing slow performance for genuine users. Bot detection ensures fair resource distribution, preventing bots from hogging server capacity and ensuring a smooth experience for your visitors.</li></ol><h4>If you want to see how its implemented in NextJs</h4><p><a href="https://github.com/AnoopNayak7/bot-detection-nextjs-botd">Click here</a></p><h3>Conclusion:</h3><p>In the ever-evolving landscape of web security, bot detection is your ally in the fight against malicious actors. With BotD seamlessly integrated into your Next.js project, you not only protect your website but also elevate the overall user experience. So, why let unwanted bots tarnish your digital presence? Embrace Bot Detection with BotD and let your Next.js application thrive securely.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8bb55d919c42" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering System Design: Databases (Part 4)]]></title>
            <link>https://blog.stackademic.com/mastering-system-design-databases-part-4-595c7c0ed231?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/595c7c0ed231</guid>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[mongodb]]></category>
            <category><![CDATA[nosql]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Sun, 07 Jan 2024 13:09:07 GMT</pubDate>
            <atom:updated>2024-04-12T00:46:44.421Z</atom:updated>
            <content:encoded><![CDATA[<p>In our last discussions on system design, we explored vital concepts like performance, scalability, and the intricate CAP Theorem (<a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-3fcfbf92c781">Part 1</a>). We also covered background jobs, DNS, and Content Delivery Networks (CDNs) in <a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-part-2-6ed6906cce31">Part 2</a>. In the previous blog, we covered Load Balancing and Application Layers <a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-part-3-222c39f4a415">Part 3</a>.</p><p>Today, we take the next step. <strong>Part 4</strong> focuses on a very important topic:</p><h3>Databases</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VnNj4pnY93ni2-YqBBQTTg.png" /></figure><h3>Databases</h3><p>Choosing the correct database for a system is a crucial decision that can greatly affect its performance, scalability, and overall success. It’s important to select the right database for several reasons:</p><ol><li><strong>Performance</strong>: Databases vary in how well they handle tasks, and selecting an unsuitable one may result in sluggish performance and delayed responses.</li><li><strong>Scalability</strong>: As the system expands and data volume grows, the database must seamlessly scale. Certain databases excel at managing substantial amounts of data, while others may need help to adapt.</li><li><strong>Data Modeling</strong>: Databases differ in their abilities to model data effectively. Choosing the right one is essential for maintaining consistent and organized data structures.</li><li><strong>Data Integrity</strong>: Various databases offer different features for upholding data integrity, such as constraint enforcement and varying levels of data security. Choosing the appropriate database ensures data reliability.</li><li><strong>Support and Maintenance</strong>: Databases come with varying community support and documentation quality levels. Opting for a database with an active community and comprehensive documentation makes it simpler to find assistance and resources for support and maintenance.</li></ol><blockquote>Overall, by choosing the right database, you can ensure that your system will perform well, scale as needed, and be maintainable in the long run.</blockquote><h3>SQL vs NoSQL</h3><p><strong>SQL databases</strong>, such as MySQL and PostgreSQL, are best suited for structured, relational data and use a fixed schema. They provide robust ACID (Atomicity, Consistency, Isolation, Durability) transactions and support complex queries and joins.</p><p><strong>NoSQL databases</strong>, such as MongoDB and Cassandra, are best suited for unstructured, non-relational data and use a flexible schema. They provide high scalability and performance for large amounts of data and are often used in big data and real-time web applications.</p><blockquote>The choice between SQL and NoSQL depends on the specific use case and requirements of the project. If you need to store and query structured data with complex relationships, an SQL database is likely a better choice. If you need to store and query large amounts of unstructured data with high scalability and performance, a NoSQL database may be a better choice.</blockquote><h3>How SQL works</h3><p>SQL databases are valuable in handling structured data, or data that has relationships between its variables and entities.</p><h4>Scalability</h4><p>In general, SQL databases can scale <em>vertically</em>, meaning you can increase the load on a server by migrating to a larger server that adds more CPU, RAM, or <a href="https://www.ibm.com/cloud/blog/hard-disk-drive-vs-solid-state-drive">SSD</a> capability. While vertical scalability is used most frequently, SQL databases can also scale horizontally through sharding or partitioning logic, although that’s not well-supported.</p><h4>Structure</h4><p>SQL database schema organizes data in relational, tabular ways, using tables with columns or attributes and rows of records. Because SQL works with such a strictly predefined schema, it requires organizing and structuring data before starting with the SQL database.</p><h4>Properties</h4><p>RDBMS, which uses SQL, must exhibit four properties, known by the acronym ACID. These ensure that transactions are processed successfully and that the SQL database has a high level of reliability:</p><ul><li><strong>Atomicity</strong>: All transactions must succeed or fail and cannot be left partially complete, even in the case of system failure.</li><li><strong>Consistency</strong>: The database must follow rules that validate and prevent corruption at every step.</li><li><strong>Isolation</strong>: Concurrent transactions cannot affect each other.</li><li><strong>Durability</strong>: Transactions are final, and even system failure cannot “roll back” a complete transaction.</li></ul><h3><em>let’s also look into some important topics about SQL:</em></h3><h4>Replication</h4><p>Replication is the process of copying data from one database to another. Replication is used to increase availability and scalability of databases.</p><p>There are two types of replication:</p><p><strong>Master-slave Replication:</strong></p><p>The master serves reads and writes, replicating writes to one or more slaves, which serve only reads. Slaves can also replicate to additional slaves in a tree-like fashion. If the master goes offline, the system can continue to operate in read-only mode until a slave is promoted to a master or a new master is provisioned.</p><p><strong>Master-master Replication:</strong></p><p>Both masters serve reads and writes and coordinate with each other on writes. If either master goes down, the system can continue to operate with both reads and writes.</p><h4>Sharding</h4><p>Sharding refers to the horizontal partitioning of a database’s data across multiple servers or nodes.</p><p>The goal of sharding is to distribute the data in a way that each server (or shard) is responsible for a subset of the data, thereby improving performance, scalability, and parallelism.</p><blockquote>When dealing with exceptionally large datasets that cannot be efficiently managed by a single server, sharding becomes a practical solution. It allows for the distribution of the data, preventing a single server from becoming overwhelmed and improving query performance.</blockquote><p>Sharding is beneficial for workloads with a high volume of both read and write operations. By distributing the data, read and write operations can be performed concurrently on different shards, reducing contention and improving throughput.</p><h4>Federation</h4><p>Federation in the context of databases refers to a strategy where data is distributed across multiple independent database instances.</p><p>Smaller databases result in more data that can fit in memory, which in turn results in more cache hits due to improved cache locality. With no single central master serializing writes you can write in parallel, increasing throughput.</p><h4>Denormalization</h4><p>Denormalization is a database optimization technique that involves intentionally introducing redundancy into a database by combining tables. Unlike normalization, which aims to eliminate redundancy and improve data integrity, denormalization sacrifices some of these benefits in favor of enhancing query performance and simplifying certain types of data retrieval.</p><p>The primary goal of denormalization is to improve the performance of read-heavy operations, such as data retrieval and reporting. By reducing the number of joins needed in queries, denormalized databases can respond more quickly to certain types of queries.</p><p>Denormalization simplifies queries by reducing the need for complex joins across multiple tables. This is particularly beneficial in scenarios where the data retrieval speed is more critical than maintaining the most normalized form of the data.</p><h4>SQL Tuning</h4><p>SQL tuning is the process of optimizing the performance of SQL queries to ensure they execute efficiently and quickly. The goal is to enhance the overall performance of database systems by improving the way SQL statements are written and executed.</p><h3>How NoSQL works</h3><p>Unlike SQL, NoSQL systems allow you to work with different data structures within a database. Because they allow a dynamic schema for unstructured data, there’s less need to pre-plan and pre-organize data, and it’s easier to make modifications. NoSQL databases allow you to add new attributes and fields, as well as use varied syntax across databases.</p><h4>Scalability</h4><p>NoSQL databases scale better horizontally, which means one can add additional servers or nodes as needed to increase load.</p><h4>Structure</h4><p>NoSQL databases are not relational, so they don’t solely store data in rows and tables. Instead, they generally fall into one of four types of structures:</p><ul><li>Column-oriented, where data is stored in cells grouped in a virtually unlimited number of columns rather than rows.</li><li>Key-value stores, use an associative array (also known as a dictionary or map) as their data model. This model represents data as a collection of key-value pairs.</li><li>Document stores, which use documents to hold and encode data in standard formats, including XML, YAML, JSON (JavaScript Object Notation), and BSON. A benefit is that documents within a single database can have different data types.</li><li>Graph databases represent data on a graph that shows how different sets of data relate to each other. Neo4j, RedisGraph (a graph module built into Redis), and OrientDB are examples of graph databases.</li></ul><h4>Properties</h4><p>While SQL calls for ACID properties, NoSQL follows the CAP theory (although some NoSQL databases — such as IBM’s DB2, MongoDB, AWS’s DynamoDB, and Apache’s CouchDB — can also integrate and follow ACID rules).</p><p>The <a href="https://www.ibm.com/cloud/learn/cap-theorem">CAP theorem</a> says that distributed data systems allow a trade-off that can guarantee only two of the following three properties (which form the acronym CAP) at any one time:</p><ul><li>Consistency: Every request receives either the most recent result or an error. MongoDB is an example of a strongly consistent system, whereas others such as Cassandra offer eventual consistency.</li><li>Availability: Every request has a non-error result.</li><li>Partition tolerance: Any delays or losses between nodes do not interrupt the system operation.</li></ul><h3>let’s also look into some important topics about NoSQL:</h3><h4>Key Value Store</h4><p>A Key-Value Store is a type of NoSQL database that organizes and stores data as a collection of key-value pairs. In this model, each data entry, or record, consists of a unique identifier (the key) and the associated data (the value). Key-Value Stores are known for their simplicity, high performance, and scalability, making them suitable for various applications with specific requirements.</p><p>when to use:</p><p><strong>Scenario 1:</strong> You need a fast and efficient caching layer to store frequently accessed data and reduce the load on your primary data store or database.</p><p>Storing the results of expensive database queries or computation for quick retrieval.</p><p><strong>Scenario 2:</strong> Managing session data in web applications where quick access to user-related information is essential.</p><p>Storing user session details, preferences, or temporary data.</p><h4>Document Store</h4><p>Document Store is a type of NoSQL database that stores and retrieves data in a format known as documents. In this context, a document refers to a self-contained piece of information that can be in various formats, such as JSON, BSON, XML, or others. Document Stores are designed to handle unstructured or semi-structured data and are known for their flexibility, scalability, and ease of development.</p><p>Document Stores are schema-less, allowing developers to store different types of documents in the same database without a predefined structure.</p><p>Document Stores are designed for horizontal scalability, meaning they can efficiently distribute data across multiple servers or nodes to handle increasing amounts of data and traffic.</p><h4>Wide Column Store</h4><p>A Wide Column Store is a type of NoSQL database that organizes data into columns rather than rows, making it particularly suitable for handling large volumes of distributed and sparse data. This database model is designed for scalability and high-performance read and write operations. One of the notable characteristics of Wide Column Stores is that they provide flexibility in defining columns for each row, allowing for varied and evolving data schemas.</p><p>Data is organized into column families, where each column family consists of rows with a unique key and columns associated with that key.</p><p>Data is stored in a column-oriented manner, meaning that each column is stored separately rather than each row.</p><p>Google introduced Bigtable as the first wide column store, which influenced the open-source HBase often-used in the Hadoop ecosystem, and Cassandra from Facebook. Stores such as BigTable, HBase, and Cassandra maintain keys in lexicographic order, allowing efficient retrieval of selective key ranges.</p><p>It is designed to handle large amounts of data and is particularly well-suited for analytical and operational workloads. Here are some scenarios where you might consider using <em>Wide Column Store</em> type of NoSQL database:</p><ol><li>Time Series Data: Bigtable is well-suited for handling time-series data, such as IoT data, log data, or any data that has a timestamp associated with it. Its design allows for efficient storage and retrieval of time-ordered data.</li><li>Analytics: If you have large-scale analytical workloads that require high-throughput and low-latency access to vast amounts of data, Bigtable can be a good choice. It provides excellent performance for read and write operations, making it suitable for real-time analytics.</li><li>High Throughput and Low Latency: Bigtable is designed for high throughput and low-latency access to data, making it suitable for applications where responsiveness is critical. This can include real-time analytics, fraud detection, and other time-sensitive operations.</li><li>Scalability: Google Cloud Bigtable scales horizontally to handle large datasets and high-throughput workloads. It automatically distributes data across multiple nodes, allowing you to scale your storage and performance needs as your application grows.</li></ol><h3>Conclusion</h3><p>In summary, this blog talked a lot about databases in making computer systems work well. Picking the right kind of database is super important because it affects how fast the system works, how much it can handle, and how long it will keep working well.</p><p>We learned about two main types of databases: SQL and NoSQL. SQL is good for organized, structured data, while NoSQL is great for handling a lot of unorganized data that needs to grow.</p><p>For SQL, we looked at how it works, how it can get bigger (or scale), and some important rules it follows to keep things reliable. We also discussed ways to make SQL work even better, like copying data to different places (replication), splitting data across servers (sharding), and simplifying things for better reading (denormalization).</p><p>Then, we jumped into NoSQL, which is more flexible and can handle different types of data. NoSQL is good at getting bigger by adding more servers. We talked about the different ways NoSQL stores data, like using keys and values, documents, or wide columns.</p><p>Finally, we covered specific types of NoSQL databases like Key-Value Stores, Document Stores, and Wide Column Stores. We even took a quick look at Google Cloud’s Bigtable as an example of a Wide Column Store, which is good for handling time-based data and big analytics tasks.</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=595c7c0ed231" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/mastering-system-design-databases-part-4-595c7c0ed231">Mastering System Design: Databases (Part 4)</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Exploring Real-Time Communication in Web Development: Short Polling vs. Long Polling]]></title>
            <link>https://medium.com/@anoopnayak1/exploring-real-time-communication-in-web-development-short-polling-vs-long-polling-ec571f5e8af8?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/ec571f5e8af8</guid>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[react]]></category>
            <category><![CDATA[full-stack]]></category>
            <category><![CDATA[polling]]></category>
            <category><![CDATA[backend]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Fri, 17 Nov 2023 10:25:53 GMT</pubDate>
            <atom:updated>2024-02-16T19:23:08.216Z</atom:updated>
            <content:encoded><![CDATA[<p>Hey there, web creator! So, you want your site to talk in real-time, huh? Cool! Let’s chat about two buddies: Short Polling and Long Polling.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_ZUBrd9zExyLc09rEfQ_Zw.png" /></figure><h3>Short Polling:</h3><p>What is Short Polling? Short polling is similar to a quick chat between a web browser and a server. Imagine you’re texting a friend, asking, “Anything new?” at regular intervals.</p><p>When to Use Short Polling? Short polling is ideal for scenarios where real-time updates aren’t critical. Think of a weather app refreshing every few minutes; short polling suits these cases well.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kJpzW9u-f2TH6bkzd4CC8A.png" /></figure><h4>Advantages:</h4><ul><li>Simplicity: Implementing short polling is straightforward, making it accessible for various applications.</li><li>Compatibility: It works seamlessly with a wide range of server environments and doesn’t require special configurations.</li></ul><h4>Disadvantages:</h4><ul><li>Excessive Requests: Frequent, periodic requests can lead to unnecessary server load and increased network traffic.</li><li>Latency: Updates are only received during the polling intervals, introducing a delay in information retrieval.</li></ul><blockquote><strong><em>Example:</em></strong> Consider a stock market ticker using short polling to update prices every 30 seconds. While it provides regular updates, there’s a slight lag between the actual market changes and when users receive the information.</blockquote><h3>Long Polling:</h3><p>What is Long Polling? Long polling is like having a sustained conversation where the server holds the line until it has something to share, avoiding constant check-ins.</p><p>When to Use Long Polling? Long polling shines when real-time updates matter, and you want to minimize unnecessary requests. Imagine a messaging app where you don’t want to miss a message but also don’t want to keep asking, “Any new messages?”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5AjxwVXCkMMyYHmRX07Tnw.png" /></figure><h4>Advantages:</h4><ul><li>Reduced Requests: The connection stays open until there’s new information, minimizing unnecessary communication.</li><li>Near Real-Time Updates: Long polling provides quicker updates compared to short polling, creating a more responsive user experience.</li></ul><h4>Disadvantages:</h4><ul><li>Resource Intensive: Maintaining long-lived connections can strain server resources, especially in environments not optimized for them.</li><li>Connection Loss Latency: If the connection is lost, there might be a delay when re-establishing it.</li></ul><blockquote><strong><em>Example:</em></strong> Imagine a live chat application utilizing long polling. Users receive messages as soon as they’re sent, creating a fluid and instantaneous conversation flow.</blockquote><h4>Choosing the Right Chat Style: Short or Long?</h4><p>Think about how often your site likes to chat and if your server is a social butterfly or more of a shy type.</p><h4>WebSockets: The Tech Rockstars</h4><p>In the tech world, WebSockets are like rockstars. They’re super efficient and keep the connection strong both ways.</p><h3>Conclusion</h3><p>In the orchestra of real-time communication, WebSockets often steal the spotlight for their efficiency in maintaining persistent, bidirectional connections. In conclusion, whether you opt for short polling or long polling depends on your web application’s unique needs. Understanding the nuances of these communication techniques empowers you to create a user experience that harmonizes with the rhythm of real-time updates. Choose wisely, and let your web pages dance to the beat of seamless, dynamic communication.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ec571f5e8af8" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering System Design: Building Scalable and Robust Architectures(Part 3)]]></title>
            <link>https://blog.stackademic.com/mastering-system-design-building-scalable-and-robust-architectures-part-3-222c39f4a415?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/222c39f4a415</guid>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[systems-thinking]]></category>
            <category><![CDATA[front-end-development]]></category>
            <category><![CDATA[scalability]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Fri, 03 Nov 2023 06:57:11 GMT</pubDate>
            <atom:updated>2024-04-12T00:46:37.841Z</atom:updated>
            <content:encoded><![CDATA[<p>In our last discussions on system design, we explored vital concepts like performance, scalability, and the intricate CAP Theorem (<a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-3fcfbf92c781">Part 1</a>). We also covered background jobs, DNS, and Content Delivery Networks (CDNs) in <a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-part-2-6ed6906cce31">Part 2</a>.</p><p>Today, we take the next step. <strong>Part 3</strong> focuses on two key areas:</p><ol><li><strong>Load Balancing</strong></li><li><strong>Application Layers</strong></li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sVwbGvRbHmorG4BPkC31SQ.png" /></figure><h3>Load Balancers</h3><p>Load balancers distribute incoming client requests to computing resources such as application servers and databases. In each case, the load balancer returns the response from the computing resource to the appropriate client. Load balancers are effective at:</p><ol><li><strong>Keep Servers Healthy</strong>: They make sure that client requests don’t reach servers that are not working properly.</li><li><strong>Prevent Overload</strong>: They distribute the incoming requests evenly to avoid putting too much stress on any one server.</li><li><strong>Avoid Breakdowns</strong>: They help your system stay up and running even if one server stops working.</li></ol><p>Load balancers can be implemented with hardware (expensive) or with software such as HAProxy. Additional benefits include:</p><ol><li><strong>SSL Handling</strong>: Load balancers can make your system more efficient by handling the complicated job of encrypting and decrypting data. This means your main servers don’t have to do this hard work, which can be costly.</li><li><strong>Certificate Management</strong>: You don’t need to install security certificates on each individual server. Load balancers handle this for you, making things simpler.</li><li><strong>Session Control:</strong> If your web apps don’t keep track of sessions, load balancers can help by making sure a specific client’s requests go to the same server. This is useful for things like online shopping carts or staying logged in.</li></ol><h3>Disadvantages of load balancer</h3><ul><li>The load balancer can become a performance bottleneck if it does not have enough resources or if it is not configured properly.</li><li>Introducing a load balancer to help eliminate a single point of failure results in increased complexity.</li><li>A single load balancer is a single point of failure, configuring multiple load balancers further increases complexity.</li></ul><h3>Let&#39;s discuss some important topics related to Load Balancers</h3><h4>Load Balancer vs Reverse Proxy</h4><p>Reverse proxy servers and load balancers are components in a client-server computing architecture. Both act as intermediaries in the communication between the clients and servers, performing functions that improve efficiency.</p><blockquote><strong><em>The basic definitions are simple:</em></strong></blockquote><blockquote>A <a href="https://www.nginx.com/resources/glossary/reverse-proxy-server"><strong>reverse proxy</strong></a> accepts a request from a client, forwards it to a server that can fulfill it, and returns the server’s response to the client.</blockquote><blockquote>A <a href="https://www.nginx.com/resources/glossary/load-balancing"><strong>load balancer</strong></a> distributes incoming client requests among a group of servers, in each case returning the response from the selected server to the appropriate client.</blockquote><p><strong>But they sound pretty similar, right?</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ccJGLih_Q0RhNYGvKkpN6w.jpeg" /></figure><p>Reverse proxies and Load balancers can seem very similar. They both sit between the people using a website and the computers that run it. They handle requests from the users and send them to the right place, then take the responses from the computers and send them back to the users. It’s common to get these two mixed up. To clarify, let’s see when and why we use them on a website.</p><h3>Let&#39;s first see what Load Balancing do?</h3><p>Load balancers are most commonly deployed when a site needs multiple servers because the volume of requests is too much for a single server to handle efficiently. Deploying multiple servers also eliminates a single point of failure, making the website more reliable. Most commonly, the servers all host the same content, and the load balancer’s job is to distribute the workload in a way that makes the best use of each server’s capacity, prevents overload on any server, and results in the fastest possible response to the client.</p><h3>What does Reverse Proxy do?</h3><p>A reverse proxy is like a bouncer for a website. It stands between the people using the site and the servers running it. When someone wants to see a webpage, the reverse proxy checks if the servers are ready. If they are, it lets the person in. If the servers are busy or not working, the reverse proxy keeps the person outside. It helps the website handle lots of visitors and stay safe from bad things on the internet.</p><h3>Types of load balancing algorithms</h3><p>A load balancer is a software or hardware device that keeps any one server from becoming overloaded. A load-balancing algorithm is the logic that a load balancer uses to distribute network traffic between servers (an algorithm is a set of predefined rules).</p><p>So there are 2 algorithms for this:</p><h4>Dynamic load balancing algorithms</h4><ol><li><strong>Least Connection</strong>: It looks at which servers are not too busy (the fewest connections) and sends traffic there. This assumes that all servers can handle about the same amount of work.</li><li><strong>Weighted Least Connection</strong>: It lets you give each server a “weight” that represents how much work it can handle. Servers with higher weights get more traffic.</li><li><strong>Weighted Response Time:</strong> This one looks at how quickly each server responds to requests and also how busy they are. It sends traffic to the servers that are both quick and not too busy, making things faster for users.</li><li><strong>Resource-Based:</strong> It checks how much “power” each server has at that moment, like how much computer memory and processing it has available. It asks each server using special software. Then, it sends traffic to the servers with the most power at the time.</li></ol><h4>Static load balancing algorithms</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UtTLSIbNyGNGKFBeI_0nFw.png" /></figure><ol><li><strong>Round Robin:</strong> It’s like taking turns. The load balancer sends traffic to each server one after the other. Imagine serving slices of pizza to friends, making sure everyone gets a fair share.</li><li><strong>Weighted Round Robin: </strong>It’s similar, but some servers get more traffic than others. It’s like giving a bit more pizza to friends who are really hungry.</li><li><strong>IP Hash: </strong>This one uses math to figure out which server should get the traffic. It looks at where the traffic is coming from and where it’s going, like checking who’s ordering pizza and where they live, and then decides which server should deliver it.</li></ol><h4>Next lets learn about Layer 7 and Layer 4 Load balancing:</h4><h4>Layer 7 Load Balancing</h4><p>Layer 7 load balancers are like smart traffic cops for websites. They look deep into the stuff people send to a website, like the words in a letter or the type of cookies they have. They decide where to send the requests based on this info. It’s like a traffic cop who not only directs cars but also reads the notes they’re carrying and then sends them to the right place.</p><p>On the other hand, layer 4 load balancing is quicker and doesn’t need as much computer power.</p><blockquote>It’s like a regular traffic cop who just looks at the cars and directs them without reading the notes inside. It works well on modern computers.</blockquote><h4>Layer 4 Load Balancing</h4><p>Layer 4 load balancers work at the transport level, which is like looking at the labels on packages. They check things like where the packages are coming from (source), where they’re going (destination), and how they’re labeled (ports). But they don’t open the packages to see what’s inside.</p><p>These load balancers also help direct packages to the right place and change the address labels</p><blockquote>A bit like a post office that forwards your mail to the right address.</blockquote><h3><strong>Scaling</strong></h3><p>Imagine you have a small online store. At first, a few customers visit, and your system runs smoothly. But as your shop becomes popular, more and more people want to buy your products. If your system can’t handle this increased demand, it’s like trying to make hundreds of pizzas with just one tiny oven. That’s when you need to scale.</p><h4>Types of Scaling:</h4><ol><li><strong>Vertical Scaling:</strong> This is like upgrading your oven to a bigger and more powerful one. You give your system more resources (like adding more memory or a faster processor) to handle the increased load. It’s good for when you need a quick boost, but there’s a limit to how big you can make your oven.</li><li><strong>Horizontal Scaling:</strong> Instead of getting a bigger oven, you add more ovens to your kitchen. This means you use multiple servers to share the workload. It’s like having many smaller ovens working together. This approach is great for handling really big parties because you can keep adding more ovens as needed.</li></ol><h4>Challenges:</h4><p>Scaling isn’t just about adding more resources. You have to make sure all the parts of your system work together smoothly and share the work fairly. It’s like managing a team of experts in a busy kitchen, where each chef has a specific job. They need to work together well to keep everything running smoothly and the food coming out perfectly.</p><h4>When to Scale:</h4><p>You scale when your system begins to operate more slowly or encounters difficulties in handling the workload. It’s similar to realizing that your system needs to grow when it can no longer manage the increased demand effectively.</p><h3>Application Layer</h3><p>Separating out the web layer from the application layer (also known as platform layer) allows you to scale and configure both layers independently. Adding a new API results in adding application servers without necessarily adding additional web servers. The single responsibility principle advocates for small and autonomous services that work together. Small teams with small services can plan more aggressively for rapid growth.</p><h4>If you are not getting what is application server and web server let me tell you what it is</h4><p>An application server is like the chef in a restaurant’s kitchen. It prepares and manages the main course, side dishes, and desserts. In the tech world, this server runs the core software or applications that process and manage data, perform calculations, and handle complex tasks. It’s responsible for the logic behind the scenes, like handling orders in a restaurant.</p><p>On the other hand, a web server is like a waiter in a restaurant. It’s the one who takes orders from customers (web browsers), serves them their requested dishes (web pages), and ensures everything is delivered to the table. In the tech world, a web server’s main job is to handle incoming web requests from users, retrieve the requested web pages or resources, and send them back to the users’ browsers for display.</p><blockquote>Together, the application server and web server form a complete restaurant experience, where the chef (application server) prepares the food, and the waiter (web server) serves it to the customers. In the tech world, they work together to deliver web applications to users.</blockquote><h3>Microservices</h3><p>In the “<strong>Application Layer</strong>” discussion, we also encounter <strong>microservices</strong>. These are like individual puzzle pieces that work together to create a bigger picture. Each piece has its unique role and communicates effectively with others to achieve a common goal.</p><p>For example, think of an online shopping platform. It could have microservices for handling user accounts, product recommendations, order processing, payment, and shipping. Each microservice focuses on its specific task, ensuring that the entire shopping experience runs smoothly. It’s similar to different departments in a store, like one dedicated to checkout, another for product display, and so on, all working together to provide a seamless shopping experience.</p><h3>Service Discovery</h3><ol><li><strong>Services Finding Each Other:</strong> Imagine you have various pieces of a puzzle, and you need them to connect to create the full picture. In the world of computer systems, different services or software components need to “find” each other to work together seamlessly.</li><li><strong>Tools like Consul, Etcd, and Zookeeper:</strong> These are like special organizers or directories that keep track of the names, addresses, and communication ports of these services. They act as a reference guide, making it easy for one service to locate and connect with another.</li><li><strong>Health Checks:</strong> Just as you’d want to make sure each puzzle piece is in good condition before using it, services need to ensure that they are working properly. Health checks are like quality checks. They verify if a service is in good shape and ready to work. These checks often use a specific web address (HTTP endpoint) to see if a service responds correctly.</li><li><strong>Key-Value Stores: </strong>Consul and Etcd have an additional feature, like a virtual notepad, where you can store important information such as configuration settings and shared data that multiple services need. It’s like having a shared document where everyone can jot down notes and refer to them when needed.</li></ol><blockquote>In summary, these tools act as organizers, ensuring that different services can easily find each other, make sure they are working correctly, and share important information when needed. It’s all about making the complex world of computer systems run smoothly and efficiently.</blockquote><h3>Conclusion</h3><p>To sum it up, in our discussions on system design, we’ve explored essential concepts like load balancing, application layers, and tools for services to work together efficiently. These ideas are crucial in creating robust and high-performance technology solutions. Keep an eye out for more insights in the upcoming parts!</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=222c39f4a415" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/mastering-system-design-building-scalable-and-robust-architectures-part-3-222c39f4a415">Mastering System Design: Building Scalable and Robust Architectures(Part 3)</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering System Design: Building Scalable and Robust Architectures(Part 2)]]></title>
            <link>https://blog.stackademic.com/mastering-system-design-building-scalable-and-robust-architectures-part-2-6ed6906cce31?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6ed6906cce31</guid>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[scalability]]></category>
            <category><![CDATA[system]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[front-end-development]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Mon, 30 Oct 2023 08:05:32 GMT</pubDate>
            <atom:updated>2024-04-12T00:45:28.352Z</atom:updated>
            <content:encoded><![CDATA[<p>In the <a href="https://medium.com/@anoopnayak1/mastering-system-design-building-scalable-and-robust-architectures-3fcfbf92c781">previous blog</a>, we delved into essential system design concepts, such as performance, scalability, latency, throughput, availability, consistency, and the CAP Theorem. We also explored various patterns related to consistency and availability. Building on this foundational knowledge, the upcoming Part 2 will dive even deeper into system design principles and strategies, equipping you with the expertise to craft robust and high-performance technology solutions that meet the demands of today’s digital world.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m1-sNlCvLcHozjAdDyunbw.png" /></figure><h3>Background Jobs</h3><p>Background jobs in system design are tasks that the system handles independently, running in the background without relying on the main system flow or any external input. They serve multiple purposes, including:</p><p>1. <strong>Maintenance Tasks:</strong> These involve activities like tidying up old data, creating reports, or backing up the database.</p><p>2. <strong>Data Handling:</strong> Background jobs efficiently manage significant data volumes, handling tasks like data import, export, or transformation.</p><p>3. <strong>Notifications:</strong> They are used for sending messages or alerts, such as email notifications or push notifications to users.</p><p>4. <strong>Complex Computations:</strong> Background jobs are ideal for performing lengthy calculations, like machine learning or data analysis, without disrupting the system’s primary operations.</p><h3>Triggers</h3><p>Background jobs can be initiated in several different ways. They fall into one of the following categories:</p><h4><strong>Event-driven triggers</strong>:</h4><p>The task is started in response to an event, typically an action taken by a user or a step in a workflow.</p><p>In this scenario, when something happens, like a user making an order, a message is put into a queue by the user interface or another part of the system. A background task keeps an ear out for new messages in the queue. When a new message arrives, the task reads it and takes the information it contains to do a job in the background. This method is called asynchronous message-based communication. It’s like leaving a note for someone to do a task, and when they see the note, they get the job done without interruptions.</p><h4><strong>Schedule-driven triggers</strong>:</h4><p>The task is invoked on a schedule based on a timer. This might be a recurring schedule or a one-off invocation that is specified for a later time.</p><p>In a scheduled trigger setup, specific tasks or actions are programmed to automatically start at predetermined times or intervals. This scheduling ensures that the tasks are executed regularly and predictably, without the need for manual initiation. It’s like setting an alarm clock to wake up at the same time every day, ensuring that your morning routine begins precisely when you want it to.</p><h4>Returning results</h4><p>Background jobs work separately from the user interface or the process that started them. They are like tasks you give with a “do it and let me know when you’re done” instruction. The initiating process doesn’t wait for the task to finish, and it doesn’t get an automatic alert when it’s completed. It’s a bit like sending a letter — you drop it in the mailbox and it gets delivered, but you’re not there to see it happen.</p><h3>Domain Name System</h3><p>DNS, which stands for Domain Name System, is like a phonebook for the internet. It helps your computer find websites by translating human-friendly web addresses (like <a href="http://www.example.com/">www.example.com</a>) into computer-friendly numbers (IP addresses).</p><p>Here are some key points in simpler terms:</p><ol><li><strong>Hierarchy</strong>: DNS is organized like a tree with a few important servers at the very top. It’s like having a big boss overseeing everything.</li><li><strong>Your ISP or Router:</strong> When you connect to the internet through your Internet Service Provider (ISP) or home router, they tell your computer which DNS server to use. This server will help you find websites.</li><li><strong>Caching</strong>: Along the way, different servers store (cache) translations to make future lookups faster. But these cached translations can become outdated.</li><li><strong>TTL</strong>: Each translation has an expiration date (Time To Live or TTL). After that time, it needs to be checked again.</li><li><strong>Records</strong>: Different types of records in DNS do different jobs.</li><li><strong>NS Record</strong>: This tells which servers are in charge of a domain or subdomain.</li><li><strong>MX Record</strong>: It helps with email by specifying the servers that receive messages.</li><li><strong>A Record</strong>: It connects a name (like a website address) to an IP address.</li><li><strong>CNAME</strong>: It’s like a shortcut, directing one name to another name or an IP address.</li><li><strong>Managed DNS Services</strong>: Companies like CloudFlare and Route 53 offer services to help you manage your DNS settings. They make it easier for you to control how your domain is translated into IP addresses.</li><li><strong>Traffic Routing</strong>: Some DNS services can also direct internet traffic in specific ways, like sending visitors to the closest server or managing website traffic.</li></ol><h3>Content Delivery Networks</h3><p>A Content Delivery Network (CDN) is like a big network of servers spread out all over the world. These servers help give people faster access to stuff on websites, like pictures, videos, and web pages. When you want to see something on a website, the CDN helps you get it quickly.</p><p>This is good for two big reasons:</p><ol><li>Stuff comes to you from a server that’s close by, so it’s faster.</li><li>The CDN does some of the work, so the website’s own servers don’t get too busy.</li></ol><h4>Types of CDN’s — Push CDN&amp; Pull CDN</h4><p>Push and pull CDNs are two ways to use Content Delivery Networks to make websites faster and more efficient. Here’s a simple explanation:</p><ol><li><strong>Pull CDN (Origin Pull):</strong></li></ol><p>Pull CDNs grab new content from your server when the first user requests the content. You leave the content on your server and rewrite URLs to point to the CDN. This results in a slower request until the content is cached on the CDN.</p><ul><li>Think of it like this: Imagine you have a shelf with your favorite books at home. When you want to read a book, you go to the shelf and take it.</li><li>In a “Pull” CDN, the CDN waits for someone to ask for a book (a file) from your shelf (your server).</li><li>When someone asks, the CDN takes the book (file) from your shelf and gives it to the person. It’s like borrowing a book from your shelf when you need it.</li></ul><p><strong>Scenario: </strong>Your website contains images that change frequently, like user-uploaded profile pictures.</p><p><strong>Usage:</strong></p><ul><li>You store these images on your web server (origin).</li><li>When a visitor requests a web page with an image, a “Pull” CDN checks if the image is on your server.</li><li>If the image exists, the CDN fetches it from your server and delivers it to the visitor.</li></ul><p><strong>Advantage:</strong></p><p>You can update or change images on your server, and the CDN automatically serves the latest versions when requested.</p><blockquote>Sites with heavy traffic work well with pull CDNs, as traffic is spread out more evenly with only recently-requested content remaining on the CDN.</blockquote><p>2. <strong>Push CDN (Origin Push):</strong></p><p>Push CDNs receive new content whenever changes occur on your server. You take full responsibility for providing content, uploading directly to the CDN and rewriting URLs to point to the CDN. You can configure when content expires and when it is updated. Content is uploaded only when it is new or changed, minimizing traffic, but maximizing storage.</p><ul><li>Now, imagine you have a friend who loves to surprise you with new books. They come to your home and put new books on your shelf, even if you didn’t ask for them.</li><li>In a “Push” CDN, you (or your friend) send books (files) to the CDN without waiting for someone to ask.</li><li>The CDN keeps these books ready for anyone who wants them, so they don’t have to come to your home (server) to get them.</li></ul><p><strong>Scenario: </strong>Your website contains a set of product images that rarely change.</p><p><strong>Usage:</strong></p><ul><li>You proactively send these product images to a “Push” CDN even if nobody has requested them yet.</li><li>The CDN stores these images and makes them readily available to visitors.</li></ul><p><strong>Advantage:</strong></p><p>Since these product images don’t change often, you send them to the CDN once, and they’re available for users without having to go back to your server repeatedly.</p><blockquote>Sites with a small amount of traffic or sites with content that isn’t often updated work well with push CDNs. Content is placed on the CDNs once, instead of being re-pulled at regular intervals.</blockquote><blockquote>In simple terms, “Pull” CDNs wait for requests and then fetch files from your server when needed, like borrowing a book from your shelf. “Push” CDNs proactively send files to the CDN, making them available for anyone who wants them, like your friend leaving books on your shelf as a surprise. Both methods have their uses depending on the situation.</blockquote><h3>Conclusion</h3><p>In this blog post, we’ve covered a range of fundamental concepts in system design, including DNS (Domain Name System), Content Delivery Networks (CDNs), and Background Jobs.</p><h4>Let’s summarize our key takeaways:</h4><p><strong>DNS (Domain Name System):</strong><br>- DNS is like the internet’s phonebook, translating human-friendly web addresses (e.g., <a href="http://www.example.com/">www.example.com</a>) into computer-friendly IP addresses.<br>- It operates in a hierarchical structure with a few crucial servers overseeing the entire system.<br>- Internet Service Providers (ISPs) or routers direct your computer to a DNS server, which helps you find websites.<br>- Cached translations are stored along the way, but they have expiration dates (TTL).<br>- Different types of DNS records perform various functions, like NS Records for domain management and A Records for connecting names to IP addresses.</p><p><strong>Content Delivery Networks (CDNs):</strong><br>- CDNs are extensive networks of servers distributed worldwide, serving content from locations closer to users to improve website performance.<br>- CDNs have two main types: “Pull” CDNs and “Push” CDNs.<br>- “Pull” CDNs retrieve content from your server when requested by a user. Content is cached and then served.<br>- “Push” CDNs receive new content when changes occur on your server. Content is uploaded proactively and made available for users.<br>- The choice between “Pull” and “Push” CDNs depends on the content’s update frequency and website traffic.</p><p><strong>Background Jobs:</strong><br>- Background jobs are tasks in system design that run independently, handling various functions without relying on the main system flow or external input.<br>- They serve purposes such as maintenance tasks, data handling, notifications, and complex computations.<br>- Background jobs can be triggered in two ways: event-driven (in response to user actions or workflow steps) or schedule-driven (based on a timer).<br>- They work separately from the main process, making them suitable for tasks where you want to initiate a job and check back later for the results.</p><p>As we continue to explore system design in the next part of this series, we’ll delve into<strong> load balancers</strong>, <strong>application layer</strong>-related topics, and more. These concepts are essential for building scalable, high-performance, and resilient technology solutions. Stay tuned for a deeper dive into the fascinating world of system design.</p><h3>Stackademic 🎓</h3><p>Thank you for reading until the end. Before you go:</p><ul><li>Please consider <strong>clapping</strong> and <strong>following</strong> the writer! 👏</li><li>Follow us <a href="https://twitter.com/stackademichq"><strong>X</strong></a><strong> | </strong><a href="https://www.linkedin.com/company/stackademic"><strong>LinkedIn</strong></a><strong> | </strong><a href="https://www.youtube.com/c/stackademic"><strong>YouTube</strong></a><strong> | </strong><a href="https://discord.gg/in-plain-english-709094664682340443"><strong>Discord</strong></a></li><li>Visit our other platforms: <a href="https://plainenglish.io/"><strong>In Plain English</strong></a><strong> | </strong><a href="https://cofeed.app/"><strong>CoFeed</strong></a><strong> | </strong><a href="https://venturemagazine.net/"><strong>Venture</strong></a><strong> | </strong><a href="https://blog.cubed.run/"><strong>Cubed</strong></a></li><li>More content at <a href="https://stackademic.com/"><strong>Stackademic.com</strong></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6ed6906cce31" width="1" height="1" alt=""><hr><p><a href="https://blog.stackademic.com/mastering-system-design-building-scalable-and-robust-architectures-part-2-6ed6906cce31">Mastering System Design: Building Scalable and Robust Architectures(Part 2)</a> was originally published in <a href="https://blog.stackademic.com/">Stackademic</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Strengthening Your Application’s Security : Gatekeeper Pattern]]></title>
            <link>https://medium.com/@anoopnayak1/strengthening-your-applications-security-gatekeeper-pattern-6748832bb1d8?source=rss-1c919ece5cb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6748832bb1d8</guid>
            <category><![CDATA[backend]]></category>
            <category><![CDATA[patterns]]></category>
            <category><![CDATA[gatekeepers]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[backend-development]]></category>
            <dc:creator><![CDATA[Anoop Nayak]]></dc:creator>
            <pubDate>Thu, 26 Oct 2023 11:53:04 GMT</pubDate>
            <atom:updated>2023-10-26T11:53:04.932Z</atom:updated>
            <content:encoded><![CDATA[<h3>Strengthening Your Application’s Security : Gatekeeper Pattern</h3><p>In the ever-evolving landscape of web applications, security stands as an uncompromising fortress. In this blog, we’re about to explore a trusty guardian — <strong>the Gatekeeper pattern</strong>. This pattern acts as a dedicated instance, safeguarding your application’s sensitive information and resources.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0_8Xo7um5-lKF0l0nr_SlQ.png" /></figure><h3>The Gatekeeper’s Role:</h3><p>Web applications often expose endpoints for clients to access their APIs. These endpoints execute tasks like authentication, authorization, validation, and request processing. They also access storage and other services on behalf of clients. However, there’s a lurking threat: if a malicious user gains access to the application’s hosting environment, it’s like leaving the castle gates open for intruders.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZI9Svnj2tEZX9a8znwg3hw.png" /></figure><h3>The Solution:</h3><p>To prevent this vulnerability, the <strong>Gatekeeper pattern</strong> comes to the rescue. It decouples the code that manages public endpoints from the code responsible for request processing and resource access. It acts as an intermediary, much like a trusted gatekeeper at the castle entrance.</p><h3>How Does it Work?</h3><p>The Gatekeeper, performs the following crucial tasks:</p><ol><li><strong>Controlled Validation:</strong> It meticulously validates all incoming requests, rejecting any that don’t meet validation criteria.</li><li><strong>Limited Risk and Exposure:</strong> Importantly, the Gatekeeper operates in a restricted mode. It doesn’t have access to sensitive credentials or keys used by the trusted host to access storage and services. Even if the Gatekeeper is compromised, the attacker can’t get hold of these critical keys.</li><li><strong>Appropriate Security:</strong> The Gatekeeper operates in a limited privilege mode, whereas the rest of the application runs with full trust to access storage and services. Even if the Gatekeeper is compromised, it can’t directly access the application services or data.</li></ol><h3>Acting as a Firewall:</h3><p>Think of the Gatekeeper as a firewall for your application. It reviews each request, validating and sanitizing it before deciding whether to pass it on to the trusted host that carries out essential tasks.</p><p>Considerations and Best Practices:</p><p>When implementing the Gatekeeper pattern, here are a few points to keep in mind:</p><ol><li><strong>Internal Endpoints:</strong> Trusted hosts should only expose internal or protected endpoints that the Gatekeeper can access. External interfaces should be off-limits.</li><li><strong>Limited Privilege Mode:</strong> The Gatekeeper should operate in a limited privilege mode, requiring separation of the Gatekeeper and the trusted host, often in separate services or virtual machines.</li><li><strong>Validation Focus: </strong>The Gatekeeper’s primary role is to validate and sanitize requests, while the trusted hosts may perform additional request validation.</li><li><strong>Secure Communication:</strong> Use secure communication channels (e.g., HTTPS, SSL, or TLS) between the Gatekeeper and trusted hosts to ensure data privacy.</li><li><strong>Performance Impact:</strong> Adding an extra layer through the Gatekeeper pattern may impact performance due to additional processing and network communication.</li><li><strong>Redundancy and Autoscaling:</strong> The Gatekeeper instance could be a single point of failure, so consider deploying redundant instances and implementing autoscaling to maintain availability.</li></ol><h3>When to Use the Gatekeeper Pattern:</h3><p>The Gatekeeper pattern is particularly useful for applications that:</p><ul><li>Handle sensitive information.</li><li>Expose services requiring robust protection against malicious attacks.</li><li>Conduct mission-critical operations that can’t tolerate disruptions.</li><li>Need request validation to be separate from their main tasks, or to centralize validation for simpler maintenance and administration.</li></ul><h3>Conclusion</h3><p>In conclusion, the Gatekeeper pattern is your application’s loyal guardian, ensuring that only the most trustworthy requests gain access to your sensitive resources. It’s a security measure that keeps your digital fortress safe in an era where security is paramount.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6748832bb1d8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>