<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Ryan Lai on Medium]]></title>
        <description><![CDATA[Stories by Ryan Lai on Medium]]></description>
        <link>https://medium.com/@forreya?source=rss-23030b811506------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*TAFqgpeyIpZDWE92Djni0g.png</url>
            <title>Stories by Ryan Lai on Medium</title>
            <link>https://medium.com/@forreya?source=rss-23030b811506------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sat, 21 Nov 2026 04:55:37 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@forreya/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Message Queues: Point-to-Point, PubSub, and Request-Reply]]></title>
            <link>https://forreya.medium.com/message-queues-point-to-point-pubsub-and-request-reply-44de03817a21?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/44de03817a21</guid>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[message-queue]]></category>
            <category><![CDATA[pub-sub]]></category>
            <category><![CDATA[message-broker]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Sat, 22 Feb 2025 14:46:53 GMT</pubDate>
            <atom:updated>2025-02-22T14:46:53.809Z</atom:updated>
            <content:encoded><![CDATA[<p>For many people, there is an expectation that application servers process all incoming queries and requests on the spot. However, message queuing is an approach that allows components to communicate asynchronously and independently by decoupling different parts of a system.</p><h3>Foundational Concepts</h3><p>Before going straight into message queue patterns and its benefits, let’s take a look at the key terminology involved:</p><ul><li><strong>Message:</strong><br>The unit of data that is transmitted. This could be anything from a simple text string or command to complex objects like files or metadata.</li><li><strong>Queue:</strong><br>An ordered list where messages are stored until they are processed. Queues ensure that messages are handled in the correct order and at the right time.</li><li><strong>Producer:</strong><br>The system or component that creates and sends messages into the queue.</li><li><strong>Consumer:</strong><br>The system or component that retrieves and processes messages from the queue.</li></ul><p>Take note of these terms carefully, as they form the backbone of all message queue implementations.</p><h3>Message Queue Patterns</h3><p>Message queues are used in different scenarios.</p><h4>1. Point-to-Point</h4><p>This is the simplest form of message queuing. In a point-to-point pattern, a <strong>single producer</strong> sends messages to a<strong> single consumer.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*m-TlAZx8HT4UVlPpRa9-fg.png" /></figure><p><strong>Example:</strong> When a user submits a web form, the data is converted into a JSON object and placed in a queue. Then at some later time, a consumer processes this message — perhaps inserting the data into a database or triggering an email notification. This allows the producer (the web form) to continue working without waiting for the consumer to complete its task.</p><h4>2. Publish-Subscribe (PubSub)</h4><p>In the PubSub model, messages are published to a channel or topic where multiple consumers can subscribe and react to the same message.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*FU_tKmQjM55XRWQ2UvszhQ.png" /></figure><p><strong>Example:</strong> In an e-commerce platform, when a new order is placed, one subscriber might update the inventory, another processes the payment, and yet another updates the order history. Each consumer performs a distinct task based on the same initial message.</p><h4>3. Request-Reply</h4><p>Here, after a consumer processes a message, it sends a response back to the original sender using a secondary queue. This pattern is useful when the original request requires a confirmation or result.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*7adkLnIxUvnQKxMmRsH47w.png" /></figure><p><strong>Example:</strong> An application may request data processing, and once the task is completed, the result is placed on a reply queue, turning the consumer into a producer for the response cycle.</p><h3>Benefits of Message Queuing</h3><p>With that consise understanding of the three main patterns, we can now appreciate the advantages message queuing brings when it comes to system architecture:</p><ol><li><strong>Decoupling:</strong><br>Because of the separation of concerns between consumers, each component is able to solely focus on a single responsibility. This means that updating one part of the system (such as a payment module consumer) doesn’t necessitate retesting or refactoring other consumers of the application to check for inadvertant effects stemming from the updates.</li><li><strong>Scalability:</strong><br>Since components operate independently, you can scale parts of your system individually. If one service, like an email processor, experiences high demand, you can add more consumers for that queue without affecting other services.</li><li><strong>Flexibility:</strong><br>Message queues allow you to modify or swap out individual components without overhauling the entire system. They also allow for the integration of different technologies and programming languages for each component, which makes it possible to individually tailor each component’s environment more easily based on their functionality.</li><li><strong>Reduced Latency:</strong><br>By decoupling the processing of messages, producers are not forced to wait for a full response cycle before moving on to other tasks. This asynchronous behavior can reduce overall latency and improve system responsiveness.</li></ol><p>Two notable message queue services are RabbitMQ and Apache Kafka. We won’t dive into these services in this article but it’s good to have a general awareness of popular real-life message queue services that exist. To put the general idea of message queuing into three words, it achieves better efficiency, scalability, and flexibility. It not only simplifies development and maintenance but also reduces latency by eliminating the need for producers to wait for a complete response cycle for every request.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=44de03817a21" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Short Introduction To Object Storage: S3, Google Cloud Storage & Azure Blob Storage]]></title>
            <link>https://forreya.medium.com/a-short-introduction-to-object-storage-s3-google-cloud-storage-azure-blob-storage-b743568dbfa3?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/b743568dbfa3</guid>
            <category><![CDATA[object-storage]]></category>
            <category><![CDATA[azure-blob-storage-client]]></category>
            <category><![CDATA[amazon-s3]]></category>
            <category><![CDATA[google-cloud-storage]]></category>
            <category><![CDATA[object-storage-services]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Wed, 12 Feb 2025 17:33:36 GMT</pubDate>
            <atom:updated>2025-02-12T17:38:09.923Z</atom:updated>
            <content:encoded><![CDATA[<p>So what is an object store? Let’s start off with a simple definition.</p><blockquote>Object storage is a storage architecture designed for managing large volumes of unstructured data.</blockquote><p>Unlike traditional file systems or relational databases, object storage does not organize data in a hierarchical structure. For example, in a file system, each folder is able to potentially contain its own set of subfolders and files. In object storage, data is stored as discrete units called <strong>objects</strong> in a flat layout. Each object typically contains:</p><ul><li>The <strong>data payload</strong> (the actual content, such as an image, video, or document).</li><li><strong>Metadata</strong> (key–value pairs describing attributes of the object, like content type, creation date, or custom tags).</li><li>A <strong>unique identifier or key</strong> that allows the object to be accessed efficiently. This means duplicate keys (or identifiers) aren’t allowed in object stores.</li></ul><p>You may have already guessed. But this mechanic is actually similar to a hash map, where each value is indexed by a unique key. The presence of this unique key in both hash maps and object stores allows for basically instantaneous data retrieval as we don’t need to traverse down a nested directory tree, even when dealing with billions of objects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zMKYuT1f_lzRutDpRz8pcQ.png" /></figure><h4>Flat Namespace</h4><p>This lack of a hierachy is known as a <strong>flat namespace, </strong>meaning that all objects (data items) are stored at the same level without a nested structure like folders and subdirectories.</p><blockquote>A <strong>namespace</strong> in computing is essentially a container or a set that holds a collection of unique identifiers (otherwise known as names). They ensure that each name within that set is unique and can be used to reference an object or resource unambiguously.</blockquote><p>It’s important to note that many cloud providers, such as Amazon S3, uses interfaces that mimic the “look and feel” of folders. However, these “folders” are simply an illusion and do not actually contain the objects within it. These so-called “folders” are simply used as part of the unique names of each object in the store. Because of this, every single folder name has to be globally unique. The folder name, in combination with an object key, provides a way for users to directly access a specific file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jXXCGCGsAmA6Rce36KbBjA.png" /></figure><h4>Update and Modification</h4><p>When we talk about files in traditional file systems. Usually these files can be read, written, and even modified in place. Object stores work rather differently. Many object storage systems treat objects as <strong>immutable</strong> to maintain data integrity. In practice, if you need to modify an object, you typically upload a new version rather than editing the original. This immutability is a property that is specifically enforced to improve data integrity and to allow for robust versioning management.</p><h4>Data Retrieval</h4><p>We already touched upon this a little, but it’s important to reiterate that data retrieval in object stores is extremely efficient. The flat structure, combined with a unique key for each object means that efficiency is maintained, even as the data scale reaches petabytes or exabytes. As you can imagine, this type of architecture is optimized for high availability and cost-effective scaling.</p><p>If we compare this with hierarchical data organization, we usually experience a degradation in latency as the number of files grow, as operations would require traversing through more and more directory levels.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M2cnhVbzfHxJw7jkINwYlA.png" /></figure><h4>API-Driven Access</h4><p>Object storage systems are typically accessed over HTTP via RESTful APIs. This type of interface is ideal for large-file use cases because:</p><ul><li><strong>Ease of Access</strong>: Clients can retrieve objects using simple HTTP requests.</li><li><strong>Scalability</strong>: APIs are able to handle massive numbers of objects without the overhead of SQL queries or complex transaction mechanics.</li><li><strong>Security</strong>: Access can be tightly controlled using tokens, access keys, and other authentication/authorization mechanisms, ensuring that only permitted applications or users can access the stored objects.</li></ul><h4>Distributed Architecture</h4><p>Object storage systems are typically designed as distributed systems. This means that when you store an object, it isn’t just saved on one disk or server — it’s replicated across multiple disks, nodes, and often even across geographically dispersed data centers. This built-in redundancy protects the system against hardware failures, natural disasters, and other localized issues. By nature, object stores are built for scale and also provides high reliability by default.</p><h4>Popular Implementations</h4><p>Let’s quickly go over some popular services that provide object storage for referential purposes:</p><ul><li><strong>Amazon S3</strong>: Perhaps the most popular object storage service. All S3 buckets must have globally unique names, as the unique identifier for each file is created from a combination of bucket name and object key.</li><li><strong>Google Cloud Storage</strong>: Provides similar capabilities to S3,.</li><li><strong>Azure Blob Storage</strong>: Optimized for storing large binary objects (blobs).</li></ul><h4>Use Cases for Object Storage</h4><p>Object storage is particularly well-suited for storing large, unstructured datasets where the primary operations are storing and retrieving complete objects rather than querying granular data within them. For example:</p><ul><li><strong>Media Storage</strong>: Storing images, videos, and audio files where the file sizes are large, and the retrieval is done via unique URLs or keys.</li><li><strong>Backup and Archive</strong>: Long-term storage of backups, logs, and archival data where data integrity and durability is vital.</li><li><strong>Content Distribution</strong>: Serving static assets for websites or applications directly via HTTP, often integrated with content delivery networks (CDNs) for fast global access.</li></ul><p>A good way to get an idea of what an object store is, is to simply think of object storage as a “fancy” Google Drive or Dropbox designed for applications at scale — one where you can instantly retrieve a file using a unique identifier without the overhead of potentially traversing the entire directory structure. And with that, see you in the next one!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b743568dbfa3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Distributed Database Replication w/ CAP and PACELC]]></title>
            <link>https://forreya.medium.com/distributed-database-replication-w-cap-and-pacelc-9aa51b793e7d?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/9aa51b793e7d</guid>
            <category><![CDATA[pacelc-theorem]]></category>
            <category><![CDATA[cap-theorem]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[database-design]]></category>
            <category><![CDATA[database-replication]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Fri, 31 Jan 2025 15:35:04 GMT</pubDate>
            <atom:updated>2025-01-31T15:35:04.827Z</atom:updated>
            <content:encoded><![CDATA[<p>When discussing database replication in distributed systems, we need to carefully consider the trade-offs between <strong>system latency, system availability, and data consistency</strong>. One of the foundational principles that guide us in making these architectural decisions is the <strong>CAP theorem</strong>, which highlights three core system properties: <strong>Consistency</strong>, <strong>Availability</strong>, and <strong>Partition Tolerance</strong>. Let’s begin by defining each of these concepts.</p><h3>Consistency</h3><p>Consistency in the CAP context differs from the typical definition you might see in ACID relational databases. Here, it means that all database nodes have the same version of data at all times. This implies a constant, unified state across the cluster of database nodes. In a system prioritizing consistency, if data is changed on one node, the update must be propagated to all other nodes before any subsequent read operation is allowed. This ensures the data returned from any one node is never out-of-date, and is perpetually consistent with every other node in the system.</p><h3>Availability</h3><p>Availability means that every request receives a (non-error) response, regardless of individual node failures or network delays, as long as the entire system hasn’t completely crashed. In other words, a system favoring availability will continue to serve read and write requests — even if that means some nodes might return out-of-sync data when certain nodes or network paths are down.</p><h3>Partition Tolerance</h3><p>Partition tolerance refers to a system’s ability to keep functioning despite network partitions or communication cuts between different nodes of the system. In real-world distributed systems, network failures are inevitable, so partition tolerance is practically non-negotiable. The system must either be able to handle these partitions or risk total downtime.</p><h3>What the CAP Theorem Really Says</h3><p>Now that we have the three letters clearly defined, let’s understand the core statement and philosophy behind the CAP theorem, which states that in the presence of a network partition, a distributed system must choose between Consistency and Availability. To put it another way, when a partition occurs, you have to choose one of two:</p><ol><li><strong>Consistency</strong>: Require all nodes to agree on the same version of the data. The system will block reads/writes to a certain connected nodes (making it unavailable) if it is unable to communicate with other nodes in the cluster.</li><li><strong>Availability</strong>: The system remains operational in the event of a network partition, responding to all requests, even if some nodes might return stale data.</li></ol><p>Since <strong>Partition Tolerance (P)</strong> is unavoidable, the “choice” boils down to <strong>Consistency</strong> vs. <strong>Availability</strong> when a partition occurs. Hence, we end up with two potential systems</p><h4>C + P</h4><p>A CP system focuses on strong consistency alongside partition tolerance. During a partition, it may block reads or writes until data can be synchronized across all nodes, ensuring everyone sees the same state. This approach often increases <strong>latency</strong>, as updates must be confirmed via responses by multiple nodes before any read/write operation completes. The result is strong consistency, but at the cost of slower responses and possible unavailability during network issues.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5JfjciocRt842_NECBY1MA.jpeg" /></figure><h4>A + P</h4><p>An AP system keeps availability high even in the face of network partitions. It responds to read and write requests immediately, eventhough some nodes may temporarily serve stale data. Over time, the system converges to a consistent state using techniques like <strong>eventual consistency</strong>, which allows it to synchronize changes asynchronously. This yields low latency and high availability but sacrifices strict real-time data consistency during network partitions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B15jYAP2vC-BHBiQmf09gA.jpeg" /></figure><h3>Introducing the PACELC Theorem</h3><p>You might already be getting a little confused with all these letters, so let’s confuse you further with yet another acronym: <strong>PACELC</strong>. This theorem refines CAP by considering what happens <strong>even when there isn’t</strong> <strong>a partition:</strong></p><ol><li><strong>PAC — </strong>If a partition occurs, the system must choose between Availability and Consistency (the standard CAP consideration).</li><li><strong>ELC — </strong>Else, when there’s no partition, the system faces a trade-off between Latency and Consistency.</li></ol><p>In other words, PACELC covers both scenarios — network partition and normal operation. Even when your system is running smoothly (no partitions), you often decide between <strong>reducing system latency</strong> or maintaining <strong>strong data consistency</strong>. Systems that pursue low latency might loosen consistency requirements, while those aiming for stronger data consistency accept higher latency in return.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vJRqREhmeGGJqtAT8lfaKg.png" /></figure><p>In my opinion, the PACELC theorem offers a more holistic view than CAP alone. It emphasizes that we face data consistency trade-offs not only during network failures but also when our system is fully operational. I recommend checking out Martin Kleppmann’s article on “CP and AP” databases and why CAP definitions don’t fully capture every nuance of modern distributed database systems.</p><p>Anyway, thanks for reading, until next time!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9aa51b793e7d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Replication & Sharding: What’s The Difference?]]></title>
            <link>https://forreya.medium.com/replication-sharding-whats-the-difference-f221a03cabaa?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/f221a03cabaa</guid>
            <category><![CDATA[database-replication]]></category>
            <category><![CDATA[sharding]]></category>
            <category><![CDATA[shard-key]]></category>
            <category><![CDATA[relational-databases]]></category>
            <category><![CDATA[horizontal-scaling]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Tue, 28 Jan 2025 16:27:30 GMT</pubDate>
            <atom:updated>2025-01-28T16:27:30.749Z</atom:updated>
            <content:encoded><![CDATA[<p>Let’s clarify one key point off the bat: while <strong>replication</strong> and <strong>sharding</strong> are often mentioned together, they are fundamentally different processes. However, both aim to achieve the same overall goals- performance, scalability, and reliability.</p><h4>Common Database Challenges</h4><p>So let’s begin with a simple scenario. Your application has been running smoothly for years on a single database server. But with the passage of time, two key issues start to emerge:</p><ol><li><strong>Too many concurrent connections</strong>: The database struggles to handle the increasing number of simultaneous queries. Perhaps your application is becoming more popular now and therefore has more traffic than ever before.</li><li><strong>Slower queries</strong>: Each transaction involves so much data that execution times start to degrade significantly.</li></ol><p>So how could you address these challenges?</p><h3>Replication</h3><p>One solution could involve creating copies of your database and distributing them across multiple servers. Although this sounds like an easy solution, this process is not as simple as it sounds. Maintaining data integrity across copies involves tradeoffs and nuances that needs to be considered.</p><h4>Leader-Follower Replication</h4><p>A common replication model is the <strong>leader-follower replication </strong>model. Here’s how it works:</p><ul><li>The <strong>leader</strong> database server is the primary source of truth. It handles all <strong>write operations</strong> and replicates its data to one or more <strong>follower</strong> servers. In many systems, the follower servers themselves are able to replicate their own data into other followers too.</li><li>Followers are read-only by design, keeping the leader as the single source of up-to-date data. This avoids data inconsistencies that could arise if followers allowed writes, as newly updated data would never actually be updated into the leader node if it were to be written into a follower.</li></ul><p>With this setup, <strong>read traffic scales effectively</strong>, making it ideal for applications where reads dominate.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nF_j5oXSoD9XA5FxI1lvaQ.png" /></figure><h4>Leader-Leader Replication</h4><p>With<strong> leader-leader replication, </strong>we allow all nodes to handle both reads and writes. Unlike the single-source data replication style in leader-follower, cloning data between multiple leaders can get very complicated. Since there are multiple write sources now, we lose some data consistency between the nodes. The main tradeoffs are:</p><ul><li><strong>Consistency issues</strong>: Since multiple nodes can write simultaneously, data inconsistency is usually larger.</li><li><strong>High latency</strong>: Having to constantly perform updates between multiple leaders can significantly slow down database performance.</li></ul><p><strong>Use case</strong>: Leader-leader replication is useful for geographically distributed systems. For example:</p><ul><li>Nodes in Asia have an Asia-based leader.</li><li>Nodes in the US have a US-based leader. Data synchronization between regions can occur asynchronously (e.g., every hour), while nodes within the same region remain more consistent by updating each other at shorter intervals, or maybe even synchronously.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5j90oN6_fKdIEQC4CjXHsg.png" /></figure><h4>Methods of Replication</h4><p><strong>Asynchronous Replication: </strong>Updates to followers are delayed and occur periodically which means followers may temporarily store slightly stale data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k-h8zGVeAWdDgvJRVTPxmg.png" /></figure><p><strong>Synchronous Replication</strong>: Updates to followers happen immediately after a write to the leader. Although data consistency will be maintained here, there will be higher latencies for write operations, especially so with globally distributed followers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u-SXFrWnKFFIhf73UwnDHA.png" /></figure><h4>Benefits of Replication</h4><p>So to wrap up database replication, let’s summarise its two primary benefits:</p><ol><li><strong>Improved Read Scalability</strong>: Having the total incoming load split across multiple followers means that the system is able to handle more concurrent read traffic.</li><li><strong>Increased Fault Tolerance</strong>: Followers act as backups. If the leader fails, a follower can take over as the new leader.</li></ol><h3>Sharding</h3><p>While <strong>replication</strong> can solve the issue of handling more concurrent connections, it doesn’t address the problem of database size. This is where <strong>sharding</strong> comes in.</p><p>As databases grow over time, a point comes when the sheer volume of data becomes a bottleneck. This growth impacts query performance, causing even basic queries to take seconds — or longer — to return results. We’re talking about datasets in the petabyte range, and as you can imagine, processing this amount of data will inevitably slow down the entire system.</p><p>So let’s talk about sharding. The thing about sharding is that the concept itself isn’t very hard to grasp. Implementing it in practice is where the real complexity lies.</p><h4>What is Sharding?</h4><p>Sharding is the process of <strong>splitting a database into smaller, more manageable parts</strong> called <strong>shards</strong>. Unlike replication, which copies data across multiple servers, sharding divides the data itself.</p><p>For instance:</p><ul><li>Imagine a massive table containing millions of rows. With sharding, you could place half of the rows on <strong>Machine 1</strong> and the other half on <strong>Machine 2</strong>.</li><li>Each shard is essentially a smaller database, allowing it to handle queries more efficiently by working with a smaller amount of data using the same amount of dedicated resources.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y3u414bdj6E6syMLlxWppA.png" /></figure><h4>Benefits of Sharding</h4><p>Sharding has two major benefits:</p><ol><li><strong>Scalability</strong>: Each shard can process its own subset of data, improving query performance.</li><li><strong>Concurrent Traffic Handling</strong>: By distributing data across multiple servers, the system can manage higher traffic without overloading any single node.</li></ol><h4>Tradeoffs of Sharding</h4><p>When we perform sharding, we lose some of the consistency that we usually get with ACID compliancy, such as:</p><ul><li><strong>Foreign key constraints</strong>: Relationships between tables may span multiple shards, making enforcement difficult.</li><li><strong>Unique constraints</strong>: Since each shard operates independently, ensuring uniqueness across all shards becomes complex.</li></ul><p>These issues require very careful planning and most of the time, custom solutions tailored to the application at the application level is needed to maintain these schema rules.</p><h4>Shard Keys</h4><p>Effective sharding is determined by <strong>how data is distributed across the shards</strong>. This is usually done using a <strong>shard key</strong>, which is a field or combination of fields that determines which shard a piece of data belongs to. Let’s look at two common approaches to defining shard keys.</p><h4>1. Range-Based Sharding</h4><p>In this method, data is divided into ranges based on the shard key. Some examples include:</p><ul><li>The shard key being the <strong>ID</strong> field. Rows are distributed to shards based on the ID value falling within specific ranges (e.g., 0–1000 for Shard 1 and 1001–2000 for Shard 2).</li><li>The shard key being the<strong> first letter of user’s</strong> <strong>last name</strong>. Rows are distributed based on the alphabetical range of the last name (e.g., A–L for Shard 1 and M–Z for Shard 2).</li></ul><p>This approach is relatively simple to implement and works well when traffic is predictable. While range-based sharding splits data evenly across shards in theory, it often results in <strong>uneven query loads</strong> in practice. This is because some shards may end up storing “hotspot” data that is accessed more frequently than others, leading to certain shards becoming overloaded while others remain underutilized.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*W2ZK9iSZEbSq_WjzXBV6RQ.png" /></figure><h4>2. Hash-Based Sharding</h4><p>Here, a hash function is applied to the shard key to determine the shard.<br>Instead of simply using a range query, a consistent hashing algorithm can distribute values more randomly across shards, which avoids the <strong>hotspot problem</strong> seen in range-based sharding. The randomness of hashing ensures that:</p><ol><li>Data distribution is more scattered across all shards.</li><li>Each shard is more likely to handle an equal share of the overall workload.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Zuo2GIjNachKC38SrzhpA.png" /></figure><h4>Handling JOIN Operations Across Shards</h4><p>One of the most complex aspects of sharding is performing <strong>JOIN operations</strong> across multiple shards.</p><ul><li>With a single table, performing data fetching is relatively straightforward.</li><li>However, as more tables and relationships come into play, querying data across shards becomes increasingly complex.</li></ul><p>Custom application logic is often needed to manage these kind of queries without experiencing a degradation in performance. This logic usually depends heavily on the application’s specific requirements and database structures.</p><p>As you can see from everything we have just discussed, horizontally scaling relational database systems can get pretty complicated. This is just the tip of the iceberg, but hopefully this article gave you a little bit of insight to how replication &amp; sharding works at the surface level.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f221a03cabaa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Rise of NoSQL: What Sets Document, Wide-Column, and Graph DBs Apart]]></title>
            <link>https://forreya.medium.com/the-rise-of-nosql-what-sets-document-wide-column-and-graph-dbs-apart-778da4b2614e?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/778da4b2614e</guid>
            <category><![CDATA[graph-database]]></category>
            <category><![CDATA[wide-column-database]]></category>
            <category><![CDATA[eventual-consistency]]></category>
            <category><![CDATA[sharding]]></category>
            <category><![CDATA[nosql]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Fri, 17 Jan 2025 17:46:37 GMT</pubDate>
            <atom:updated>2025-01-17T17:48:16.911Z</atom:updated>
            <content:encoded><![CDATA[<p>If you’re reading this article, chances are you already have some familiarity with Relational Database Management Systems (RDBMS). These systems, often referred to simply as relational databases, rely on SQL to query and organize data in structured, interrelated tables. NoSQL databases, as the name suggests, moves away from SQL and the concept of rigid relationships between data. This means operations like JOIN—a staple in relational databases—are off the table.</p><p>Now why would we even want to trade the structure and reliability of SQL for something seemingly less connected? The answer lies in the limitations that NoSQL databases were designed to overcome. Let’s see what these are.</p><h3>Scalability</h3><p>One of the most significant limitations NoSQL databases addresses is scalability. Traditional relational databases, although powerful, struggle to scale efficiently when the data load grows past a certain level.</p><p><em>Why is that?</em></p><p>Let’s break it down.</p><p>Relational databases, due to their strict adherence to ACID compliance, are quite difficult to scale efficiently. For most traditional systems, the easiest way to scale up is typically through <strong>vertical scaling</strong> — which means adding more resources (like CPU, RAM, or storage) to a single database server.</p><p>When it comes to <strong>horizontal scaling</strong>, where data is distributed across multiple servers, one might think that it would be as simple as splitting things up. In reality, it’s not that straightforward. Let’s walk through an example to see why.</p><h3>Splitting a Relational Database</h3><p>Imagine we divide the data in a relational database into two parts: one half goes to <strong>Shard 1</strong>, and the other half to <strong>Shard 2</strong>. At first glance, this seems manageable, but immediately we can notice a few glaring points to consider.</p><h4><strong>Routing Data Correctly</strong></h4><p>Splitting the database into “shards” (segments of data) introduces a new complexity: <strong>how do we ensure we route queries to the correct shard</strong>. For example, what if a query needs data stored on Shard 1 but instead gets routed to Shard 2?</p><ul><li><strong>How is this usually solved?</strong> Shard routing often relies on techniques like consistent hashing, but implementing and maintaining these systems can be pretty complex.</li></ul><p>Now let’s jump onto the next jarring issue.</p><h4><strong>JOINs Across Shards</strong></h4><p>Relational databases thrive on relationships and rely heavily on JOIN operations to combine data from different tables. However, when data is spread across multiple shards:</p><ul><li>A query involving a JOIN might need data from both Shard 1 and Shard 2. This results in <strong>cross-shard queries</strong>, which are highly resource-intensive and can negate the performance benefits of sharding.</li><li><strong>Enforcing constraints</strong> such as foreign keys, unique keys, etc, becomes a nightmare. These relationships now span multiple independent shards, and maintaining them requires significant overhead and constant synchronization.</li></ul><p>By this point, hopefully it’s clear to you why relational databases struggle with horizontal scaling. Their architecture is designed for consistency and structure, but the tradeoff is that distributing data dynamically is very complex.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*-vrkTOuYZWXTJ-k67eTnTA.png" /></figure><p><strong>Let’s continue by exploring the various types of NoSQL Databases.</strong></p><h3><strong>Key-Value Stores</strong></h3><p>Some might argue that key-value stores don’t even qualify as “real” databases. The main argument? They are often <strong>in-memory systems</strong> designed to minimize read and write latencies rather than serving as primary data storage solutions. Unlike traditional relational databases, key-value stores don’t rely on disk storage, focusing instead on simplicity and speed. At their core, they are essentially glorified hash maps.</p><p>Here’s an example of how data might look in a key-value store:</p><ol><li><strong>ID</strong> → nx87tshx</li><li><strong>Name</strong> → Froggy</li><li><strong>Address</strong> → { &quot;street&quot;: &quot;Green Road&quot;, &quot;town&quot;: &quot;Toad Town&quot;, &quot;postCode&quot;: &quot;GREEN12&quot; }</li></ol><p>As you can see, key-value stores are structured similarly to JSON objects:</p><ul><li>Each <strong>key</strong> is unique, and its associated <strong>value</strong> can be anything — from simple data types like strings or integers to more complex structures like objects or nested data.</li><li>However, they <strong>don’t inherently support relationships</strong> between keys or enforce constraints (e.g., no schemas or foreign keys).</li></ul><h4><strong>Benefits of Key-Value Stores</strong></h4><p>If your workload involves <strong>frequent reads and writes of simple data</strong>, and <strong>speed</strong> is the primary concern, key-value stores are an excellent solution. This makes them ideal for use cases such as:</p><ul><li><strong>Caching Layers</strong>: Often paired with a primary database to improve response times.</li><li><strong>Session Management</strong>: Storing user sessions in web applications.</li></ul><p>Some of the most widely used key-value stores include <strong>Redis, Memcached and etcd.</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*H_KRb_cjLPWRjuKqRM4GtQ.png" /></figure><h3>Document Databases</h3><p>Next up, we have <strong>document databases</strong>, also known as document stores. Think of these as the next level up from key-value stores. While they share some similarities, document databases introduce a more structured way of organizing data. In a way, they bridge the gap between key-value simplicity and relational database complexity.</p><p>In a document database, data is organized into <strong>collections</strong> (analogous to tables in relational databases), which contain <strong>documents</strong> (similar to rows). Here’s an example of how a document might look:</p><pre>{<br>    &quot;id&quot;: &quot;nx87tshx&quot;,<br>    &quot;name&quot;: &quot;Froggy&quot;,<br>    &quot;email&quot;: &quot;froggyluvsflies@example.com&quot;,<br>    &quot;age&quot;: 3,<br>    &quot;address&quot;: {<br>        &quot;street&quot;: &quot;Green Road&quot;,<br>        &quot;town&quot;: &quot;Toad Town&quot;,<br>        &quot;postCode&quot;: &quot;GREEN12&quot;<br>    }<br>}</pre><p>Notice anything familiar? It’s essentially a JSON object, and that’s no coincidence. Many developer love this kind of data format due to it’s readability and ease of use.</p><h4><strong>Benefits of Document Stores</strong></h4><p>The biggest advantages of document stores are:</p><ol><li><strong>Scalability</strong>: Like other NoSQL databases, document stores are designed to handle massive data loads and can scale horizontally with ease.</li><li><strong>Flexibility</strong>: They are schema-flexible, meaning you can mix and match data types, add or remove fields, and modify the structure of your data without needing to predefine a strict schema.</li></ol><p>These characteristics make document databases perfect for scenarios where data structures tend to evolve over time or where rigid schemas are a limitation. The most widely used document database is undoubtedly <strong>MongoDB</strong>, a name which has become synonymous with this type of database model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/750/1*wMh0weakVwAowiMPLZ5HaA.jpeg" /></figure><h3>Wide-Column Databases</h3><p>Wide-column databases are built to handle <strong>massive scale</strong>, and are particularly optimized for workloads involving <strong>high-volume writes and reads</strong>. They are often used in scenarios where the data grows rapidly and requires fast ingestion, such as time-series data. For instance, a system logging sensor readings every second or storing user activity data in real-time is a perfect fit for this database model.</p><h4><strong>SSTables</strong></h4><p>Wide-column databases store data in <strong>SSTables</strong> (Sorted String Tables). These are<strong> immutable files </strong>that are:</p><ul><li><strong>Sorted</strong>: Ensuring efficient reads.</li><li><strong>Persistent</strong>: Stored on disk for durability.</li></ul><p>However, their <strong>immutability</strong> means that when data is updated or deleted, the existing SSTables aren’t actually modified. Instead:</p><ul><li><strong>New SSTables are created </strong>containing the updated or deleted data.</li><li>The system retains old versions of data, leading to <strong>multiple overlapping SSTables.</strong></li></ul><p>This makes wide-column databases less suited for workloads involving <strong>frequent updates or deletes</strong>, as these operations introduce additional latency and complexity.</p><h4><strong>Background Compaction: The Cleanup Process</strong></h4><p>To address the issues that arise with immutability, a process known as <strong>background compaction</strong> is employed. This involves:</p><ol><li><strong>Consolidating Data</strong>: Merging overlapping SSTables to achieve a smaller number of SSTables.</li><li><strong>Discarding Tombstones</strong>: Removing deleted data and outdated table versions that are no longer needed.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/641/1*3BGSgLJrYC70UJNmJfk4ZA.png" /></figure><h4>Schema Variants</h4><p>Wide-column databases come in two variants:</p><ol><li><strong>Schema-defined</strong>: A predefined structure for rows and columns.</li><li><strong>Schema-less</strong>: Allows flexibility in defining columns for each row, similar to document databases.</li></ol><p>This adaptability means they can be tailored for a range of use-cases, ranging from structured to semi-structured data.</p><p>The most well-known wide-column databases includwe <strong>Cassandra, </strong>which is an open-source database inspired by Google’s Bigtable. Incidentally, Bigtable itself is a highly powerful and distributed database system, used internally by Google for large-scale applications.</p><h3>Graph-Based Databases</h3><p>Despite the term NoSQL being often used interchangably with non-relational, graph-based databases are actually all about <strong>relationships</strong>. Consider a social media platform like Twitter or Instagram: every user has followers and follows others. Interlinked relationships such as these are best represented using a <strong>graph structure</strong>, which models entities and their connections intuitively.</p><p>In a graph database, the data is structured as:</p><ul><li><strong>Vertices</strong>: Represent entities such as users, products, or places.</li><li><strong>Edges</strong>: Represent relationships between entities, like follows, likes, etc.</li><li><strong>Properties</strong>: Attributes of vertices or edges, such as a user’s name or the type of relationship between two users.</li></ul><h4>The Main Benefit</h4><p>The key strength of graph-based databases is their ability to handle <strong>relationship queries</strong> efficiently. While relational databases struggle with joins across multiple tables as relationships deepen further down to multiple derivatives, graph databases are purpose-built for such queries. In other words, they are optimized for querying and analyzing deeply linked data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8bJL1LN0UTVLz8zqUQLgxA.png" /></figure><h4>Real-World Use Cases</h4><p>To reiterate, graph databases are <strong>all about relationships between entities</strong>. This makes them extremely suitable for scenarios such as:</p><ol><li><strong>Social Networks: </strong>Find all friends of a user within 3 degrees of separation (mutual friends).</li><li><strong>Recommendation Engines: </strong>Suggest products, movies, or friends based on relationships and preferences.</li><li><strong>Fraud Detection: </strong>Identify suspicious transaction patterns or unusual activity using graph traversal and analytics.</li></ol><h3>Simplifying Horizontal Scaling</h3><p>Now then, back to the big picture. As we talked about early on in the article, horizontal scaling in relational databases is challenging, largely due to the constraints imposed by ACID compliance. While this guarantees data reliability and consistency, they complicate scaling across multiple nodes.</p><p>NoSQL databases, on the other hand, take a different approach:</p><ul><li><strong>Relaxed ACID Compliance</strong>: Strict consistency is often traded for eventual consistency (which we will talk about in a second), making distributed operations simpler.</li><li><strong>Schema Flexibility</strong>: NoSQL’s schema-flexible design allows data to be distributed and scaled easier without the enforcement of rigid constraints complicating the process.</li><li><strong>Built-In Sharding</strong>: Most NoSQL databases come with automatic sharding mechanisms, which manages data distribution, query routing, and load balancing off the shelf.</li></ul><h4>Eventual Consistency</h4><p>We briefly touched upon this a second ago, but let’s talk about one of the defining characteristics of many NoSQL databases: <strong>eventual consistency</strong>.</p><p><strong>How It Works<br></strong>Imagine a NoSQL database replicated (not split) across two nodes, <strong>Node 1</strong> and <strong>Node 2</strong>:</p><ol><li>A write operation occurs on <strong>Node 1</strong> (the leader).</li><li>The updated data doesn’t immediately replicate to <strong>Node 2</strong> (the follower).</li><li>If a read query is performed on <strong>Node 2</strong> before the update arrives, <strong>slightly outdated data</strong> will be returned.</li><li>Periodically, the leader synchronizes changes with its followers, ensuring that all nodes eventually reach the same state — this is what <strong>eventual consistency </strong>is.</li></ol><p>Usually, all write operations typically go through the leader node. Follower nodes are simply read-only replicas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*u7HQREM2g26FuVhchRfzrA.png" /></figure><h4>Trade-Offs</h4><p>NoSQL databases accept that some users may see <strong>slightly outdated data</strong> for a short period of time. However, this trade-off is usually acceptable for scenarios where real-time accuracy isn’t critical, such as:</p><ul><li><strong>Follower counts</strong> on social media platforms.</li><li><strong>The number of likes</strong> on a post or video.</li></ul><p>Through replication and eventual consistency, NoSQL databases can handle significantly higher read traffic compared to relational databases. However, the system’s write capacity is still limited by the leader node, so this model is ideal for use-cases prioritizing read scalability over write scalability.</p><h4>A Note on Scaling Relational Databases</h4><p>I wanted to clarify something before we end. Relational databases, in fact, <strong>can</strong> scale horizontally using techniques like sharding and replication. However, implementing them often requires:</p><ul><li><strong>Significant architectural changes</strong>: Sharding strategies or database replication mechanisms must be carefully designed and tailored to the specific use case.</li><li><strong>Custom application-level logic</strong>: Developers need to explicitly manually manage data distribution, routing, and consistency on the application level, adding a ton of additional complexity to the system.</li></ul><p>In contrast, most NoSQL databases offer these features <strong>out of the box.</strong></p><h3>Why NoSQL Was Introduced</h3><p>To summarize, here’s what NoSQL databases bring to the table:</p><ol><li><strong>High Scalability — </strong>Ideal for systems with massive amounts of data and high traffic due the simplified and often automated nature of horizontal scaling.</li><li><strong>Flexibility — </strong>Schema-flexible designs allow data structures to evolve without requiring costly migrations, especially in scenarios where rigid table structures aren’t especially needed.</li><li><strong>Global Distribution — </strong>Ideal for systems that require geographically distributed nodes for lower geographical latencies, all thanks to simpler horizontal scaling mechanisms!</li></ol><p>And that’s a wrap! Hopefully this gave you a clear understanding of how NoSQL databases differ from their relational counterparts and why they’ve become so popular in this current day and age. Thanks for reading, see you in the next one!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=778da4b2614e" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Relational Databases: B+ Trees, SQL, Transactions, and ACID Compliance]]></title>
            <link>https://forreya.medium.com/relational-databases-b-trees-sql-transactions-and-acid-compliance-a57ee9e2d781?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/a57ee9e2d781</guid>
            <category><![CDATA[relational-databases]]></category>
            <category><![CDATA[sql]]></category>
            <category><![CDATA[database-transaction]]></category>
            <category><![CDATA[dbms]]></category>
            <category><![CDATA[acid-properties]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Sat, 11 Jan 2025 14:40:44 GMT</pubDate>
            <atom:updated>2025-01-20T07:15:13.794Z</atom:updated>
            <content:encoded><![CDATA[<p>One of the primary methods for storing data in applications is by using databases. Among the many types available, relational databases are used for organizing data into structured tables with clearly defined relationships. These databases are managed using a specialized query language known as SQL (Structured Query Language). Systems that combine relational databases with SQL are referred to as Relational Database Management Systems (RDBMS), and these will be the focus of this article.</p><h3>The Mechanics</h3><p>To begin, relational databases store their data on disk for <strong>data persistence</strong>, which ensures that the data remains intact in the event of the database crashing or going offline. Under the hood, relational databases rely on a data structure called the <strong>B+ tree</strong> for efficient data organization and retrieval. You can think of it as an optimized variation of the binary tree.</p><h4>What is a B+ Tree?</h4><p>Unlike a binary tree, where each node can have at most two children, a B+ tree allows each node to have multiple children (up to <strong>M children</strong>, where M is the “order” of the tree).</p><p>The B+ tree is composed of two types of nodes: <strong>internal nodes</strong> and <strong>leaf nodes</strong>. Internal nodes function as guides, containing keys (values) that direct the traversal to the appropriate child node. This process continues until reaching the leaf nodes, which store the actual data or references to it. A key feature of the B+ tree is that all leaf nodes are linked together in a sorted <strong>linked list</strong>, making it easy to perform range queries by traversing through them in order.</p><p>Another advantage of the B+ tree is its <strong>sorted order</strong>: keys within each node are kept in ascending order, making it easy to quickly decide where to traverse down next in order to find the desired data. Since each node can hold multiple keys (or values), the tree stays relatively short and balanced, which keeps searching fast by minimizing the number of levels that need to be traversed.</p><p>To summarize the key advantages of B+ trees:</p><ul><li><strong>Fast Lookups</strong>: The sorted keys and hierarchical structure enable quick navigation to the target data.</li><li><strong>Efficient Range Queries</strong>: The sorted linked list structure of leaf nodes makes retrieving a range of values very straightforward and efficient.</li><li><strong>Balanced and Compact</strong>: The ability to store multiple keys per node keeps the tree balanced and reduces its height, resulting in fewer layers to traverse through during searches.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_GkRfef1owvibeEjdyMjYQ.png" /></figure><h3>Tables</h3><p>In relational databases, data is organized into <strong>tables</strong>, which are defined by a <strong>schema</strong>. The schema specifies the structure of the table, including the fields (columns) and their data types. For example, the following query creates a table called Frog with three fields: PassportNumber, Name, and Age.</p><pre>CREATE TABLE Frog (<br>    PassportNumber varchar(50),<br>    Name varchar(100),<br>    Age int<br>);</pre><p>In this table, the actual data entries are referred to as <strong>rows</strong>. Each row represents a single instance of data that abides by the structure set by the table’s schema. For example, a frog named ‘Jack’ with an age of 100 would be a row in the Frog table.</p><h4>Table Relationships</h4><p>Multiple tables can also have relationships between them. For instance, imagine a world where frogs can own houses. In this case, a separate table named House could be created with the following schema:</p><pre>CREATE TABLE House (<br>    OwnerPassportNumber varchar(50),<br>    Address varchar(100)<br>);</pre><p>In this setup, we could enforce a rule requiring that every OwnerPassportNumber in the House table must correspond to a valid PassportNumber in the Frog table. This ensures that each house has an owner who is an existing frog. In SQL, a rule like this is called a <strong>constraint</strong>, and specifically, this type of relationship is implemented using a <strong>foreign key</strong>, which establishes a link between two tables. In our example, the OwnerPassportNumber field in the House table would act as the foreign key, ensuring that it always references a valid row in the Frog table.</p><p>Additionally, every table requires a <strong>primary key</strong> to uniquely identify each row. A primary key can consist of one or more columns, but it must ensure that no two rows share the same value for the key. In the Frog table, the PassportNumber column is a natural choice for the primary key, as each frog must have a unique passport number. This unique identifier allows for relational databases to have efficient data organization and also allows them to setup relationships between tables, as we have just seen.</p><p>These rules and constraints form the backbone of relational databases, and is the reason why relationships between tables are even possible — something that non-relational databases typically cannot achieve. Leveraging these relationships, relational databases are able to support complex queries, such as <strong>JOIN operations</strong>, which combine data from multiple interrelated tables.</p><p>For example, let’s take a look at the following query:</p><pre>SELECT Frog.PassportNumber, House.Address<br>FROM Frog, House<br>WHERE Frog.PassportNumber = House.OwnerPassportNumber;</pre><p>In this query, we are accessing data from two tables by joining them through the common field PassportNumber. The query returns the passport number and the address of the frog that owns the house. While this example is a little contrived, it helps serve as a simple example of how table relations can help us craft more complex queries.</p><h3><strong>Database Transactions</strong></h3><p>An important concept when it comes to relational databases are <strong>transactions</strong>, which represents a single unit of work in a database. They consist of a sequence of one or more operations executed on the database. Transactions allow developers to group related operations, ensuring they are executed in a controlled manner. This is a little abstract for now, so let’s see what this means exactly.</p><ol><li><strong>BEGIN</strong>: A keyword that marks the start of a transaction.</li><li><strong>Update</strong>: Modify a specific frog’s data (e.g., changing its name or age).</li><li><strong>Insert</strong>: Add a new frog to the database.</li><li><strong>Select</strong>: Retrieve a set of frogs based on specific criteria (e.g., frogs older than 50 years).</li><li><strong>COMMIT</strong>: Marks the successful completion of the transaction, ensuring all changes are saved permanently.</li></ol><p>If any step within the transaction fails, the system ensures the database remains in its previous consistent state by <strong>rolling back</strong> any changes made during the transaction.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1000/1*bIicvPFDqSPM9T_cvnZ47Q.png" /></figure><h3>The ACID Properties</h3><p>Most relational databases adhere to the ACID properties — Atomicity, Consistency, Isolation, and Durability. These properties ensure that relational databases maintain data integrity even during system failures or concurrent operations. Let’s go through the acronym letter-by-letter.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*TFqehoWUxqGG7MpyzbnPMQ.png" /></figure><h4>1. Atomicity (A)</h4><p>Atomicity ensures that a transaction is <strong>all or nothing</strong>. Either every operation within a transaction completes successfully, or none of them do. If any operation fails, the database reverts to its previous state as if the transaction never occurred. For example:</p><ul><li>If a transaction involves updating a frog’s data and inserting a new record, both actions must succeed for the transaction to commit. If one fails, neither change will persist.</li></ul><h4>2. Consistency (C)</h4><p>Consistency ensures that the database remains in a valid state before and after a transaction. It enforces all rules, constraints, and relationships defined in the database schema. For instance:</p><ul><li>The column name in the Frog table may have a rule that it cannot contain NULL values. Consistency ensures that no transaction violates this rule.</li></ul><p>Som examples of constraints include:</p><ul><li>Unique values for specific columns.</li><li>Enforcing foreign key relationships between tables.</li></ul><h4>3. Isolation (I)</h4><p>Isolation ensures that concurrent transactions do not interfere with each other. Even if multiple transactions are happening simultaneously, each one operates as though it is the only one running.</p><p>Without proper isolation, issues like <strong>dirty reads</strong>, <strong>phantom reads</strong>, or <strong>non-repeatable reads</strong> can occur. For example:</p><ul><li>Transaction 1 increments the age of a frog named Greg from 20 to 22.</li><li>While Transaction 1 is in progress, Transaction 2 increments Greg’s age by 2.</li><li>If Transaction 1 fails and is rolled back, Greg’s age should remain at 20. However, without isolation, Transaction 2 might rely on the uncommitted value (22) and save Greg’s age as 24 instead of 22. This inconsistency is called a <strong>dirty read</strong>.</li></ul><p>To prevent such issues, relational databases often serialize transactions to enforce isolation by ensuring that multiple transactions are executed in an order that produces the same result as if they were run one at a time.</p><h4>4. Durability (D)</h4><p>Durability ensures that once a transaction is committed, its changes are <strong>permanently saved</strong> to disk. This means that even if the system crashes after a transaction is committed, the data will not be lost.</p><p>For example:</p><ul><li>When you commit a transaction to update a frog’s age, the change is written to disk and guaranteed to persist — even if the database system crashes immediately afterward.</li></ul><p>It’s important to note that uncommitted transactions are not guaranteed to persist. Only after a <strong>COMMIT</strong> does the database ensure that the changes are saved permanently.</p><p>And that’s about it! I don’t like to write summaries if they aren’t needed so apologies for the abrupt endi-</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a57ee9e2d781" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Mechanics of Consistent Hashing & Stateful Load Distribution]]></title>
            <link>https://forreya.medium.com/the-mechanics-of-consistent-hashing-stateful-load-distribution-f2b8d6da1ed7?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/f2b8d6da1ed7</guid>
            <category><![CDATA[consistent-hashing]]></category>
            <category><![CDATA[load-balancing]]></category>
            <category><![CDATA[hash-function]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[load-balancer]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Tue, 07 Jan 2025 17:08:19 GMT</pubDate>
            <atom:updated>2025-01-07T17:08:19.498Z</atom:updated>
            <content:encoded><![CDATA[<p>The way in which we distribute traffic across servers depends on the application and the nature of the data we are handling. One effective method of load balancing is to use a method known as consistent hashing. Let’s begin with a simplified example to understand its benefits and scenarios where it shines.</p><p>To illustrate consistent hashing, consider a simplified example: mapping users to servers based on their IP addresses. Suppose our application has 3 servers, and a user with an IP address of 9 sends a request. Using a basic hashing function — such as taking the remainder of the IP divided by the total number of servers — we can assign the user to a server. In this case, an IP address of 9 will be mapped to the first server (server 0), as 9 mod 3 is 0. The result ensures that a user with the same IP will always be mapped to a specific server. This deterministic routing is ideal for certain systems, as we will see in a minute.</p><h4>Benefits of Consistent Hashing</h4><p>Now what scenarios exactly would consistent hashing be useful in? In architectures like most RESTful APIs, we know that server interactions are designed to be stateless in nature- and therefore consistent routing would be unnecessary. However, in <strong>stateful systems </strong>where each server individually maintains internal state, maybe using an in-memory Redis cache, consistent hashing becomes crucial.</p><p><em>Why?</em></p><p>Going back to our previous example, having a stateful systems means that requests mapped to <strong>Server 0</strong> will find their cached data only on that server. If routing is inconsistent, subsequent requests may be routed to different servers, making cached state unavailable.</p><p>With a different method like round robin, the server that a user is allocated to could vary from request to request, meaning its previous state would not be be guaranteed to still be available in a subsequent request. All in all, the main point is that consistent hashing is essential for systems where maintaining continuity in user-server mapping is critical.</p><h4>Server Removal</h4><p>What happens if a server is removed? Let’s revisit our 3-server example. Removing <strong>Server 2</strong> ideally redistributes its load evenly across the remaining servers while preserving mappings for users of <strong>Server 0</strong> and <strong>Server 1</strong>. However, with a simple hashing function, reducing the total servers actually alters the hash distribution. For instance:</p><ul><li>Previously, 9 mod 3 = 0, mapping to <strong>Server 0</strong>.</li><li>With only two servers, 9 mod 2 = 1, now routing to <strong>Server 1</strong>.</li></ul><p>This change would disrupt the mapping, as users will lose access to their cached states on their original servers, which means this situation is not ideal in stateful systems. So how can we optimize this?</p><h4>Improving the Simplified Model</h4><p>To address the issue above, consistent hashing maps servers onto a <strong>circular space and takes advantage of the angular properties of a circle</strong>. Here’s how it works:</p><ol><li>Servers are represented as points on a circle, e.g., at 0°, 120°, and 240°.</li><li>Requests are hashed to a point on the circle’s circumference.</li><li>Requests are routed to the next server in the <strong>clockwise direction</strong>.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BhfT9GdM7SAnoFZh0dcj1g.png" /></figure><p>Instead of directly hashing and mapping requests to a specific server, consistent hashing will now instead map requests to some point on a circle’s circumference. Let’s say a user named Charles is mapped to somewhere between server A and B. On the diagram, this would be one of the blue squares. The mapping function will map Charles’s requests to the closest server (or node) in the clockwise direction. So in this case, Charles would be mapped to server A.</p><blockquote>From this point on, we will be referring to the servers as “nodes” to generalize the example as consistent hashing doesn’t just apply to application servers, but also to database partitions and other forms of distributed systems.</blockquote><p>Before, we were simply modding the IP address by the number of servers. With consistent hashing, we are mapping to a circle and therefore the hashing function would change to instead mod by 360. This way, requests are consistently mapped to somewhere along the circle’s circumference. If we now remove Node A, all the yellow requests will still be routed to Node B, and all the red requests will still be routed to Node C. By doing this, we are able to maintain user mapping consistency with the remaining servers- unlike before. Where will the blue users go though? Well, since Node A will now no longer exist- all these requests will now be mapped to Node C, as that would be the new closest server in the clockwise direction. In an ideal world, we would want these requests who ‘lost’ their original server to be distributed evenly among Node C and Node B. Although we won’t go into the solutions for this issue in this article, it’s important to be aware of the fact that solutions for this do exist.</p><h4>Adding Nodes</h4><p>Now what about the alternate scenario? What if we added a new server somewhere on the circle? Let’s say we add a Node D halfway between Node A and Node C. In this scenario, roughly half of the requests will continue to be mapped to Node C but the other half will now be re-mapped to the new Node D, meaning these users will lose access to the cached data on Node C in subsequent requests. But, if we did add a new server, we probably did that to scale up the system anyways so it would be fine to lose some cached data to be re-distributed to Node D as a side-effect. At least the majority of users, other than the small portion that now goes to Node D, are still being routed consistently to their original designated servers as before, minimising routing changes.</p><blockquote>Remember, if users don’t actually need to consistently be mapped to the same server- a simpler approach like round robin would be sufficient. In a lot of cases, the complexity of implementation that comes with consistent hashing is not worth it if we don’t actually require the consistent routing that comes with it.</blockquote><h4>Real-Life Implementations</h4><p>One popular type of system that uses consistent hashing is CDNs, as you CDN servers serve cached data, and therefore it would be in their best interest to maintain individual user routing when it comes to serving static content. Another use case is databases partitions. In some cases, we might split a database’s data up into multiple “partitions” in order to handle more traffic. In cases like these, we would need a way to evenly split and distribute the requests in a consistent way across partitions, and also to ensure minimal disruption when partitions are added or removed.</p><h4>Extra Bits</h4><p>I thought I would quickly mention another hashing algorithm that is similar to consistent hashing- rendezvous hashing. Conceptually, it’s pretty similar in what it attempts to accomplish, although it differs in it’s mechanism. While rendezvous hashing is simpler in most scenarios, you could pretty much get by with just understanding consistent hashing.</p><p>To finally top off this article, I’ll add some keywords that are important to be familiar with when it comes to consistent hashing.</p><ol><li>Hash Key — A unique identifier (e.g., an IP address) that determines a user’s unique mapping to a specific node.</li><li>Hash Function — For pedagogical purposes, this was modding the IP address by the number of servers or by 360 when using the circular method. However, in real life this would usually be some industry-grade hashing function such as SHA (Secure-Hashing Algorithm) or some variation of that.</li><li>Nodes — This could be servers such as CDN servers, or even database partitions or shards. The term node is a general definition that refers to all these items.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f2b8d6da1ed7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Forward & Reverse Proxies: How This Relates To Load Balancers]]></title>
            <link>https://forreya.medium.com/forward-reverse-proxies-how-this-relates-to-load-balancers-f8a75aa8a024?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/f8a75aa8a024</guid>
            <category><![CDATA[load-balancer]]></category>
            <category><![CDATA[forward-proxy]]></category>
            <category><![CDATA[load-balancing]]></category>
            <category><![CDATA[reverse-proxy]]></category>
            <category><![CDATA[osi-model]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Wed, 11 Dec 2024 14:58:11 GMT</pubDate>
            <atom:updated>2024-12-11T14:58:11.856Z</atom:updated>
            <content:encoded><![CDATA[<p>So let’s jump right into it with the introduction of proxies. Like many things in programming, proxies comes in two main types: forward and reverse.</p><h4>Forward Proxies</h4><p>Most of the time, when people talk about proxies, they’re usually referring to forward proxies. Let’s walk through an example.</p><p>Imagine you’re a client trying to send a request to a destination server. Pretty simple. Now, let’s introduce a middle server between you and that destination server. This middle server, the proxy, would take your request and forward it to the actual destination server on your behalf. The twist is that the proxy acts as if it’s the one making the request, so the destination server doesn’t know the request initially came from you. It essentially hides your IP address and identity.</p><p><em>This sounds a little contrived, why do we need this “middle-layer”?</em></p><p>Well, imagine your computer isn’t allowed to access a certain server because of restrictions imposed by your country, workplace, or school. A forward proxy can get around this by sending the request on your behalf, effectively bypassing those restrictions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dctBMFpOt9KXPGsQbCe_ng.png" /></figure><p>Forward proxies can also block requests. For example, in schools or offices, it’s quite common to have systems that connect all devices to a forward proxy in order to block access to social media sites like Instagram or Facebook. This is what’s called a corporate proxy.</p><p>Another popular use for forward proxies is with VPNs, as they hide your identity —making it seem like your requests are coming from a completely different country in order to bypass governmental restrictions.</p><h4>Reverse Proxies</h4><p>While forward proxies hide the client, reverse proxies do the opposite: they hide the destination server. So what does this actually mean?</p><p>As a user, you might send your request to a reverse proxy. This proxy, however, doesn’t actually handle your request itself. Instead, it forwards your request to the real destination server, which processes the initial request. The key thing to note is that you, the client, have no idea the destination server even exists. All you know is that you’ve sent a request to the reverse proxy and received a response back. On the flip side, the destination server often knows about you, but not always.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*T-9M7iACkdl-bbluudDEXw.png" /></figure><p>A common example of a reverse proxy is a <strong>Content Delivery Network (CDN)</strong>. When you request static content from an application, such as an image or a video, a CDN server might fulfill that request directly. However, if the specific content isn’t currently cached on the CDN server, it would then forward the request to the origin server to retrieve the data. This entire internal fetching process is invisible to you, the client, because it’s managed seamlessly behind the scenes. From the client’s perspective, the only thing that matters is receiving a response to their request — it’s irrelevant whether the response came from the CDN’s cache or the origin server.</p><p>Another popular example is load balancers, which we’ll talk about next.</p><h4>Load Balancers</h4><p>Conceptually, understanding load balancers is fairly straightforward: they distribute incoming network traffic across multiple servers. However, their implementation in real-world scenarios can be challenging due to the complexities of networking.</p><p>In a lot of modern applications, client requests are initially directed to a load balancer. The load balancer’s primary responsibility is to distribute these requests across multiple backend servers. These servers usually run identical applications or APIs as a result of horizontal scaling to manage higher traffic volumes. If you remember from a previous article, this scalability is partly made easier if API requests are stateless, as is common with RESTful API design.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0nas7E2ailsJPQMTTQ7zEw.png" /></figure><p>All in all, the key here is that load balancers evenly distribute requests in order to ensure that resources are used efficiently and the application remains responsive, even under heavy load.</p><h4>Methods to Distribute Traffic</h4><p>It’s important to be aware of the commonly used options for distributing incoming requests:</p><ol><li><strong>Round Robin</strong><br>This is as simple as it gets. Requests are sent to servers in a fixed order: first to Server 1, then Server 2, and so on, looping back to Server 1 when it reaches the end.</li><li><strong>Weighted Round Robin</strong><br>There are cases where an application’s servers might not all be equally powerful. Some might have more resources or processing power. Weighted round robin takes this into account. Let’s say you have three servers and the first one is twice as powerful as the others, the load balancer in turn will send double the traffic to the stronger server compared to the other two. In other words, 50% of incoming traffic will go to the first server, 25% to the second, and 25% to the third.</li><li><strong>Least Connections</strong><br>Unlike simply cycling through servers, the “Least Connections” method considers the number of active connections each server has. Just because a connection was established first doesn’t necessarily mean it will finish sooner than later connections — some requests take longer to process. As a result, this method balances the workload by directing incoming requests to the server with the fewest active connections at any given time. By doing so, it ensures that no server becomes a bottleneck due to long-running connections, which helps balance the workload more dynamically.</li><li><strong>Location-Based</strong><br>We can also load balance based on user location. If a user makes a request from Southeast Asia, for instance, the load balancer might route it to a server located in Southeast Asia to reduce latency.</li><li><strong>Hashing</strong><br>Another common method for distributing traffic is by using hashing. This involves hashing the values of certain fields in a request, like the client’s IP address or request content. The resulting hash determines which server gets the request. This approach is great for randomising the flow of requests which helps to evenly distribute traffic across all of the application’s servers.</li></ol><h4>Types of Load Balancers</h4><p>Load balancers can be categorized into two main types: <strong>Layer 4</strong> and <strong>Layer 7</strong>. These correspond to different levels of the OSI model (Open Systems Interconnection) and determine how the load balancer processes traffic.</p><ul><li><strong>Layer 4 Load Balancers</strong>: Operate at the transport layer (TCP/UDP).</li><li><strong>Layer 7 Load Balancers</strong>: Operate at the application layer.</li></ul><blockquote>The OSI model is a theoretical framework that serves as a guideline for how communication over networks should happen, breaking it into seven layers. Think of it as a “conceptual” version of the TCP/IP model.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/685/1*cUjKbPQJe5ffHuNc3JcIxQ.png" /></figure><ol><li><strong>Layer 4 Load Balancers </strong><br>Layer 4 load balancers function at the transport layer, similar to the TCP/UDP protocols in the TCP/IP model. Their primary focus is speed and efficiency and therefore solely use basic information like the destination IP address to decide where to route traffic, which allows it to balance traffic using methods such as <strong>location-based balancing</strong> or<strong> round robin. </strong>However, this simplicity means that Layer 4 load balancers do not have access to the actual application data or the specific resource a user is requesting. They can only process and modify transport-level information. For example, a Layer 4 load balancer would rewrite the “destination IP address” field in the request from it’s own address to the address of the destination server when forwarding the request.</li><li><strong>Layer 7 Load Balancers</strong><br>Unlike Layer 4, Layer 7 load balancers operate at the application layer, which allows them to understand the content of incoming requests. This enables them to make intelligent routing decisions based on the type of resource being requested. For instance, imagine a system where <strong>Server A</strong> exclusively handles “Cat” resources, while <strong>Server B</strong> manages “Authentication” related functionality. A Layer 7 load balancer has the ability to analyze the application data in each request and route it to the appropriate server based on the resource type. This deep level of insight makes Layer 7 load balancers much more flexible and powerful, as they can distribute traffic more intelligently. The tradeoff, however, is increased complexity and network overhead. Because Layer 7 balancers inspect the content of each request, they need to decrypt incoming data and re-encrypt it before forwarding it to the destination server. This process requires two separate TCP connections to be established: one between the client and the load balancer, and another between the load balancer and the destination server. This additional processing introduces some additional network overhead.</li></ol><h4>Addressing the Single Point of Failure</h4><p>In a previous article, we discussed how horizontally scaling multiple servers in an application improves both <strong>availability</strong> and <strong>reliability</strong>. However, even with multiple servers, having a single load balancer in the mix still creates a <strong>single point of failure</strong>. If the load balancer goes down, the entire system would become inaccessible. So, how can we mitigate this risk?</p><p><strong>Solutions to Address Load Balancer Failures:</strong></p><ol><li><strong>Deploy Multiple Load Balancers</strong>:<br>Running multiple load balancers simultaneously provides redundancy. It ensures that the system remains operational even if one fails.</li><li><strong>Use Backup Load Balancers</strong>:<br>Backup load balancers provide a safety net in the case where the primary load balancer goes offline. In cases like this, all incoming traffic would automatically be redirected to a backup load balancer.</li></ol><h4>Load Balancer Capacity</h4><p>Typically, load balancers are designed to handle very high volumes of traffic and throughput since their primary function is nothing but forwarding requests. Because of this, it’s usually quite unlikely for a load balancer to become overloaded. However, planning for the unlikely is key when it comes to building resilient systems.</p><h4>Real-World Implementations</h4><p>Before we wrap up, let’s discuss some real-life examples of load balancers, one of which is <strong>Nginx</strong>, which offers a wide range of features, including load balancing. Additionally, many cloud providers, such as <strong>AWS</strong> and <strong>Google Cloud Platform (GCP)</strong>, offer out-of-the-box load balancers, which greatly simplifies the deployment and management process.</p><p>So if you’re interested in load balancing, it’s worth looking into these tools and services to see how they can be utilized in your own applications.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f8a75aa8a024" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Push & Pull CDNs: How They Work?]]></title>
            <link>https://forreya.medium.com/push-pull-cdns-how-they-work-1ea54e7701d3?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/1ea54e7701d3</guid>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[cdn]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[content-delivery-network]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Mon, 09 Dec 2024 17:06:07 GMT</pubDate>
            <atom:updated>2024-12-11T13:14:08.036Z</atom:updated>
            <content:encoded><![CDATA[<h3>Push &amp; Pull CDNs: How They Work</h3><p>As application designers, we love caching data — it’s one of the most efficient ways to improve performance. Content Delivery Networks (CDNs) take caching to the next level by storing content <strong>physically</strong> closer to end users. Instead of making requests that could potentially span across the entire globe, CDNs allow us to serve users more efficiently by setting up geographically distributed servers.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JPoDzFWjxiNZKJTkNOMlPQ.png" /></figure><p>The most effective CDN servers are those located closest to the end users. For example, if the majority of our users are in <strong>Malaysia and the United Kingdom</strong>, we would ideally deploy CDN servers in <strong>Southeast Asia</strong> and <strong>Europe</strong>. This proximity reduces latency for individual requests, making the overall experience faster for users. Additionally, CDNs lighten the load on the origin servers by distributing the content among their servers, significantly reducing the need for direct requests to the origin.</p><h4>Benefits</h4><p>There are three main benefits of utilizing CDNs:</p><ol><li><strong>Reduced Latency:</strong> CDNs bring the data physically closer to users, which minimizes the time taken for requests to travel across networks.</li><li><strong>Improved Scalability:</strong> By offloading traffic to CDN servers, the origin servers handle fewer requests, making the system capable of supporting a larger user base.</li><li><strong>Improved Availability:</strong> The distributed nature of CDNs ensures that even if one server goes down, others can take over, keeping the content-serving system operational.</li></ol><p>In technical terms, CDNs reduce system latency while increasing overall system availability, making it a win-win for both users and developers.</p><h4>Static Content</h4><p>CDNs are designed primarily to serve <strong>static content</strong>, which refers to data that doesn’t change frequently. This includes:</p><ul><li><strong>JavaScript files</strong></li><li><strong>CSS files</strong></li><li><strong>Images and videos</strong></li><li><strong>Documents (e.g., PDFs)</strong></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_I12SktPU49iSGm5xFtVPQ.png" /></figure><p>Since static content doesn’t require frequent updates, it’s ideal for caching across multiple locations. However, dynamic content (e.g., user-specific data, application logic or authentication-related code) typically requires newer technologies like <strong>edge servers</strong>, which allow for dynamic content delivery. We’ll leave edge servers for another article.</p><h4>Push vs. Pull CDNs</h4><p>Now, they are a couple different types of CDNs. Namely, <strong>Push CDNs</strong> and <strong>Pull CDNs</strong>. Let’s use an example to illustrate the differences between the two. Imagine you’ve just uploaded a new image to your origin server for your application.</p><p><strong>Push CDN:</strong><br>With a push CDN, you actively <strong>push</strong> the new image to all CDN servers across the globe as soon as the image is uploaded to the origin server. This ensures that the content is immediately available everywhere but requires careful management to avoid stale data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yTN7TWobFoQqsTI-LLbFeA.png" /></figure><p>Push CDNs are ideal in the following scenarios:</p><ul><li><strong>Immediate Availability Worldwide Required — </strong>All content is proactively pushed to every CDN server, ensuring it’s ready to be served anywhere immediately.</li><li><strong>Traffic is Predictable — </strong>Push CDNs require pre-planned content distribution. Predictable traffic ensures the right content is in the right place at the right time.</li><li><strong>More Control Over Caching — </strong>With push CDNs, you manually control when and how content is updated across servers.</li></ul><p><strong>Pull CDN:</strong><br>With a pull CDN, the image isn’t uploaded to the CDN servers immediately. Instead, when a user requests the image through a CDN server for the first time, the server retrieves the image from the origin, serves it to the user, and caches it locally for subsequent requests. This approach is particularly effective for applications where content popularity varies by region. For example, an image frequently accessed in <strong>Asia</strong> will be cached on local CDN servers there, while other regions may not store it unless users request it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*J3LKcY7uug8rLKemjorokg.png" /></figure><p>Pull CDNs are better suited in the following scenarios:</p><ul><li><strong>Dynamic or Frequently Updated Content — </strong>Pull CDNs retrieve and cache content on demand, reducing the risk of serving outdated data.</li><li><strong>User Behavior Varies Across Regions — </strong>Pull CDNs cache content only in the regions where it is requested, optimizing storage and performance.</li><li><strong>Simpler Setup — </strong>Pull CDNs handle caching automatically, reducing the need for manual management.</li></ul><h4>Cache-Control Headers in HTTP Requests</h4><p>In HTTP, the Cache-Control header plays a critical role in how data is cached. One of its possible values is public, which allows the content to be cached by external middle-layer servers like CDNs (instead of just the user’s browser).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/820/1*tMpXJbUA9EbyRzwW5fZ0Nw.png" /></figure><p>This is particularly relevant for <strong>pull CDNs</strong>:</p><ul><li>When a pull CDN requests static content from the origin server, the presence of thepublic keyword in the Cache-Control header tells the CDN that it’s allowed to cache the content.</li><li>As a result, future requests for the same content can be served directly from the CDN, bypassing requests to the origin server.</li></ul><p>Now, whether you choose a <strong>push CDN</strong> or a <strong>pull CDN</strong> depends on your specific use case and context- so it’s important to understand the fundamental differences between the two.</p><p>While this article focused solely on static content, the emergence of <strong>edge computing</strong> is expanding the capabilities of CDNs to handle dynamic content. This represents quite an exciting future when it comes to content delivery. In fact, I might just write another article on edge servers if I don’t get too lazy!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1ea54e7701d3" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Caching: Data Input and Eviction Strategies]]></title>
            <link>https://forreya.medium.com/caching-data-input-and-eviction-strategies-8603cd7f7433?source=rss-23030b811506------2</link>
            <guid isPermaLink="false">https://medium.com/p/8603cd7f7433</guid>
            <category><![CDATA[caching]]></category>
            <category><![CDATA[system-design-interview]]></category>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[cache]]></category>
            <category><![CDATA[software-development]]></category>
            <dc:creator><![CDATA[Ryan Lai]]></dc:creator>
            <pubDate>Tue, 19 Nov 2024 18:29:54 GMT</pubDate>
            <atom:updated>2024-11-19T18:29:54.360Z</atom:updated>
            <content:encoded><![CDATA[<p>Caching- a fundamental concept that spans across various levels of software development. If you are familiar with the inner workings of a computer, you may already recognize its importance within a single machine. Namely, a CPU cache stores frequently accessed RAM data, allowing for faster retrieval and significantly improving the computer’s performance due to the cache’s lower latency compared to RAM.</p><p>This principle extends beyond singular machines and is widely applied throughout system design. In essence, caching a technique that involves creating a copy of a subset of data from a primary storage source and storing it in a faster-access system to improve overall performance.</p><h3>Caching From the Client’s Perspective</h3><p>You can be observe caching directly in the network tab of your browser’s developer tools. For a hands-on learning experience, follow along with the following steps on any large-scale company’s website.</p><ol><li>Find a file that rarely changes, such as a JavaScript file responsible for making the webpage dynamic.</li><li><strong>Initial Request </strong>— When you load the page for the first time, observe the file’s metrics in the network tab. The <strong>Time</strong> column shows how long it took for the browser to download the file from the server. The <strong>Size</strong> column displays the file’s size. In the example below, a 30.4 KB JavaScript file took 92ms to download from the server.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mXICjYgEix7AGNGPzS-sEg.png" /></figure><p>3.<strong> Subsequent Request</strong> — If you reload the page. This time, the <strong>Size</strong> column for the same file will display “(memory cache)” or “(disk cache).” This means the file was cached locally during the initial request. Instead of making another HTTP request, the browser retrieves the file directly from the cache.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hKdP0Xek9Uz__CBZKOU4yQ.png" /></figure><p>4. Since this time, the file was served from the cache rather than the server, this is known as a <strong>Cache HIT</strong>. The browser finds the data locally (in memory or on disk) and retrieves it, which is much faster than making a new network request.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/508/1*8FHu7o5QpJmqX7nZPI3nSg.png" /></figure><p>5. A specific HTTP response header you should be familiar with is called cache-control. This header governs caching rules, one of which is the duration for which a file can be cached for. For example, cache-control: max-age=3600 specifies that the file can be cached for 3600 seconds (1 hour) before it is considered stale and a new network request is needed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/538/1*ld6F_iI0hkSlXQwHDzlmpA.png" /></figure><p>In the example above, the cache used was stored in the computer’s memory (RAM). However, browsers can also use the disk for caching files.</p><p><em>But isn’t disk slower than RAM and CPU cache?</em></p><p>Yes, disk access is slower than RAM and CPU cache. However, it is still <strong>orders of magnitude faster</strong> than making an HTTP request over the internet. Disk caching strikes a good balance between speed and persistence, which makes it a very good option for storing data locally.</p><h4>Hits vs. Misses</h4><p>When dealing with HTTP requests, a client uses the following information to determine whether it can use cached data:</p><ol><li><strong>CACHE HIT</strong> — If the requested data already <strong>exists on the client’s disk or memory and it itsn’t stale</strong>, the client retrieves the data from the cache.</li><li><strong>CACHE MISS — </strong>If<strong> </strong>the requested data either <strong>doesn’t exist in the cache, or exists but has expired (is “stale”)</strong>. In this scenario, a new network request is necessary.</li></ol><h4>Cache Ratio</h4><p>The <strong>cache ratio</strong> measures the percentage of cache reads that result in cache hits. The formula for calculating the cache ratio is:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3997WvtM6Bv8PtLGbF1aVg.png" /></figure><p>A higher cache ratio is desirable because it indicates that the cache is being utilized effectively.</p><h3>Caching From The Server’s Perspective</h3><p>Now let’s optimize server-side performance with caching using the example of a <strong>cat adoption website</strong>. On this site, users can upload and view details about cats available for adoption.</p><p><strong>Current Workflow<br></strong>Each time a cat is uploaded, the server writes the data to a persistent database, such as PostgreSQL, via a network query. While this approach works, retrieving data repeatedly from the database can lead to slower response times due to disk-based access latency.</p><p><strong>Optimized Workflow with Caching<br></strong>To improve average data retrieval speed, we introduce an <strong>in-memory key-value store</strong>, such as Redis, to cache frequently accessed data. Redis stores data in memory (RAM), allowing for much quicker access compared to disk-based databases like PostgreSQL. However, it’s important to note that Redis has some limitations:</p><ul><li><strong>Non-persistence:</strong> Data in memory is lost on server restarts.</li><li><strong>Limited storage:</strong> RAM capacity is smaller than disk storage.</li></ul><p>Although Redis can run as a seperate service on the same server as the main server that serves users, it typically runs as a separate service that is accessible over the network, similar to the PostgreSQL database.</p><h4><strong>Logic Flow</strong></h4><ol><li><strong>User Requests Data — </strong>A user visits the website and requests information about a specific cat using its unique cat ID.</li><li><strong>Check Cache First — </strong>The server checks Redis to see if the cat data is already cached. Let’s say this time around, the data isn’t in Redis, so the server queries PostgreSQL to retrieve it.</li><li><strong>Retrieve and Cache — </strong>After fetching the data from PostgreSQL, the server responds to the user and stores the retrieved data in Redis as a key-value pair. For example, the key could be the unique cat ID, and the value could contain the relevant details about the cat. This step ensures that future requests for the same cat can be served directly from Redis instead of querying PostgreSQL again.</li><li><strong>Subsequent Request (Cache Hit) — </strong>If another user subsequently requests data for the same cat, the server will check Redis and find the cached data (Cache Hit). The data is retrieved directly from Redis and sent to the user without querying PostgreSQL, significantly reducing response time.</li></ol><h4>Advantages of Server-Side Caching</h4><ul><li>Data retrieval from Redis (in-memory) is much faster than disk-based database queries.</li><li><strong>Reduced Database Load:</strong> Frequently accessed data no longer generates repeated queries to PostgreSQL.</li></ul><p>Combining the usage of persistent databases for long-term storage and in-memory caches like Redis for quick access is a commonly used server-side technique for making a system more performant.</p><h4>Frequently Accessed Data</h4><p>Since memory (RAM) is significantly more limited in size than disk storage, it’s crucial to carefully select which data to store in the cache. In most cases, only a subset of the data — determined by its access frequency — is cached.</p><p>Using the example of the cat adoption website, we might identify users whose uploaded cats has consistently become the most popular. By focusing on this metric, the cache can prioritize storing data for cats created by these high-impact users. This means that although most cats won’t be cached, the strategy will remain highly efficient as the majority of site visitors are likely to view only a small selection of popular cats. By caching this frequently accessed portion of data, the system can achieve a substantial performance boost by increasing throughput and reducing the load on the database.</p><h3>Approaches to Inputting Cache Data</h3><h4>1. Write-Around Caching</h4><p>Write-around caching involves writing data only to the main database (persistent storage) without updating the cache. The cache only gets updated later on when the data is read. This method is useful when we want to speed up write operations but increases the likelihood of cache misses for recently written data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*G1BORGgT9GeLkzl0hwPUcQ.png" /></figure><p><strong>Example:</strong><br>A user uploads a new cat profile to the adoption site. The server writes this data directly to PostgreSQL (on disk) but does not add it to Redis (in memory). If another user requests this cat’s data shortly afterward, Redis won’t have it, resulting in a cache miss. The server will then query PostgreSQL, retrieve the data, and add it to Redis for future requests.<br>This method is useful when write operations are frequent but immediate reads are less likely.</p><h4>2. Write-Through Caching</h4><p>Write-through caching basically ensures that every write operation updates both the cache and the main database simultaneously. This approach means that the cache and database are always synchronized upon a write operation, which helps to minimize cache misses. However, this comes at the cost of increased write latency since every write involves two storage systems.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9izIiFe4luwjoLo9K-jVTA.png" /></figure><p><strong>Example:</strong><br>When a user uploads a new cat profile, the server writes the data to PostgreSQL (on disk) and immediately updates Redis (in memory) with the same data. This guarantees that if another user requests this cat’s data, it’s already in the cache, resulting in a cache hit. This method is best when we want to ensure data consistency and is ideal when cache misses must be avoided, even though it incurs higher write overhead.</p><h4>3. Write-Back Caching</h4><p>Write-back caching involves writing data only to the cache initially and deferring the write to the main database. The database is only updated later, either periodically or when the cached data is evicted (we’ll get to this in a second). This approach optimizes write speeds and reduces cache misses but comes with the risk of data loss if the cache fails before the main database is updated.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dE8xhAA57mXjnP3IIv_2Wg.png" /></figure><p><strong>Example:</strong><br>When a user uploads a new cat profile, the server writes the data to Redis (in memory) but delays updating PostgreSQL (on disk). If another user requests the same cat data, it’s already in Redis, ensuring a cache hit. Later, the server writes the cached data to PostgreSQL either during periodic updates or when the cache evicts the data. While this approach provides faster write operations, it risks data loss if the cache crashes before the data is written to the database. For critical data, such as user-submitted profiles on the cat adoption site, this risk is unacceptable. Losing data due to a server crash can harm the site’s functionality and user trust, making write-back caching only suitable for non-critical data where occasional losses are tolerable.</p><h4>Choosing the Best Cache Input Approach</h4><p>To give a quick rundown of what we just discussed:</p><ul><li><strong>Write-Around</strong>: Best for scenarios with frequent writes but infrequent immediate reads.</li><li><strong>Write-Through</strong>: Ideal for cases where data consistency and low cache misses are essential, even at the cost of increased write latency.</li><li><strong>Write-Back</strong>: Suited for non-critical, high-speed applications where some data loss is tolerable.</li></ul><h3>Eviction Policies</h3><p>Now that we’ve covered how to insert data into the cache, let’s explore some strategies to removing data when the cache reaches its memory limit. Since RAM is finite, eviction policies play a critical role when it comes to optimizing the <strong>cache-hit ratio</strong>. A well-chosen eviction policy ensures the cache retains the most valuable data, improving performance by minimizing disk-based operations.</p><h4>Random</h4><p>Let’s begin with the simplest one. This policy evicts a randomly selected piece of data when the cache is full. It is simple to implement but lacks any prioritization based on data usage patterns. As a result, it does nothing to improve the cache-hit ratio and is generally suboptimal.</p><h4>FIFO (First In, First Out)</h4><p>In FIFO, the oldest data in the cache (the first item inserted) is evicted first. While straightforward, FIFO treats all data equally, disregarding whether it is frequently or recently accessed. The only consideration is the age of the data.</p><h4>LRU (Least Recently Used)</h4><p>LRU is a widely used caching policy, supported by systems like Redis. It evicts data that hasn’t been accessed for the longest time. Each cached item is tagged with a timestamp recording its last access, and the item with the oldest timestamp is removed when the cache is full.</p><p>This prioritizes recent activity by ensuring frequently accessed data stays in the cache. This makes it well-suited for dynamic applications where recency is a strong indicator of relevance. One downside to this is that it may evict data with high long-term value but low recent usage. Another thing to note is that, compared to FIFO, there is a slight additional overhead when it comes to maintaining and updating access timestamps.</p><h4>LFU (Least Frequently Used)</h4><p>LFU removes the data accessed the fewest number of times. Each cached item is paired with a counter that tracks how many times it has been accessed, and the item with the smallest counter is removed when eviction is necessary.</p><p>This method retains data with the highest cumulative importance, based on total usage counts, which makes it useful for scenarios where long-term popularity matters more than recency. However, this risks retaining outdated data with historically high access counts but little current relevance. Similarly to LRU, it also requires additional overhead to maintain and update access counters for each item.</p><h4>Which Eviction Policy for the Cat Adoption Website?</h4><p>Let’s now consider the best option for the cat adoption website.</p><ul><li><strong>Random</strong> is immediately unsuitable, as it offers no optimization.</li><li><strong>FIFO</strong>, while simple, is also inadequate since it ignores access frequency or recency, which are key metrics for optimizing cache efficiency.</li></ul><h4><strong>Comparing LRU and LFU</strong></h4><ul><li><strong>LFU</strong>: Imagine a cat profile that was highly popular last year but has since fallen out of relevance. Under LFU, this profile might remain in the cache indefinitely because of its historically high access count. Meanwhile, a newly trending cat profile with fewer cumulative views could be evicted, even though it’s currently more important.</li><li><strong>LRU</strong>: In this same scenario, the older, less relevant cat profile would be evicted because its “recent access” timestamp would be relatively older. The cache would prioritize retaining data for cats that are currently trending.</li></ul><p>Given the dynamic nature of user interest in cat profiles, <strong>LRU is the better choice </strong>as<strong> </strong>it ensures that recent trends dominate the cache, making it more aligned with user behavior and improving the cache-hit ratio for the most actively used data.</p><p>So to finish off, while caching can involve many intricate details and sophisticated implementations, the core concept is pretty simple: it aims to speed up systems by minimizing the frequency of slower operations. To achieve this, we make trade-offs, often introducing additional system complexity in exchange for significantly improved performance.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8603cd7f7433" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>