<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jigar Patel on Medium]]></title>
        <description><![CDATA[Stories by Jigar Patel on Medium]]></description>
        <link>https://medium.com/@jogi47?source=rss-796cb6be2ade------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*11g8boM8wh2VVIzxlDNAyQ.jpeg</url>
            <title>Stories by Jigar Patel on Medium</title>
            <link>https://medium.com/@jogi47?source=rss-796cb6be2ade------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 19:55:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@jogi47/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[The Architecture of Data: From Raw Logs to Real-Time Systems (Part 3)]]></title>
            <link>https://levelup.gitconnected.com/the-architecture-of-data-from-raw-logs-to-real-time-systems-part-3-24da2cf97a25?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/24da2cf97a25</guid>
            <category><![CDATA[technical-interview]]></category>
            <category><![CDATA[database-design]]></category>
            <category><![CDATA[data-engineering]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[big-data]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Mon, 23 Feb 2026 17:31:01 GMT</pubDate>
            <atom:updated>2026-02-23T17:31:01.768Z</atom:updated>
            <content:encoded><![CDATA[<h4>A guide to batch processing, stream processing, and building data systems that deserve your trust.</h4><p><strong>TL;DR</strong> Parts I and II covered how to store data and distribute it across machines. Part III asks a different question: once you have data, how do you transform it, react to it in real-time, and compose multiple systems into something that actually works?</p><ul><li><strong>Batch Processing</strong>: The Unix philosophy applied to planetary-scale data. MapReduce, dataflow engines, and why your output is always derived from your input.</li><li><strong>Stream Processing</strong>: What happens when data never stops arriving. Message brokers, Kafka, change data capture, and event sourcing as the foundation for real-time systems.</li><li><strong>The Hard Parts of Streaming</strong>: Time is broken, joins are complex, and fault tolerance requires choosing between microbatching, checkpointing, and idempotency.</li><li><strong>Building Systems That Deserve Trust</strong>: No single database does everything. Derived data, the log as source of truth, and why integrity matters more than timeliness.</li></ul><p>If you’ve read Parts I and II, this is where it all comes together. The goal isn’t to master every tool — it’s to understand how the pieces compose into systems that are correct, maintainable, and honest about their guarantees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*cCCIULs99G4qQqUAgD_cyA.png" /></figure><h3>The Unix Philosophy at Planetary Scale</h3><p>Before MapReduce existed, the Unix command line had already figured out the core idea.</p><p>Want the five most requested URLs from a web server log?</p><pre>cat /var/log/nginx/access.log |<br>    awk &#39;{print $7}&#39; |<br>    sort |<br>    uniq -c |<br>    sort -rn |<br>    head -n 5</pre><p>Six programs. Each does one thing. Connected by pipes. The user composes them however they want. None of the tools know about each other.</p><p>This is the Unix philosophy: small tools, uniform interfaces (text in, text out), and composability. It works beautifully — until your data doesn’t fit on one machine.</p><p>MapReduce is the Unix philosophy applied to distributed computing.</p><h4>Map, Shuffle, Reduce</h4><p>Every MapReduce job has two functions you write:</p><p><strong>Map</strong> takes one input record and emits zero or more key-value pairs. <strong>Reduce</strong> takes a key and all its associated values, then produces an aggregated result. Between them, the framework handles the hard part: shuffling data across machines so all values for the same key end up at the same reducer.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*YdmEeA2yKK_sV4FqKuB9Dw.png" /></figure><p>The input lives on HDFS — Hadoop’s distributed file system — where files are split into 128 MB blocks and replicated across machines. MapReduce reads from HDFS, processes in parallel, and writes back to HDFS. The mapper runs on the same machine that holds the data whenever possible, avoiding network transfer.</p><p>Why is this fault-tolerant? Three properties:</p><ol><li><strong>Inputs are immutable.</strong> If a mapper crashes, just rerun it on the same input.</li><li><strong>Functions are deterministic.</strong> Same input always produces same output.</li><li><strong>Intermediate results go to disk.</strong> If a reducer fails, fetch the mapper outputs again.</li></ol><p>The scheduler detects failures via heartbeats, restarts tasks on other machines, and the job completes as if nothing happened. This is why MapReduce could run on thousands of commodity machines — it expected failures and designed around them.</p><h4>Joins: The Hard Part</h4><p>The interesting MapReduce patterns aren’t simple aggregations. They’re joins.</p><p><strong>Sort-merge join</strong>: Both datasets emit records keyed by the join field. The shuffle groups them together. The reducer sees all records for a given key from both datasets and combines them. Secondary sorting ensures the smaller “dimension” record (e.g., user profile) arrives before the larger “fact” records (e.g., user’s activity events), so the reducer can hold one profile in memory while streaming through millions of activities.</p><p><strong>Broadcast hash join</strong>: When one dataset is small enough to fit in memory, load it into a hash table on every mapper. Each mapper looks up the join key locally. No shuffle needed — massively faster.</p><p><strong>The skew problem</strong>: What if one key has vastly more data than others? A celebrity with 50 million followers makes one reducer process 50 million records while others finish in minutes. Solutions: sample to find hot keys, then either salt them (append random suffixes to spread across reducers) or use a broadcast join for just the hot keys. Databases don’t handle this automatically — it’s your problem.</p><h4>Why MapReduce Lost</h4><p>MapReduce was groundbreaking. It was also slow.</p><p>The fundamental issue: <strong>materialization of intermediate state</strong>. Every mapper writes to disk. Every reducer reads from disk. A multi-stage pipeline means: Map → Write to HDFS → Read from HDFS → Map → Write to HDFS → Read from HDFS → Reduce. All that disk I/O for data that’s only used once.</p><p>Add to this: the rigid map-then-reduce structure (what if you need map-map-reduce-map?), JVM startup overhead per job, and the inability to pipeline data between stages.</p><h4>Dataflow Engines: The Better Answer</h4><p>Spark, Flink, and Tez replaced MapReduce’s rigid two-phase model with a DAG (Directed Acyclic Graph) of operators. Any operator can connect to any other. Intermediate results stay in memory. The entire pipeline runs as a single job.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HCElPDCj2xEYL2NCCJ_8pg.png" /></figure><p>The performance difference: 10–100x faster than MapReduce for multi-stage pipelines.</p><p>But what about fault tolerance? MapReduce’s intermediate files on disk made recovery trivial. Spark solved this with <strong>lineage</strong>: each dataset tracks the chain of transformations that produced it. If a partition is lost, Spark traces back through the lineage to the nearest checkpoint or source data and recomputes. For long pipelines, periodic checkpoints to HDFS prevent recomputing from scratch.</p><p>The high-level story is compelling: we went from writing raw Java map/reduce functions (low-level, verbose) to Pig and Cascading (mid-level, composable) to Hive and Spark SQL (high-level, declarative). Write SQL, let the optimizer choose broadcast vs sort-merge joins, partition counts, and filter pushdown. The circle closed — we’re back to declarative queries, but now they run across thousands of machines.</p><blockquote><em>Batch output is derived data. The input is the source of truth. You can always delete the output and recompute it.</em></blockquote><p>This is the key insight of batch processing. Your output is disposable. Your input is sacred. If something goes wrong, recompute. This makes batch pipelines remarkably robust — you can’t corrupt data you can recreate.</p><h3>When Data Never Stops Arriving</h3><p>Batch processing works on bounded datasets. But what about data that has no end? User clicks, sensor readings, financial transactions — they arrive continuously. You don’t want to wait until midnight to process today’s data.</p><p>This is stream processing: handling an unbounded flow of events as they arrive.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aTYXFRrgWYRx9cCG18X66Q.png" /></figure><p>An <strong>event</strong> is an immutable record of something that happened: a user clicked, a payment was processed, a temperature was measured. Events have timestamps, are self-contained, and never change. Once something happened, it happened.</p><p>The question is: how do you get events from producers to consumers?</p><h4>The Two Kinds of Message Brokers</h4><p><strong>Traditional brokers</strong> (RabbitMQ, ActiveMQ, SQS) work like a post office. You send a letter, the post office holds it, the recipient picks it up, and the letter is destroyed. Messages are deleted after acknowledgment. If the consumer crashes before acknowledging, the message is redelivered. Good for task queues where each job should be processed exactly once.</p><p><strong>Log-based brokers</strong> (Kafka, Kinesis, Pulsar) work like a newspaper archive. Every message is appended to a persistent, ordered log. Consumers track their own position (offset) in the log. Messages are never deleted (until a retention period expires). Multiple consumers can read independently at different speeds.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7kX64kPiu42UyGJ-0_SclQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RAeJrlnQ9SOk9SQ5rGTSNg.png" /></figure><p>Kafka won the architecture wars for event streaming because of that replay capability. When you discover a bug in your consumer, you can fix the code and reprocess from the beginning. When you add a new derived system, it can consume the entire history. This changes the game.</p><h4>Kafka: The Log at the Center</h4><p>Kafka organizes data into <strong>topics</strong>, each split into <strong>partitions</strong>. A producer assigns messages to partitions (by key hash or round-robin). Within a partition, messages are strictly ordered and identified by offset.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*L5t_y3C-tQjTiH9x9VlNVQ.png" /></figure><p>Each partition goes to exactly one consumer within a group. Different consumer groups read independently, each maintaining their own offsets. Scaling? Add more partitions and consumers.</p><p>The offset mechanism makes failure recovery simple: if a consumer crashes, it restarts from the last committed offset. The trade-off is at-least-once delivery by default — a message might be processed twice if the consumer crashes after processing but before committing. Exactly-once requires additional mechanisms we’ll cover later.</p><h4>Change Data Capture: The Database as a Stream</h4><p>Here’s an insight that changes how you think about architecture: every write to a database is an event. INSERT, UPDATE, DELETE - these are a stream of changes.</p><p><strong>Change Data Capture (CDC)</strong> extracts this stream from the database’s replication log:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hFFzDpJPCuPD6p2CVpomxQ.png" /></figure><p>Debezium, Maxwell, and native database features (PostgreSQL logical replication, MongoDB change streams) power this pattern. Every derived system — cache, search index, warehouse — becomes a consumer of the change stream. Add a new consumer without touching the primary database. If a consumer falls behind, it catches up by replaying from its last offset.</p><p>This is more reliable than dual-writes (writing to both the database and Elasticsearch simultaneously), which can leave systems inconsistent if one write succeeds and the other fails.</p><h4>Event Sourcing: Remembering Everything</h4><p>CDC captures changes from a database that stores current state. Event sourcing flips this: <strong>store the events as the source of truth, derive the current state from them</strong>.</p><p>Think of it like recording every chess move instead of photographing the board. The photograph tells you where the pieces are now. The move log tells you how they got there, enables replaying the game to any point, and allows different people to derive different analyses from the same game.</p><p>A shopping cart in a traditional database: {items: [{sku: &quot;XYZ&quot;, qty: 2}], total: 20}. You don&#39;t know if the user added two at once, added one then another, or added three and removed one.</p><p>A shopping cart as event sourcing: CartCreated → ItemAdded(XYZ) → ItemAdded(ABC) → ItemRemoved(ABC) → ItemAdded(XYZ). Complete audit trail. Time travel. Multiple views from the same events.</p><p>The trade-offs are real: replaying all events to reconstruct current state is slow (solve with periodic snapshots), event schema evolution is tricky, and deleting data (GDPR “right to be forgotten”) conflicts with immutable event logs.</p><h4>Commands vs Events</h4><p>An important distinction that event sourcing makes explicit: a <strong>command</strong> is a request to do something (“AddItemToCart”). It’s imperative, it might fail validation, and it happens before the system decides whether to accept it. An <strong>event</strong> is a fact that something happened (“ItemWasAddedToCart”). It’s past tense, it has already been validated, and it’s immutable.</p><p>Commands flow in. The system validates them against business rules. If accepted, the system produces events. If rejected (item out of stock, insufficient funds), no event is generated. This separation keeps the event log clean — it only contains things that actually happened, never requests that were denied.</p><p><strong>CQRS</strong> (Command Query Responsibility Segregation) pairs naturally with event sourcing: write commands to the event store, project events into separate read-optimized views. An Elasticsearch index for search, a Redis cache for fast lookups, a PostgreSQL table for relational queries — all derived from the same event stream, each optimized for its access pattern.</p><h3>The Hardest Problems in Stream Processing</h3><p>Having a stream of events is one thing. Processing them correctly is another. Three problems make stream processing fundamentally harder than batch: time, joins, and failure.</p><h4>Time Is Broken</h4><p>When counting events per minute, which minute do you mean?</p><p><strong>Event time</strong>: when the event actually occurred (timestamp in the event). <strong>Processing time</strong>: when the event reaches your stream processor.</p><p>These are not the same. A mobile user clicks at 10:00:00 but the device is offline. The event arrives at 10:15:00. If you count by processing time, it’s in the wrong minute. If you count by event time, you need to handle events that arrive late.</p><p>Processing time is easy but wrong. Event time is correct but hard.</p><h4>Windows: Grouping the Unbounded</h4><p>To aggregate a stream, you need to define <strong>windows</strong> — time ranges that group events:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*56x4dqXSm1ynk95kyu0L0g.png" /></figure><p>The question that makes windowing hard: when do you emit results? Events arrive late. A tumbling window for 10:00–10:01 might receive an event with event time 10:00:55 at processing time 10:03:00.</p><p><strong>Watermarks</strong> are the answer: a declaration that “we believe no more events with timestamp &lt; T will arrive.” When the watermark passes the window boundary, emit the result. Late events after the watermark are either dropped, trigger corrections, or are routed to a late-events side output.</p><p>Watermarks are a trade-off. Aggressive watermarks (close windows quickly) give lower latency but drop more late events. Conservative watermarks (wait longer) catch more late events but increase latency. Think of it as “last call” at a bar — you have to decide when to stop accepting orders for a given time period.</p><h4>Stream Joins: Three Flavors of Hard</h4><p>Joining streams is more complex than joining database tables because data arrives over time and you’re working with incomplete information.</p><p><strong>Stream-stream join</strong>: Match events from two streams within a time window. Calculate click-through rates by joining impression events with click events by ad_id within 5 seconds. Both streams need state: buffer impressions, match incoming clicks against buffered impressions, expire unmatched impressions after the window.</p><p><strong>Stream-table join</strong> (enrichment): Enrich events with reference data. A user activity event arrives; look up the user’s profile to add their tier and region. Options: query the database per event (slow), load the table into memory and refresh periodically (stale), or subscribe to the table’s CDC stream so the local copy is always current.</p><p><strong>Table-table join</strong>: Both sides are CDC streams. Maintain a materialized join that updates when either table changes. When an order arrives, look up the customer. When a customer updates their address, re-emit all their orders with the new address.</p><h4>Fault Tolerance: Three Approaches</h4><p>Stream processors run for months or years. Crashes will happen. How do you recover without losing or duplicating data?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1B64lUlwkQ1_3QhS8C-wNg.png" /></figure><p>Microbatching reuses batch semantics: process a 1-second batch, commit, continue. If the batch fails, reprocess it. Simple reasoning, but your minimum latency equals your batch interval.</p><p>Flink’s checkpointing is more elegant: inject barrier markers into the stream. When an operator receives barriers from all inputs, it snapshots its state. On failure, restore the last checkpoint, rewind the input to the checkpointed offset, and continue. The snapshot is consistent across all operators without stopping processing.</p><p>Idempotency sidesteps the problem: if processing an event twice produces the same result, duplicates don’t matter. Include the event ID in the output: INSERT ... ON CONFLICT (event_id) DO NOTHING. Now retries are safe.</p><blockquote><em>Exactly-once is about the effect, not the processing. The processor might process an event twice, but the output looks like it was processed once.</em></blockquote><p>End-to-end exactly-once requires three things: replayable input (Kafka), deterministic processing, and idempotent or transactional output. Miss any one and you get at-least-once or at-most-once instead.</p><h4>Choosing a Framework</h4><p>The three major stream processing frameworks make different trade-offs:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UctqXIdlhqcc0wmtrK8zxw.png" /></figure><p>Kafka Streams stands out because it’s a library, not a cluster — deploy it as a regular application. Flink has the most sophisticated event-time processing and watermark handling. Spark Streaming unifies batch and streaming under one API. Choose based on your existing infrastructure and the complexity of your time semantics.</p><h3>Building Systems That Deserve Trust</h3><p>Here’s the uncomfortable reality of modern data architecture: no single database does everything well. Your application probably needs a primary database, a cache, a search index, and an analytics warehouse. They all need the same data, represented differently.</p><p>How do you keep them in sync?</p><h4>Two Roads to Synchronization</h4><p><strong>Distributed transactions</strong> (the synchronous approach): wrap all writes in a two-phase commit. Write to PostgreSQL, Redis, and Elasticsearch atomically. If any system is slow or unavailable, everything blocks. Brittle, slow, and every system must support the 2PC protocol.</p><p><strong>Derived data</strong> (the asynchronous approach): write to one primary source, derive everything else from a change stream.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qKmn68z1sE_hro5lAGt6Jg.png" /></figure><p>Each derived ASYNCHRONOUSLY from the stream.<br>Can fail and recover independently.<br>Add new consumers without touching the primary.<br>Replay the stream to rebuild any view.</p><p>The trade-off: eventual consistency instead of immediate consistency. The search index might be a few seconds behind the primary database. For most applications, this is fine. For the ones where it isn’t, read the user’s own writes from the primary.</p><h4>The Log as Source of Truth</h4><p>The key insight that makes derived data work: if all writes go through a single ordered log, every derived system can process events in the same order and eventually reach the same state.</p><p>This is state machine replication: same inputs + same order + deterministic function = same output. Even if the search index is behind the cache, they’re processing the same events in the same order. They’ll converge.</p><p>Kafka becomes the “central nervous system” of the architecture. Every service writes events to Kafka. Every derived system consumes from Kafka. Data flows are visible, auditable, and replayable. Adding a new system means adding a new consumer — no changes to existing producers.</p><h4>Unbundling the Database</h4><p>A traditional database bundles everything: storage engine, indexes, query optimizer, transaction manager, replication, caching. Convenient but inflexible. You can’t swap out one component or scale them independently.</p><p>The unbundled approach composes specialized systems: Kafka for durable ordering, HDFS for bulk storage, RocksDB for fast key-value access, Elasticsearch for search, Spark for batch computation, Flink for stream processing. Each does one thing well. Together they form a “database” that’s more flexible than any monolith.</p><p>This is the Unix philosophy again, applied to data infrastructure.</p><h4>The Dataflow Mindset</h4><p>Instead of thinking “application talks to database,” think in terms of dataflow: inputs flow through transformation functions to produce derived outputs. Like a spreadsheet — when cell A1 changes, every formula that depends on it automatically updates.</p><p>The separation is clean:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VgW9xGruyjYB48_MIB7Bzw.png" /></figure><p><strong>Write path</strong>: User action → validate → append to event log. <br><strong>Derivation</strong>: Event stream → apply transformation → update materialized views. <br><strong>Read path</strong>: Query → look up precomputed view → return response.</p><p>The write path optimizes for validation and durability. The read path optimizes for query speed. New read views can be added without touching the write path. Clear audit trail of every change.</p><h4>Correctness: The End-to-End Argument</h4><p>Lower-level systems provide building blocks, but correctness must be enforced at the application level.</p><p>Consider: “Transfer $100 from Alice to Bob.” TCP ensures packets arrive. Transactions ensure atomicity within the database. But what if the client crashes after sending the request and before receiving the response? Did the transfer happen? The client doesn’t know. Retrying might transfer twice.</p><p>The solution is <strong>idempotency keys</strong>: the client generates a unique ID for each operation. The server checks if it’s seen that ID before. If yes, return the stored result. If no, execute the operation and store the result. Now retries are safe. This can’t be solved at a lower layer — it must be end-to-end.</p><h4>Integrity vs Timeliness</h4><p>Two properties that are often conflated but shouldn’t be:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6RGuQ11Dj5G5wD4LTXHnwg.png" /></figure><p><strong>Integrity</strong>: Data is correct. Constraints are not violated. Money doesn’t disappear. Usernames stay unique. Inventory doesn’t go negative. Violations are serious — data corruption is often irreversible.</p><p><strong>Timeliness</strong>: Users see up-to-date state. The dashboard shows current numbers. Profile updates appear immediately. Violations are annoying but not catastrophic.</p><blockquote><em>Integrity must always be maintained. Timeliness can be relaxed.</em></blockquote><p>This is a liberating insight. You don’t need synchronous replication and serializable isolation everywhere. You need them for integrity-critical operations (financial transfers, uniqueness constraints). For everything else, eventual consistency is fine. The art is knowing which parts of your system need strong guarantees and which can tolerate weaker ones.</p><p>And even with strong guarantees: trust, but verify. Hardware fails. Software has bugs. Humans misconfigure things. Build in continuous auditing — checksums for bit-level corruption, reconciliation between systems, invariant checks (“sum of all balances must equal X”), and alertable anomalies. Double-entry bookkeeping exists for a reason.</p><h4>The Responsibility We Carry</h4><p>Data systems aren’t neutral. The same techniques that power useful applications can be misused.</p><p>Personalized recommendations become filter bubbles. Fraud detection becomes surveillance. Credit scoring becomes discrimination. The feedback loops are insidious: a predictive policing algorithm sends more officers to a neighborhood, more arrests happen, the data “confirms” the prediction, more officers are sent. The algorithm is self-reinforcing, amplifying the bias baked into the original data.</p><p>Data is not just an asset, it’s a liability. Every stored record is a breach risk. Every dataset is subject to subpoena. Every profile is a privacy exposure. The question shouldn’t be “what data can we collect?” but “what is the minimum data needed to provide this service?”</p><p>Users should control their data: access it, correct it, delete it, port it. These aren’t just GDPR requirements — they’re basic respect for the people whose data you hold. As engineers who build these systems, the question we should ask isn’t just “does it work?” but “is it right?”</p><h3>Putting It All Together</h3><p>Parts I, II, and III form a complete picture. Part I: how to store and model data on a single machine. Part II: how to distribute data across machines and maintain guarantees. Part III: how to derive new data, process it in real-time, and compose systems that work correctly.</p><p>The recurring theme across all three parts: <strong>trade-offs are everything</strong>.</p><ul><li>Batch processing trades latency for throughput and simplicity</li><li>Stream processing trades simplicity for lower latency</li><li>Derived data trades immediate consistency for independence and flexibility</li><li>Strong guarantees trade performance for correctness</li></ul><p>Four questions to ask about any data pipeline:</p><ol><li><strong>What is the source of truth?</strong> Is there one authoritative system, or can truth be reconstructed by replaying a log?</li><li><strong>What are the derived systems?</strong> Which views are derived from the source, and how do they stay in sync?</li><li><strong>What happens when something fails?</strong> Can the pipeline recover by replaying? Are operations idempotent? Is there state that can be lost?</li><li><strong>What guarantees does the application actually need?</strong> Not “what guarantees would be nice” — what guarantees would cause real damage if violated?</li></ol><h4>The Meta-Lesson</h4><p>The data landscape changes constantly. MapReduce was revolutionary in 2004 and obsolete by 2015. New tools appear every year. But the underlying principles don’t move:</p><ul><li>Immutable inputs + deterministic functions = safe recomputation</li><li>Total ordering through a log enables derived state</li><li>Idempotency makes retries safe</li><li>The end-to-end argument means correctness lives in the application</li><li>Integrity must never be compromised; timeliness can be relaxed</li></ul><p>Understanding these principles makes you resilient to tool churn. You’ll recognize that the “new” streaming framework is implementing the same checkpointing strategy Flink pioneered. You’ll see that the “new” event-driven architecture is CQRS with a different name. You’ll know that the “new” data mesh is unbundled databases with organizational boundaries.</p><p>Principles outlast implementations. Understanding outlasts memorization. And the best data system is one that’s honest about what it guarantees and what it doesn’t.</p><h3>Closing Thoughts</h3><p>This series covered the 12 chapters of “Designing Data-Intensive Applications” across three blog posts. The book is dense. The real world is denser.</p><p>If there’s one takeaway from the entire journey, it’s this: <strong>every system is a composition of trade-offs, and the dangerous ones are the trade-offs you didn’t know you were making.</strong></p><p>The engineer who assumes their database provides serializable isolation when it’s actually running snapshot isolation will ship write skew bugs. The engineer who assumes replication is synchronous when it’s asynchronous will lose data. The engineer who assumes events are processed exactly once when the system only guarantees at-least-once will corrupt downstream state.</p><p>The antidote is specificity. Know your isolation level. Know your replication mode. Know your delivery guarantees. Read the documentation, not the marketing page. Test under failure conditions, not just happy paths.</p><p>And remember that behind every data system are humans — the engineers who build them, the operators who run them, and the people whose data flows through them. Build systems that respect all three.</p><p><strong>What’s the most surprising thing you’ve learned about how your data pipeline actually works under the hood?</strong></p><p><strong>Have you experienced the gap between “exactly-once” marketing claims and at-least-once reality?</strong></p><p><strong>What’s your approach to keeping derived data in sync - CDC, event sourcing, or something else entirely?</strong></p><p><em>This blog is based on Part III (Derived Data) of “Designing Data-Intensive Applications” by Martin Kleppmann — the final chapters that tie everything together and ask not just “can we build it?” but “should we, and how do we do it right?”</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24da2cf97a25" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-architecture-of-data-from-raw-logs-to-real-time-systems-part-3-24da2cf97a25">The Architecture of Data: From Raw Logs to Real-Time Systems (Part 3)</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Architecture of Data: When One Machine Isn’t Enough (Part 2)]]></title>
            <link>https://levelup.gitconnected.com/the-architecture-of-data-when-one-machine-isnt-enough-part-2-8fa25cd12afb?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/8fa25cd12afb</guid>
            <category><![CDATA[principal-engineer]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[distributed-systems]]></category>
            <category><![CDATA[information-architecture]]></category>
            <category><![CDATA[database-design]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Wed, 11 Feb 2026 17:30:47 GMT</pubDate>
            <atom:updated>2026-02-15T08:22:39.509Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A guide to replication, partitioning, transactions, and the hard truths of distributed systems.</em></h4><p><strong>TL;DR</strong> When your data outgrows a single machine, a new class of problems emerges. This blog covers the five chapters that every engineer building distributed systems needs to internalize:</p><ul><li><strong>Replication</strong>: Keeping copies of data on multiple machines — and dealing with the inevitable lag, conflicts, and split-brain scenarios.</li><li><strong>Partitioning</strong>: Splitting data across nodes so no single machine becomes a bottleneck — and why this is harder than it sounds.</li><li><strong>Transactions</strong>: The guarantees we take for granted, the fine print nobody reads, and the isolation levels that silently let bugs through.</li><li><strong>Distributed System Failures</strong>: Networks lie, clocks drift, and processes pause without warning. The third option between “works” and “doesn’t” is “maybe.”</li><li><strong>Consistency and Consensus</strong>: What it actually takes for multiple machines to agree on something — and why that’s one of the hardest problems in computer science.</li></ul><p>If you build systems that run on more than one machine, this is the reality you’re operating in. The goal isn’t to be scared — it’s to know exactly what can go wrong so you can design around it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*vBqSxWSLiQ8Pbqg4EBM46A.png" /></figure><h3>Keeping Copies in Sync</h3><p>Why would you store the same data on multiple machines? Three reasons: keep data geographically close to users (latency), survive machine failures (availability), and handle more read traffic (throughput).</p><p>Simple enough. The problem isn’t copying data once — it’s handling changes to replicated data.</p><p>Think of it like a chain of restaurants that all cook from one recipe book. The original chef writes new recipes. Other kitchens follow along. But what happens when the original chef writes a new recipe while the others are still preparing the last one? What happens when two chefs in different locations both modify the same recipe?</p><p>This is replication.</p><h4>Leader-Follower: The Default</h4><p>The most common approach is straightforward: one node is the leader, all writes go through it, and followers copy the changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*OBSevPxGjudDwrAFJpRm7g.png" /></figure><p>PostgreSQL, MySQL, MongoDB, Kafka — they all support this model. Writes go to the leader. Reads can go to any replica.</p><p>The critical question is: does the leader wait for followers to confirm the write?</p><p><strong>Synchronous replication</strong>: the head chef waits for every kitchen to confirm they’ve written down the recipe before moving on. Guaranteed consistency, but one slow kitchen blocks everything.</p><p><strong>Asynchronous replication</strong>: the head chef sends the recipe and immediately starts the next one. Fast, but if the head chef’s kitchen burns down before the recipe reaches everyone, it’s lost forever.</p><p>Most systems use <strong>semi-synchronous</strong>: one follower is synchronous (guarantees at least one backup), the rest are async. It’s the pragmatic middle ground.</p><h4>What Happens When the Leader Dies?</h4><p>Failover — promoting a follower to leader — sounds straightforward. It isn’t.</p><p>First, you have to detect the failure. Most systems use a timeout: if the leader doesn’t respond for 30 seconds, assume it’s dead. But maybe it’s just slow, or there’s a network partition. Choose the timeout carefully — too short and you trigger unnecessary failovers, too long and users experience prolonged outages.</p><p>Second, you have to choose a new leader. The best candidate is the follower with the most up-to-date data. But “most up-to-date” is imprecise when replication is asynchronous.</p><p>Third, you have to reconfigure the system. Clients must send writes to the new leader. Other followers must consume from the new leader. And when the old leader comes back, it must become a follower and discard any unreplicated writes.</p><p>What can go wrong? Almost everything:</p><ul><li><strong>Lost writes</strong>: If the old leader had unreplicated writes, they’re discarded when it rejoins as a follower. This violates durability expectations.</li><li><strong>Split brain</strong>: Two nodes both think they’re the leader. Both accept writes. Data diverges. This is one of the most dangerous failure modes in distributed systems.</li><li><strong>Cascading failures</strong>: An ill-timed failover can overload the new leader, triggering another failover.</li></ul><p>Many operations teams prefer manual failover because of these risks. Automatic failover is convenient, but the edge cases can be catastrophic.</p><h4>The Three Faces of Replication Lag</h4><p>When replication is asynchronous, followers can fall behind. This creates three specific anomalies that are worth internalizing because you will encounter all of them:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0uKhml58byaC9KQEMs_1jg.png" /></figure><p>These aren’t theoretical problems. They happen in production at scale. The classic solution is transactions — let the database handle these guarantees. But many distributed databases abandoned transactions for performance reasons, pushing this complexity to application developers.</p><h4>Multi-Leader: When One Leader Isn’t Enough</h4><p>If you have datacenters on multiple continents, forcing all writes through a single leader means cross-ocean latency on every write. Multi-leader replication gives each datacenter its own leader.</p><p>The use case extends beyond datacenters. Your phone’s calendar app working offline? That’s multi-leader replication — your phone is a “leader” that syncs later. Google Docs with multiple editors? Same pattern.</p><p>The big problem: <strong>write conflicts</strong>. Two leaders accept conflicting writes, and when they replicate to each other, chaos ensues.</p><p>Conflict resolution strategies range from crude to sophisticated:</p><ol><li><strong>Last write wins (LWW)</strong>: Highest timestamp wins. Simple, but silently drops data. Cassandra uses this.</li><li><strong>Merge values</strong>: Combine both writes (“title A/B”). Works for some data types.</li><li><strong>Record the conflict</strong>: Store both versions, let the application or user resolve later.</li><li><strong>CRDTs</strong>: Data structures mathematically designed for automatic conflict resolution.</li></ol><p>The best strategy? Avoid conflicts entirely. If all writes for a particular record always go through the same leader, conflicts can’t happen. Route users to their “home” datacenter.</p><h4>Leaderless: The Quorum Approach</h4><p>What if nobody is the leader? Dynamo-style databases (Cassandra, Riak) take this approach: write to multiple replicas directly, read from multiple replicas, and use the most recent value.</p><p>The math is simple. With n replicas, if w replicas acknowledge a write and r replicas are queried for a read, then as long as <strong>w + r &gt; n</strong>, the read and write sets overlap. At least one node in the read set has the latest write.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uU_X0AN516NtnFXVefexqg.png" /></figure><p>Common configuration: n=3, w=2, r=2. Can tolerate one node failure for both reads and writes.</p><p>But even with quorums, you can get stale reads. Concurrent writes, sloppy quorums (writes going to non-designated nodes during outages), and read/write timing all create edge cases. Dynamo-style databases are optimized for workloads that can tolerate eventual consistency.</p><h3>Splitting the Data</h3><p>Replication gives you copies. Partitioning splits the data into pieces so no single machine stores everything.</p><p>Think of it like an encyclopedia split into volumes. Volume A-D on one shelf, E-H on another. Each shelf handles its own portion. If you need “Aardvark,” you know exactly which shelf.</p><p>But how do you decide which data goes to which partition?</p><h4>Key-Range vs Hash Partitioning</h4><p>Two fundamental strategies, each with a clear trade-off:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ipxPVO6hDHtX6k8cNxweyA.png" /></figure><p>Key-range example: if you partition sensor data by timestamp, all of today’s writes hit one partition. Hot spot.</p><p>Hash partitioning example: hash the user ID to distribute evenly. But now “find all activity from March” requires querying every partition.</p><p>Cassandra offers a hybrid: <strong>compound keys</strong>. Hash the first part (user_id) for distribution, use the second part (timestamp) for sorting within a partition. Query “all posts by user-123 from March” efficiently — hash finds the partition, timestamp finds the range.</p><p>Even with good partitioning, hot keys remain a problem. When a celebrity with millions of followers posts, that one key overloads its partition. The fix is application-level: add random suffixes to hot keys (celebrity-123_001 through celebrity-123_100), spread writes across partitions, combine on read. Databases don’t handle this automatically.</p><h4>The Secondary Index Problem</h4><p>Primary key lookups go to one partition. But what about SELECT * FROM cars WHERE color = &#39;red&#39;? Red cars are scattered across all partitions.</p><p>Two approaches:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*otvLlO-Xg3lA3XwaJQrOAw.png" /></figure><p>MongoDB, Elasticsearch, and Cassandra use local indexes. DynamoDB supports global indexes. The trade-off is write speed versus read speed — again, no free lunch.</p><h4>Rebalancing and Routing</h4><p>When you add a node, data must move. The key rule: <strong>never use hash(key) mod N</strong>. Changing N reassigns almost every key.</p><p>Better strategies: create many more partitions than nodes (Riak, Elasticsearch), let partitions split dynamically (HBase, MongoDB), or fix partitions per node (Cassandra). Many systems use human-in-the-loop rebalancing because automatic rebalancing combined with automatic failure detection can cause cascading failures.</p><p>Once data is partitioned, clients need to find the right node. Three options: contact any node and let it forward (Cassandra’s gossip protocol), use a routing tier (ZooKeeper-based), or make the client partition-aware. Most production systems use a coordination service like ZooKeeper that nodes register with and clients subscribe to.</p><h3>The Guarantees We Take for Granted</h3><p>A transaction is a contract between your application and the database. But like most contracts, the fine print matters more than the headline.</p><blockquote><em>Transaction isolation is principally about avoiding race conditions.</em></blockquote><h4>ACID Deconstructed</h4><p>ACID sounds like a single guarantee. It’s actually four separate promises, and each is weaker than you think:</p><p><strong>Atomicity</strong> — Not about concurrency (that’s isolation). Atomicity means all-or-nothing: if a transaction fails partway through, all changes are rolled back. The database uses a write-ahead log so it can undo partial writes after a crash.</p><p><strong>Consistency</strong> — The most misleading letter. Consistency in ACID means your application’s invariants hold (account balances don’t go negative, email addresses are unique). But this is <strong>your application’s responsibility</strong>, not the database’s. The database just gives you the tools.</p><p><strong>Isolation</strong> — Concurrent transactions don’t interfere with each other. In theory, this means serializability — the result is the same as if transactions ran one at a time. In practice, most databases use weaker isolation because serializability is expensive.</p><p><strong>Durability</strong> — Committed data survives crashes. Implemented through write-ahead logs and replication. But perfect durability doesn’t exist — if every disk and every backup fails simultaneously, data is gone.</p><h4>The Isolation Levels Nobody Understands</h4><p>This is where most production bugs hide. Each isolation level prevents certain anomalies while allowing others:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3iOWyTGRVLaWA-2RVW456A.png" /></figure><h4>Lost Updates: The Race Condition You’ve Already Written</h4><p>Two transactions read the same value, modify it, and write back. One modification disappears.</p><pre>-- Transaction 1                    -- Transaction 2<br>SELECT counter FROM stats;          SELECT counter FROM stats;<br>-- counter = 42                     -- counter = 42<br>UPDATE stats                        UPDATE stats<br>SET counter = 43;                   SET counter = 43;<br>-- Expected final value: 44<br>-- Actual final value: 43 (one increment lost!)</pre><p>Solutions, from simplest to most general:</p><ol><li><strong>Atomic operations</strong>: UPDATE stats SET counter = counter + 1 - let the database handle it</li><li><strong>Explicit locking</strong>: SELECT ... FOR UPDATE - lock the row while you&#39;re modifying it</li><li><strong>Automatic detection</strong>: PostgreSQL and Oracle’s snapshot isolation detect lost updates and abort one transaction</li><li><strong>Compare-and-set</strong>: UPDATE ... WHERE counter = 42 - only update if the value hasn&#39;t changed</li></ol><h4>Write Skew: The Bug That Passes Code Review</h4><p>Write skew is more subtle than lost updates. Two transactions read the same data, each makes a valid decision based on what they read, but they write to <strong>different</strong> objects. The combined result violates a constraint.</p><p>The classic example: a hospital requires at least one doctor on call. Alice and Bob are both on call. Both check the count (2), both conclude they can leave, both update their own record. Nobody is on call.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zYhCnuvaZ5UQ-4Sfw4yVIw.png" /></figure><p>You can’t fix this with atomic operations or automatic detection because no single write conflicts with another — they write to different rows. The same pattern shows up in meeting room double-bookings, username registration races, and financial overdrafts across multiple systems.</p><p>Only serializable isolation prevents write skew.</p><p>A related problem is <strong>phantoms</strong>: when a write in one transaction changes the result of a search query in another transaction. Meeting room double-bookings are a phantom problem — both transactions check “is the room free?”, both see no bookings, both insert a booking. You can’t lock rows that don’t exist yet. The workaround is <strong>materializing conflicts</strong> — pre-creating objects to lock (e.g., a row for every room/time slot). But that’s ugly. Serializable isolation is cleaner.</p><h4>Three Roads to Serializability</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UH4aggTC9xQRuQhTBHVQyQ.png" /></figure><p>VoltDB and Redis use serial execution. Traditional databases used 2PL for decades. PostgreSQL 9.1+ offers SSI. Each makes different performance trade-offs.</p><p>The takeaway: “serializable” is a single guarantee, but the three implementations have wildly different performance characteristics. Know which one your database uses.</p><h3>The Harsh Reality of Distributed Systems</h3><p>On a single computer, things are deterministic. An operation either works or fails completely. In a distributed system, a third option exists: <strong>maybe</strong>. Some parts work while others fail. And you can’t tell which is which.</p><p>This is the fundamental shift. Not “things can fail” — every engineer knows that. The shift is “things can partially fail in ways that look like success.”</p><h4>Unreliable Networks</h4><p>When you send a request and don’t get a response, you have no idea what happened. The request could be lost, queued, or processed with the response lost. The remote node could be crashed, slow, or paused. You cannot distinguish between these cases.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lFAs_9Bya56wl1fKF9CYaQ.png" /></figure><p>The only tool you have is timeouts. But there’s no correct timeout value. Too short: you declare alive nodes dead and trigger unnecessary failovers. Too long: users wait forever during actual failures.</p><p>Internet networks are fundamentally asynchronous — packets share capacity, queues add variable delays, and there’s no upper bound on latency. This is unlike telephone networks, which reserve dedicated bandwidth per connection. The internet chose packet switching because it’s more efficient for bursty data traffic. The cost is unpredictable latency.</p><h4>Unreliable Clocks</h4><p>Every computer has a clock, but clocks on different machines disagree. NTP synchronization helps, but network latency makes it imprecise — typical error is several milliseconds.</p><p>This breaks “last write wins.” If Node A’s clock is slightly fast and Node B’s clock is slightly slow, Node A’s write gets a higher timestamp even though Node B actually wrote later. Data is silently lost because of clock skew.</p><p>It’s like people in different time zones comparing wall clocks to decide who called first. The comparison is meaningless without a shared reference frame.</p><p>Two types of clocks exist for good reason:</p><ul><li><strong>Time-of-day clocks</strong>: “What time is it?” — can jump backward due to NTP adjustments, not reliable for ordering</li><li><strong>Monotonic clocks</strong>: “How much time has elapsed?” — always moves forward, perfect for measuring durations, meaningless across machines</li></ul><p>Google’s Spanner solved this with TrueTime: instead of pretending clocks are precise, it returns a confidence interval. “The current time is between 14:30:00.120 and 14:30:00.126.” Then it waits out the uncertainty before committing. This requires GPS receivers and atomic clocks in every datacenter — not something most of us have.</p><h4>Process Pauses: The Invisible Danger</h4><p>A process can be frozen for an arbitrary amount of time without knowing it. Garbage collection pauses, VM live migration, OS context switches, disk I/O waits — all can freeze your process for seconds.</p><p>This creates the <strong>fencing token problem</strong>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ylW1anyLEgXOukMx1WgokQ.png" /></figure><p>Without fencing tokens, Client A would corrupt Client B’s work. The lock service issues monotonically increasing tokens, and the storage layer rejects writes with outdated tokens. This is why coordination services like ZooKeeper exist — they implement these patterns so you don’t have to get them wrong.</p><h4>Truth Is Defined by the Majority</h4><p>In a distributed system, a node cannot trust its own judgment. It might think it’s the leader, but a network partition could mean the other nodes have already elected a new leader. If the majority of nodes believe you’re dead, you’re dead — even if you’re still running.</p><p>This is why quorum decisions matter: a majority (more than half) of nodes must agree. In a 5-node cluster, 3 nodes can form a quorum. The minority side of a network partition must stop accepting writes to avoid split-brain.</p><h4>Safety and Liveness: The Two Properties</h4><p>Every distributed algorithm cares about two kinds of properties:</p><p><strong>Safety</strong>: “Nothing bad happens.” If violated, you can point to a specific moment when it broke. Examples: at most one node holds the lock at any time. Sequence numbers always increase. All nodes see the same value.</p><p><strong>Liveness</strong>: “Something good eventually happens.” Cannot point to a specific moment of violation because “eventually” hasn’t happened yet. Examples: requests eventually receive responses. The algorithm eventually completes.</p><p>The practical rule: safety must always be preserved, even during failures. Liveness can be weakened during failures. It’s OK for the system to be slow during a network partition, but it must never return wrong results. “Slow but correct” beats “fast but wrong.”</p><p>Most practical systems assume a <strong>partially synchronous</strong> model (network behaves well most of the time but can be slow) with <strong>crash-recovery</strong> failures (nodes can crash and come back). This is realistic for cloud environments. Byzantine faults — nodes actively lying or sending corrupted data — are only relevant for blockchain systems, aerospace, or multi-organization deployments where participants don’t trust each other.</p><h3>Putting It All Together</h3><blockquote><em>The CAP theorem is sometimes presented as ‘pick 2 out of 3.’ Unfortunately, putting it this way is misleading.</em></blockquote><p>The real choice: during a network partition (which will happen), do you prioritize consistency (return errors rather than stale data) or availability (always return something, even if it’s stale)? When there’s no partition, you can have both.</p><p>But CAP is only the beginning. The deeper lesson is about the spectrum of guarantees and their costs.</p><h4>The Spectrum of Consistency</h4><p><strong>Linearizability</strong> is the strongest guarantee: make it look like there’s only one copy of the data. Like a single variable in a single-threaded program — once a read returns a new value, all subsequent reads must return that value or newer.</p><p>When do you need it? Leader election (only one leader at a time), uniqueness constraints (only one user with a given username), cross-channel coordination (image must be stored before processor receives the “process it” message).</p><p>The cost: coordination between nodes on every operation. Cross-datacenter latency. Limited throughput.</p><p>Which replication methods can provide linearizability? Single-leader can, but only if you read from the leader (or use synchronous replication to followers). Multi-leader cannot — concurrent writes on different leaders means no total order. Leaderless with quorums looks like it might, but timing issues with concurrent reads and writes can still violate linearizability even with strict quorums. Consensus algorithms (Raft, Paxos, Zab) are designed specifically for this — it’s why etcd and ZooKeeper use them.</p><p><strong>Causal consistency</strong> is weaker but often sufficient. It only orders events that could have affected each other. Two independent operations can proceed in parallel without coordination. This is the strongest model that doesn’t require synchronization across all nodes.</p><p><strong>Eventual consistency</strong> is the weakest useful guarantee: if you stop writing and wait long enough, all replicas will converge. No promise about how long “long enough” is.</p><h4>Consensus: The Hardest Problem</h4><p>Multiple nodes agreeing on a value sounds simple. It’s one of the hardest problems in distributed systems. The requirements: all nodes decide the same value, each decides at most once, the decision was actually proposed by someone, and every non-crashed node eventually decides.</p><p><strong>Two-phase commit (2PC)</strong> is the classic approach: a coordinator asks all participants “can you commit?”, waits for all to say yes, then tells everyone to commit. The fatal flaw: if the coordinator crashes after participants have voted yes but before sending the commit decision, participants are stuck. They can’t commit (didn’t get the message) and can’t abort (might have been commit). They hold locks and wait for the coordinator to recover. This is why 2PC is called a “blocking” protocol.</p><p><strong>Raft</strong> is the modern answer. One node is elected leader for a term. The leader handles all client requests, replicates log entries to followers, and commits once a majority acknowledge. If the leader fails, followers detect the timeout, start an election for a new term, and the candidate with the most up-to-date log wins.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*X5kipim29HtcCfVOHZD_vg.png" /></figure><p>Why it’s safe: election requires a majority vote. Commit requires a majority acknowledgment. Two majorities in the same cluster always overlap, so the new leader always knows about committed entries.</p><p><strong>Coordination services</strong> (ZooKeeper, etcd, Consul) implement consensus so your application doesn’t have to. They provide linearizable key-value storage, distributed locks, leader election, service discovery, and change notifications. They’re designed for small amounts of critical coordination data — not for general data storage.</p><h4>A System Is Only as Strong as Its Guarantees</h4><blockquote><em>A system is only as strong as the guarantees it actually provides, not the ones you assume it provides.</em></blockquote><p>Four questions to ask about any distributed data system:</p><ol><li><strong>What replication model does it use?</strong> Leader-follower, multi-leader, or leaderless — and what are the lag implications?</li><li><strong>How is data partitioned?</strong> Key-range or hash — and what happens to secondary index queries?</li><li><strong>What isolation level is actually running?</strong> Not what the documentation says. What the default configuration provides.</li><li><strong>What happens during a network partition?</strong> Does it prioritize consistency or availability? Does it even handle partitions gracefully?</li></ol><h4>The Human Element</h4><p>The impossibility results in distributed systems aren’t reasons to give up. They’re reasons to be precise about what your system actually guarantees.</p><p>Most applications don’t need linearizability everywhere. Most transactions don’t need serializable isolation. Most data doesn’t need synchronous replication. The art is knowing which parts of your system need strong guarantees and which can tolerate weaker ones.</p><p>The most common failure mode isn’t a clever distributed systems edge case. It’s an engineer who didn’t understand what isolation level their database was running, or who assumed replication was synchronous when it wasn’t, or who didn’t account for clock skew in their timestamp-based ordering.</p><p>Understanding beats assumptions. Explicit guarantees beat implicit ones. And the best distributed system is the one whose failure modes your team understands.</p><h3>The Road Ahead</h3><p>Parts I and II cover the foundations and the challenges of distributed data. Part III shifts perspective: instead of asking how to store and retrieve data reliably, it asks how to <strong>derive</strong> new data from existing data.</p><p><strong>Batch processing: </strong>MapReduce and dataflow engines like Spark. Take massive datasets and transform them. The Unix philosophy applied to distributed systems: small tools, composable pipelines, immutable inputs.</p><p><strong>Stream processing: </strong>What if the data never stops arriving? Message brokers, event sourcing, change data capture. Processing data in real-time as it flows through the system.</p><p><strong>The future of data systems: </strong>The boundaries between databases, caches, and message queues are blurring. The real question isn’t which tool to use — it’s how to compose multiple tools into a system that provides the guarantees your application needs.</p><p>The theme remains the same: understand the trade-offs, make them explicitly, and design for the failure modes that matter.</p><p><strong>What’s the most subtle distributed systems bug you’ve encountered in production?</strong></p><p><strong>Have you ever been burned by assuming stronger isolation guarantees than your database actually provides?</strong></p><p><strong>What coordination problems have you solved with ZooKeeper, etcd, or similar tools — and what would you do differently?</strong></p><p><em>This blog is based on Part II (Distributed Data) of “Designing Data-Intensive Applications” by Martin Kleppmann, the chapters that separate engineers who build distributed systems from engineers who understand them.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8fa25cd12afb" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-architecture-of-data-when-one-machine-isnt-enough-part-2-8fa25cd12afb">The Architecture of Data: When One Machine Isn’t Enough (Part 2)</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Architecture of Data: Building Systems That Last (Part 1)]]></title>
            <link>https://levelup.gitconnected.com/the-architecture-of-data-building-systems-that-last-part-1-ccb02dbf039b?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/ccb02dbf039b</guid>
            <category><![CDATA[database-design]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[principal-engineer]]></category>
            <category><![CDATA[data-model]]></category>
            <category><![CDATA[solution-architect]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Mon, 09 Feb 2026 15:37:51 GMT</pubDate>
            <atom:updated>2026-02-15T08:04:26.820Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>A comprehensive guide to designing data-intensive applications that scale gracefully and evolve without breaking.</em></h4><p><strong>TL;DR</strong> Every data system is a series of trade-offs. This blog covers the foundations you need to make informed decisions:</p><ul><li><strong>Reliability, Scalability, Maintainability</strong>: The three pillars that separate systems that thrive from those that collapse under their own weight.</li><li><strong>Data Models</strong>: Choosing between relational, document, and graph models — and why that choice constrains everything else.</li><li><strong>Storage Engines</strong>: What happens after you click “save” — from hash indexes to LSM-trees to B-trees.</li><li><strong>Encoding and Evolution</strong>: How to change your system while it’s running without breaking everything.</li></ul><p>If you’re a developer who designs systems, chooses databases, or makes architectural decisions, this is the foundation you need. The goal isn’t to memorize implementations — it’s to build a mental framework for evaluating trade-offs.</p><p>Let’s dive in.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/928/1*BkwaNJSbvUDHSmOm9H_U4g.png" /></figure><h3>What Actually Makes a System “Good”?</h3><p>Most systems work. The question is: do they keep working when things go wrong? Do they handle growth without requiring a complete rewrite? Can new engineers understand them without a six-month onboarding process?</p><p>These questions point to three fundamental concerns that matter more than any feature list:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*_TPUv-AGT_sguPbo0Ncr0g.png" /></figure><h4>Reliability: The Art of Failing Gracefully</h4><p>Here’s a truth that surprises many engineers: hardware failures aren’t your biggest reliability threat. Human errors are.</p><p>Studies consistently show that configuration errors by operators cause the leading outages — not disk failures, not network partitions, not software bugs. Humans are unreliable. The most reliable systems acknowledge this and design around it.</p><p>What does designing for human error look like?</p><ul><li><strong>Well-designed APIs</strong> that make it easy to do the right thing and hard to do the wrong thing</li><li><strong>Sandbox environments</strong> where mistakes can happen safely</li><li><strong>Fast rollback</strong> capabilities when mistakes happen in production</li><li><strong>Detailed monitoring</strong> that tells you what happened, not just that something went wrong</li></ul><p>Reliability is like a building’s structural integrity. You don’t notice it until it fails. The goal isn’t to prevent all faults — that’s impossible. The goal is to build <strong>fault-tolerant</strong> systems that prevent faults from becoming failures.</p><h4>Scalability: The Twitter Problem</h4><p>Scalability isn’t a binary property. You can’t say “this system is scalable” or “this system doesn’t scale.” The real question is more nuanced: if the system grows in a particular way, what are our options for coping?</p><p>Consider Twitter’s timeline challenge. They have two main operations:</p><ul><li><strong>Post a tweet</strong>: About 4.6k requests/second on average</li><li><strong>View home timeline</strong>: About 300k requests/second</li></ul><p>The naive approach is simple: when a user requests their timeline, query the database for everyone they follow, fetch their recent tweets, and merge them. This is called <strong>fan-out on read</strong>.</p><pre>SELECT tweets.*, users.*<br>FROM tweets<br>JOIN users ON tweets.sender_id = users.id<br>JOIN follows ON follows.followee_id = users.id<br>WHERE follows.follower_id = current_user<br>ORDER BY tweets.created_at DESC<br>LIMIT 100;</pre><p>But this query gets slow fast. If you follow 1,000 people, that’s a lot of data to merge on every timeline view.</p><p>Twitter’s alternative: <strong>fan-out on write</strong>. When someone posts a tweet, immediately insert it into the home timeline cache of all their followers. Now reading is fast — just look up the cache.</p><p>But what about celebrities? Justin Bieber has 30+ million followers. One tweet means 30 million cache insertions. That’s not sustainable.</p><p>Twitter’s solution is a hybrid: fan-out on write for most users, fan-out on read for celebrities. The lesson isn’t about Twitter’s specific solution. It’s that scalability depends entirely on your access patterns, and there’s no one-size-fits-all answer.</p><h4>Percentiles: Why Averages Lie</h4><p>If your average response time is 200ms, that tells you almost nothing about user experience.</p><p>Use <strong>percentiles</strong> instead:</p><ul><li><strong>p50 (median)</strong>: Half of requests are faster, half are slower</li><li><strong>p95</strong>: 95% of requests are faster than this</li><li><strong>p99</strong>: 99% of requests are faster than this</li></ul><p>Here’s why tail latencies matter: users with slow responses are often your most valuable customers. They have more data, more purchases, more history. A single slow backend call can delay an entire user request. If a page makes 100 backend calls, even p99.99 becomes relevant because the probability that at least one is slow becomes significant.</p><p>The average is a lie that hides the experience of your most important users.</p><h4>Maintainability: The Long Game</h4><p>Here’s an uncomfortable truth: the majority of software cost isn’t initial development. It’s maintenance — fixing bugs, keeping systems operational, adapting to new platforms, adding features, repaying technical debt.</p><blockquote><em>“Simplicity is prerequisite for reliability.” — Edsger Dijkstra</em></blockquote><p>Maintainability comes down to three principles:</p><ol><li><strong>Operability</strong>: Can your operations team keep it running? Good monitoring, reasonable defaults, documentation that actually exists.</li><li><strong>Simplicity</strong>: Can new engineers understand it? Complexity is the enemy. Every abstraction that hides unnecessary detail is a gift to future maintainers.</li><li><strong>Evolvability</strong>: Can you make changes without rewriting everything? This depends on good abstractions and clean boundaries between modules.</li></ol><p>Systems aren’t static. Requirements change constantly. The systems that last are the ones designed for change.</p><h3>Choosing Your Data Model is Choosing Your Constraints</h3><p>Data models are perhaps the most important decision in software development. They have a profound effect on how we think about the problem we’re solving.</p><p>Think of choosing a data model like choosing a city layout. Manhattan’s grid makes navigation simple and predictable. Boston’s organic streets evolved around existing paths and buildings. Each optimizes for different things. Neither is universally better — but each makes certain things easy and others difficult.</p><h4>The Relational Model: 50 Years of Dominance</h4><p>Edgar Codd proposed the relational model in 1970. Decades later, despite countless challengers, it still dominates. Why?</p><ul><li><strong>General-purpose</strong>: Works for many use cases</li><li><strong>Powerful query optimization</strong>: The database figures out how to execute your query efficiently</li><li><strong>Mature ecosystem</strong>: Tools, expertise, reliability</li></ul><p>The power of relational databases lies in <strong>normalization</strong> — storing each piece of information once and referencing it by ID.</p><p>Consider storing a user’s city. You could store “Seattle” as a string in every row. But if the city name changes, you need to update every row. Different spellings might creep in. You’re wasting storage.</p><p>The normalized approach: store “Seattle” once with an ID, and reference that ID everywhere. Single source of truth. Update once, reflected everywhere.</p><p>The cost? Joins. To reassemble the data, you need to query multiple tables and combine them. This trade-off — storage efficiency and consistency versus query complexity — is at the heart of relational design.</p><h4>The Document Model: Flexibility and Locality</h4><p>What if your data is naturally hierarchical? A user profile with positions, education, and interests. A blog post with comments and tags. An e-commerce order with line items.</p><p>Document databases store these as self-contained documents:</p><pre>{<br>    &quot;user_id&quot;: 123,<br>    &quot;name&quot;: &quot;Alice&quot;,<br>    &quot;positions&quot;: [<br>        { &quot;title&quot;: &quot;Engineer&quot;, &quot;company&quot;: &quot;Acme&quot; },<br>        { &quot;title&quot;: &quot;Senior Engineer&quot;, &quot;company&quot;: &quot;Globex&quot; }<br>    ],<br>    &quot;education&quot;: [<br>        { &quot;school&quot;: &quot;MIT&quot;, &quot;degree&quot;: &quot;CS&quot; }<br>    ]<br>}</pre><p>The advantages are compelling:</p><ul><li><strong>No impedance mismatch</strong>: JSON maps directly to application objects</li><li><strong>Better locality</strong>: All related data in one place, one query gets everything</li><li><strong>Schema flexibility</strong>: Different documents can have different structures</li></ul><p>But here’s the catch: <strong>schema-on-read vs schema-on-write</strong>.</p><p>Relational databases enforce schema when you write. If the data doesn’t match the schema, the write fails. This is like customs at the border — strict validation at entry.</p><p>Document databases often enforce schema when you read. You write whatever you want; your application handles the structure (or lack thereof) when reading. This is like trusting tourists and auditing randomly.</p><p>Neither is universally better. Schema-on-write catches errors early but makes evolution harder. Schema-on-read is flexible but pushes validation into application code.</p><p>Documents struggle with <strong>many-to-many relationships</strong>. If Alice recommends Bob, and you want to show Alice’s profile with Bob’s name, you need to:</p><ol><li>Fetch Alice’s document</li><li>Extract Bob’s user_id</li><li>Fetch Bob’s document</li><li>Combine in application code</li></ol><p>You’re doing a join manually. Often slower and more error-prone than letting the database handle it.</p><h4>The Graph Model: When Connections Are the Data</h4><p>Sometimes the relationships between entities are as important as the entities themselves. Social networks. Fraud detection. Road networks. Recommendation engines.</p><p>Graph databases model data as vertices (nodes) and edges (relationships):</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7CwZMuSAd2HD-BqdDae8UQ.png" /></figure><p>The key insight: any vertex can have an edge to any other vertex. There’s no schema restricting what can be connected. This flexibility is powerful for modeling complex, evolving domains.</p><p>Graph query languages like Cypher make traversing relationships natural:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RkBeLQh-NfoFvrrAs1xfFw.png" /></figure><p>This finds all people born anywhere within the United States — whether directly in a state, or in a city within a state. The *0.. means &quot;follow any number of WITHIN edges.&quot; You don&#39;t need to know how deep the hierarchy goes.</p><h4>The Convergence</h4><p>Here’s the interesting trend: the lines are blurring. PostgreSQL has excellent JSON support. MongoDB added joins. Graph extensions appear everywhere.</p><p>The future likely has multi-model databases that support multiple approaches. But understanding the fundamental trade-offs remains essential — because even in a multi-model world, you’re still making choices about how to structure your data.</p><p><strong>The question to ask</strong>: What are your access patterns? Do you load whole documents or join across entities? Are relationships first-class citizens or afterthoughts? How often does the schema change?</p><h3>What Happens After You Click “Save”</h3><p>A database needs to do two things: store data when you give it data, and give data back when you ask for it. But how it does this internally has profound implications for performance.</p><p>Here’s the fundamental trade-off: <strong>you cannot optimize for everything simultaneously</strong>. A system optimized for writes will be slower at reads. A system optimized for reads will be slower at writes. Understanding this trade-off helps you choose the right tool.</p><p>Think of it like a library versus a printing press. A library organizes books for finding them. A printing press organizes for producing them. A library that’s also a printing press has to compromise.</p><h4>The World’s Simplest Database</h4><p>Let’s start with the simplest possible database — two bash functions:</p><pre>db_set() {<br>    echo &quot;$1,$2&quot; &gt;&gt; database<br>}<br><br>db_get() {<br>    grep &quot;^$1,&quot; database | sed -e &quot;s/^$1,//&quot; | tail -n 1<br>}</pre><p>db_set appends to a file. O(1) - incredibly fast. db_get scans the entire file. O(n) - painfully slow with millions of records.</p><p>This is the fundamental tension. Writes are easy — just append. Reads require searching.</p><p>The solution is <strong>indexes</strong> — additional data structures that help locate data quickly. But indexes slow down writes because every write must update the index. This is why databases don’t index everything by default.</p><h4>Hash Indexes: Simple and Fast</h4><p>The simplest index: keep an in-memory hash map where keys point to byte offsets in the data file.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*XvolXQ3JbH212k1N-ax9wg.png" /></figure><p>Read: look up key in hash map, seek to offset, read value. Write: append to file, update hash map.</p><p>This is essentially what Bitcask (Riak’s storage engine) does. It works surprisingly well for workloads where you update the same keys frequently but don’t have too many distinct keys.</p><p>The limitations are significant: the hash table must fit in memory, and range queries are expensive (you can’t efficiently find all keys between X and Y).</p><h4>LSM-Trees: Optimized for Writes</h4><p>What if we sort the data? This is the insight behind <strong>Log-Structured Merge-Trees (LSM-Trees)</strong>.</p><p>The algorithm:</p><ol><li><strong>Writes</strong> go to an in-memory balanced tree (the <strong>memtable</strong>)</li><li>When the memtable gets big, write it to disk as a sorted file (an <strong>SSTable</strong>)</li><li><strong>Reads</strong> check the memtable, then SSTables from newest to oldest</li><li><strong>Background compaction</strong> merges SSTables to reduce file count</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VUmS-p19KG5kKwG2usqSTQ.png" /></figure><p>The magic of sorting: merging sorted files is efficient (like the merge step in merge sort). And you don’t need to keep all keys in memory — a sparse index pointing to every few thousand keys is enough.</p><p>LSM-trees power Cassandra, LevelDB, RocksDB, and HBase. They excel at write-heavy workloads because writes are sequential (just append to the memtable, then write sorted files).</p><p>The cost? Reads may need to check multiple SSTables. Bloom filters help — they can quickly tell you “this key definitely doesn’t exist in this file,” avoiding unnecessary disk reads.</p><h4>B-Trees: The Read-Optimized Classic</h4><p>B-trees have been the standard for almost all relational databases for over 40 years. They work differently.</p><p>Instead of append-only logs, B-trees use fixed-size <strong>pages</strong> (typically 4KB) organized in a tree structure:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*9rKj0Dw8JIrVyT3sYvW2PA.png" /></figure><p>To find a key: start at the root, find the right range, follow the pointer to the child page, repeat until you reach a leaf.</p><p>The branching factor (number of children per page) is typically several hundred. This means a 4-level tree can store enormous amounts of data, and any lookup requires at most 4 page reads.</p><p>Unlike LSM-trees, B-trees modify pages in place. This requires careful handling: a <strong>Write-Ahead Log (WAL)</strong> ensures that if the system crashes mid-write, the database can recover.</p><h4>The Trade-off</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lHdsZ5mjCJIkw2DvddYk7g.png" /></figure><p>There’s no “best” storage engine. Write-heavy workloads favor LSM-trees. Read-heavy workloads favor B-trees. Transactional requirements might favor B-trees for their simpler concurrency model.</p><p><strong>The real answer: test with your actual workload.</strong></p><h4>OLTP vs OLAP: Two Different Worlds</h4><p>Not all database workloads are alike. <strong>OLTP (Online Transaction Processing)</strong> and <strong>OLAP (Online Analytical Processing)</strong> have fundamentally different access patterns:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*mP0T98YtI3KF8LSl76QjxQ.png" /></figure><p>This is why data warehouses exist. You don’t want analytical queries slowing down your production database. You extract, transform, and load (ETL) data into a separate system optimized for analytics.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*NoihTyqHYiH2uUAfhusazA.png" /></figure><h4>Column-Oriented Storage: The Analytics Secret Weapon</h4><p>Analytics queries typically scan millions of rows but only need a few columns. Row-oriented storage is wasteful — you load entire rows just to use 3 columns.</p><p>Column-oriented storage stores each column separately:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0022OuOXTcJsnijEukAFBA.png" /></figure><p>Now a query that needs only date, product, and quantity loads only those columns. Massive I/O savings.</p><p>Better yet, column storage compresses extremely well. Values in a column are often similar, enabling techniques like bitmap encoding that reduce data size dramatically.</p><p>This is why systems like Redshift, Snowflake, and BigQuery dominate analytics workloads.</p><h3>The Art of Evolving Systems While They’re Running</h3><p>Applications change. Features are added. Requirements evolve. This means data formats need to change too.</p><p>But in a large system, you can’t upgrade everything at once. During a rolling deployment, some servers run new code while others run old code. Mobile apps might never update. This means <strong>different versions of your code will be running simultaneously</strong>, reading and writing potentially different formats.</p><p>The question is: how do you ensure old and new code can coexist?</p><h4>The Two Types of Compatibility</h4><p><strong>Backward compatibility</strong>: New code can read data written by old code. This is usually easier — you control the new code, so you know what to expect.</p><p><strong>Forward compatibility</strong>: Old code can read data written by new code. This is trickier — old code must gracefully ignore fields it doesn’t understand.</p><p>You need both. During a rolling upgrade, new servers receive data from old servers (backward compatibility), and old servers receive data from new servers (forward compatibility).</p><h4>The Problem with JSON</h4><p>JSON is everywhere, but it has real problems for evolving systems:</p><ol><li><strong>Number ambiguity</strong>: JavaScript uses double-precision floats for all numbers. Integers greater than 2⁵³ lose precision.</li><li><strong>No binary support</strong>: Binary data must be Base64-encoded, increasing size by 33%.</li><li><strong>No enforced schema</strong>: You never really know what fields to expect.</li></ol><p>JSON is like natural language — flexible and expressive, but prone to misunderstandings. It’s fine for debugging and configuration. For high-throughput internal services, binary formats are often better.</p><h4>Schema-Based Binary Encoding</h4><p>Protocol Buffers, Thrift, and Avro use schemas to enable much more compact encoding.</p><p>Protocol Buffers example:</p><pre>message Person {<br>    required string user_name       = 1;<br>    optional int64  favorite_number = 2;<br>    repeated string interests       = 3;<br>}</pre><p>Notice the numbers (1, 2, 3)? These are <strong>field tags</strong>. In the encoded data, we store these numbers instead of field names. The result: the same data that’s 81 bytes as JSON is only 33 bytes as Protocol Buffers.</p><p>But the real magic is <strong>schema evolution</strong>.</p><p>Adding a field:</p><pre>message Person {<br>    required string user_name       = 1;<br>    optional int64  favorite_number = 2;<br>    repeated string interests       = 3;<br>    optional string email           = 4;  // NEW<br>}</pre><ul><li>Old code reading new data sees tag 4, doesn’t recognize it, skips it. Forward compatibility.</li><li>New code reading old data doesn’t find tag 4, uses the default. Backward compatibility.</li></ul><p>The rules are simple:</p><ul><li>Never change a field tag number</li><li>Only add optional fields</li><li>Never remove required fields</li><li>Never reuse tag numbers</li></ul><h4>Avro: A Different Approach</h4><p>Avro takes a radically different approach: <strong>no field tags at all</strong>. Fields are matched by name.</p><p>The key concept is <strong>writer’s schema vs reader’s schema</strong>. The encoder has a schema. The decoder has a (possibly different) schema. Avro compares them at read time and resolves differences.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jEd_hx0d4Q65gekuH3T88g.png" /></figure><p>The schema resolution works like a translator who knows both dialects. Fields present in both schemas are matched by name. Fields in the writer’s schema but not the reader’s are ignored. Fields in the reader’s schema but not the writer’s use default values.</p><p>This sounds complex, but it’s powerful for dynamically generated schemas. If your schema comes from a database schema, you don’t want to manually assign tag numbers. Avro’s name-based matching is more natural.</p><p><strong>When to use which:</strong></p><ul><li><strong>Protocol Buffers/Thrift</strong>: Strict schemas, code generation, clear versioning</li><li><strong>Avro</strong>: Dynamic schemas, Hadoop ecosystem, schema changes frequently</li><li><strong>JSON</strong>: Human readability, external APIs, debugging</li></ul><h4>Three Modes of Data Flow</h4><p>Data flows between systems in different ways, each with different compatibility implications:</p><p><strong>Through databases</strong>: Data outlives code. Records written years ago need to be readable today. When rewriting old records, preserve unknown fields — don’t discard them.</p><p><strong>Through services (REST/RPC)</strong>: Servers may be upgraded before clients. API versioning through URLs or headers helps manage this.</p><p><strong>Through messages</strong>: Message brokers like Kafka decouple senders from receivers. The broker stores messages, providing buffering and durability. This asynchronous model is more resilient but requires careful attention to message format evolution.</p><h3>Putting It All Together</h3><blockquote><em>“There is no silver bullet.” — Fred Brooks</em></blockquote><p>Every architectural decision is a trade-off. The goal isn’t finding the “best” solution — it’s finding the right trade-offs for your context.</p><h4>The Trade-off Mindset</h4><p>Here’s a mental model that helps: for every architectural decision, ask “What am I giving up?”</p><p>Choose a document database for schema flexibility? You’re giving up easy many-to-many relationships. Choose an LSM-tree for write throughput? You’re giving up predictable read latency. Choose Protocol Buffers for compact encoding? You’re giving up human readability.</p><p>None of these are wrong choices. They’re trade-offs. The danger isn’t making trade-offs — it’s making them without realizing what you’re trading away.</p><p><strong>The questions to ask before choosing:</strong></p><ol><li><strong>What are the access patterns?</strong> Read-heavy or write-heavy? Point lookups or range scans?</li><li><strong>How does the data relate to itself?</strong> Independent documents or interconnected graph? One-to-many or many-to-many?</li><li><strong>How will the schema evolve?</strong> Stable for years or changing weekly? Controlled by you or by external systems?</li><li><strong>What’s the consistency requirement?</strong> Financial transaction or social media feed? “Eventually consistent” or “immediately correct”?</li><li><strong>What’s the team’s expertise?</strong> The best system is the one you can operate. A theoretically perfect system that your team can’t debug at 3 AM is worse than a simpler system they understand.</li></ol><h4>The Human Element</h4><blockquote><strong>Systems are built by humans, operated by humans, and used by humans.</strong></blockquote><p>This seems obvious, but it has profound implications. The most elegant distributed algorithm is worthless if no one on your team understands it. The most performant storage engine is worthless if you can’t diagnose why it’s slow.</p><p>Simplicity is a feature, not a compromise. Every layer of abstraction you add must justify its existence. Every dependency you introduce must earn its place. Every configuration option you expose becomes something that can be misconfigured.</p><p>Documentation isn’t a nice-to-have — it’s part of the system. A system without documentation is a system that only exists in the heads of its original authors. When those authors leave, the system becomes legacy code that everyone fears to touch.</p><p>Operability isn’t a nice-to-have either. Monitoring, alerting, debugging tools, deployment automation — these aren’t afterthoughts. They’re essential infrastructure that determines whether your 3 AM pages are “quick fix and back to bed” or “4-hour war room.”</p><p>The best architecture is the one that lets your team sleep at night.</p><h3>The Road Ahead</h3><p>These foundations — reliability, scalability, maintainability, data models, storage engines, and encoding — are just the beginning. They set the stage for the harder problems that arise when you move beyond a single machine.</p><p><strong>What comes next:</strong></p><p><strong>Replication</strong> — How do you keep copies of your data on multiple machines? Leader-follower, multi-leader, leaderless architectures each have different trade-offs. What happens when a replica falls behind? What happens when a leader fails?</p><p><strong>Partitioning</strong> — When your data doesn’t fit on one machine, how do you split it? Hash-based partitioning, range-based partitioning, and the challenge of secondary indexes across partitions.</p><p><strong>Transactions</strong> — What does “ACID” actually mean? Isolation levels from “read uncommitted” to “serializable,” and why most applications run at weaker levels than they think.</p><p><strong>Distributed Systems Challenges</strong> — Networks are unreliable. Clocks drift. Nodes fail silently. The ways things can go wrong in distributed systems are surprisingly subtle and often counterintuitive.</p><p><strong>Consistency and Consensus</strong> — What does “consistency” even mean? Linearizability, causal consistency, eventual consistency. And how do systems like Raft and Paxos achieve agreement across unreliable networks?</p><p>Each of these builds on the foundations we’ve covered. Understanding data models helps you reason about replication conflicts. Understanding storage engines helps you understand why certain partitioning strategies work better. Understanding encoding helps you think about message formats in distributed protocols.</p><p><strong>The meta-lesson:</strong> these foundations don’t change much. Specific databases come and go. Specific tools gain and lose popularity. But the fundamental trade-offs — read vs write optimization, consistency vs availability, simplicity vs flexibility — remain constant.</p><p>Understanding “why” a system works the way it does makes you adaptable to new technologies. Memorizing “how” a specific tool works makes you obsolete when that tool fades.</p><p>Understanding beats memorization. Principles outlast implementations.</p><p><strong>What trade-off decisions are you currently wrestling with in your systems?</strong></p><p><strong>Have you experienced the pain of a data model choice that made sense initially but became a constraint later?</strong></p><p><strong>I’d love to hear about your experiences with schema evolution — what worked and what caused midnight pages?</strong></p><p><em>This blog is based on the foundational chapters of “Designing Data-Intensive Applications” by Martin Kleppmann — arguably the most important book for anyone building systems that need to scale.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ccb02dbf039b" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/the-architecture-of-data-building-systems-that-last-part-1-ccb02dbf039b">The Architecture of Data: Building Systems That Last (Part 1)</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Data Source Patterns: A Developer’s Guide to Choosing the Right Data Access Strategy]]></title>
            <link>https://medium.com/codetodeploy/mastering-data-source-patterns-a-developers-guide-to-choosing-the-right-data-access-strategy-ded5e444c758?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/ded5e444c758</guid>
            <category><![CDATA[clean-architecture]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[database-design]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[solution-architect]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Wed, 28 Jan 2026 04:39:24 GMT</pubDate>
            <atom:updated>2026-01-28T04:39:24.189Z</atom:updated>
            <content:encoded><![CDATA[<h4>Simple Rules for Smarter Data Layer Decisions</h4><p><a href="https://medium.com/codetodeploy">CodeToDeploy</a></p><h4>Every ORM You’ve Used Implements One of These Four Patterns</h4><p>Every time you write User.find(1) in Rails, User::find(1) in Laravel, or userRepository.findOne(1) in TypeORM, you&#39;re using a Data Source pattern whether you know it or not. These patterns are the invisible architecture behind every database interaction in your application.</p><p>Understanding these patterns is not just academic knowledge. <strong>It’s the difference between choosing the right tool for your project and fighting your framework for years to come.</strong></p><p>In this comprehensive guide, we’ll explore the four fundamental Data Source patterns from Martin Fowler’s legendary book “Patterns of Enterprise Application Architecture” (POEAA). By the end, you’ll be able to identify which pattern your favorite framework uses, understand why certain architectural decisions were made, and most importantly, know when to use each pattern in your own projects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*pKiGj6g0MOPqz58JqdUpjA.png" /></figure><h3>What you will learn from this blog</h3><ul><li>The fundamental problem these patterns solve and why it matters for your application’s longevity.</li><li>How each of the four Data Source patterns works, with practical TypeScript examples you can run.</li><li>Which popular frameworks use which patterns, so you can make informed technology choices.</li><li>When to use each pattern and, equally important, when NOT to use them.</li><li>Common anti-patterns that lead to unmaintainable code and how to avoid them.</li></ul><p><strong>6,500+ </strong>Tech Professional Courses. Upgrade your skills before opportunities slip away.<br><a href="https://www.dpbolvw.net/click-101641947-17135603"> <em>Free 10-day access.</em></a></p><h3>The Core Problem: Two Worlds That Speak Different Languages</h3><p>Before diving into patterns, we need to understand the problem they solve. Consider this analogy:</p><blockquote><em>Imagine your application as a city, and your database as a foreign country. Your objects speak one language (methods, inheritance, polymorphism), while the database speaks another entirely (tables, rows, foreign keys, SQL). Every time your application needs data, it’s like sending a diplomat to negotiate between these two worlds.</em></blockquote><p>This fundamental tension is called <strong>Object-Relational Impedance Mismatch</strong>. Objects are hierarchical, can have behavior, and reference each other through pointers. Relational databases are flat, store only data, and reference each other through foreign keys.</p><p>Here’s a simple example of this mismatch:</p><pre>// In your application, you think in objects<br>class Order {<br>  customer: Customer;        // Direct reference to another object<br>  items: OrderItem[];        // Collection of objects<br>  calculateTotal(): number { // Behavior lives with data<br>    return this.items.reduce((sum, item) =&gt; sum + item.price, 0);<br>  }<br>}<br>// But in the database, you have flat tables<br>// orders: id, customer_id, created_at<br>// order_items: id, order_id, product_id, quantity, price<br>// No methods, no direct references, just foreign keys</pre><p><strong>The Data Source patterns are different strategies for bridging this gap.</strong> Each pattern makes different trade-offs between simplicity, flexibility, and separation of concerns. Understanding these trade-offs is crucial for making the right architectural decisions.</p><p>Let me walk you through each pattern, starting from the simplest and progressing to the most sophisticated.</p><h3>Pattern 1: Table Data Gateway</h3><h3>The Simplest Bridge Between Objects and Tables</h3><p>Table Data Gateway is the most straightforward approach to database access. Think of it as a <strong>single receptionist who handles all requests for one department</strong>. You have one gateway class per database table, and this gateway handles all SQL operations for that table.</p><p>The key characteristic of Table Data Gateway is that it returns raw data (records or rows), not domain objects. It’s purely about data access, nothing more.</p><blockquote><em>“A Table Data Gateway holds all the SQL for accessing a single table or view: selects, inserts, updates, and deletes. Other code calls its methods for all interaction with the database.” — Martin Fowler</em></blockquote><h3>How It Works</h3><p>Let’s build a product catalog system using Table Data Gateway:</p><pre>// Types for the table rows - matching database schema exactly<br>interface ProductRow {<br>  id: string;<br>  name: string;<br>  price: number;<br>  category_id: string;<br>  stock: number;<br>}<br><br>// TABLE DATA GATEWAY - handles ALL database access for products table<br>class ProductGateway {<br>  // Find methods - return raw data<br>  find(id: string): ProductRow | undefined {<br>    // SQL: SELECT * FROM products WHERE id = ?<br>    return database.products.get(id);<br>  }<br>  findAll(): ProductRow[] {<br>    // SQL: SELECT * FROM products<br>    return Array.from(database.products.values());<br>  }<br>  findByCategory(categoryId: string): ProductRow[] {<br>    // SQL: SELECT * FROM products WHERE category_id = ?<br>    return Array.from(database.products.values())<br>      .filter(p =&gt; p.category_id === categoryId);<br>  }<br>  findByPriceRange(minPrice: number, maxPrice: number): ProductRow[] {<br>    // SQL: SELECT * FROM products WHERE price BETWEEN ? AND ?<br>    return Array.from(database.products.values())<br>      .filter(p =&gt; p.price &gt;= minPrice &amp;&amp; p.price &lt;= maxPrice);<br>  }<br>  // Insert - takes individual parameters, not objects<br>  insert(name: string, price: number, categoryId: string, stock: number): string {<br>    // SQL: INSERT INTO products (id, name, price, category_id, stock) VALUES (?, ?, ?, ?, ?)<br>    const id = `p-${Date.now()}`;<br>    database.products.set(id, { id, name, price, category_id: categoryId, stock });<br>    return id;<br>  }<br>  // Update - takes all fields<br>  update(id: string, name: string, price: number, categoryId: string, stock: number): void {<br>    // SQL: UPDATE products SET name = ?, price = ?, category_id = ?, stock = ? WHERE id = ?<br>    database.products.set(id, { id, name, price, category_id: categoryId, stock });<br>  }<br>  // Partial updates for specific fields<br>  updateStock(id: string, newStock: number): void {<br>    // SQL: UPDATE products SET stock = ? WHERE id = ?<br>    const product = database.products.get(id);<br>    if (product) product.stock = newStock;<br>  }<br>  // Delete<br>  delete(id: string): void {<br>    // SQL: DELETE FROM products WHERE id = ?<br>    database.products.delete(id);<br>  }<br>  // Aggregate methods<br>  countByCategory(categoryId: string): number {<br>    // SQL: SELECT COUNT(*) FROM products WHERE category_id = ?<br>    return this.findByCategory(categoryId).length;<br>  }<br>}</pre><p><strong>Usage Example</strong></p><pre>const productGateway = new ProductGateway();<br><br>// Find returns raw data, not objects<br>const laptop = productGateway.find(&quot;p1&quot;);<br>console.log(laptop); // { id: &quot;p1&quot;, name: &quot;Laptop&quot;, price: 999, ... }<br>// You work with plain data<br>const electronics = productGateway.findByCategory(&quot;c1&quot;);<br>electronics.forEach(p =&gt; console.log(`${p.name}: $${p.price}`));<br>// Insert returns the new ID<br>const newId = productGateway.insert(&quot;Keyboard&quot;, 75, &quot;c1&quot;, 100);</pre><h3>When to Use Table Data Gateway</h3><p><strong>Best suited for:</strong></p><ul><li>Simple CRUD applications with minimal business logic</li><li>Scripts and batch processing jobs</li><li>Reporting and data export tools</li><li>Quick prototypes and MVPs</li><li>Applications where performance is critical (avoiding object overhead)</li></ul><p><strong>Real-world implementations:</strong></p><ul><li><strong>Knex.js</strong> query builder in Node.js follows this pattern</li><li>Raw <strong>JDBC</strong> usage in Java</li><li>Direct <strong>PDO</strong> usage in PHP</li><li><strong>Database/sql</strong> package in Go</li></ul><h3>The Anti-Pattern to Avoid</h3><p><strong>Never add business logic to a Table Data Gateway.</strong> The moment you start writing methods like calculateTotalRevenue() or applyDiscount() in your gateway, you&#39;re mixing concerns. The gateway should only know about SQL, not about your business rules.</p><pre>// BAD - Business logic in gateway<br>class ProductGateway {<br>  applyBulkDiscount(categoryId: string, percent: number): void {<br>    const products = this.findByCategory(categoryId);<br>    products.forEach(p =&gt; {<br>      const newPrice = p.price * (1 - percent / 100);<br>      this.updatePrice(p.id, newPrice);<br>    });<br>  }<br>}<br><br>// GOOD - Keep gateway pure, business logic elsewhere<br>class DiscountService {<br>  constructor(private gateway: ProductGateway) {}<br>  applyBulkDiscount(categoryId: string, percent: number): void {<br>    const products = this.gateway.findByCategory(categoryId);<br>    products.forEach(p =&gt; {<br>      const newPrice = p.price * (1 - percent / 100);<br>      this.gateway.updatePrice(p.id, newPrice);<br>    });<br>  }<br>}</pre><h3>Pattern 2: Row Data Gateway</h3><h3>One Object Per Database Row</h3><p>Row Data Gateway takes a different approach. Instead of one gateway handling all rows, <strong>you have one gateway instance per row in the database</strong>. Each gateway object corresponds to exactly one record.</p><p>The crucial distinction from Active Record (which we’ll cover next) is that Row Data Gateway contains <strong>only data access logic, no domain logic</strong>. Domain logic stays in separate domain objects that use the gateway.</p><p>Think of it as a <strong>personal assistant assigned to each record</strong>. The assistant knows how to fetch and update that specific record, but doesn’t make business decisions.</p><h3>How It Works</h3><p>Let’s build an order management system:</p><pre>// ROW DATA GATEWAY - one instance per row, handles only data access<br>class OrderGateway {<br>  // Data fields matching database columns<br>  public id: string = &quot;&quot;;<br>  public customerId: string = &quot;&quot;;<br>  public total: number = 0;<br>  public status: string = &quot;&quot;;<br>  public createdAt: string = &quot;&quot;;<br><br>  // Static finder that returns gateway instances<br>  static find(id: string): OrderGateway | null {<br>    const row = database.orders.get(id);<br>    if (!row) return null;<br>    const gateway = new OrderGateway();<br>    gateway.id = row.id;<br>    gateway.customerId = row.customer_id;<br>    gateway.total = row.total;<br>    gateway.status = row.status;<br>    gateway.createdAt = row.created_at;<br>    return gateway;<br>  }<br>  static findByCustomer(customerId: string): OrderGateway[] {<br>    const gateways: OrderGateway[] = [];<br>    for (const row of database.orders.values()) {<br>      if (row.customer_id === customerId) {<br>        const gateway = new OrderGateway();<br>        gateway.id = row.id;<br>        gateway.customerId = row.customer_id;<br>        gateway.total = row.total;<br>        gateway.status = row.status;<br>        gateway.createdAt = row.created_at;<br>        gateways.push(gateway);<br>      }<br>    }<br>    return gateways;<br>  }<br>  // Instance methods for persistence<br>  insert(): void {<br>    if (!this.id) this.id = `o-${Date.now()}`;<br>    database.orders.set(this.id, {<br>      id: this.id,<br>      customer_id: this.customerId,<br>      total: this.total,<br>      status: this.status,<br>      created_at: this.createdAt,<br>    });<br>  }<br>  update(): void {<br>    database.orders.set(this.id, {<br>      id: this.id,<br>      customer_id: this.customerId,<br>      total: this.total,<br>      status: this.status,<br>      created_at: this.createdAt,<br>    });<br>  }<br>  delete(): void {<br>    database.orders.delete(this.id);<br>  }<br>}</pre><h3>Separating Domain Logic</h3><p>Here’s where Row Data Gateway shines. <strong>Your domain object uses the gateway but doesn’t know about the database:</strong></p><pre>// Domain object - contains business logic, no database knowledge<br>class Order {<br>  private gateway: OrderGateway;<br><br>  constructor(gateway: OrderGateway) {<br>    this.gateway = gateway;<br>  }<br>  // Domain getters<br>  get id(): string { return this.gateway.id; }<br>  get total(): number { return this.gateway.total; }<br>  get status(): string { return this.gateway.status; }<br>  // Business logic - pure domain rules<br>  canBeCancelled(): boolean {<br>    return this.gateway.status === &quot;pending&quot;;<br>  }<br>  cancel(): void {<br>    if (!this.canBeCancelled()) {<br>      throw new Error(&quot;Only pending orders can be cancelled&quot;);<br>    }<br>    this.gateway.status = &quot;cancelled&quot;;<br>    this.gateway.update();<br>  }<br>  ship(): void {<br>    if (this.gateway.status !== &quot;pending&quot;) {<br>      throw new Error(&quot;Only pending orders can be shipped&quot;);<br>    }<br>    this.gateway.status = &quot;shipped&quot;;<br>    this.gateway.update();<br>  }<br>  applyDiscount(percent: number): void {<br>    const discount = this.gateway.total * (percent / 100);<br>    this.gateway.total = this.gateway.total - discount;<br>    this.gateway.update();<br>  }<br>}</pre><p><strong>Usage Example</strong></p><pre>// Find using gateway<br>const orderGateway = OrderGateway.find(&quot;o1&quot;);<br>if (orderGateway) {<br>  // Wrap in domain object for business logic<br>  const order = new Order(orderGateway);<br>  console.log(`Can be cancelled: ${order.canBeCancelled()}`);<br>  order.applyDiscount(10);<br>  console.log(`After 10% discount: $${order.total}`);<br>}<br>// Create new order<br>const newGateway = new OrderGateway();<br>newGateway.customerId = &quot;c3&quot;;<br>newGateway.total = 500;<br>newGateway.status = &quot;pending&quot;;<br>newGateway.createdAt = new Date().toISOString().split(&quot;T&quot;)[0];<br>newGateway.insert();</pre><h3>When to Use Row Data Gateway</h3><p><strong>Best suited for:</strong></p><ul><li>When you want clear separation between data access and business logic</li><li>Legacy systems where you’re gradually introducing domain objects</li><li>When different developers work on persistence vs. business logic</li><li>Systems that need to be tested without a database</li></ul><p><strong>Real-world note:</strong> Row Data Gateway is less common as a standalone pattern today. It’s often seen as a stepping stone toward Data Mapper or as an internal implementation detail within larger patterns.</p><h3>The Anti-Pattern to Avoid</h3><p><strong>Don’t mix domain logic into the gateway.</strong> If you find yourself adding methods like canBeCancelled() directly in OrderGateway, you&#39;re drifting toward Active Record. Keep the gateway focused solely on CRUD operations.</p><h3>Pattern 3: Active Record</h3><h3>Objects That Know How to Save Themselves</h3><p>Active Record is perhaps the most widely used pattern in web development. It combines data and database operations into a single class. <strong>The object knows how to persist itself</strong> — it’s both the data and the data access layer.</p><p>This is the pattern you’re using when you write user.save() in Rails or $user-&gt;save() in Laravel.</p><blockquote><em>“An object that wraps a row in a database table or view, encapsulates the database access, and adds domain logic on that data.” — Martin Fowler</em></blockquote><p>The key insight is that Active Record trades architectural purity for developer productivity. It’s convenient, intuitive, and fast to develop with.</p><h3>How It Works</h3><p>Let’s build a user management system:</p><pre>// ACTIVE RECORD - combines data and database operations<br>class User {<br>  public id: string = &quot;&quot;;<br>  public name: string = &quot;&quot;;<br>  public email: string = &quot;&quot;;<br>  public createdAt: Date = new Date();<br><br>  // Static finder methods - class-level queries<br>  static find(id: string): User | null {<br>    const row = database.users.get(id);<br>    if (!row) return null;<br>    return User.fromRow(row);<br>  }<br>  static findByEmail(email: string): User | null {<br>    for (const row of database.users.values()) {<br>      if (row.email === email) {<br>        return User.fromRow(row);<br>      }<br>    }<br>    return null;<br>  }<br>  static all(): User[] {<br>    return Array.from(database.users.values()).map(row =&gt; User.fromRow(row));<br>  }<br>  static create(attributes: { name: string; email: string }): User {<br>    const user = new User();<br>    user.id = `user-${Date.now()}`;<br>    user.name = attributes.name;<br>    user.email = attributes.email;<br>    user.createdAt = new Date();<br>    user.save();<br>    return user;<br>  }<br>  // Private helper for row mapping<br>  private static fromRow(row: { id: string; name: string; email: string; created_at: string }): User {<br>    const user = new User();<br>    user.id = row.id;<br>    user.name = row.name;<br>    user.email = row.email;<br>    user.createdAt = new Date(row.created_at);<br>    return user;<br>  }<br>  // Instance method to save - the signature Active Record move<br>  save(): void {<br>    if (!this.id) {<br>      this.id = `user-${Date.now()}`;<br>    }<br>    database.users.set(this.id, {<br>      id: this.id,<br>      name: this.name,<br>      email: this.email,<br>      created_at: this.createdAt.toISOString(),<br>    });<br>  }<br>  delete(): void {<br>    database.users.delete(this.id);<br>  }<br>  // Domain logic methods - mixed with data access<br>  updateEmail(newEmail: string): void {<br>    if (!newEmail.includes(&quot;@&quot;)) {<br>      throw new Error(&quot;Invalid email format&quot;);<br>    }<br>    this.email = newEmail;<br>    this.save();  // Automatic persistence!<br>  }<br>  isNewUser(): boolean {<br>    const oneWeekAgo = new Date();<br>    oneWeekAgo.setDate(oneWeekAgo.getDate() - 7);<br>    return this.createdAt &gt; oneWeekAgo;<br>  }<br>  getDisplayName(): string {<br>    return `${this.name} &lt;${this.email}&gt;`;<br>  }<br>}</pre><p><strong>Usage Example — Clean and Intuitive</strong></p><pre>// Create users - feels natural<br>const alice = User.create({ name: &quot;Alice&quot;, email: &quot;alice@example.com&quot; });<br>const bob = User.create({ name: &quot;Bob&quot;, email: &quot;bob@example.com&quot; });<br><br>// Find and use<br>const foundUser = User.find(alice.id);<br>console.log(foundUser?.getDisplayName()); // &quot;Alice &lt;alice@example.com&gt;&quot;<br>// Query by attributes<br>const byEmail = User.findByEmail(&quot;bob@example.com&quot;);<br>console.log(byEmail?.name); // &quot;Bob&quot;<br>// Update - automatic save!<br>alice.updateEmail(&quot;alice.new@example.com&quot;);<br>// List all<br>User.all().forEach(user =&gt; console.log(user.getDisplayName()));<br>// Domain logic<br>console.log(`Is Alice new? ${alice.isNewUser()}`);</pre><h3>When to Use Active Record</h3><p><strong>Best suited for:</strong></p><ul><li>CRUD-heavy applications with simple domain logic</li><li>Rapid prototyping and MVPs</li><li>Applications where database schema closely matches domain model</li><li>Small to medium-sized applications</li><li>When developer productivity is prioritized over architectural purity</li></ul><p><strong>Real-world implementations:</strong></p><ul><li><strong>Ruby on Rails</strong> — The pattern that made Active Record famous</li><li><strong>Laravel Eloquent</strong> — PHP’s most popular ORM</li><li><strong>Django ORM</strong> — Python’s default choice</li><li><strong>Sequelize</strong> — Node.js Active Record implementation</li></ul><h3>The Anti-Pattern to Avoid</h3><p><strong>Beware of “Fat Models.”</strong> As your application grows, Active Record classes tend to accumulate more and more methods. A User class might start with save() and find(), but soon it has sendWelcomeEmail(), calculateLifetimeValue(), syncWithCRM(), and dozens more.</p><pre>// BAD - Fat model anti-pattern<br>class User {<br>  // ... 50 finder methods<br>  static findByEmail() {}<br>  static findByName() {}<br>  static findActiveInLastMonth() {}<br>  static findWithSubscription() {}<br>  // ...<br><br>  // ... 30 business methods<br>  sendWelcomeEmail() {}<br>  calculateLifetimeValue() {}<br>  syncWithCRM() {}<br>  generatePasswordResetToken() {}<br>  validateTwoFactorAuth() {}<br>  // ...<br>  // This class is now 500+ lines and touches email, billing, auth, CRM...<br>}</pre><p><strong>When your Active Record class exceeds 200 lines or touches multiple concerns, it’s time to consider Data Mapper or extract service objects.</strong></p><p>Another subtle issue: <strong>Active Record couples your domain model to your database schema.</strong> If you rename a column, you might need to update code across your application. This coupling becomes painful in long-lived systems.</p><h3>Pattern 4: Data Mapper</h3><h3>Complete Separation of Concerns</h3><p>Data Mapper takes the opposite approach from Active Record. <strong>Domain objects have absolutely no knowledge of the database.</strong> A separate mapper layer handles all persistence logic.</p><p>This is the pattern used by enterprise ORMs like Hibernate, Entity Framework, and Doctrine. It requires more code but provides maximum flexibility.</p><blockquote><em>“A layer of Mappers that moves data between objects and a database while keeping them independent of each other and the mapper itself.” — Martin Fowler</em></blockquote><p>Think of Data Mapper as having a <strong>professional translator</strong> between your domain objects and the database. The domain objects speak only in business terms; they don’t even know a database exists.</p><h3>How It Works</h3><p>Let’s build a user system with proper separation:</p><pre>// DOMAIN OBJECT - completely ignorant of persistence<br>class User {<br>  constructor(<br>    public id: string,<br>    public name: string,<br>    public email: string,<br>    private passwordHash: string<br>  ) {}<br><br>// Pure domain logic - no database code whatsoever<br>  updateName(newName: string): void {<br>    if (newName.length &lt; 2) {<br>      throw new Error(&quot;Name must be at least 2 characters&quot;);<br>    }<br>    this.name = newName;<br>  }<br>  changeEmail(newEmail: string): void {<br>    if (!newEmail.includes(&quot;@&quot;)) {<br>      throw new Error(&quot;Invalid email format&quot;);<br>    }<br>    this.email = newEmail;<br>  }<br>  verifyPassword(password: string): boolean {<br>    // In real app, would use bcrypt comparison<br>    return this.passwordHash === `hash_${password}`;<br>  }<br>  getPasswordHash(): string {<br>    return this.passwordHash;<br>  }<br>}</pre><p>Notice something important: <strong>the User class has no </strong><strong>save() method, no static </strong><strong>find() methods, no database logic at all.</strong> It&#39;s a pure domain object that only knows about business rules.</p><p>Now, the mapper:</p><pre>// DATA MAPPER - handles all persistence<br>class UserMapper {<br><br>  // Map from domain object to database row<br>  private toRow(user: User): { id: string; name: string; email: string; password_hash: string } {<br>    return {<br>      id: user.id,<br>      name: user.name,<br>      email: user.email,<br>      password_hash: user.getPasswordHash(),<br>    };<br>  }<br>  // Map from database row to domain object<br>  private toDomain(row: { id: string; name: string; email: string; password_hash: string }): User {<br>    return new User(row.id, row.name, row.email, row.password_hash);<br>  }<br>  // Finder methods<br>  find(id: string): User | null {<br>    const row = database.users.get(id);<br>    if (!row) return null;<br>    return this.toDomain(row);<br>  }<br>  findByEmail(email: string): User | null {<br>    for (const row of database.users.values()) {<br>      if (row.email === email) {<br>        return this.toDomain(row);<br>      }<br>    }<br>    return null;<br>  }<br>  findAll(): User[] {<br>    return Array.from(database.users.values()).map(row =&gt; this.toDomain(row));<br>  }<br>  // Persistence methods - explicit, not automatic<br>  insert(user: User): void {<br>    const row = this.toRow(user);<br>    database.users.set(user.id, row);<br>  }<br>  update(user: User): void {<br>    if (!database.users.has(user.id)) {<br>      throw new Error(`User ${user.id} not found`);<br>    }<br>    const row = this.toRow(user);<br>    database.users.set(user.id, row);<br>  }<br>  delete(user: User): void {<br>    database.users.delete(user.id);<br>  }<br>}<br>// Factory for creating new users (optional but recommended)<br>class UserFactory {<br>  static create(name: string, email: string, password: string): User {<br>    const id = `user-${Date.now()}`;<br>    const passwordHash = `hash_${password}`; // In real app, use bcrypt<br>    return new User(id, name, email, passwordHash);<br>  }<br>}</pre><p><strong>Usage Example, Explicit but Clear</strong></p><pre>const userMapper = new UserMapper();<br><br>// Create users - domain objects don&#39;t know about database<br>const alice = UserFactory.create(&quot;Alice&quot;, &quot;alice@example.com&quot;, &quot;secret123&quot;);<br>const bob = UserFactory.create(&quot;Bob&quot;, &quot;bob@example.com&quot;, &quot;password456&quot;);<br>// Mapper handles persistence - explicit save<br>userMapper.insert(alice);<br>userMapper.insert(bob);<br>// Find and modify<br>const foundUser = userMapper.findByEmail(&quot;alice@example.com&quot;);<br>if (foundUser) {<br>  foundUser.updateName(&quot;Alice Smith&quot;);<br>  foundUser.changeEmail(&quot;alice.smith@example.com&quot;);<br>  userMapper.update(foundUser); // Must explicitly save!<br>}<br>// Domain logic works without database<br>console.log(`Password correct: ${alice.verifyPassword(&quot;secret123&quot;)}`);<br>// List all<br>userMapper.findAll().forEach(user =&gt; {<br>  console.log(`${user.name} (${user.email})`);<br>});</pre><h3>When to Use Data Mapper</h3><p><strong>Best suited for:</strong></p><ul><li>Complex domain logic (Domain-Driven Design)</li><li>Long-lived enterprise applications</li><li>When domain model differs significantly from database schema</li><li>Applications requiring high testability</li><li>Teams with separate database and domain specialists</li><li>Microservices with complex business rules</li></ul><p><strong>Real-world implementations:</strong></p><ul><li><strong>Hibernate</strong> (Java) — The gold standard for Data Mapper</li><li><strong>Entity Framework</strong> (C#) — Microsoft’s enterprise ORM</li><li><strong>Doctrine</strong> (PHP) — The enterprise choice for PHP</li><li><strong>TypeORM</strong> (Node.js) — Repository pattern follows Data Mapper principles</li><li><strong>SQLAlchemy</strong> (Python) — When used with explicit sessions</li></ul><h3>The Anti-Pattern to Avoid</h3><p><strong>Anemic Domain Model.</strong> The danger with Data Mapper is going too far in the other direction. If your domain objects become just data containers with getters and setters, and all logic lives in services, you’ve created an “anemic domain model.”</p><pre>// BAD - Anemic domain model<br>class User {<br>  public id: string = &quot;&quot;;<br>  public name: string = &quot;&quot;;<br>  public email: string = &quot;&quot;;<br>  // Just getters/setters, no behavior<br>}<br><br>class UserService {<br>  // ALL logic lives here - User is just a data bag<br>  updateName(user: User, newName: string) { ... }<br>  changeEmail(user: User, newEmail: string) { ... }<br>  verifyPassword(user: User, password: string) { ... }<br>  // This is procedural programming disguised as OOP<br>}</pre><p><strong>The domain object should still contain business logic.</strong> The mapper handles persistence; the domain object handles business rules.</p><h3>The Evolution Path: From Simple to Sophisticated</h3><p>Just as we evolved through levels in creational patterns, there’s a natural progression in data source patterns. Your application’s needs should guide which pattern you choose.</p><h3>Level 0: Raw SQL</h3><p>You start with raw SQL queries scattered throughout your code. Quick to write, but maintenance nightmare.</p><pre>// Level 0 - SQL everywhere<br>function getUser(id: string) {<br>  return db.query(&quot;SELECT * FROM users WHERE id = ?&quot;, [id]);<br>}</pre><h3>Level 1: Table Data Gateway</h3><p>You centralize all SQL for each table into gateway classes. Queries are organized, but you’re still working with raw data.</p><pre>// Level 1 - Organized queries<br>const user = userGateway.find(id);<br>console.log(user.name); // Still raw data</pre><h3>Level 2: Active Record</h3><p>You want objects with behavior. Active Record gives you that with minimal ceremony.</p><pre>// Level 2 - Objects with behavior<br>const user = User.find(id);<br>user.updateEmail(&quot;new@email.com&quot;); // Behavior + automatic save</pre><h3>Level 3: Data Mapper</h3><p>Your domain becomes complex. Business rules don’t map cleanly to tables. You need the flexibility of Data Mapper.</p><pre>// Level 3 - Full separation<br>const user = userMapper.find(id);<br>user.applyBusinessRule();<br>userMapper.update(user); // Explicit persistence<br><br></pre><h4><strong>How to Decide: The Decision Tree</strong></h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DgS0_lGqEo5uTOaYLIR5wA.png" /><figcaption><strong>How to Decide: The Decision Tree</strong></figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7Jf4UYHWGe_xxRDha42rtA.png" /><figcaption>Pattern Comparison: Side by Side</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KshVG2VrdgqlGs_o-yTsZA.png" /><figcaption>Framework Deep Dive: What Are You Actually Using?</figcaption></figure><h3>Common Mistakes and Anti-Patterns</h3><h3>1. Using Active Record for Complex Domains</h3><p><strong>The Symptom:</strong> Your model classes have 50+ methods, touch multiple database tables, and contain business logic that spans multiple concepts.</p><p><strong>The Problem:</strong> Active Record couples your domain to your database. When business rules become complex, this coupling creates a tangled mess.</p><p><strong>The Fix:</strong> Graduate to Data Mapper. Extract domain objects that focus on business rules, and use mappers for persistence.</p><h3>2. The “God Gateway” Anti-Pattern</h3><p><strong>The Symptom:</strong> Your Table Data Gateway has methods for every possible query combination, growing to hundreds of methods.</p><pre>// Anti-pattern: 100+ methods in one gateway<br>class UserGateway {<br>  findByEmail() {}<br>  findByEmailAndStatus() {}<br>  findByEmailAndStatusAndCreatedAfter() {}<br>  findByNameStartingWith() {}<br>  findByNameStartingWithAndEmail() {}<br>  // ... 95 more methods<br>}</pre><p><strong>The Fix:</strong> Use a query builder pattern or specification pattern for complex queries.</p><h3>3. Anemic Domain Models with Data Mapper</h3><p><strong>The Symptom:</strong> Your domain objects are just data containers. All logic lives in services.</p><p><strong>The Problem:</strong> You’ve essentially created a procedural system disguised as OOP. The benefits of Data Mapper are wasted.</p><p><strong>The Fix:</strong> Put behavior where the data is. Domain objects should enforce business invariants.</p><h3>4. Repository Pattern Confusion</h3><p><strong>The Symptom:</strong> You have classes called “repositories” that are really just Active Record wrappers.</p><p><strong>The Problem:</strong> Repository pattern (from DDD) implies Data Mapper underneath. If your repository just delegates to Active Record, you’re adding a layer without benefit.</p><p><strong>The Fix:</strong> Either embrace Active Record directly or implement proper Data Mapper with repositories.</p><h3>5. Over-Engineering Simple CRUD</h3><p><strong>The Symptom:</strong> You’ve built a full Data Mapper infrastructure with Unit of Work, Identity Map, and Lazy Loading for a simple blog application.</p><p><strong>The Problem:</strong> You’ve traded development velocity for architectural purity you don’t need.</p><p><strong>The Fix:</strong> Start simple. Use Active Record for CRUD apps. Evolve to Data Mapper when complexity demands it.</p><h3>Summary: Choosing Your Pattern</h3><p>Let’s distill this into actionable guidance.</p><p><strong>Choose Table Data Gateway when:</strong></p><ul><li>You’re writing scripts, batch jobs, or reports</li><li>Performance is critical and object overhead matters</li><li>You’re comfortable working with raw data structures</li></ul><p><strong>Choose Row Data Gateway when:</strong></p><ul><li>You’re migrating from raw SQL to a more object-oriented approach</li><li>You want object representation but strict separation from domain logic</li><li>You’re building a custom ORM or data layer</li></ul><p><strong>Choose Active Record when:</strong></p><ul><li>Your domain is primarily CRUD</li><li>You want rapid development and high productivity</li><li>Your database schema closely matches your domain model</li><li>You’re building an MVP or prototype</li><li>Your team prefers convention over configuration</li></ul><p><strong>Choose Data Mapper when:</strong></p><ul><li>Your domain logic is complex (good fit for DDD)</li><li>Your domain model differs significantly from your database schema</li><li>You need to test domain logic without a database</li><li>You’re building a long-lived enterprise application</li><li>You have separate teams for domain and persistence concerns</li></ul><p><strong>The key insight is this:</strong> These patterns are not about “good” versus “bad.” They’re about choosing the right level of abstraction for your specific problem. Active Record isn’t inferior to Data Mapper; it’s optimized for different use cases.</p><blockquote><em>“The best architects understand that form follows function. The pattern you choose should serve your application’s needs, not the other way around.” — Martin Fowler</em></blockquote><p><strong>Remember:</strong> You can start with Active Record and evolve to Data Mapper as complexity grows. Many successful applications have done exactly this. The important thing is recognizing when it’s time to graduate.</p><h3>Thank you for being a part of the community</h3><p><em>Before you go:</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*d9QTaaaxboQP_gKSLedW_w.png" /></figure><p>👉 Be sure to <strong>clap</strong> and <strong>follow</strong> the writer ️👏<strong>️️</strong></p><p>👉 Follow us: <a href="https://www.linkedin.com/in/code-to-deploy-3784391b9/"><strong>Linkedin</strong></a>| <a href="https://medium.com/codetodeploy"><strong>Medium</strong></a></p><p>👉 CodeToDeploy Tech Community is live on Discord — <a href="https://discord.gg/ZpwhHq6D"><strong>Join now!</strong></a></p><p><strong>Note:</strong> This Post may contain affiliate links.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ded5e444c758" width="1" height="1" alt=""><hr><p><a href="https://medium.com/codetodeploy/mastering-data-source-patterns-a-developers-guide-to-choosing-the-right-data-access-strategy-ded5e444c758">Mastering Data Source Patterns: A Developer’s Guide to Choosing the Right Data Access Strategy</a> was originally published in <a href="https://medium.com/codetodeploy">CodeToDeploy</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Software Design: With the Power of Domain-Driven Design & Clean Architecture — Chapter 3]]></title>
            <link>https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-3-02f84f3dd7c6?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/02f84f3dd7c6</guid>
            <category><![CDATA[stock-trading]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[software-design]]></category>
            <category><![CDATA[clean-architecture]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Mon, 26 Jan 2026 04:57:52 GMT</pubDate>
            <atom:updated>2026-01-26T16:04:47.583Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Mastering Software Design: With the Power of Domain-Driven Design &amp; Clean Architecture — Chapter 3</strong></h3><h4><em>Building Scalable, Maintainable, and Future-Proof Systems with DDD and Clean Architectur</em></h4><p><strong>TL;DR:</strong> This is the final chapter of our Domain-Driven Design journey. In Chapter 1, we learned the foundations: Entities, Value Objects, Services, and Modules. In Chapter 2, we explored structural patterns: Aggregates, Factories, and Repositories. Now, we complete the picture with two powerful pattern families. <strong>Supple Design</strong> patterns (Intention-Revealing Interfaces, Side-Effect-Free Functions, Assertions, Standalone Classes, and Specifications) make your code flexible and expressive. <strong>Strategic Design</strong> patterns (Bounded Contexts, Context Maps, Shared Kernels, Customer/Supplier, Conformist, Anti-Corruption Layers, Open Host Services, and Published Language) help you manage complexity in large systems with multiple teams. By mastering these 13 patterns, you’ll have the complete toolkit to build enterprise-grade software that stands the test of time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WR_cEtsn86LJ94U9amGFzw.png" /></figure><p>We’ve come a long way in this series. Chapter 1 gave us the vocabulary — the nouns and verbs of our domain model. Chapter 2 taught us how to structure these elements into coherent units with clear lifecycles. But there’s still more to the story.</p><p>As systems grow, two challenges emerge. First, individual models become rigid and difficult to change — what Eric Evans calls “code rot.” Second, different parts of the organization develop conflicting interpretations of the same concepts. A single monolithic model breaks down.</p><p>This chapter addresses both challenges. <strong>Supple Design</strong> keeps individual models flexible and expressive. <strong>Strategic Design</strong> provides the organizational patterns for managing complexity across an entire enterprise.</p><p>Let’s begin with making our models supple.</p><h3><strong>Part 1: Supple Design Patterns</strong></h3><blockquote>“A lot of overengineering has been justified in the name of flexibility. But more often than not, excessive layers of abstraction and indirection get in the way. Look at the design of software that really empowers the people who handle it; you will usually see something simple. Simple is not easy.” — Eric Evans</blockquote><p>Supple Design is about creating models that are a joy to work with, code that bends without breaking, that reveals its intent, and that invites confident refactoring. It’s the difference between code that fights you and code that flows.</p><p>Over time, even well-designed systems become rigid. Quick fixes pile up. Developers fear changing anything because the implications are unclear. Supple Design patterns combat this entropy by making code self-documenting, predictable, and composed of independent, testable pieces.</p><p>Let’s explore the five patterns that achieve this.</p><h3><strong>Intention-Revealing Interface</strong></h3><p>The first and most fundamental Supple Design pattern is <strong>Intention-Revealing Interface</strong>. The principle is simple: names should describe <strong>what</strong> something does, not <strong>how</strong> it does it.</p><p>When you read a method name, you should immediately understand its purpose without needing to read the implementation. When you see a class name, you should know what concept it represents.</p><pre>// BAD: Names reveal implementation, not intent<br>class DataProcessor {<br>  process(data) { }<br>  calc(items) { }<br>  doWork(input) { }<br>}<br><br>class Handler {<br>  handle(request) { }<br>  execute(command) { }<br>}</pre><pre>// GOOD: Names reveal domain intent<br>class TradeOrderBook {<br>  submitMarketOrder(traderId, stockSymbol, quantity) { }<br>  submitLimitOrder(traderId, stockSymbol, quantity, limitPrice) { }<br>  cancelOrder(orderId) { }<br>  getActiveOrders(traderId) { }<br>}<br><br>class PortfolioValuator {<br>  calculateCurrentValue(portfolio) { }<br>  calculateUnrealizedGains(portfolio) { }<br>  calculateDiversificationScore(portfolio) { }<br>}</pre><p>The good examples use domain language. A trader understands “submitMarketOrder” and “cancelOrder.” These names make the code self-documenting.</p><h4><strong>Benefits</strong></h4><ul><li><strong>Reduces cognitive load: </strong>Developers understand usage without reading implementation.</li><li><strong>Enables domain conversations: </strong>Developers and domain experts can discuss code using the same language.</li><li><strong>Catches design flaws: </strong>If you can’t name something clearly, perhaps the design is confused.</li></ul><p>This pattern seems obvious, yet it’s frequently violated. Take the time to choose names that speak your domain’s language.</p><h3><strong>Side-Effect-Free Functions</strong></h3><p>The second pattern separates <strong>queries </strong>(return information) from <strong>commands</strong> (change state). Functions that only compute and return results, without modifying anything — are called <strong>Side-Effect-Free Functions</strong>.</p><blockquote>“Operations that return results without producing side effects are called functions. A function can be called multiple times and return the same value each time.” — Eric Evans</blockquote><h4><strong>Pure Functions in Practice</strong></h4><p>Value Objects naturally embody this pattern. All their operations return new instances rather than modifying the original:</p><pre>class Money {<br>  constructor(amount, currency = &#39;USD&#39;) {<br>    this.amount = amount;<br>    this.currency = currency;<br>    Object.freeze(this); // Immutable<br>  }<br><br>  // All operations return NEW instances - no side effects<br>  add(other) {<br>    this.assertSameCurrency(other);<br>    return new Money(this.amount + other.amount, this.currency);<br>  }<br><br>  subtract(other) {<br>    this.assertSameCurrency(other);<br>    return new Money(this.amount - other.amount, this.currency);<br>  }<br><br>  multiply(factor) {<br>    return new Money(this.amount * factor, this.currency);<br>  }<br><br>  percentage(percent) {<br>    return new Money(this.amount * (percent / 100), this.currency);<br>  }<br><br>  // Query methods - pure functions<br>  isGreaterThan(other) {<br>    return this.amount &gt; other.amount;<br>  }<br><br>  equals(other) {<br>    return this.amount === other.amount &amp;&amp; this.currency === other.currency;<br>  }<br><br>  assertSameCurrency(other) {<br>    if (this.currency !== other.currency) {<br>      throw new Error(`Currency mismatch: ${this.currency} vs ${other.currency}`);<br>    }<br>  }<br>}</pre><p>You can also create pure calculation services:</p><pre>class TradingFeeCalculator {<br>  // Pure function - same inputs always produce same output<br>  calculateCommission(tradeValue, traderTier) {<br>    const rates = {<br>      &#39;retail&#39;: 0.01,      // 1%<br>      &#39;premium&#39;: 0.005,    // 0.5%<br>      &#39;institutional&#39;: 0.001 // 0.1%<br>    };<br>    const rate = rates[traderTier] || rates[&#39;retail&#39;];<br>    return tradeValue.multiply(rate);<br>  }<br><br>  calculateTotalCost(tradeValue, commission, taxes) {<br>    return tradeValue.add(commission).add(taxes);<br>  }<br><br>  // Complex calculation built from simple pure functions<br>  calculateTradeBreakdown(stockPrice, quantity, traderTier, taxRate) {<br>    const tradeValue = stockPrice.multiply(quantity);<br>    const commission = this.calculateCommission(tradeValue, traderTier);<br>    const taxes = tradeValue.multiply(taxRate);<br>    const total = this.calculateTotalCost(tradeValue, commission, taxes);<br><br>    return {<br>      tradeValue,<br>      commission,<br>      taxes,<br>      total<br>    };<br>  }<br>}</pre><h4><strong>Benefits</strong></h4><ul><li><strong>Easy to test: </strong>No setup or teardown needed. Given inputs, verify outputs.</li><li><strong>Safe to parallelize: </strong>No shared state to corrupt.</li><li><strong>Cacheable: </strong>Same inputs always produce same outputs (memoization).</li><li><strong>Easy to reason about: </strong>No hidden dependencies or surprises.</li></ul><p>Separate your commands (which change state) from your queries (which compute). Make as many operations as possible into pure functions.</p><h3><strong>Assertions</strong></h3><p>The third pattern makes implicit rules explicit. <strong>Assertions </strong>encode invariants, pre-conditions, and post-conditions directly in your code.</p><blockquote>“State post-conditions of operations and invariants of classes and AGGREGATES. If ASSERTIONS cannot be coded directly in your programming language, write automated unit tests for them.” — Eric Evans</blockquote><h4><strong>Types of Assertions</strong></h4><ul><li><strong>Invariants: </strong>Conditions that must always be true.</li><li><strong>Pre-conditions: </strong>What must be true before an operation.</li><li><strong>Post-conditions: </strong>What will be true after an operation.</li></ul><h4><strong>Implementing Assertions</strong></h4><pre>class Assert {<br>  static notNull(value, message) {<br>    if (value === null || value === undefined) {<br>      throw new Error(message);<br>    }<br>  }<br><br>  static positive(value, message) {<br>    if (typeof value !== &#39;number&#39; || value &lt;= 0) {<br>      throw new Error(message);<br>    }<br>  }<br><br>  static state(condition, message) {<br>    if (!condition) {<br>      throw new Error(`Invalid state: ${message}`);<br>    }<br>  }<br><br>  static that(condition, message) {<br>    if (!condition) {<br>      throw new Error(`Assertion failed: ${message}`);<br>    }<br>  }<br>}</pre><p>Now apply them to a TradeOrder:</p><pre>class TradeOrder {<br>  constructor(orderId, traderId) {<br>    Assert.notNull(orderId, &#39;Order ID is required&#39;);<br>    Assert.notNull(traderId, &#39;Trader ID is required&#39;);<br><br>    this._orderId = orderId;<br>    this._traderId = traderId;<br>    this._positions = [];<br>    this._status = &#39;draft&#39;;<br>  }<br><br>  addPosition(stockSymbol, quantity, pricePerShare) {<br>    // PRE-CONDITIONS<br>    this.assertModifiable();<br>    Assert.notNull(stockSymbol, &#39;Stock symbol is required&#39;);<br>    Assert.positive(quantity, &#39;Quantity must be positive&#39;);<br>    Assert.positive(pricePerShare.amount, &#39;Price must be positive&#39;);<br><br>    const positionsBefore = this._positions.length;<br><br>    this._positions.push({<br>      stockSymbol,<br>      quantity,<br>      pricePerShare<br>    });<br><br>    // POST-CONDITION<br>    Assert.that(<br>      this._positions.length === positionsBefore + 1,<br>      &#39;Position should have been added&#39;<br>    );<br>  }<br><br>  execute() {<br>    // PRE-CONDITIONS<br>    this.assertModifiable();<br>    Assert.that(this._positions.length &gt; 0, &#39;Cannot execute empty order&#39;);<br><br>    this._status = &#39;executed&#39;;<br>    this._executedAt = new Date();<br><br>    // POST-CONDITIONS<br>    Assert.state(this._status === &#39;executed&#39;, &#39;Order should be executed&#39;);<br>    Assert.notNull(this._executedAt, &#39;Execution time should be set&#39;);<br><br>    // INVARIANT check<br>    this.assertInvariants();<br>  }<br><br>  assertModifiable() {<br>    Assert.state(<br>      this._status === &#39;draft&#39;,<br>      &#39;Cannot modify a non-draft order&#39;<br>    );<br>  }<br><br>  assertInvariants() {<br>    // Business invariant: executed orders must have positions<br>    if (this._status === &#39;executed&#39;) {<br>      Assert.that(<br>        this._positions.length &gt; 0,<br>        &#39;Executed order must have at least one position&#39;<br>      );<br>    }<br>  }<br>}</pre><h4><strong>Benefits</strong></h4><ul><li><strong>Self-documenting: </strong>The code tells you exactly what’s expected.</li><li><strong>Fail fast: </strong>Invalid states are caught immediately, not later in obscure places.</li><li><strong>Confidence in refactoring: </strong>Assertions catch mistakes during development.</li></ul><p>Make your domain’s rules explicit. Don’t let invalid states silently corrupt your system.</p><h3>Standalone Classes</h3><p>The fourth pattern promotes <strong>low coupling</strong>. A <strong>Standalone Class</strong> is one that can be understood and used in isolation, with minimal dependencies on other classes.</p><blockquote>“Low coupling is fundamental to object design. When you can, go all the way. Eliminate all other concepts from the picture. Then the class will be self-contained and can be studied and understood alone.” — Eric Evans</blockquote><p>Value Objects are naturally standalone. They encapsulate a concept completely:</p><pre>// StockSymbol - completely self-contained<br>class StockSymbol {<br>  constructor(symbol) {<br>    const trimmed = symbol.trim().toUpperCase();<br>    if (!StockSymbol.isValid(trimmed)) {<br>      throw new Error(`Invalid stock symbol: ${symbol}`);<br>    }<br>    this._value = trimmed;<br>    Object.freeze(this);<br>  }<br><br>  static isValid(symbol) {<br>    return /^[A-Z]{1,5}$/.test(symbol);<br>  }<br><br>  get value() { return this._value; }<br><br>  equals(other) {<br>    return other instanceof StockSymbol &amp;&amp; this._value === other._value;<br>  }<br><br>  toString() { return this._value; }<br>}<br><br>// Percentage - standalone with rich behavior<br>class Percentage {<br>  constructor(value) {<br>    if (value &lt; 0 || value &gt; 100) {<br>      throw new Error(&#39;Percentage must be between 0 and 100&#39;);<br>    }<br>    this._value = value;<br>    Object.freeze(this);<br>  }<br><br>  static of(value) { return new Percentage(value); }<br><br>  get value() { return this._value; }<br><br>  applyTo(amount) {<br>    return amount * (this._value / 100);<br>  }<br><br>  applyToMoney(money) {<br>    return money.multiply(this._value / 100);<br>  }<br><br>  complement() {<br>    return new Percentage(100 - this._value);<br>  }<br><br>  toString() { return `${this._value}%`; }<br>}<br><br>// DateRange - self-contained with useful operations<br>class DateRange {<br>  constructor(start, end) {<br>    if (start &gt; end) {<br>      throw new Error(&#39;Start date must be before end date&#39;);<br>    }<br>    this._start = new Date(start);<br>    this._end = new Date(end);<br>    Object.freeze(this);<br>  }<br><br>  contains(date) {<br>    return date &gt;= this._start &amp;&amp; date &lt;= this._end;<br>  }<br><br>  overlaps(other) {<br>    return this._start &lt;= other._end &amp;&amp; this._end &gt;= other._start;<br>  }<br><br>  get durationInDays() {<br>    const diff = this._end - this._start;<br>    return Math.ceil(diff / (1000 * 60 * 60 * 24));<br>  }<br>}</pre><p>These classes have no external dependencies. They can be tested, reused, and understood completely on their own.</p><h4>Benefits</h4><ul><li><strong>Easy to understand: </strong>No need to trace through dependencies.</li><li><strong>Easy to test: </strong>No mocks or stubs required.</li><li><strong>Highly reusable: </strong>Can be used in any context.</li><li><strong>Reduce complexity: </strong>Each class is a self-contained unit of meaning.</li></ul><p>Strive to make as many classes as possible standalone. Extract concepts into Value Objects that carry their own weight.</p><h3>Specification</h3><p>The fifth and final Supple Design pattern elevates business rules to first-class objects. A <strong>Specification</strong> is a predicate that determines whether an object satisfies certain criteria.</p><blockquote><em>“Business rules often do not fit the responsibility of any of the obvious ENTITIES or VALUE OBJECTS, and their variety and combinations can overwhelm the basic meaning of the domain object. But moving the rules out of the domain layer is even worse, since the domain code no longer expresses the model.” — Eric Evans</em></blockquote><h4>The Specification Pattern</h4><pre>// Base specification with composition support<br>class Specification {<br>  isSatisfiedBy(candidate) {<br>    throw new Error(&#39;Must implement isSatisfiedBy&#39;);<br>  }<br><br>  and(other) {<br>    return new AndSpecification(this, other);<br>  }<br><br>  or(other) {<br>    return new OrSpecification(this, other);<br>  }<br><br>  not() {<br>    return new NotSpecification(this);<br>  }<br>}<br><br>class AndSpecification extends Specification {<br>  constructor(left, right) {<br>    super();<br>    this.left = left;<br>    this.right = right;<br>  }<br><br>  isSatisfiedBy(candidate) {<br>    return this.left.isSatisfiedBy(candidate) &amp;&amp; this.right.isSatisfiedBy(candidate);<br>  }<br>}<br><br>class OrSpecification extends Specification {<br>  constructor(left, right) {<br>    super();<br>    this.left = left;<br>    this.right = right;<br>  }<br><br>  isSatisfiedBy(candidate) {<br>    return this.left.isSatisfiedBy(candidate) || this.right.isSatisfiedBy(candidate);<br>  }<br>}<br><br>class NotSpecification extends Specification {<br>  constructor(spec) {<br>    super();<br>    this.spec = spec;<br>  }<br><br>  isSatisfiedBy(candidate) {<br>    return !this.spec.isSatisfiedBy(candidate);<br>  }<br>}</pre><h4>Stock Trading Specifications</h4><pre>// Individual specifications<br>class HighValueTradeSpecification extends Specification {<br>  constructor(minValue) {<br>    super();<br>    this.minValue = minValue;<br>  }<br><br>  isSatisfiedBy(order) {<br>    return order.totalValue.amount &gt;= this.minValue;<br>  }<br>}<br><br>class ActiveOrderSpecification extends Specification {<br>  isSatisfiedBy(order) {<br>    return order.status === &#39;draft&#39; || order.status === &#39;pending&#39;;<br>  }<br>}<br><br>class PremiumTraderSpecification extends Specification {<br>  isSatisfiedBy(trader) {<br>    return trader.tier === &#39;premium&#39; || trader.tier === &#39;institutional&#39;;<br>  }<br>}<br><br>class VerifiedTraderSpecification extends Specification {<br>  isSatisfiedBy(trader) {<br>    return trader.isVerified &amp;&amp; trader.kycComplete;<br>  }<br>}<br><br>// Composed specifications - readable business rules!<br>const eligibleForMarginTrading = new PremiumTraderSpecification()<br>  .and(new VerifiedTraderSpecification());<br><br>const requiresComplianceReview = new HighValueTradeSpecification(100000)<br>  .and(new ActiveOrderSpecification());<br><br>// Usage in repository<br>class TradeOrderRepository {<br>  findSatisfying(specification) {<br>    return this.orders.filter(order =&gt; specification.isSatisfiedBy(order));<br>  }<br>}<br><br>// Usage<br>const highValueActiveOrders = orderRepository.findSatisfying(<br>  new HighValueTradeSpecification(50000).and(new ActiveOrderSpecification())<br>);</pre><h4>Benefits</h4><ul><li><strong>Explicit business rules: </strong>Rules are named objects, not scattered conditionals.</li><li><strong>Composable: </strong>Build complex rules from simple ones using AND, OR, NOT.</li><li><strong>Reusable: </strong>Same specification works in repositories, services, and validation.</li><li><strong>Testable: </strong>Each specification can be tested in isolation.</li></ul><p>Whenever you find yourself writing complex conditional logic, consider whether a Specification would make the rule clearer.</p><h3>Part 2: Strategic Design Patterns</h3><p>While Supple Design makes individual models flexible, <strong>Strategic Design</strong> addresses a different challenge: how do we manage complexity across an entire enterprise with multiple teams and subsystems?</p><h3>When Do You Need Strategic Design?</h3><p>As systems grow, a single unified model becomes impractical. Different departments have different interpretations of the same terms. “Order” means something different to traders than to the settlement team. “Customer” means something different to marketing than to compliance.</p><p>Strategic Design provides patterns for:</p><ul><li>Defining clear boundaries around different models.</li><li>Managing relationships between those boundaries.</li><li>Protecting your model from external influences.</li></ul><h3>Bounded Context</h3><p>The most important Strategic Design pattern is the <strong>Bounded Context</strong>. It defines an explicit boundary within which a domain model exists, with a specific Ubiquitous Language that applies only within that context.</p><blockquote><em>“A BOUNDED CONTEXT delimits the applicability of a particular model so that team members have a clear and shared understanding of what has to be consistent and how it relates to other CONTEXTS.” — Eric Evans</em></blockquote><h4>The Same Term, Different Meanings</h4><p>In a stock trading platform, “Order” means different things in different contexts:</p><pre>// TRADING CONTEXT: Order = trade execution request<br>namespace TradingContext {<br>  class Order {<br>    constructor(orderId, traderId, stockSymbol, quantity, orderType) {<br>      this.orderId = orderId;<br>      this.traderId = traderId;<br>      this.stockSymbol = stockSymbol;<br>      this.quantity = quantity;<br>      this.orderType = orderType; // &#39;market&#39;, &#39;limit&#39;, &#39;stop&#39;<br>      this.status = &#39;pending&#39;;<br>    }<br><br>    execute(executionPrice) {<br>      this.executionPrice = executionPrice;<br>      this.status = &#39;executed&#39;;<br>      this.executedAt = new Date();<br>    }<br>  }<br>}<br><br>// SETTLEMENT CONTEXT: Order = payment and delivery instruction<br>namespace SettlementContext {<br>  class SettlementOrder {<br>    constructor(settlementId, tradeReference, buyerAccount, sellerAccount) {<br>      this.settlementId = settlementId;<br>      this.tradeReference = tradeReference;<br>      this.buyerAccount = buyerAccount;<br>      this.sellerAccount = sellerAccount;<br>      this.settlementDate = this.calculateT2Settlement();<br>      this.status = &#39;pending&#39;;<br>    }<br><br>    calculateT2Settlement() {<br>      // T+2 settlement rule<br>      const date = new Date();<br>      date.setDate(date.getDate() + 2);<br>      return date;<br>    }<br>  }<br>}<br><br>// COMPLIANCE CONTEXT: Order = audit record<br>namespace ComplianceContext {<br>  class OrderAuditRecord {<br>    constructor(recordId, orderReference, traderId, riskScore) {<br>      this.recordId = recordId;<br>      this.orderReference = orderReference;<br>      this.traderId = traderId;<br>      this.riskScore = riskScore;<br>      this.flaggedForReview = riskScore &gt; 0.7;<br>      this.reviewStatus = &#39;pending&#39;;<br>    }<br>  }<br>}</pre><p>Each context has its own model, its own language, and its own team. They share an identifier (order reference) but not a data structure.</p><h4>Benefits</h4><ul><li><strong>Team autonomy: </strong>Each team owns their context completely.</li><li><strong>Clear language: </strong>Terms have precise meaning within each context.</li><li><strong>Focused models: </strong>Each model solves one problem well.</li><li><strong>Independent deployment: </strong>Contexts can evolve at their own pace.</li></ul><h3>Context Map</h3><p>Once you have multiple Bounded Contexts, you need to understand how they relate. A <strong>Context Map</strong> is a visual documentation of all contexts and their integration patterns.</p><h4>Stock Trading Platform Context Map</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VnM7YI0mi1Cw8EWMCRYlXQ.png" /><figcaption>Stock Trading Platform Context Map</figcaption></figure><p>The map shows:</p><ul><li><strong>Customer/Supplier</strong> between Trading and Pricing (Trading’s needs influence Pricing’s roadmap).</li><li><strong>Anti-Corruption Layer</strong> between Trading and Legacy System.</li><li><strong>Published Language</strong> from Market Data to all consumers.</li><li><strong>Shared Kernel</strong> between Settlement and Reporting.</li><li><strong>Conformist</strong> relationship where Reporting adopts Compliance’s model directly.</li></ul><h3>Shared Kernel</h3><p>A <strong>Shared Kernel</strong> is a small, explicitly agreed-upon subset of the domain model that two or more teams share. It’s code that both teams own together.</p><pre>// SHARED KERNEL - used by Settlement, Reporting, and Trading contexts<br>namespace SharedKernel {<br>  class TraderId {<br>    constructor(value) {<br>      this.value = value;<br>      Object.freeze(this);<br>    }<br>    equals(other) { return this.value === other.value; }<br>  }<br>class StockSymbol {<br>    constructor(value) {<br>      this.value = value.toUpperCase();<br>      Object.freeze(this);<br>    }<br>  }<br>class Money {<br>    constructor(amount, currency = &#39;USD&#39;) {<br>      this.amount = amount;<br>      this.currency = currency;<br>      Object.freeze(this);<br>    }<br>    add(other) { return new Money(this.amount + other.amount, this.currency); }<br>  }<br>class TradeReference {<br>    constructor(value) {<br>      this.value = value;<br>      Object.freeze(this);<br>    }<br>  }<br>}</pre><p>Both Settlement and Reporting contexts import and use these types directly. No translation needed.</p><h4>Trade-offs</h4><ul><li><strong>Pro</strong>: No translation code, single source of truth.</li><li><strong>Con</strong>: Changes require coordination between teams, tight coupling.</li></ul><p>Use Shared Kernel sparingly — only for truly fundamental concepts that are stable and well-understood.</p><h3>Customer/Supplier</h3><p>In a <strong>Customer/Supplier</strong> relationship, the downstream team (Customer) depends on the upstream team (Supplier). The Customer’s requirements influence the Supplier’s development priorities.</p><pre>// SUPPLIER: Pricing Context<br>namespace PricingContext {<br>  // Public contract - designed with customers in mind<br>  class PriceQuote {<br>    constructor(stockSymbol, bidPrice, askPrice, timestamp, validUntil) {<br>      this.stockSymbol = stockSymbol;<br>      this.bidPrice = bidPrice;<br>      this.askPrice = askPrice;<br>      this.timestamp = timestamp;<br>      this.validUntil = validUntil;<br>    }<br>  }<br>class PricingService {<br>    getQuote(stockSymbol) {<br>      // Returns PriceQuote for customers<br>    }<br>    // Added at Trading Context&#39;s request<br>    getBulkQuotes(stockSymbols) {<br>      return stockSymbols.map(s =&gt; this.getQuote(s));<br>    }<br>    // Added at Trading Context&#39;s request<br>    subscribeToUpdates(stockSymbol, callback) {<br>      // Real-time price updates<br>    }<br>  }<br>}<br>// CUSTOMER: Trading Context<br>namespace TradingContext {<br>  class TradingService {<br>    constructor(pricingService) {<br>      this.pricingService = pricingService;<br>    }<br>    async executeMarketOrder(traderId, stockSymbol, quantity) {<br>      const quote = await this.pricingService.getQuote(stockSymbol);<br>      // Use the quote to execute trade<br>    }<br>  }<br>}</pre><p>The Customer/Supplier pattern requires good communication. The Customer team can request features, and the Supplier team considers those requests in their roadmap.</p><h3>Conformist</h3><p>Sometimes the upstream team has no interest in accommodating downstream needs. In a <strong>Conformist</strong> relationship, the downstream team simply adopts the upstream model as-is.</p><pre>// UPSTREAM: Market Data Provider (external, won&#39;t change for us)<br>const MarketDataAPI = {<br>  getStockQuote: (symbol) =&gt; ({<br>    sym: symbol,<br>    bid: 150.25,<br>    ask: 150.30,<br>    vol: 1000000,<br>    ts: &#39;2024-03-15T10:30:00Z&#39;<br>  })<br>};<br><br>// CONFORMIST: Our system uses their model directly<br>namespace TradingContext {<br>  class MarketDataClient {<br>    async getQuote(symbol) {<br>      // We conform to their structure<br>      const data = await MarketDataAPI.getStockQuote(symbol);<br>      // Using their field names directly<br>      return {<br>        symbol: data.sym,<br>        bidPrice: data.bid,<br>        askPrice: data.ask,<br>        volume: data.vol,<br>        timestamp: data.ts<br>      };<br>    }<br>  }<br>}</pre><p>This is pragmatic when the upstream model is good enough and translation provides little value.</p><h3>Anti-Corruption Layer</h3><p>When the upstream model is problematic — legacy code, poor design, or conflicting terminology — you need an <strong>Anti-Corruption Layer</strong> (ACL). This translation layer protects your clean model from external corruption.</p><pre>// LEGACY SYSTEM: Awful design we inherited<br>namespace LegacyTradingSystem {<br>  // Cryptic field names, magic numbers, string dates<br>  function fetchTrade(id) {<br>    return {<br>      trd_id: id,<br>      stk_cd: &#39;AAPL&#39;,<br>      qty: 100,<br>      px: 15025,          // Price in cents!<br>      typ: 1,             // 1=Buy, 2=Sell<br>      sts: &#39;C&#39;,           // &#39;C&#39;=Complete, &#39;P&#39;=Pending, &#39;X&#39;=Cancelled<br>      trd_dt: &#39;20240315&#39;  // YYYYMMDD string<br>    };<br>  }<br>}<br><br>// ANTI-CORRUPTION LAYER: Translates legacy to our clean model<br>namespace ACL {<br>  class TradeTranslator {<br>    static toDomain(legacy) {<br>      return new Trade(<br>        new TradeId(legacy.trd_id),<br>        new StockSymbol(legacy.stk_cd),<br>        legacy.qty,<br>        Money.fromCents(legacy.px),<br>        TradeTranslator.translateType(legacy.typ),<br>        TradeTranslator.translateStatus(legacy.sts),<br>        TradeTranslator.parseDate(legacy.trd_dt)<br>      );<br>    }<br>    static translateType(code) {<br>      const types = { 1: &#39;buy&#39;, 2: &#39;sell&#39; };<br>      return types[code] || &#39;unknown&#39;;<br>    }<br>    static translateStatus(code) {<br>      const statuses = { &#39;C&#39;: &#39;complete&#39;, &#39;P&#39;: &#39;pending&#39;, &#39;X&#39;: &#39;cancelled&#39; };<br>      return statuses[code] || &#39;unknown&#39;;<br>    }<br>    static parseDate(dateStr) {<br>      const year = parseInt(dateStr.substring(0, 4));<br>      const month = parseInt(dateStr.substring(4, 6)) - 1;<br>      const day = parseInt(dateStr.substring(6, 8));<br>      return new Date(year, month, day);<br>    }<br>  }<br>  class LegacyTradeFacade {<br>    async getTrade(tradeId) {<br>      const legacy = await LegacyTradingSystem.fetchTrade(tradeId);<br>      return TradeTranslator.toDomain(legacy);<br>    }<br>  }<br>}<br>// OUR CLEAN DOMAIN: Never sees the ugliness<br>namespace TradingContext {<br>  class TradeService {<br>    constructor(legacyFacade) {<br>      this.legacyFacade = legacyFacade;<br>    }<br>    async getTradeDetails(tradeId) {<br>      // Gets clean domain objects<br>      const trade = await this.legacyFacade.getTrade(tradeId);<br>      // Works with proper types<br>      console.log(`${trade.type} ${trade.quantity} ${trade.stockSymbol} at ${trade.price}`);<br>    }<br>  }<br>}</pre><p>The ACL is the <strong>only</strong> place that knows about the legacy system’s quirks. If the legacy system changes, only the ACL needs to update.</p><h3>Open Host Service</h3><p>An <strong>Open Host Service</strong> provides a well-defined, stable API for external consumers. It’s how your context exposes itself to the world.</p><pre>namespace MarketDataContext {<br>  // Internal domain model (rich, complex)<br>  class StockQuote {<br>    constructor(symbol, bidPrice, askPrice, volume, lastTrade) {<br>      // Complex internal representation<br>    }<br>calculateSpread() { }<br>    isLiquid() { }<br>  }<br>  // PUBLISHED LANGUAGE: Simple DTOs for external consumption<br>  const PublishedLanguage = {<br>    StockQuoteDTO: class {<br>      constructor(symbol, bid, ask, volume, spread, timestamp) {<br>        this.symbol = symbol;<br>        this.bid = bid;<br>        this.ask = ask;<br>        this.volume = volume;<br>        this.spread = spread;<br>        this.timestamp = timestamp;<br>        this.apiVersion = &#39;2.0&#39;;<br>      }<br>    }<br>  };<br>  // OPEN HOST SERVICE: Stable API<br>  class MarketDataService {<br>    // GET /api/v2/quotes/{symbol}<br>    getQuote(symbol) {<br>      const internal = this.repository.findBySymbol(symbol);<br>      return this.toDTO(internal);<br>    }<br>    // GET /api/v2/quotes/bulk<br>    getBulkQuotes(symbols) {<br>      return symbols.map(s =&gt; this.getQuote(s));<br>    }<br>    // WebSocket /api/v2/stream<br>    subscribeToQuotes(symbols, callback) {<br>      // Real-time updates<br>    }<br>    toDTO(internal) {<br>      return new PublishedLanguage.StockQuoteDTO(<br>        internal.symbol,<br>        internal.bidPrice,<br>        internal.askPrice,<br>        internal.volume,<br>        internal.calculateSpread(),<br>        new Date().toISOString()<br>      );<br>    }<br>  }<br>}</pre><p>The internal model can evolve freely. The Published Language (DTOs) provides stability for external consumers.</p><h3>Published Language</h3><p>The <strong>Published Language</strong> is the shared vocabulary that all integrating parties understand. It’s the contract between contexts.</p><pre>namespace PublishedLanguage {<br>  // Version control<br>  const VERSION = &#39;2.0.0&#39;;<br>// Standard events used across all contexts<br>  class TradeExecutedEvent {<br>    constructor(tradeId, traderId, stockSymbol, quantity, price, executedAt) {<br>      this.eventType = &#39;trade.executed&#39;;<br>      this.eventId = crypto.randomUUID();<br>      this.schemaVersion = VERSION;<br>      this.occurredAt = new Date().toISOString();<br>      this.data = {<br>        tradeId,<br>        traderId,<br>        stockSymbol,<br>        quantity,<br>        price: { amount: price.amount, currency: price.currency },<br>        executedAt<br>      };<br>    }<br>  }<br>  class OrderPlacedEvent {<br>    constructor(orderId, traderId, stockSymbol, quantity, orderType) {<br>      this.eventType = &#39;order.placed&#39;;<br>      this.eventId = crypto.randomUUID();<br>      this.schemaVersion = VERSION;<br>      this.occurredAt = new Date().toISOString();<br>      this.data = { orderId, traderId, stockSymbol, quantity, orderType };<br>    }<br>  }<br>  // Shared data structures<br>  class MoneyDTO {<br>    constructor(amount, currency) {<br>      this.amount = amount;<br>      this.currency = currency;<br>    }<br>  }<br>  class AddressDTO {<br>    constructor(line1, line2, city, state, postalCode, country) {<br>      this.line1 = line1;<br>      this.line2 = line2;<br>      this.city = city;<br>      this.state = state;<br>      this.postalCode = postalCode;<br>      this.country = country;<br>    }<br>  }<br>}</pre><p>When Trading Context executes a trade, it publishes a TradeExecutedEvent using the Published Language. Settlement, Compliance, and Reporting contexts all understand this event format.</p><h3>Part 3: The Complete Picture</h3><p>We’ve covered 13 patterns across two families. Let’s see how they work together in a complete system.</p><h3>A Stock Trading Platform Architecture</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tDEUZeJlaYIU3RvsEy4HAA.png" /><figcaption>A Stock Trading Platform Architecture</figcaption></figure><p><strong>Within each context</strong> (Trading, Pricing, Settlement):</p><ul><li>Supple Design patterns keep the model flexible.</li><li>Intention-Revealing Interfaces make code readable.</li><li>Side-Effect-Free Functions enable safe calculations.</li><li>Assertions protect invariants.</li><li>Specifications encode business rules.</li><li>Standalone Classes reduce complexity.</li></ul><p><strong>Between contexts</strong>:</p><ul><li>Bounded Contexts define clear boundaries.</li><li>Context Maps document relationships.</li><li>Shared Kernel provides common types.</li><li>Customer/Supplier negotiates interfaces.</li><li>Anti-Corruption Layer protects from legacy.</li><li>Open Host Service exposes stable APIs.</li><li>Published Language enables event-driven integration.</li></ul><h3>Conclusion</h3><p>This completes our journey through Domain-Driven Design.</p><p><strong>In Chapter 1</strong>, we learned the building blocks: Entities give us identity, Value Objects give us meaning, Services encapsulate operations, and Modules organize our code.</p><p><strong>In Chapter 2</strong>, we learned structure: Aggregates define consistency boundaries, Factories handle complex creation, and Repositories abstract persistence.</p><p><strong>In Chapter 3</strong>, we learned flexibility and scale: Supple Design keeps individual models maintainable, and Strategic Design manages the relationships between them.</p><p>But DDD is more than a collection of patterns. It’s a mindset — a commitment to deeply understanding the business domain and expressing that understanding in code.</p><blockquote><em>“The critical complexity of most software projects is in understanding the domain itself.” — Eric Evans</em></blockquote><p>The patterns we’ve explored are tools. The real skill is knowing when and how to apply them. Start small. Identify the core domain of your system — the part that truly differentiates your business. Apply tactical patterns there first. As the system grows, introduce strategic patterns to manage the complexity.</p><p>Most importantly, never stop learning from domain experts. The best software is built through collaboration between those who understand the business and those who understand the code.</p><p>Thank you for joining me on this journey. Now go build something extraordinary.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=02f84f3dd7c6" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-3-02f84f3dd7c6">Mastering Software Design: With the Power of Domain-Driven Design &amp; Clean Architecture — Chapter 3</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Software Design: With the Power of Domain-Driven Design & Clean Architecture — Chapter 2]]></title>
            <link>https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-2-ce319f3e337f?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/ce319f3e337f</guid>
            <category><![CDATA[system-design-concepts]]></category>
            <category><![CDATA[domain-driven-development]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[technical-architecture]]></category>
            <category><![CDATA[design-patterns]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Mon, 05 Jan 2026 03:22:32 GMT</pubDate>
            <atom:updated>2026-01-08T01:35:29.756Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Mastering Software Design: With the Power of Domain-Driven Design &amp; Clean Architecture — Chapter 2</strong></h3><h4>The Three Pillars That Separate Amateur Code from Production-Grade Systems</h4><p><strong>TL;DR:</strong> In the previous chapter, we explored the foundational building blocks of Domain-Driven Design — Entities, Value Objects, Services, and Modules. These concepts help us model the static aspects of our domain. Now, we move into the advanced patterns that bring structure and lifecycle management to our domain model. <strong>Aggregates</strong> define transactional boundaries and protect business invariants. <strong>Factories</strong> encapsulate complex object creation logic. <strong>Repositories</strong> provide a collection-like interface for retrieving and persisting aggregates. Together, these three patterns form the backbone of any well-designed DDD system. By the end of this chapter, you’ll understand how to design consistent, maintainable, and testable domain models.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ptknru6q032zbiqTIfW2jw.png" /></figure><p>In Chapter 1, we established a solid foundation with Entities, Value Objects, Services, and Modules. If you haven’t read it yet, I encourage you to do so before diving into this chapter.</p><p>Now, we’re ready to tackle the patterns that truly bring a domain model to life. These aren’t just theoretical concepts — they’re the patterns that separate amateur implementations from production-grade systems used by companies like Amazon, Netflix, and Uber.</p><p>Let’s begin with perhaps the most important pattern in DDD — the Aggregate.</p><h3><strong>Aggregate</strong></h3><p>An <strong>Aggregate</strong> is a cluster of domain objects that are treated as a single unit for data changes. Think of it as a consistency boundary — a fortress that protects the integrity of your business rules.</p><p>Every Aggregate has a single entry point called the <strong>Aggregate Root</strong>. External objects can only reference and interact with the Aggregate through its root. Internal objects are hidden from the outside world, protected by the root’s vigilant guardianship.</p><blockquote>“An AGGREGATE is a cluster of associated objects that we treat as a unit for the purpose of data changes. Each AGGREGATE has a root and a boundary. The boundary defines what is inside the AGGREGATE. The root is a single, specific ENTITY contained in the AGGREGATE.”</blockquote><blockquote>— Eric Evans</blockquote><p>The concept might seem abstract at first, so let me illustrate with a real-world analogy.</p><p><strong>The Car Analogy</strong></p><p>Consider a car. You interact with a car through its steering wheel, pedals, and dashboard — these are the “entry points.” You don’t directly manipulate the pistons, fuel injectors, or transmission gears. The car’s internal mechanisms are encapsulated, protected from misuse, and designed to maintain consistency (you can’t engage reverse while moving forward at high speed).</p><p>Similarly, in software, the Aggregate Root acts as the steering wheel of your domain objects. All modifications to internal entities must go through the root, ensuring business rules are never violated.</p><p><strong>Key Characteristics of Aggregates</strong></p><ul><li><strong>Single Entry Point: </strong>Only the Aggregate Root has global identity and can be referenced from outside. Internal entities have local identity meaningful only within the aggregate.</li><li><strong>Transactional Boundary: </strong>One transaction should modify only one aggregate. If you need to modify multiple aggregates, use eventual consistency through domain events.</li><li><strong>Invariant Protection: </strong>The Aggregate Root is responsible for maintaining all business invariants. No external force can put the aggregate into an invalid state.</li><li><strong>Reference by ID: </strong>Aggregates should reference other aggregates by their identity (ID), not by direct object reference. This keeps aggregates loosely coupled and independently loadable.</li></ul><p><strong>Stock Trading Example: TradeOrder Aggregate</strong></p><p>Let me translate this into our stock trading domain. A <strong>TradeOrder</strong> is an excellent example of an Aggregate Root. It contains <strong>OrderLines</strong> (the individual stock positions being traded) as internal entities.</p><pre>// Value Object: Money<br>class Money {<br>  constructor(amount, currency = &#39;USD&#39;) {<br>    if (amount &lt; 0) throw new Error(&#39;Amount cannot be negative&#39;);<br>    this.amount = amount;<br>    this.currency = currency;<br>    Object.freeze(this);<br>  }<br><br>  add(other) {<br>    return new Money(this.amount + other.amount, this.currency);<br>  }<br><br>  multiply(factor) {<br>    return new Money(this.amount * factor, this.currency);<br>  }<br><br>  toString() {<br>    return `${this.currency} ${this.amount.toFixed(2)}`;<br>  }<br>}<br><br>// Internal Entity: OrderLine (only accessible through TradeOrder)<br>class OrderLine {<br>  constructor(lineId, stockSymbol, stockName, quantity, pricePerShare) {<br>    if (quantity &lt;= 0) throw new Error(&#39;Quantity must be positive&#39;);<br>    this._lineId = lineId;<br>    this._stockSymbol = stockSymbol;<br>    this._stockName = stockName;<br>    this._quantity = quantity;<br>    this._pricePerShare = pricePerShare;<br>  }<br><br>  get lineId() { return this._lineId; }<br>  get stockSymbol() { return this._stockSymbol; }<br>  get quantity() { return this._quantity; }<br><br>  get subtotal() {<br>    return this._pricePerShare.multiply(this._quantity);<br>  }<br><br>  // Internal method - called only by Aggregate Root<br>  updateQuantity(newQuantity) {<br>    if (newQuantity &lt;= 0) throw new Error(&#39;Quantity must be positive&#39;);<br>    this._quantity = newQuantity;<br>  }<br>}<br><br>// AGGREGATE ROOT: TradeOrder<br>class TradeOrder {<br>  constructor(orderId, traderId) {<br>    this._orderId = orderId;           // Global identity<br>    this._traderId = traderId;          // Reference to Trader aggregate by ID<br>    this._lines = [];<br>    this._status = &#39;draft&#39;;<br>    this._lineIdCounter = 0;<br>  }<br><br>  get orderId() { return this._orderId; }<br>  get traderId() { return this._traderId; }<br>  get status() { return this._status; }<br><br>  // Returns read-only view of lines (protects internal state)<br>  get lines() { return [...this._lines]; }<br><br>  get totalValue() {<br>    return this._lines.reduce(<br>      (sum, line) =&gt; sum.add(line.subtotal),<br>      new Money(0)<br>    );<br>  }<br><br>  get isEmpty() { return this._lines.length === 0; }<br><br>  // All modifications go through the Aggregate Root<br>  addPosition(stockSymbol, stockName, pricePerShare, quantity) {<br>    this.assertDraft();<br><br>    // Check if stock already exists in order<br>    const existingLine = this._lines.find(l =&gt; l.stockSymbol === stockSymbol);<br>    if (existingLine) {<br>      existingLine.updateQuantity(existingLine.quantity + quantity);<br>      return existingLine.lineId;<br>    }<br><br>    // Create new line with local identity<br>    const lineId = `line-${++this._lineIdCounter}`;<br>    this._lines.push(new OrderLine(lineId, stockSymbol, stockName, quantity, pricePerShare));<br>    return lineId;<br>  }<br><br>  removePosition(lineId) {<br>    this.assertDraft();<br>    const index = this._lines.findIndex(l =&gt; l.lineId === lineId);<br>    if (index === -1) throw new Error(`Position ${lineId} not found`);<br>    this._lines.splice(index, 1);<br>  }<br><br>  // Execute order - transitions state and enforces invariants<br>  execute() {<br>    this.assertDraft();<br><br>    // Enforce business invariants<br>    if (this.isEmpty) {<br>      throw new Error(&#39;Cannot execute empty order&#39;);<br>    }<br><br>    this._status = &#39;executed&#39;;<br>    this._executedAt = new Date();<br>  }<br><br>  cancel() {<br>    if (this._status === &#39;executed&#39;) {<br>      throw new Error(&#39;Cannot cancel executed order&#39;);<br>    }<br>    this._status = &#39;cancelled&#39;;<br>  }<br><br>  assertDraft() {<br>    if (this._status !== &#39;draft&#39;) {<br>      throw new Error(&#39;Cannot modify non-draft order&#39;);<br>    }<br>  }<br>}</pre><p><strong>Notice several important patterns in this code:</strong></p><p>1. <strong>OrderLine is internal: </strong>You cannot create an OrderLine directly. It’s created through the TradeOrder’s `addPosition` method.</p><p>2. <strong>Invariants are enforced: </strong>You cannot execute an empty order. You cannot modify an executed order.</p><p>3. <strong>Reference by ID: </strong>The TradeOrder references the Trader by `traderId`, not by a direct Trader object. This is crucial for maintaining aggregate independence.</p><p>4. <strong>State transitions are controlled: </strong>The order follows a state machine (draft → executed or cancelled), and the root controls all transitions.</p><p><strong>The Golden Rules of Aggregates<br></strong>Eric Evans provides clear guidelines that should govern your aggregate design:</p><p>1. <strong>One transaction = One aggregate: </strong>Never modify multiple aggregates in a single transaction. If you need to update multiple aggregates, use domain events and eventual consistency.</p><p>2. <strong>Keep aggregates small: </strong>Large aggregates lead to concurrency issues and performance problems. If an aggregate is getting too large, consider splitting it.</p><p>3. <strong>Reference other aggregates by identity: </strong>Don’t hold direct references to other aggregate roots. Store their IDs instead.</p><p>4. <strong>Eventual consistency between aggregates: </strong>Accept that different aggregates may be temporarily inconsistent. This is a feature, not a bug.</p><blockquote>“Cluster the ENTITIES and VALUE OBJECTS into AGGREGATES and define boundaries around each. Choose one ENTITY to be the root of each AGGREGATE, and control all access to the objects inside the boundary through the root.”</blockquote><blockquote>— Eric Evans</blockquote><h3><strong>Factory</strong></h3><p>Now that we understand Aggregates and their boundaries, let’s address a common challenge: <strong>How do we create these complex aggregates?</strong></p><p>Creating an aggregate isn’t always straightforward. It might require validating inputs, setting up relationships between internal entities, enforcing creation rules, or initializing the aggregate into a valid starting state. Putting all this logic inside the aggregate itself pollutes its primary responsibility — maintaining business invariants.</p><p>This is where the <strong>Factory</strong> pattern comes in.</p><p>A <strong>Factory</strong> is an object or method responsible for creating complex aggregates and entities. It encapsulates the knowledge needed to create valid domain objects, especially when creation involves complex logic or multiple objects.</p><blockquote>“A car engine is an intricate piece of machinery, with dozens of parts collaborating to perform the engine’s responsibility: to turn a shaft. One could imagine trying to design an engine block that could grab on to a set of pistons and insert them into its cylinders, spark plugs that would find their sockets and screw themselves in. But it seems unlikely that such a complicated machine would be as reliable or as efficient as our typical engines are. Instead, we accept that something else will assemble the pieces.”</blockquote><blockquote>— Eric Evans</blockquote><p>The analogy is perfect. Just as a car is assembled by a factory (robotic or human), not by the car itself, domain objects should be created by dedicated factories, not by the objects themselves.</p><p><strong>Types of Factories in DDD</strong></p><p>DDD recognizes several types of factories, each suited for different scenarios:</p><p><strong>1. Factory Method: </strong>A method on an entity or aggregate that creates related objects.</p><pre>class TradeOrder {<br>  // Factory method for creating a copy (for reordering)<br>  static createReorder(originalOrder, newOrderId) {<br>    const newOrder = new TradeOrder(newOrderId, originalOrder.traderId);<br>    for (const line of originalOrder.lines) {<br>      newOrder.addPosition(<br>        line.stockSymbol,<br>        line.stockName,<br>        line._pricePerShare,<br>        line.quantity<br>      );<br>    }<br>    return newOrder;<br>  }<br><br>  // Factory method for reconstituting from persistence<br>  static reconstitute(orderId, traderId, lines, status, executedAt) {<br>    const order = new TradeOrder(orderId, traderId);<br>    order._lines = lines;<br>    order._status = status;<br>    order._executedAt = executedAt;<br>    return order;<br>  }<br>}</pre><p><strong>2. Standalone Factory Class: </strong>A dedicated class responsible for creating aggregates.</p><pre>class TradeOrderFactory {<br>  constructor() {<br>    this.orderCounter = 0;<br>    this.lineCounter = 0;<br>  }<br><br>  // Create a new market order<br>  createMarketOrder(traderId, stockSymbol, stockName, currentPrice, quantity) {<br>    const orderId = this.generateOrderId();<br>    const order = new TradeOrder(orderId, traderId);<br>    order.addPosition(stockSymbol, stockName, currentPrice, quantity);<br>    return order;<br>  }<br><br>  // Create an order with multiple positions<br>  createPortfolioOrder(traderId, positions) {<br>    if (positions.length === 0) {<br>      throw new Error(&#39;Portfolio order must have at least one position&#39;);<br>    }<br><br>    const orderId = this.generateOrderId();<br>    const order = new TradeOrder(orderId, traderId);<br><br>    for (const position of positions) {<br>      order.addPosition(<br>        position.stockSymbol,<br>        position.stockName,<br>        new Money(position.price),<br>        position.quantity<br>      );<br>    }<br><br>    return order;<br>  }<br><br>  // Create a limit order (simplified example)<br>  createLimitOrder(traderId, stockSymbol, stockName, limitPrice, quantity) {<br>    const orderId = this.generateOrderId();<br>    const order = new TradeOrder(orderId, traderId);<br>    order._orderType = &#39;limit&#39;;<br>    order._limitPrice = limitPrice;<br>    order.addPosition(stockSymbol, stockName, limitPrice, quantity);<br>    return order;<br>  }<br><br>  generateOrderId() {<br>    return `ORD-${Date.now()}-${++this.orderCounter}`;<br>  }<br>}</pre><p><strong>3. Abstract Factory: </strong>When you need to create families of related objects. This is useful when different contexts require different variations of the same aggregate type.</p><pre>// Abstract Factory for different trading tiers<br>class TradingAccountFactory {<br>  createWelcomePackage() { throw new Error(&#39;Must implement&#39;); }<br>  createCommissionStructure() { throw new Error(&#39;Must implement&#39;); }<br>  createTradingLimits() { throw new Error(&#39;Must implement&#39;); }<br>}<br><br>class RetailTradingFactory extends TradingAccountFactory {<br>  createWelcomePackage() {<br>    return { message: &#39;Welcome to trading!&#39;, bonus: new Money(0) };<br>  }<br><br>  createCommissionStructure() {<br>    return { perTrade: new Money(9.99), percentage: 0 };<br>  }<br><br>  createTradingLimits() {<br>    return { maxOrderValue: new Money(50000), marginAllowed: false };<br>  }<br>}<br><br>class InstitutionalTradingFactory extends TradingAccountFactory {<br>  createWelcomePackage() {<br>    return { message: &#39;Welcome, Institutional Partner!&#39;, bonus: new Money(1000) };<br>  }<br><br>  createCommissionStructure() {<br>    return { perTrade: new Money(0), percentage: 0.001 }; // 0.1% of trade value<br>  }<br><br>  createTradingLimits() {<br>    return { maxOrderValue: new Money(10000000), marginAllowed: true };<br>  }<br>}</pre><p><strong>When to Use Factories</strong></p><p>Factories are not always necessary. Simple object creation can happen directly. Use factories when:</p><ul><li><strong>Object creation is complex: </strong>Multiple steps, validations, or dependencies.</li><li><strong>Creation involves multiple related objects: </strong>An aggregate with internal entities.</li><li><strong>Creation rules need enforcement: </strong>Business rules that must be satisfied during creation.</li><li><strong>Different creation paths exist: </strong>Multiple ways to create the same type of object.</li><li><strong>Reconstitution from persistence: </strong>Rebuilding objects from stored data.</li></ul><h4>Factory vs Constructor</h4><p>A common question: Why not just use constructors?</p><p>Constructors are fine for simple objects. But they have limitations:</p><p>- Constructors must return an instance of their class (no polymorphism).<br>- Complex validation logic in constructors makes them hard to test.<br>- Multiple creation scenarios require overloaded constructors, which becomes messy.<br>- Constructors can’t express creation in domain language.</p><p>Factories solve these problems by providing named creation methods that express intent in domain language: <em>createMarketOrder</em>, <em>createLimitOrder</em>, <em>createReorder</em>.</p><h3><strong>Repository</strong></h3><p>We’ve covered how to model aggregates (Aggregates) and how to create them (Factories). Now comes the final piece of the puzzle: <strong>How do we store and retrieve these aggregates?</strong></p><p>Enter the <strong>Repository</strong> pattern.</p><p>A <strong>Repository</strong> is a mechanism for encapsulating storage, retrieval, and search behavior that emulates a collection of objects. From the domain’s perspective, a repository behaves like an in-memory collection — you add objects to it, remove objects from it, and query for objects matching certain criteria.</p><blockquote>“A REPOSITORY represents all objects of a certain type as a conceptual set (usually emulated). It acts like a collection, except with more elaborate querying capability. Objects of the appropriate type are added and removed, and the machinery behind the REPOSITORY inserts them or deletes them from the database.”</blockquote><blockquote>— Eric Evans</blockquote><p>The beauty of the Repository pattern is that it hides all persistence details from the domain. Your domain logic doesn’t know (or care) whether data is stored in PostgreSQL, MongoDB, or a flat file. It just asks the repository for objects and trusts they’ll appear.</p><p><strong>Key Principles of Repositories</strong></p><ol><li><strong>One Repository per Aggregate Root: </strong>Only aggregate roots have repositories. You never have a repository for internal entities like `OrderLine`. To access an OrderLine, you must first retrieve its parent TradeOrder through the OrderRepository.</li><li><strong>Query Methods Use Domain Language: </strong>Repository methods should be expressed in the language of the domain, not in database terminology.</li></ol><pre>// Good - Domain language<br>orderRepository.findActiveOrdersByTrader(traderId);<br>orderRepository.findPendingLimitOrders();<br>customerRepository.findPremiumTraders();<br><br>// Bad - Database language<br>orderRepository.selectWhereStatusEqualsAndTraderIdEquals(&#39;active&#39;, traderId);<br>orderRepository.executeQuery(&#39;SELECT * FROM orders WHERE...&#39;);</pre><p><strong>3. Returns Fully Reconstituted Aggregates: </strong>Repositories don’t return raw data or partial objects. They return complete, valid aggregate instances ready for use. The Factory pattern is often used internally by repositories to reconstitute aggregates from stored data.</p><p><strong>4. Abstracts Persistence Mechanism: </strong>The repository interface is defined in the domain layer, but the implementation lives in the infrastructure layer. This allows you to swap databases without changing domain code.</p><p><strong>Stock Trading Example: TradeOrderRepository</strong></p><pre>// Interface defined in Domain Layer<br>class TradeOrderRepository {<br>  save(order) { throw new Error(&#39;Must implement&#39;); }<br>  findById(orderId) { throw new Error(&#39;Must implement&#39;); }<br>  delete(orderId) { throw new Error(&#39;Must implement&#39;); }<br><br>  // Domain-specific queries<br>  findByTraderId(traderId) { throw new Error(&#39;Must implement&#39;); }<br>  findActiveOrders() { throw new Error(&#39;Must implement&#39;); }<br>  findPendingLimitOrders() { throw new Error(&#39;Must implement&#39;); }<br>  findOrdersPlacedBetween(startDate, endDate) { throw new Error(&#39;Must implement&#39;); }<br><br>  // Collection semantics<br>  count() { throw new Error(&#39;Must implement&#39;); }<br>  exists(orderId) { throw new Error(&#39;Must implement&#39;); }<br>}<br><br>// Implementation in Infrastructure Layer (In-Memory for demo)<br>class InMemoryTradeOrderRepository extends TradeOrderRepository {<br>  constructor() {<br>    super();<br>    this.orders = new Map();<br>  }<br><br>  save(order) {<br>    this.orders.set(order.orderId, order);<br>    console.log(`[OrderRepo] Saved order: ${order.orderId}`);<br>  }<br><br>  findById(orderId) {<br>    return this.orders.get(orderId) || null;<br>  }<br><br>  delete(orderId) {<br>    this.orders.delete(orderId);<br>    console.log(`[OrderRepo] Deleted order: ${orderId}`);<br>  }<br><br>  findByTraderId(traderId) {<br>    return Array.from(this.orders.values())<br>      .filter(order =&gt; order.traderId === traderId);<br>  }<br><br>  findActiveOrders() {<br>    return Array.from(this.orders.values())<br>      .filter(order =&gt; order.status === &#39;draft&#39;);<br>  }<br><br>  findPendingLimitOrders() {<br>    return Array.from(this.orders.values())<br>      .filter(order =&gt; order._orderType === &#39;limit&#39; &amp;&amp; order.status === &#39;draft&#39;);<br>  }<br><br>  findOrdersPlacedBetween(startDate, endDate) {<br>    return Array.from(this.orders.values())<br>      .filter(order =&gt; {<br>        if (!order._executedAt) return false;<br>        return order._executedAt &gt;= startDate &amp;&amp; order._executedAt &lt;= endDate;<br>      });<br>  }<br><br>  count() {<br>    return this.orders.size;<br>  }<br><br>  exists(orderId) {<br>    return this.orders.has(orderId);<br>  }<br>}</pre><p><strong>Repository vs Data Access Object (DAO)</strong></p><p>Many developers confuse Repositories with DAOs. While they seem similar, they serve different purposes:</p><p><strong>Repository vs DAO (in simple terms)</strong></p><ul><li><strong>Focus: </strong>A Repository works with <strong>aggregate roots</strong>, while a DAO works directly with <strong>database tables or records</strong>.</li><li><strong>Language: </strong>Repositories speak the <strong>language of the domain</strong>, whereas DAOs are expressed in terms of <strong>database operations</strong>.</li><li><strong>Returns: </strong>A Repository returns <strong>domain objects</strong>, while a DAO typically returns <strong>DTOs or raw data</strong>.</li><li><strong>Abstraction: </strong>Repositories model data as a <strong>collection of objects</strong>, whereas DAOs expose <strong>CRUD-style operations</strong>.</li><li><strong>Layer</strong><br>Repositories are <strong>domain-centric</strong>, while DAOs are <strong>data-centric</strong>.</li></ul><p>A DAO might have methods like <em>`insertOrder()`</em>, <em>`updateOrderStatus()`</em>, <em>`selectOrdersByCustomerId()`</em>. A Repository has methods like <em>`save()`</em>, <em>`findActiveOrders()`</em>, <em>`findByTraderId()`</em>.</p><p>The Repository provides a higher-level abstraction that speaks the language of the domain.</p><p><strong>The Specification Pattern</strong></p><p>For complex queries, DDD offers the <strong>Specification</strong> pattern. Specifications encapsulate query criteria as first-class objects that can be combined.</p><pre>// Specification interface<br>class Specification {<br>  isSatisfiedBy(candidate) { throw new Error(&#39;Must implement&#39;); }<br><br>  and(other) {<br>    return new AndSpecification(this, other);<br>  }<br><br>  or(other) {<br>    return new OrSpecification(this, other);<br>  }<br>}<br><br>// Concrete specifications<br>class HighValueOrderSpecification extends Specification {<br>  constructor(minValue) {<br>    super();<br>    this.minValue = minValue;<br>  }<br><br>  isSatisfiedBy(order) {<br>    return order.totalValue.amount &gt;= this.minValue;<br>  }<br>}<br><br>class ActiveOrderSpecification extends Specification {<br>  isSatisfiedBy(order) {<br>    return order.status === &#39;draft&#39;;<br>  }<br>}<br><br>// Combined specification<br>class AndSpecification extends Specification {<br>  constructor(left, right) {<br>    super();<br>    this.left = left;<br>    this.right = right;<br>  }<br><br>  isSatisfiedBy(candidate) {<br>    return this.left.isSatisfiedBy(candidate) &amp;&amp; this.right.isSatisfiedBy(candidate);<br>  }<br>}<br><br>// Usage<br>const highValueActive = new HighValueOrderSpecification(10000)<br>  .and(new ActiveOrderSpecification());<br><br>const matchingOrders = allOrders.filter(o =&gt; highValueActive.isSatisfiedBy(o));</pre><p>This pattern is particularly powerful for building dynamic query interfaces where users can combine multiple filter criteria.</p><h3><strong>Connecting the Dots: The Complete Picture</strong></h3><p>Now that we’ve explored Aggregates, Factories, and Repositories individually, let’s see how they work together in a typical workflow.</p><p><strong>The Lifecycle of a TradeOrder</strong></p><pre>1. CREATION (Factory)<br>   └── TradeOrderFactory.createMarketOrder(traderId, stockSymbol, price, quantity)<br>       └── Returns a new TradeOrder aggregate in valid state<br><br>2. MODIFICATION (Aggregate Root)<br>   └── order.addPosition(stockSymbol, stockName, price, quantity)<br>   └── order.removePosition(lineId)<br>   └── All modifications go through the Aggregate Root<br><br>3. PERSISTENCE (Repository)<br>   └── orderRepository.save(order)<br>       └── Repository stores the aggregate<br><br>4. RETRIEVAL (Repository → Factory)<br>   └── orderRepository.findById(orderId)<br>       └── Repository uses Factory internally to reconstitute aggregate<br>       └── Returns fully reconstructed TradeOrder<br><br>5. STATE TRANSITION (Aggregate Root)<br>   └── order.execute()<br>       └── Aggregate Root validates invariants and changes state<br><br>6. PERSISTENCE (Repository)<br>   └── orderRepository.save(order)<br>       └── Updated state is persisted</pre><p><strong>This flow ensures:</strong></p><p>- Objects are always created in a valid state (Factory).<br>- Business invariants are always maintained (Aggregate).<br>- Persistence is abstracted from domain logic (Repository).<br>- The domain model remains clean and focused on business rules.</p><p><strong>Example: Complete Trading Workflow</strong></p><pre>// Setup<br>const orderFactory = new TradeOrderFactory();<br>const orderRepository = new InMemoryTradeOrderRepository();<br><br>// 1. Trader places a market order<br>const order = orderFactory.createMarketOrder(<br>  &#39;trader-001&#39;,<br>  &#39;AAPL&#39;,<br>  &#39;Apple Inc.&#39;,<br>  new Money(175.50),<br>  100<br>);<br><br>// 2. Add more positions to the order<br>order.addPosition(&#39;GOOGL&#39;, &#39;Alphabet Inc.&#39;, new Money(140.25), 50);<br>order.addPosition(&#39;MSFT&#39;, &#39;Microsoft Corp.&#39;, new Money(378.91), 75);<br><br>// 3. Save to repository<br>orderRepository.save(order);<br><br>// 4. Later, retrieve the order<br>const retrievedOrder = orderRepository.findById(order.orderId);<br>console.log(`Order ${retrievedOrder.orderId}: ${retrievedOrder.totalValue}`);<br><br>// 5. Execute the trade<br>retrievedOrder.execute();<br>orderRepository.save(retrievedOrder);<br><br>// 6. Query for trader&#39;s history<br>const traderOrders = orderRepository.findByTraderId(&#39;trader-001&#39;);<br>console.log(`Trader has ${traderOrders.length} orders`);</pre><h3><strong>Conclusion</strong></h3><p>In this chapter, we’ve explored three powerful patterns that form the structural backbone of any DDD implementation:</p><ul><li><strong>Aggregates: </strong>define consistency boundaries and protect business invariants. They ensure that your domain model is always in a valid state by controlling all access to internal entities through a single root.</li><li><strong>Factories: </strong>encapsulate complex object creation logic. They express creation in domain language and ensure that aggregates are always born in a valid state.</li><li><strong>Repositories: </strong>provide a collection-like abstraction over persistence. They speak the language of the domain and hide the complexity of data storage from your domain model.</li></ul><p>Together, these patterns enable you to build systems that are:</p><ul><li><strong>Consistent: </strong>Business rules are enforced at all times.</li><li><strong>Maintainable: </strong>Each pattern has a single responsibility.</li><li><strong>Testable: </strong>Factories and Repositories can be easily mocked.</li><li><strong>Scalable: </strong>Aggregates define natural boundaries for distribution.</li></ul><blockquote>“When we approach system design with an emphasis on domain modeling, we start with a rich model and then implement the technical infrastructure to support it. The domain should be the heart of the system.”</blockquote><blockquote>— Eric Evans</blockquote><p>In the next chapter, we’ll explore <strong>Supple Design </strong>patterns (Intention-Revealing Interfaces, Side-Effect-Free Functions, Assertions, and Specifications) alongside <strong>Strategic Design </strong>patterns (Bounded Contexts, Context Maps, and Anti-Corruption Layers). These patterns help us design systems that are not just correct, but also elegant and adaptable to change.</p><p>Until then, I encourage you to experiment with the patterns we’ve discussed. Take a domain you’re familiar with, identify your aggregates, design your factories, and implement your repositories. The best way to master DDD is through practice.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ce319f3e337f" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-2-ce319f3e337f">Mastering Software Design: With the Power of Domain-Driven Design &amp; Clean Architecture — Chapter 2</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering Software Design: With the Power of Domain-Driven Design & Clean Architecture -Chapter 1]]></title>
            <link>https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-1-42b45146334e?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/42b45146334e</guid>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[software-architecture]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[clean-architecture]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Wed, 12 Mar 2025 01:06:48 GMT</pubDate>
            <atom:updated>2025-03-28T01:53:18.217Z</atom:updated>
            <content:encoded><![CDATA[<h4>Building Scalable, Maintainable, and Future-Proof Systems with DDD and Clean Architecture</h4><p><strong>TL;DR </strong>What separates ordinary software from the extraordinary? How do you measure the true worth of a piece of code? What do big tech giants prioritize in low-level programming? These are fundamental questions every new programmer asks after building multiple standalone web apps and backends.</p><p>The goal of this blog series is to <strong>understand the engineering discipline</strong> behind designing <strong>resilient and maintainable software</strong>. Developing software in silos is easy, but when you step back and see the bigger picture, you realize its true impact — leading to world-class ecosystems like <strong>SAP, Jira, Zoho, and Slack</strong>.</p><p>Building software that lasts for decades and adapts to business pivots requires strong engineering principles. These systems are <strong>extendable</strong>, allowing consumers to <strong>create and integrate their own solutions seamlessly</strong>.</p><p>Today, you’ll learn about Domain-Driven Design (DDD) — an irreplaceable approach for solving business problems through digital innovation. I encourage you to read this blog and get inspired!</p><p>This article will be divided into multiple chapters. In the first few chapters, I will explain the theory of Domain-Driven Design (DDD). Later, we’ll dive into a <strong>case study</strong> to understand it better. Finally, we’ll <strong>compare DDD with Clean Architecture</strong> to see how they complement each other.</p><p>It’s going to be an exciting journey — <strong>let’s dive in!</strong></p><p><strong>Software development discipline</strong> can be divided into different levels. You can’t jump directly into Domain-Driven Design (DDD) without the necessary foundational knowledge. <strong>Each level builds upon the previous one</strong>, providing the conceptual understanding required to master DDD.</p><h3>The ladder of mastering software development discipline</h3><ul><li><strong>Level 1: Clean Code</strong> → Writing readable, maintainable, and efficient code.</li><li><strong>Level 2: Design Patterns</strong> → The 23 design patterns that solve 90% of common software development problems. These form the backbone of reusable and maintainable object-oriented code.[1]</li><li><strong>Level 3: Clean Architecture</strong> → This book presents real-world examples and case studies to demonstrate how to design systems that are <strong>robust, testable, and easy to evolve over time</strong>.</li><li><strong>Level 4: Domain-Driven Design (DDD)</strong> → A software design approach that aligns development with complex business domains.</li><li><strong>Level 5: Distributed Systems &amp; Scalability</strong> → Real programming begins when we recognize that a single machine is insufficient, requiring us to design for <strong>scale, reliability, and fault tolerance</strong>.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*00oexoSKyYtIwI9oQ6jAAg.png" /></figure><p>In this article series, we’ll <strong>focus on DDD and Clean Architecture</strong>. Let me know in the comments if you’d like more insights from other levels!</p><p>Eric Evans’ <em>Domain-Driven Design (DDD)</em> is more than just a methodology — it’s a mindset that enables the creation of resilient, long-lasting, and scalable digital products. Throughout this article, I will share relevant quotes from the book.</p><blockquote><strong><em>“A good domain model can be incredibly valuable but it’s not something that’s easy to make. Few people can do it well and it&#39;s very hard to teach. “— Eric Evans</em></strong></blockquote><p>If you haven’t read the book yet, I highly encourage you to do so — especially if you’re serious about building high-quality products.</p><h3><strong>Why Follow DDD Standards for Serious Digital Products</strong></h3><p><strong>1. Resilience and Adaptability: </strong>Technology stacks evolve, business needs change, and market demands shift. DDD helps software stay <strong>aligned with business realities</strong> by structuring code around the <strong>core domain</strong> rather than transient technical details. By focusing on <strong>bounded contexts, aggregates, and domain events</strong>, the system remains flexible, allowing <strong>seamless adaptation to new challenges</strong> without breaking its foundational architecture.</p><p><strong>2. Code That Lasts for Decades: </strong>Well-structured <strong>domain models</strong> don’t just solve immediate problems — they <strong>stand the test of time</strong>. Unlike purely technical architectures that become obsolete, DDD ensures that <strong>business logic remains central</strong>, making refactoring and technology upgrades smoother. Many legacy systems fail not because of outdated tech, but due to <strong>poorly modeled business logic</strong>. DDD prevents this by <strong>keeping code meaningful and relevant</strong>.</p><p><strong>3. Building an Ecosystem, Not Just a Product: </strong>A serious digital product should <strong>extend beyond itself</strong> — it should become a platform that integrates seamlessly into a larger ecosystem. DDD fosters<strong> </strong>clear boundaries<strong> (bounded contexts)</strong> and well-defined APIs, making it easier to <strong>interact with other services, third-party tools, and future applications</strong>. When software is built with strategic design in mind, it doesn’t just serve a single purpose. It evolves into an ecosystem that scales and sustains itself over time.</p><p>By following DDD, we’re not just writing code — we’re <strong>architecting digital businesses</strong> that <strong>last, scale, and adapt</strong>.</p><p>Let’s first understand the <strong>key elements of DDD</strong>. We will go through each element one by one and explore its significance.</p><h3>What is Modeling?</h3><p>Let’s understand modeling with the example of a <strong>map</strong> — whether physical or digital (like Apple Maps or Google Maps). A map is a perfect example of modeling because it represents real-world geography in a <strong>simplified and structured</strong> way, capturing only the relevant details while omitting unnecessary complexities.</p><p>A map can never be <strong>100% accurate</strong> to the real world. If it were, it would have to become the real world itself — defeating the purpose of having a map in the first place. This is exactly why we use models: <strong>to simplify reality and make it easier to understand and navigate</strong>.</p><p>Similarly, in <strong>software development</strong>, modeling is the process of creating an <strong>abstract representation</strong> of a real-world system, focusing only on its most <strong>essential aspects</strong> while ignoring unnecessary details. The level of detail in a model depends on the <strong>requirements and objectives</strong> of the system being designed.</p><h3>The <strong>business team</strong> or <strong>domain experts</strong> are the ones who truly understand the problem domain.</h3><p>Developers may be experts in coding, but without <strong>deep collaboration</strong> with business stakeholders, they risk building solutions that don’t align with real-world needs. <strong>DDD emphasizes that software design should be driven by domain knowledge</strong>, which resides with business experts.</p><p>The role of developers is to work closely with them, using <strong>Ubiquitous Language</strong> to create models that accurately represent business processes and rules. This collaboration ensures that the system reflects <strong>real business needs</strong> rather than just technical assumptions.</p><h3><strong>What is Ubiquitous Language?</strong></h3><p>In <strong>Domain-Driven Design (DDD)</strong>, <strong>Ubiquitous Language</strong> is a common, well-defined vocabulary shared between <strong>developers, domain experts, analysts, and stakeholders</strong>. It ensures that everyone speaks the same language, reducing misunderstandings and aligning the software with business needs.</p><p><strong>Stock Trading Example<br></strong>In a stock trading system, different teams might use different terms for the same concept. Without a shared language, developers might name things based on technical intuition rather than how traders actually think.</p><p>Instead of using generic technical terms like:<br> ❌ TradeRecord<br> ❌ StockData<br> ❌ UserInfo</p><p>We align with the traders’ terminology and use:<br><strong>OrderBook</strong> — A collection of buy and sell orders at different price levels.<br><strong>Market Order</strong> — An order that executes immediately at the best available price.<br><strong>Limit Order</strong> — An order to buy or sell at a specific price or better.<br><strong>Slippage</strong> — The difference between expected and actual execution price.<br><strong>Liquidity Provider</strong> — An entity that ensures there are enough buy/sell orders in the market.</p><p>By enforcing <strong>Ubiquitous Language</strong>, the system’s model directly reflects real-world trading concepts, making communication between developers and domain experts <strong>seamless, accurate, and efficient</strong>.</p><h3><strong>Requirement Elicitation</strong></h3><p>A great quote from the book — <em>User Stories Applied: For Agile Software Development</em> by Mike Cohn on requirement elicitation is:</p><blockquote><strong>“The best way to get the right requirements is to involve users early and often. A good user story captures what is needed, but great software comes from conversations and collaboration.” — Mike Cohn</strong></blockquote><p>This highlights the essence of agile requirement gathering — continuous interaction with users to ensure the software meets real needs.</p><p>This aligns well with <strong>Domain-Driven Design (DDD)</strong> because it emphasizes that <strong>requirements are not just collected but discovered through collaboration</strong> between developers, domain experts, and stakeholders. Instead of rigid documentation, requirement elicitation in DDD is an <strong>iterative, conversation-driven process</strong> where real business needs shape the software model.</p><p><strong>Requirement elicitation </strong>is not just about gathering a list of features — it’s about deeply understanding the <strong>business domain</strong>, identifying <strong>core business problems</strong>, and collaboratively shaping the software model.</p><p><strong>How Requirement Elicitation Works in DDD</strong></p><ol><li><strong>Engage Domain Experts</strong> — The primary source of requirements is the <strong>domain experts</strong> (traders in stock trading, doctors in healthcare, etc.). They provide real-world knowledge.</li><li><strong>Use Ubiquitous Language</strong> — Discussions should be in <strong>business terms</strong>, avoiding unnecessary technical jargon.</li><li><strong>Explore Bounded Contexts</strong> — Clearly define <strong>where</strong> and <strong>how</strong> different business concepts apply. For example, in stock trading, “Order” might mean something different for a retail trader vs. an institutional broker.</li><li><strong>Identify Core Domain and Supporting Domains</strong> — Focus on <strong>what differentiates the business</strong> from competitors (core domain) while leveraging off-the-shelf solutions for generic aspects.</li><li><strong>Event Storming</strong> — A collaborative workshop where developers and business experts map out <strong>key events</strong> in the business process (e.g., “Order Placed”, “Trade Executed”).</li><li><strong>Iterative Refinement</strong> — Requirements are not static. They evolve as the business and system understanding grows.</li></ol><p><strong>Requirement elicitation</strong> focuses on <strong>business behavior and rules</strong>, not just technical specifications. It leads to a <strong>rich domain model</strong>, making software flexible and aligned with real-world needs.</p><p>Now, let’s dive into the core of Domain-Driven Design. We will explore the fundamental elements that shape and define DDD.</p><h3><strong>Entity</strong></h3><p>an <strong>Entity</strong> is a core domain object that has a <strong>unique identity</strong> that persists over time, regardless of changes to its attributes. Entity represent Identity, any particular object in your system that is unique.</p><p>below quote from the book well explains it. Let me write it below as no other explanation can justify it better.</p><blockquote>“In our typical conception, a person (to continue with the nontechnical example) has an identity that stretches from birth to death and even beyond. That person’s physical attributes transform and ultimately disappear. The name may change. Financial relationships come and go. There is not a single attribute of a person that cannot change; yet the identity persists. Am I the same person I was at age five? This kind of metaphysical question is important in the search for effective domain models. Slightly rephrased: Does the user of the application care if I am the same person I was at age five?”</blockquote><p>There can be different identity with same attributes.</p><blockquote>“Consider transactions in a banking application. Two deposits of the same amount to the same account on the same day are still distinct transactions, so they have identity and are ENTITIES. On the other hand, the amount attributes of those two transactions are probably instances of some money object. These values have no identity, since there is no usefulness in distinguishing them. In fact, two objects can have the same identity without having the same attributes or even, necessarily, being of the same class.”</blockquote><h4><strong>Key Characteristics of an Entity</strong></h4><ol><li><strong>Has a Unique Identity</strong> — Unlike <strong>Value Objects</strong>, which are defined by their attributes, <strong>Entities are distinguished by a unique identifier (e.g., TradeID, UserID).</strong></li><li><strong>Mutable Attributes</strong> — The state of an Entity can change over time, but its identity remains the same.</li><li><strong>Lifespan &amp; Tracking</strong> — Entities have a lifecycle and need to be tracked throughout the system.</li><li><strong>Defined by Identity, Not Attributes</strong> — Two entities with the same attributes are still different if they have different identities.</li></ol><p>At first glance, it may seem that all objects we deal with have some kind of identity, making them <strong>Entities</strong>. So, does that mean most (if not all) objects in a system must be Entities?</p><p><strong>No.</strong> Entities come with <strong>too much liability</strong> — they require careful management of identity and lifecycle. Whenever possible, you should aim to transition an object <strong>away</strong> from being an Entity if it <strong>doesn’t truly require an identity</strong>.</p><p>This brings us to the next key element of <strong>DDD</strong>.</p><h3><strong>Value Object</strong></h3><p>A <strong>Value Object</strong> in DDD is an immutable, identity-less object that represents a <strong>concept</strong> rather than a distinct entity. Unlike entities, which are defined by a unique ID, <strong>value objects are identified by their attributes</strong>.</p><h4><strong>Key Characteristics of Value Objects</strong></h4><ol><li><strong>No Identity</strong> — Two value objects with the same attributes are considered equal (e.g., two Money objects with the same currency and amount).</li><li><strong>Immutable</strong> — Once created, their state cannot change; modifications require creating a new instance.</li><li><strong>Encapsulates Business Logic</strong> — Often used for calculations, validation, or ensuring consistency within the domain.</li></ol><h4><strong>Example</strong></h4><p>A Money object in a financial system:</p><pre>class Money {<br>  constructor(amount, currency) {<br>    this.amount = amount;<br>    this.currency = currency;<br>    Object.freeze(this); // Ensures immutability<br>  }<br>  equals(other) {<br>    return this.amount === other.amount &amp;&amp; this.currency === other.currency;<br>  }<br>}</pre><p>Here, Money is a value object because two Money(100, “USD”) instances are identical regardless of where they are used.</p><p>Value Objects <strong>improve code clarity, maintainability, and correctness</strong>, making them a powerful tool in DDD for <strong>modeling domain logic effectively</strong>.</p><p><strong>Value objects</strong> are well-suited as <strong>attributes</strong> of an identity. For example, a <strong>Person</strong> can be an <strong>Entity</strong>, but their <strong>Name</strong> can be a <strong>Value Object</strong> since it doesn’t require a unique identity.</p><p>Consider <strong>DTOs (Data Transfer Objects)</strong> as <strong>Value Objects</strong> — they are <strong>transient</strong> and don’t need identity management. Similarly, a <strong>CreateUser</strong> object, which is responsible for user creation via a <strong>factory method</strong>, is also a <strong>Value Object</strong>.</p><p>Value Objects can <strong>reference Entities</strong>, as explained in the following quote from the book.</p><blockquote>“VALUE OBJECTS can even reference ENTITIES. For example, if I ask an online map service for a scenic driving route from San Francisco to Los Angeles, it might derive a Route object linking L.A. and San Francisco via the Pacific Coast Highway. That Route object would be a VALUE, even though the three objects it references (two cities and a highway) are all ENTITIES.”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/731/1*vOVeA2yW0K8mdHlwsx2r6A.png" /><figcaption>Route is Value Object &amp; City, Highway are Entities</figcaption></figure><ul><li><strong>City</strong> — Entity (has a unique identity through CityId)</li><li><strong>Highway</strong> — Entity (has a unique identity through HighwayId)</li><li><strong>Route</strong> — Value object because: <br>- Defined by its attributes (Distance, EstimatedTime)<br>- No unique identity — two identical routes would be considered equal<br>- References City and Highway entities to establish context</li></ul><h4>Entity or Value Object? — Context makes the decision</h4><p>In Domain-Driven Design (DDD), the <strong>context</strong> determines whether a concept is modeled as a <strong>Value Object</strong> or an <strong>Entity</strong>. The same object can behave differently in different bounded contexts.</p><p>For example, in a <strong>government system</strong> tracking <strong>name changes and historical records</strong>, a <strong>name</strong> might be modeled as an <strong>Entity</strong> because it requires <strong>versioning, tracking previous names, and associating different name formats with identity proofs</strong>. Here, the name has <strong>a lifecycle and a unique identity</strong> beyond its attributes.</p><p>However, in another system — such as a <strong>customer management system</strong> — a name might be a <strong>Value Object</strong> because it is simply an immutable attribute of a person, without requiring historical tracking.</p><p>Thus, <strong>context is crucial</strong> in determining whether a concept should be treated as a <strong>Value Object or an Entity</strong>, ensuring that the model aligns with the business needs of the domain.</p><p>So, we’ve learned about <strong>Entities</strong> and <strong>Value Objects</strong> — but it’s important to understand that a system consists of many such elements.</p><p><strong>Example in Stock Trading Domain:<br>Entity:</strong> TradeOrder (has OrderID, which uniquely identifies it even if attributes change)<br><strong>Value Object:</strong> Price (if Price changes, it’s a different value, not the same object)</p><p>A system is typically <strong>divided into Modules</strong>, each containing multiple classes and components. What truly makes these elements impactful is <strong>understanding how they relate to one another</strong>.</p><p>This is where <strong>Modeling</strong> comes into play. It explains the domain objects and their relationship. Below is example model where we related entity and value object</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*DbGDx6TGJ5BJdCDE8TmQIg.png" /><figcaption>TradeOrder is Entity &amp; Price is Value Object</figcaption></figure><p>In the context of Domain-Driven Design (DDD), understanding <strong>Entities and Value Objects</strong> allows us to model the <strong>static aspects</strong> of a system — these are the <strong>persistent domain objects</strong> that define the structure of our domain. If you examine a typical database, every table and row represents either an <strong>Entity</strong> (with a unique identity) or a <strong>Value Object</strong> (defined by its attributes).</p><p>These objects persist beyond the runtime of the system; even if the system is inactive, they <strong>remain stored in a database</strong> or another form of persistent storage.</p><p>However, a complete domain model isn’t just about static objects. This leads us to the <strong>dynamic part of the system — Domain Services</strong>. Services encapsulate <strong>business logic that doesn’t naturally fit within an Entity or Value Object</strong>, enabling <strong>behavior-driven interactions between domain elements</strong>.</p><h3><strong>SERVICE</strong></h3><p>In DDD, a <strong>Service</strong> is a domain layer construct that encapsulates <strong>business logic</strong> that doesn’t naturally belong to an <strong>Entity </strong>or <strong>Value Object</strong>. Unlike entities, services <strong>don’t have state or identity</strong>; they operate on domain objects to perform domain-specific operations.</p><p>Don’t confuse <strong>Domain Services with Microservices</strong>. A Domain Service is stateless and applies business logic to raw values from domain objects. In contrast, a Microservice represents a complete, independent service that communicates with other services or objects over web protocols like HTTP or TCP.</p><p>Eric Evans strongly emphasizes <strong>first attempting to encapsulate behavior within an Entity or Value Object</strong> whenever possible. By doing so, the operation gains a <strong>clear owner</strong>, potentially becoming an <strong>attribute or method</strong> rather than an independent service. This approach <strong>simplifies the domain model</strong>, keeping behavior closely tied to the relevant domain object and allowing developers to focus on more complex aspects of the system.</p><p>However, when a <strong>business operation, transformation, or rule does not naturally belong to a single Entity or Value Object</strong>, it becomes a strong <strong>candidate for a Domain Service</strong>.</p><h4><strong>Characteristics of a Service in DDD</strong></h4><ol><li><strong>Stateless</strong> — Does not hold persistent state, only operates on domain objects.</li><li><strong>Encapsulates Business Logic</strong> — Contains complex logic that spans multiple entities.</li><li><strong>Independent from Entities</strong> — Provides behavior that doesn’t fit into any single domain object.</li></ol><p>By carefully <strong>distinguishing between behavior that belongs in an Entity/Value Object versus a Service</strong>, the domain model remains <strong>clean, maintainable, and aligned with business logic</strong>. Services <strong>keep the domain model clean</strong>, ensuring that <strong>Entities and Value Objects remain focused on their core responsibilities</strong> while complex logic is handled in a dedicated, reusable way.</p><pre>class CurrencyConversionService {<br>  convert(amount, fromCurrency, toCurrency) {<br>    // Fetch exchange rate and perform conversion logic<br>    return new Money(amount * this.getExchangeRate(fromCurrency, toCurrency), toCurrency);<br>  }<br>}</pre><p>A <strong>Service</strong> can operate at any layer of the system or across multiple layers simultaneously.</p><p>Eric Evans provides a great example of a <strong>service interacting with all three domains</strong>. (See the attached image.) A single service can span the <strong>Domain, Application, and Infrastructure layers</strong>, ensuring seamless coordination between them.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*EpMc_dxIWGqZfd3qis9CQw.png" /><figcaption>Service interacting with all three domains</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/925/1*jQyocpdo3O2sXe2uyrG2BQ.png" /><figcaption>Service interacting with all three domains</figcaption></figure><h3>Module (aka. Packages)</h3><p>Now that we have covered <strong>Entities, Value Objects, and Services</strong>, our domain model is starting to take shape. However, to build a truly <strong>manageable and scalable ecosystem</strong>, we need a <strong>high-level organizational structure</strong> that categorizes these domain objects effectively.</p><p>This brings us to the <strong>fourth key element of DDD — Packages (or Modules).</strong></p><blockquote>“MODULES give people two views of the model: They can look at detail within a MODULE without being overwhelmed by the whole, or they can look at relationships between MODULES in views that exclude interior detail.”</blockquote><p>Packages help in <strong>grouping related domain objects</strong> within bounded contexts, ensuring that <strong>Entities, Value Objects, and Services</strong> remain well-structured and easy to maintain. By defining <strong>clear module boundaries</strong>, we enhance the <strong>cohesion</strong> of related components while minimizing unnecessary dependencies, leading to a <strong>more maintainable and scalable domain model</strong>.</p><h4><strong>Key Characteristics of Modules</strong></h4><ol><li><strong>Encapsulation</strong> — Groups related domain objects, ensuring that internal details remain hidden from other modules.</li><li><strong>Separation of Concerns</strong> — Helps break down complex domains into smaller, more manageable units.</li><li><strong>Improved Maintainability</strong> — Prevents a monolithic domain model by creating logical divisions.</li><li><strong>Clear Boundaries</strong> — Supports bounded contexts by ensuring domain concepts don’t bleed into unrelated areas.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/903/1*LwNcg0XbI_EL6pWcjuuq0Q.png" /><figcaption>Each component represents a separate module or package</figcaption></figure><p>Module should be loosely coupled and highly cohesive. Modules are good to define the context.</p><blockquote>“When you place some classes together in a MODULE, you are telling the next developer who looks at your design to think about them together. If your model is telling a story, the MODULES are chapters. The name of the MODULE conveys its meaning. These names enter the UBIQUITOUS LANGUAGE. “Now let’s talk about the ‘customer’ module,” you might say to a business expert, and the context is set for your conversation.”</blockquote><p>Each <strong>module</strong> operates independently but interacts with others via <strong>well-defined APIs</strong>, ensuring <strong>scalability</strong> and <strong>fault tolerance</strong> in the system. <strong>DDD principles help keep the model aligned with business logic</strong>, making the system <strong>adaptive and resilient</strong>.</p><h4>How Module Communication Works</h4><p>Communication between modules or packages isn’t freeform — it <strong>follows strict standards</strong>. Each module should be independently developed and deployed, with versioning in place to avoid conflicts. To achieve this, modules must <strong>communicate through well-defined interfaces</strong> using a shared protocol. Different teams can work on separate modules without needing to understand the inner workings of others, as long as they follow the <strong>agreed communication standards</strong>.</p><p>The <strong><em>Clean Architecture</em></strong> book emphasizes this principle. Over time, these modules can evolve into independent microservices, <strong>introducing additional constraints</strong>. Communication will shift from <strong>direct in-process calls to network-based protocols</strong> like TCP, HTTP, or asynchronous messaging via a message broker.</p><h3>Conclusion</h3><p>Now that we have explored the foundational building blocks of <strong>Domain-Driven Design (DDD)</strong> — <strong>Entities, Value Objects, Services, and Modules</strong> — we have established a strong understanding of the core domain modeling principles.</p><p>Next, we move into <strong>more advanced concepts</strong>: <strong>Aggregates, Factories, and Repositories</strong>, which help in structuring complex domains, managing object creation, and ensuring consistency across the system.</p><p>That concludes this chapter. In the next one, we will <strong>dive deeper into these advanced concepts</strong> — stay tuned!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=42b45146334e" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/mastering-software-design-with-the-power-of-domain-driven-design-clean-architecture-chapter-1-42b45146334e">Mastering Software Design: With the Power of Domain-Driven Design &amp; Clean Architecture -Chapter 1</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[3-Step Framework to Learn Almost Any Subject with ChatGPT]]></title>
            <link>https://levelup.gitconnected.com/3-step-framework-to-learn-almost-any-subject-with-chatgpt-60ff309783f0?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/60ff309783f0</guid>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[framework]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[learning]]></category>
            <category><![CDATA[chatgpt]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Wed, 15 Nov 2023 14:46:58 GMT</pubDate>
            <atom:updated>2023-11-15T14:46:58.973Z</atom:updated>
            <content:encoded><![CDATA[<h3>3-Step Framework To Learn Almost Any Subject With ChatGPT</h3><h4>It helps learners formulate meaningful questions for ChatGPT, fostering curiosity-driven learning in specific subject areas.</h4><p>The problem I often observe among many students and learners is their difficulty in formulating questions for ChatGPT. This framework is designed to help you overcome that barrier. ChatGPT will generate questions for you, and these won’t be random queries about a given subject or field. Instead, they will be carefully curated questions within the specific branch of the subject you want to explore further. You can spend an entire day and night navigating through the various steps of this framework, and you’ll never tire of learning because it offers an intuitive and curiosity-driven approach to education.</p><p>The example I’ve provided in this essay is from a computer science background, simply to demonstrate the technicality of this framework. You can apply it to any subject in the world that you want to learn. Personally, I have a great fondness for physics, and it has greatly assisted me in gaining a deeper understanding of various subjects.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3KL29chIiiVClmwAuwyabg.jpeg" /><figcaption>Image created with DALL-E</figcaption></figure><h4><strong>Step 1 — Provide a subject name and request its chapters with associated weightage:</strong></h4><p>Use below prompt and request different chapters of the given subject with weights assigned to each chapter. ChatGPT will provide a response listing all the fundamental chapters of that subject along with their descriptions. This will serve as your starting point, and assigning weights to each category will help you prioritize your study of the subject.</p><p><strong>Prompt:</strong><em> Act as a Subject Index. I will provide you with a specific subject name, and your task is to provide me with the chapters (along with descriptions) that need to be learned to excel in that subject. Please include as many different chapters as possible. Additionally, inform me of the weightage of each chapter, which indicates the level of importance of that category within that particular subject. I will supply you with subject names, and you will respond as requested. I will ensure that I only provide Subject Names without engaging in any other chit-chat.</em></p><p>Ensure that the subject name is descriptive enough so that ChatGPT can understand.</p><p><strong>Here are some examples:</strong> <em>Software Architect, PMP (Project Managment Professional), AWS SAA, Backend Engineering in Software Development etc…</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/765/1*2vgeN9r55BPeopjmGG7g0A.jpeg" /><figcaption>Image Credit: ChatGPT</figcaption></figure><h4><strong>Step 2 — Specify a chapter name and request to engage in a continuous examination round:</strong></h4><p>Use below prompt and instruct ChatGPT to administer an exam for a specific chapter. You will select the chapter names from the above prompt response. If the chapter name is self-explanatory, use only the chapter name. If not, provide it with the subject name as well.</p><p>ChatGPT will then proceed to ask you questions one by one, and you are required to select the correct option. Subsequently, it will provide an explanation for the correct answer.</p><p><strong>Prompt:<em> </em></strong><em>Act as an Examiner for the topic: {topic_name} of {subject_name}. You will present topic-related questions, each with four answer options, one at a time. After I provide an answer by selecting one option, you will then present the next question. Please ensure that you provide a detailed correct answer before proceeding to the next question. Additionally, keep track of my score: a correct answer earns me +1 mark, while a wrong answer results in a -1 mark deduction. Please mention my total score before moving on to the next question. Do not cease asking questions until I explicitly request you to do so. You should also note that I am well-versed in this topic, so feel free to challenge me with the most difficult questions you can find.</em></p><p><strong>Here are few examples:</strong> System Design and Modeling for Software Architect<strong>, </strong>Project Scope Management of PMP, Design Cost-Optimized Architectures of AWS SAA, Databases and Data Modeling etc…</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/787/1*qJTo9QMu32GpbaK91C43Mw.jpeg" /><figcaption>Image Credit: ChatGPT</figcaption></figure><h4><strong>Step 3 — Establish separate threads to study each chapter, allowing your future self to conduct rapid revisions:</strong></h4><p>Strive to provide correct answers. If you’re unsure or not familiar with an answer, copy the same question that ChatGPT asked you and paste it into a different thread. Alternatively, rephrase the questions to ensure you understand the entire concept, including the provided options and how they relate to the correct answers.</p><p>Comparing different options with each other can help you reason and make sense of the right answer. It’s crucial to ensure that you have a comprehensive understanding of the entire concept before moving on to the next question.</p><p><strong>Here are few examples:</strong> What is the primary role of a software architect in the software development process?, What is the concept of “float” in project scheduling, and how does it affect project scheduling and time management?, etc….</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/785/1*_dP7JcGzRc2-7FUqIDSa-Q.jpeg" /><figcaption>Image Credit: ChatGPT</figcaption></figure><h4>If you are not yet bored, read further to know the benefits of this framework:</h4><ol><li>All you need to do is provide a subject name for the area you want to improve or increase your knowledge in. ChatGPT will take care of the rest.</li><li>It creates an intuitive way of learning. You’ll always be eager for the next question. <strong>Knowing the question first, contemplating the correct answer, and then learning about it makes the process highly intuitive.</strong></li><li>Make sure you save or export all the threads to a secure place. This will enable you to conduct quick revisions in the future.</li><li>Reflect on the weightage of each category to determine its priority.</li><li><strong>Be creative with your choice of subject names.</strong> I open Udemy and use different course names as subject names to input into ChatGPT. It is working really good for me. I am always curious to learn physics, I have learned so many new subject with this framework. e.g. String theory, Quantum physics etc…</li><li>If you come across new topics or subjects that evolve rapidly, <strong>create a separate thread for Bing search using the co-pilot feature in the Microsoft Edge browser.</strong> So far, it has been the most effective method for researching such topics online. It provides better results with original article links.</li><li><strong>While exploring answers for any topic, you may encounter a keyword that could be a subject itself. Feel free to ask for chapters of that subject from Step 1.</strong> This way, you can create a never-ending loop that will expand your latticework of knowledge.</li></ol><p>Alway Remember.. key to profound learning is asking the right questions!</p><blockquote>“If I had an hour to solve a problem and my life depended on the solution, I would spend the first 55 minutes determining the proper question to ask for once I know the proper question, I could solve the problem in less than five minutes,” — Einstein.</blockquote><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60ff309783f0" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/3-step-framework-to-learn-almost-any-subject-with-chatgpt-60ff309783f0">3-Step Framework to Learn Almost Any Subject with ChatGPT</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How to Apply Creational Design Patterns to Craft Flexible Software]]></title>
            <link>https://levelup.gitconnected.com/how-to-apply-creational-design-patterns-to-craft-flexible-software-2caa09e540ed?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/2caa09e540ed</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[design-patterns]]></category>
            <category><![CDATA[software-engineering]]></category>
            <category><![CDATA[domain-driven-design]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Tue, 10 Oct 2023 17:28:15 GMT</pubDate>
            <atom:updated>2025-03-28T01:58:07.234Z</atom:updated>
            <content:encoded><![CDATA[<h4>A 6-Step Refactoring Process with a Case Study to Explore Design Patterns: Factory, Abstract Factory, Builder.</h4><h4><em>Creating clean designs ultimately boils down to an intuitive approach</em></h4><p>To make your code clean, you begin by relocating its noise elsewhere. Then, you patiently wait until you detect any signal within that noise. As soon as you identify it, you filter out that signal and relocate the remaining noise elsewhere. You continue this process, progressively moving the noise further away from the callee to the caller. Ultimately, it’s the client or the client’s client who deals with that noise until they discover patterns and clean it up by relocating its noise elsewhere.<br><strong>If this sounds a bit abstract, let’s dive deep to learn it step by step.</strong></p><h4><strong>What you will learn from this blog?</strong></h4><ul><li>How creational design patterns can effectively separate the <em>hard-wiring</em> of your code from your core business logic.</li><li>How various creational design patterns are interconnected and which pattern is suitable for different stages of your software’s evolution.</li><li>How to leverage abstraction as a tool to introduce discipline into your procedural code.</li><li>Recognizing the right time to refactor your code and understanding the role of the business domain in making key decisions.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*PKcwpAN8Z7E1mDiTDvNyTQ.png" /><figcaption>Mastering creational design patterns for effective code evolution</figcaption></figure><p>Below is the quote from Eric Evans’ “Domain-Driven Design” book — It is a profound analogy that beautifully illustrates the importance of separating the creation of objects from their actual use. It’s bit long quote but you have to give your full attention as it holds the essence of creational designs patterns.</p><blockquote>A car engine is an intricate piece of machinery, with dozens of parts collaborating to perform the engine’s responsibility: to turn a shaft. One could imagine trying to design an engine block that could grab on to a set of pistons and insert them into its cylinders, spark plugs that would find their sockets and screw themselves in. But it seems unlikely that such a complicated machine would be as reliable or as efficient as our typical engines are. Instead, we accept that something else will assemble the pieces. Perhaps it will be a human mechanic or perhaps it will be an industrial robot. Both the robot and the human are actually more complex than the engine they assemble. The job of assembling parts is completely unrelated to the job of spinning a shaft. The assemblers function only during the creation of the car — you don’t need a robot or a mechanic with you when you’re driving. Because cars are never assembled and driven at the same time, there is no value in combining both of these functions into the same mechanism. Likewise, assembling a complex compound object is a job that is best separated from whatever job that object will have to do when it finished.</blockquote><blockquote>— Eric Evans</blockquote><p>Once you understand that car driving is not same as car manufacuring, <strong>hence the object your program consume doesn’t care who creates it,</strong> you’ll be ready to grasp the concept of factory methods. Without this fundamental understanding, using creational patterns might seem confusing or overwhelming.</p><h3>Introduction</h3><p>One of the fundamental and perhaps the most crucial aspects of any software is its maintainability. <strong>It is a key attribute of software quality and is essential for the long-term viability and cost-effectiveness of software applications.</strong> To ensure that your software lasts for decades and remains flexible and extensible, low-level design plays a pivotal role.</p><p>Design patterns are the backbone of low-level software design. They were introduced by the Gang of Four (GoF) in the 1990s and have become some of the most popular design patterns in the software development world. In this blog, we will delve into creational design patterns, accompanied by interesting examples to help you better comprehend them.</p><figure><img alt="“What is nice about this pat­terns is that it offers reuse in a way that is less risky than frame­works. Build­ing a frame­work is high-risk and a sig­nif­i­cant invest­ment. Pat­terns let you reuse design ideas and con­cepts inde­pen­dent­ly of con­crete code.”" src="https://cdn-images-1.medium.com/max/940/1*W6f1VwKU-z4vfqYvJ-9T-g.png" /></figure><p>According to the Gang of Four, there are five creational design patterns. In this blog, we will focus on learning three of these design patterns, as the remaining two are relatively easier to grasp. So, let’s dive in and gain a deeper understanding of these essential concepts.</p><h3>Our Case Study</h3><p>Let’s consider the HR Payroll system of an IT organization as a sample software to explore these design patterns. They use this system for managing the payroll of its employees. This system encompasses various modules, as you would find in any other payroll system. However, for the purpose of this blog, we will concentrate on the module responsible for calculating employee incentives.</p><p><strong>We’ll start our low-level design with the simplest architecture and gradually progress to more complex structures.</strong> <br>Throughout this journey, we will document our decision-making process and ultimately arrive at an advanced architecture that adheres to clean architecture principles.</p><h3>Level 0</h3><p>Our Incentive Module begins (and ends) with just one file, which we’ll refer to as `service.xy` (where `xy` represents the file extension of your favorite programming language). It’s important to note that this service file handles more than just calculating incentives; clients will utilize this service for various other purposes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/507/1*L2w66L9pqistAeOovxEjMw.png" /></figure><p>In above figure, you can observe a method named `getIncentiveOf`, which takes one argument called `employee_id`. Based on this employee_id, the method determines the respective incentive for the employee as of the current date. Clients will invoke this method to retrieve the incentive amount for that employee in the current year.</p><p>While this may seem straightforward at a high level, let’s take a closer look at what’s happening within this method. <strong>Calculating incentives is far from easy; it involves intricate business logic and numerous parameters.</strong> It’s crucial to segregate the concerns of each step in this calculation to keep them <em>Independent</em> and <em>Testable</em>. The last thing anyone wants is to compromise the integrity of the payroll system of any organisation. So, let’s get serious about low-level design and take it to the next level.</p><h3>Level 1</h3><p>The most obvious step is to separate our Incentive services into different file. Clients will create an object of the Incentive class and make calls to find the incentive. This approach allows us to encapsulate the incentive calculation logic into separate methods, making the code more modular and easier to maintain. It would look something like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/556/1*AjcJs-osHkyOl5JBfIbf-w.png" /></figure><h3>The Bigger Picture</h3><p>To provide some context, let’s consider this module as a single-tier web application where we have an `IncentiveController` that requests the incentive of a particular employee to pass it on to another system. This client, acting like our “postman”, is responsible for executing the work. In essence, we don’t concern ourselves with what they will do with this information; <strong>our sole focus is to determine the incentive of that employee. To find this incentive, the controller needs to connect to a database, retrieve the required information, and process it.</strong></p><p>As mentioned earlier, calculating incentives is a complex task involving various parameters. Some of these include employee attendance, which directly impacts the incentive. Additionally, there may be project performance data, Number of lines of code committed by a developer and contributions to open-source libraries. For the HR department, factors like employee retention come into play. It’s important to note that all this information may reside in different databases, and some data might even need to be fetched from third-party services.</p><pre>func getIncentiveOf(employee_id) -&gt; Rupees {<br> // Connect to the database<br> // Retrieve the minimal required information of this employee and their KPI from the database.<br> // Depending on certain conditions, we may need additional information from the database.<br> // The incentive calculation algorithm varies based on department and experience.<br> // We might need to connect to different databases to obtain specific data.<br> // We might need to connect to third-party services to acquire certain data.<br> // Process all the parameters <br> // Provide the current year&#39;s incentive to the client<br> }</pre><p>Additionally, there can be customized settings for individual employees. <strong>However, as this blog focuses on design patterns rather than payroll system specifics, let’s abstract these details as KPIs (Key Performance Indicator). Whenever we mention KPIs, please consider one of the conditions mentioned above.</strong> This approach allows us to place a stronger emphasis on the concepts of design patterns.</p><h3>Level 2</h3><p>Now it’s time to divide the `Incentive` class into different classes. One obvious way to do this is to group them by their KPIs. Each KPI functions differently and has its own unique settings. Creating separate KPI classes for each KPI is a sound approach. This not only makes each KPI independently testable but also ensures that if the software owner requests changes to a specific KPI calculation, we can make those changes without affecting other KPIs.</p><p>The `Incentive` class will create objects of each KPI class, request incentives for each KPI, and then sum up all the incentives to send to the client. The final structure would look something like this.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*MyLlaK4WK3Knjc0Hgu3_Tw.png" /></figure><p>It’s important to note that the `Incentive` class needs to first identify all the KPIs applicable to a particular employee. Based on this list of KPIs, the `Incentive` class will select the appropriate KPI class to request the incentive from.</p><h3>Level 3</h3><p>As you can see, we have now divided the logic into different classes and clearly defined each one’s responsibility. However, that’s not the end of our design journey. If we look inside each KPI, we encounter another condition: the employee’s department. KPI standards can vary depending on the department, and these variations affect the incentive system differently. <strong>It’s time to further separate the logic based on departments. So, we will create classes for each KPI according to the departments within the organization.</strong> The structure might look something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*97l8NO1zV1W5FVkxDMtxVQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/927/1*VWZXsGPp-9gS8lybKy035w.png" /></figure><h4>To summarize how things work</h4><ol><li>When a client calls `Incentive` with an `employee_id`, `Incentive` fetches the list of applicable KPIs for that employee.</li><li>It then makes a call to each KPI class. While calling a specific KPI class, it checks the employee’s department and, based on that, creates an object of the appropriate `KPI+Dep` class and calls its method to calculate the incentive for that KPI.</li><li>The calculated value is then passed back to the `Incentive` class. `Incentive` class will sum up incentive amount from each KPI and give that to Client.</li></ol><h3>Level 4</h3><p>At this stage, we can identify some issues with our system. We’ve ended up with many different classes that essentially perform the same task at a high level but with different implementations. <strong>It’s time to introduce abstraction to bring some discipline into our design.</strong></p><p>Furthermore, there are numerous dependencies between classes, and the code isn’t as modularized as it should be. We’ve created a chain of object creation where each class depends on the previous one. To ensure our code is more modular and extensible for future additions of new KPIs, we should introduce the <em>Creational Design Patterns</em>.</p><p>The creational design pattern simplifies object creation by centralizing it in a Factory method. This method takes responsibility for creating the appropriate objects, relieving intermediate classes from making those decisions. In our case, the KPI classes were responsible for checking the employee’s department and creating the corresponding object. <strong>This approach is not sustainable for the long-term durability of the software.</strong></p><p>Another advantage of introducing the Creational pattern is that it reduces dependencies. Previously, all objects were aware of each other, creating a chain of object creation. With the Factory method, it handles all the wiring, eliminating the need for intermediate classes to perform this task.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*AT7N6qa8E80ZFl8qkvhAbw.png" /></figure><h4>Our desired end goal is to have the appropriate `KPI+Dep` class provided to the `Incentive` class.</h4><p>The `Incentive` class shouldn’t concern itself with the concrete classes it’s working with. To achieve that, we’re moving the logic for creating all objects back to the Factory. It will check the employee’s department and create the respective object, passing it to `Incentive`. Now, the `Incentive` class doesn’t need to know about the concrete objects it’s working with.</p><p>In the new design, `Incentive` class will simply call the `findKPIIncentive(type)` method to obtain the appropriate KPI object. These KPI objects will have a `getIncentive` method to provide the incentive for that specific KPI.</p><p>Instead of having the client use a single `Incentive` object to fetch employee incentives, it would be more organized for the client to create different objects for different requests. This approach ensures that we don’t encounter shared properties that could potentially corrupt our data. This aligns with the programming notion of <em>Pure Functions</em>, where one function, for a given input, produces the same output.</p><p>It’s worth noting that the logic regarding how many KPIs exist and what each individual KPI’s incentive calculation entails is still within the `getIncentiveOfEmployee` method. For now, we’ll leave it there as it’s not part of the isolation process we’re currently focusing on.</p><h3>Realization on Creational Design Patterns</h3><p>This scenario exemplifies the typical use case of a Creational Pattern. It extracts all the creation logic from other business logic, providing us with the respective object we need. In the example above, the `Incentive` class, given an `employee_id` and the type of KPI required, relies on the Factory to determine the correct object to pass back to the `Incentive` class.</p><p>The `Incentive` class treats all KPIs uniformly, without going into the concrete classes. This design choice makes the system highly extensible.</p><h4>If you introduce a new KPI with either complex or straightforward logic, you can follow these steps:</h4><ol><li>Create a `KPI+Dep` class for the new KPI and ensure it conforms to the KPI interface.</li><li>Add support for the new KPI type to the Factory method. This type will be passed by the `Incentive` class.</li><li>In the `Incentive` class, maintain a list of KPI names/types, and extend it to accommodate the new KPI type.</li></ol><p>This approach ensures that your system can easily adapt to changes and additions of new KPIs, making it more robust and maintainable in the long run.</p><h3>Level 5</h3><p>Everything appears to be going well so far. <strong>We’ve realized that factories are like mysterious black boxes where you provide minimal information and, in return, receive the objects you need.</strong> However, these factories tend to contain a lot of procedural code. As we continue to add more KPIs, this code complexity can increase significantly. It’s a good idea to introduce some discipline within these classes by incorporating abstraction, which we’ll refer to as the <em>Abstract Factory Pattern</em>.</p><h4><strong>What do I mean by “procedural code,” let me elaborate</strong></h4><p>Our factory isn’t merely checking the department and creating the respective department concrete class; it’s doing much more than that. Each department KPI requires some properties that still need to be initialized. While it’s possible to initialize these properties inside the KPI class, why should we do that? Instead, we can demand a fully configured class from the factory, so all the KPI class needs to do is run the operation, retrieve data, and perform business logic. Thus, the factory has a lot more going on, depending on the department KPI class it needs to create. To better understand this, check the code snippet inside the factory method:</p><pre>func createKPIofType(type: String) -&gt; KPI {<br> // Check for KPI type and initialize the corresponding family of KPI classes.<br> // Check for employee department to initialize the family of KPI classes.<br> // Check for employee experience to initialize the properties of a particular KPI class.<br> // Check for different third-party service and different database connections required to initialize a particular KPI with specified properties.<br> // The returned KPI class will be fully configured and ready to be consumed by the Incentive class. So, every complication related to creating the KPI class is handled in this factory method.<br>}</pre><p>As you can see in the below figure, rather than keeping the creation logic inside the factory class, we’ve introduced abstraction in the form of an interface called `AbstractFactory`. This interface is implemented by each department-specific factory.</p><p>A more effective approach is to have the `Factory` as an abstract class and the department factories as concrete classes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/772/1*74UhIO0fS49rVsLSxjGejQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0qBdRFAjZFha-3XAPIbBFw.png" /></figure><h4><strong>How do we determine which factory to create in the `Incentive` class?</strong></h4><p>It’s better to let the client decide this. The client will pass the respective Factory object to `Incentive`, and the `Incentive` class will use this factory to create KPI objects. The end result looks something like this:</p><p>In essence, we’ve divided the creation logic for all KPIs department-wise. This means that each of our concrete factory classes only has to deal with one department, providing us with a cleaner, more focused playground for working with each department at a time.</p><p><strong>This level of abstraction and separation of concerns greatly enhances the maintainability and scalability of our system</strong>, making it easier to add new KPIs and departments in the future.</p><h3>Level 6</h3><p>Moving forward, let’s dive into the <em>Builder Pattern</em>. Imagine we encounter a bit more complexity in our incentive calculation. <strong>Specifically, different algorithms are applied based on the employee’s experience.</strong> For example, if a department employee has more than 5 years of experience, we need to consider different properties of KPI classes. Currently, this logic resides within the Factory KPI classes, which introduces some procedural code. It would be beneficial to introduce some structure to handle this.</p><p>In this setup, the `Incentive` class will hold a reference to the `Director` class. `Incentive` will initialize the `Director` with an `AbstractFactory` object provided by the Client. Depending on the experience of the employee, the `Director` will provide the appropriate KPI objects. The end result will resemble this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IG6Gk0xEWVyJGYURgz3Wog.png" /></figure><p>All of the procedural logic is now contained within the `Director` class. The `createKPIofType` method will look something like this:</p><pre>func createKPIofType(type: String) -&gt; KPI {<br> if (type == &quot;kpi1&quot;) {<br>   if (isSerniorProfile()) {<br>     return makeKPI1forSenior()<br>   } else {<br>     return makeKPI1forJunior()<br>   }<br> } else if (type == &quot;kpi2&quot;) {<br> // Repeat the same steps for other KPI types…<br> } else if (type == &quot;kpi3&quot;) {<br> // …<br> }<br> // Handle other KPI types…<br>}</pre><p>Now, the Factory classes are primarily responsible for constructing individual KPI objects for senior or junior employees. If we introduce new KPIs or need to modify the existing logic for current KPIs, the Factory class owners will know where to make changes without altering the wiring or structural aspects.</p><p>By incorporating the Builder pattern in our design, we’ve further enhanced the maintainability and flexibility of our system, particularly in handling different algorithms based on employee experience.</p><h3>Summary</h3><p>We started on a journey from a simple and minimal architecture to a complex one. Along the way, we made crucial design decisions aligned with our business goals. <strong>These decisions can’t be made in isolation, it should always align with nature of software or buisness needs.</strong></p><p>For the same HR payroll system, different decisions would have been made if our business needs were different (In this case, our logic for calculating incentives differed). Keep in mind that needs may change over time, but the right decisions will always provide room for changes with ease.</p><h4><strong>Let’s examine these decisions</strong></h4><p>We grouped business classes by KPI, then further grouped them by employee department, and finally, these KPI+Department combinations were categorized by years of experience. <br>This is a business-driven choice, and while it remains adaptable, such decisions are something we can abstract and discuss with domain owners who have a clear understanding of potential future modifications to the incentive calculation formula.</p><h4><strong>What if you realize that two KPIs are very similar, with only minor logic differences?</strong></h4><p>Would you consider combining them into one class? The answer is NO. This principle aligns with the <em>Single Responsibility Principle</em>. While you may be tempted to group them by logic, remember that business rules take precedence. In the future, changes may widen the gap between these two KPIs or confuse new developers as architecture rules are not kept the same across the software. So, choose your designs diligently.</p><p>As we impose discipline in our code, <strong>it aids in scaling our software development.</strong> By creating more isolated components, we can have separate development teams work on specific logic without concern for the bigger picture. Consider any significant open-source software project, and you’ll find hundreds of developers in their isolated rooms.</p><p>At each level, we were picking some kind of hard-wiring logic and throwing it out of our closed system. Of course somebody has to be owner of that and there is no right place then factories. <strong>To make any system independant — provide them with data which they work upon to get back the output.</strong> How they acquire that data? They don’t care; it’s their client’s responsibility to provide the right data.</p><p><strong>Abstractions are the means to transform procedural code into disciplined code.</strong> Objects dealing with these abstracted classes don’t need to be aware of implementation details. When you realize this, you’ll see that all design patterns essentially serve the same purpose: providing different ways to abstract and structure code.</p><p>Let’s end this article with a quote from Christopher Alexander, who inspire Eric Gamma to make such beautiful book on Design Patterns.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/940/1*ZyeTKuyven4-I-mXOGnWZw.png" /></figure><p><em>About me: </em><a href="https://www.linkedin.com/in/jigar-patel-2a26b560/">I’m Jigar</a>, a software designer at <a href="https://www.yudiz.com/">Yudiz Solutions</a>. I’m an avid reader and have a keen interest in exploring the IT industry. During my free time, I explore different topics of Theoretical Physics.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2caa09e540ed" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/how-to-apply-creational-design-patterns-to-craft-flexible-software-2caa09e540ed">How to Apply Creational Design Patterns to Craft Flexible Software</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Standing on the shoulders of Giants]]></title>
            <link>https://levelup.gitconnected.com/standing-on-the-shoulders-of-giants-03a2df0e0768?source=rss-796cb6be2ade------2</link>
            <guid isPermaLink="false">https://medium.com/p/03a2df0e0768</guid>
            <category><![CDATA[science]]></category>
            <category><![CDATA[javascript]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[physics]]></category>
            <category><![CDATA[history]]></category>
            <dc:creator><![CDATA[Jigar Patel]]></dc:creator>
            <pubDate>Tue, 10 Oct 2023 17:27:09 GMT</pubDate>
            <atom:updated>2023-11-22T03:59:03.858Z</atom:updated>
            <content:encoded><![CDATA[<h3>Standing on the Shoulders of Giants</h3><h4>From Physics to Programming to Human Nature..</h4><p>Sir Isaac Newton, the renowned physicist and mathematician, in a letter to fellow scientist Robert Hooke. Newton wrote this phrase in a letter to Hooke in 1676. The letter goes as follows:</p><p>“If I have seen further it is by <strong>standing on the shoulders of Giants.</strong>”</p><p>The context of this statement is important. Newton was acknowledging that his scientific discoveries and advancements were made possible by building upon the work of those who came before him. In essence, he was expressing gratitude to the scientists and thinkers who had laid the foundation for his own groundbreaking discoveries. Newton leveraged the work of Galileo, Kepler, René Descartes, and Robert Hooke through his lifetime.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*05Rv5Ro4oLVxf66XIOXDWQ.jpeg" /><figcaption>Image created with DALL·E</figcaption></figure><p>Now, let’s take a look at the world around us. <strong>How much have we been leveraging the giants to see further?</strong></p><p><strong>Here is one case study with an example of JavaScript Runtime.</strong> Later, we will come back to these great scientists, I promise.</p><h4><strong><em>February 8, 2005</em></strong></h4><p>Google Maps was officially launched by Google. JavaScript engines of that era were relatively slow in executing code, which affected the performance of web applications, including mapping services.</p><p>Google needed a fast JavaScript engine to make Google Maps responsive and efficient. Google had already started making their own JavaScript Engine (later named V8).</p><p>However, it did not start from scratch. <strong>V8 initially began as a project to improve the JavaScript engine used in WebKit</strong>, the open-source rendering engine that powers Apple’s Safari browser. Google aimed to contribute its improvements to the WebKit project.</p><h4><strong><em>September 2, 2008</em></strong></h4><p>Google Chrome was officially launched. It was introduced as a web browser designed to provide a faster, more efficient, and secure browsing experience.</p><p>It was introduced with the V8 JavaScript engine. This was a significant milestone in JavaScript engine development, as it introduced a high-performance just-in-time (JIT) compiler. This compiler dramatically improved the execution speed of JavaScript code, making web applications faster and more responsive.</p><p>Although V8 was developed by Google, at the time of the launch, <strong>Chrome was still using WebKit as its Browser Rendering Engine</strong>.</p><h4><strong><em>May 27, 2009</em></strong></h4><p>In less than a year after Chrome’s launch, Node was introduced by Ryan Dahl, <strong>leveraging the power of V8</strong>. Node.js is one of the most well-known and widely used server-side JavaScript runtimes. It allows developers to build scalable and high-performance network applications.</p><h4><strong><em>April 3, 2013</em></strong></h4><p>Google announced <strong>the Blink rendering engine, a fork of WebKit</strong>. This marked a significant shift away from WebKit as the core rendering engine for Chrome. Blink was created to allow Google more control over the development and optimization of the rendering engine.</p><h4><strong><em>July 29, 2015</em></strong></h4><p>Microsoft Edge was introduced. This web browser was developed by Microsoft to replace Internet Explorer and is designed to be faster, more efficient, and more feature-rich than its predecessor.</p><p>Microsoft Edge used a new rendering engine called EdgeHTML and a new JavaScript engine called ChakraCore. ChakraCore was the open-source version of the Chakra engine, and it was designed to be more lightweight and adaptable.</p><p><strong>ChakraCore was borrowed from Internet Explorer</strong>, which had been using it since 2009.</p><h4><strong><em>January 15, 2020</em></strong></h4><p>Microsoft made a significant change by adopting the Chromium rendering engine, which is used in Google Chrome, for its Edge browser. <br>This transition resulted in the <strong>abandonment of EdgeHTML and ChakraCore in favor of Chromium’s Blink rendering engine and V8 JavaScript engine.</strong></p><h4><strong><em>May 13, 2020</em></strong></h4><p>Eleven years after Node.js, Deno was introduced by Ryan Dahl, the same individual who created Node.js. (Deno was officially announced and released as an open-source project in June 2018).</p><p>It gained attention as a potential successor or alternative to Node.js, addressing some of the design and security issues that had become apparent in the earlier Node.js runtime. <strong>Deno was leveraging the same V8 engine as Node.js.</strong></p><h4><strong><em>September 22, 2023</em></strong></h4><p>Bun 1.0 was introduced by Jarred Sumner. Bun is a new JavaScript runtime that is designed to be faster, leaner, and more modern than Node.js. <strong>Bun’s speed has been a major focus for its developers and uses JavaScriptCore</strong>.</p><p>JavaScriptCore is the JavaScript engine used in Apple’s Safari web browser and various other Apple products and platforms. It is an open-source component of the WebKit project, which is a browser engine that renders web content in applications like Safari.</p><p>Bun is significantly faster than Node and Deno (as per benchmark tests), utilizing the same JavaScript engine which Google abandoned once.</p><h4>Each of these historical events didn’t happen in isolation</h4><p>At each step, one was leveraging other giants see further. Some might observe this journey and assume there is competition between them, but it is not competition; it is evolution — for a better web experience. Everybody is leveraging the giants, whether it’s Microsoft, Apple, Google, or even solo developers.</p><p>When we see something good, it’s not for us to feel envy but for us to adopt and evolve. <strong>Life is not a race where we compete with each other. It is more like a Journey where each solved puzzle leads us to find bigger puzzles to solve. </strong>When you understand this profoundly, your view of the world will change. Instead of thinking that everybody is coming to get you, you’ll see that everybody is trying to make you better.</p><h4>Coming back to our beloved scientists,</h4><p>When Einstein ran calculations on high-speed particles, they didn’t confirm Newtonian physics. Einstein’s theories of relativity did not prove Newtonian physics wrong but rather extended our understanding of the physical world, particularly in situations involving extreme speeds and strong gravitational fields. Einstein redefined the definition of gravity, as he stood on the shoulders of Newton and many other scientists of his time.</p><p>There is no point in arguing who is best or whether Einstein proved Newton wrong. Even arguing that Einstein stole the idea from Newton is the wrong argument. <strong>Creativity is in connecting the same things in different ways.</strong></p><blockquote>“Creativity is just connecting things. When you ask creative people how they did something, they feel a little guilty because they didn’t really do it. They just saw something. It seemed obvious to them after a while; that’s because they were able to connect experiences they’ve had and synthesize new things.” — Steve Jobs</blockquote><p>Newton did this when he saw the discoveries of Galileo.</p><p>Einstein did this when he saw the discoveries of Newton.</p><p>Ryan Dahl did this when he looked at the V8 engine.</p><p>Jarred Sumner did this when he looked at Node.js and its limitations.</p><p>Google did this when they saw the limitations of the JavaScript runtime.</p><p>Microsoft did this when they saw that Chakra could not give them what they wanted.</p><h4>So, to conclude, we’ve learned two things:</h4><ol><li>When you see something good, don’t compete with it; find a way to leverage it to make improvements in your work.</li><li>When you do that, never feel guilty, as you are doing what humans have been doing for hundreds of years.</li></ol><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=03a2df0e0768" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/standing-on-the-shoulders-of-giants-03a2df0e0768">Standing on the shoulders of Giants</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>