<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Caner Demirci on Medium]]></title>
        <description><![CDATA[Stories by Caner Demirci on Medium]]></description>
        <link>https://medium.com/@caner.demirci?source=rss-ee1d4a1caf0e------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*1K93Lo64JdB_50wimc8fLQ.png</url>
            <title>Stories by Caner Demirci on Medium</title>
            <link>https://medium.com/@caner.demirci?source=rss-ee1d4a1caf0e------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:31:10 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@caner.demirci/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Multi Node Blockchain Simulation Article Series]]></title>
            <link>https://medium.com/@caner.demirci/multi-node-blockchain-simulation-article-series-5beebeae510f?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5beebeae510f</guid>
            <category><![CDATA[blockchain-technology]]></category>
            <category><![CDATA[blockchain-development]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Mon, 22 Sep 2025 17:01:06 GMT</pubDate>
            <atom:updated>2025-09-22T17:01:06.530Z</atom:updated>
            <content:encoded><![CDATA[<p>This series shows how I built a small <strong>multi-node blockchain</strong> in Go. Each post is short and hands-on. By the end, you can run several nodes on your machine, mine blocks, and send coins. The code uses <strong>gRPC</strong> for fast, clear communication between nodes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cnnOs58mgmWF7o-otVWRiA.png" /></figure><h3>The Posts</h3><h4><strong>1) Blocks</strong></h4><p>What is a block? A block is a “box” of transactions that also points to the previous block. This creates the chain.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-1-blocks-en-37c530a9c887">Multi Node Blockchain Simulation Part 1: Blocks (EN)</a></p><h4><strong>2) Transactions &amp; UTXO</strong></h4><p>How do coins move? A transaction spends old outputs and creates new ones. We track spendable coins with a simple UTXO set.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-2-transactions-and-utxo-en-9b1a6310f58b">Multi Node Blockchain Simulation Part 2: Transactions and UTXO (EN)</a></p><h4><strong>3) Proof of Work</strong></h4><p>How is a block mined? The miner keeps guessing a number (nonce) until the block’s hash meets a target. That’s Proof of Work.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-3-proof-of-work-en-8d90a34e2151">Multi Node Blockchain Simulation Part 3: Proof of Work (EN)</a></p><h4><strong>4) Wallet</strong></h4><p>How do you own coins? You create a key pair and an address. You sign transactions to prove the coins are yours.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-4-wallet-en-286d9e21cb54">Multi Node Blockchain Simulation Part 4: Wallet (EN)</a></p><h4><strong>5) Database</strong></h4><p>How do nodes remember state? We store the chain and UTXO data on disk so a node can restart safely and quickly.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-5-database-en-1feac91549f9">Multi Node Blockchain Simulation Part 5: Database (EN)</a></p><h4><strong>6) Network</strong></h4><p>How do nodes talk? With gRPC. One node runs a server; others connect as clients. They share new blocks and transactions and stay in sync.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-6-network-en-73f39ceff952">Multi Node Blockchain Simulation Part 6: Network (EN)</a></p><h4><strong>7) Build Simulation</strong></h4><p>How do you see it all working? Start multiple nodes, mine a block, send coins, and watch the network update in real time.</p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-7-build-simulation-en-4be7866c5ffc">Multi Node Blockchain Simulation Part 7: Build Simulation (EN)</a></p><h3>Code repository</h3><p><a href="https://github.com/cnrdmrci/blockchain-app">GitHub - cnrdmrci/blockchain-app: Multiple Node Blockchain Simulation with Mining and Wallet Module</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5beebeae510f" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi Node Blockchain Simulation Part 7: Build Simulation (EN)]]></title>
            <link>https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-7-build-simulation-en-4be7866c5ffc?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/4be7866c5ffc</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[blockchain-technology]]></category>
            <category><![CDATA[blockchain-development]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Sun, 21 Sep 2025 15:50:50 GMT</pubDate>
            <atom:updated>2025-09-21T15:50:50.089Z</atom:updated>
            <content:encoded><![CDATA[<p><em>You can access the first article from the following link;</em></p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-1-blocks-en-37c530a9c887">Multi Node Blockchain Simulation Part 1: Blocks (EN)</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MbQzdiUOe4Wn4rKA" /><figcaption>Photo by <a href="https://unsplash.com/@theshubhamdhage?utm_source=medium&amp;utm_medium=referral">Shubham Dhage</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Get Blockchain Simulation App</h3><pre>git clone https://github.com/cnrdmrci/blockchain-app</pre><h3>App Usage</h3><pre>$ go run main.go</pre><pre>Usage:<br>  createwallet -----------------------------------&gt; Create a new wallet<br>  listaddresses ----------------------------------&gt; List wallet addressses<br>  createblockchain -address ADDRESS --------------&gt; Create a blockchain and sends genesis reward to address<br>  updateblockchain -------------------------------&gt; Update blockchain via other nodes<br>  printblockchain --------------------------------&gt; Print the blocks in the blockchain<br>  printlastblock ---------------------------------&gt; Print last block  <br>  removelastblock --------------------------------&gt; Remove last block from the blockchain  <br>  getbalance -address ADDRESS --------------------&gt; Get the balance for an address<br>  send -from FROM -to TO -amount AMOUNT -mine ----&gt; Send amount of coins. Then -mine flag is set, mine off of this node<br>  reindexutxo ------------------------------------&gt; Rebuilds the UTXO set<br>  startnode -miner ADDRESS -----------------------&gt; Start a node with TxID specified in NODE_ID env. var. -miner enables mining</pre><h3>Create Wallet</h3><h4>Node 1</h4><pre>$ export NODE_ID=3000<br>$ go run main.go createwallet</pre><pre>New address is: 15uQXdzyY8Bki98UQgeXcLtXaDhW4szBjp</pre><h4>Node 2</h4><pre>$ export NODE_ID=4000<br>$ go run main.go createwallet</pre><pre>New address is: 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS</pre><h4>Node 3</h4><pre>$ export NODE_ID=5000<br>$ go run main.go createwallet</pre><pre>New address is: 1NEnVj4RVVjjZjRqoXBBqoCV5BXneMo97S</pre><h3>Initialize Blockchain</h3><h4>Node 1</h4><pre>$ go run main.go createblockchain -address 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS</pre><pre>Nonce found: 563481<br>Block Hash : 000000421470170d61f247b3b9e587debc37387b936fac018a474950a4ab8ad2<br>Genesis block created.</pre><h3>Print Blockchain</h3><h4>Node 1</h4><pre>$ go run main.go printblockchain</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*grtlf2thVZK5PJneUW3Y0A.png" /></figure><h3>Get Wallet Balance and Check Reward</h3><h4>Node 1</h4><pre>$ go run main.go getbalance -address 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS</pre><pre>Balance of 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS: 20 </pre><h3>Start Full Node and Miner Node</h3><h4>Node 1</h4><pre>$ go run main.go startnode</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gLMrHmiKr40H2FFaXb0U-g.png" /></figure><h4>Node 3</h4><pre>$ go run main.go startnode -miner 1NEnVj4RVVjjZjRqoXBBqoCV5BXneMo97S</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ERywfCcYlS5lp3KpfRc6sQ.png" /></figure><h3>Send Funds</h3><h4>Node 2</h4><pre>$ go run main.go updateblockchain<br>$ go run main.go send -from 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS -to 15uQXdzyY8Bki98UQgeXcLtXaDhW4szBjp -amount 5</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*lhDP8_H29V9_QQ6kwEs8Pg.png" /></figure><h4>Node 1 (Full Node)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*A1JW5rJUsmG6kEuJekjiMg.png" /></figure><h4>Node 2 (Miner Node)</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vNQlADv4Kv-cjyiOPZVozA.png" /></figure><h3>Get Final Balances</h3><h4>Node 2</h4><pre>$ go run main.go updateblockchain<br>$ go run main.go getbalance -address 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS<br>$ go run main.go getbalance -address 1NEnVj4RVVjjZjRqoXBBqoCV5BXneMo97S<br>$ go run main.go getbalance -address 15uQXdzyY8Bki98UQgeXcLtXaDhW4szBjp</pre><pre>Balance of 14sLJH78gy27wSEaWDTHNrhUxLa3yXYVzS: 15<br>Balance of 1NEnVj4RVVjjZjRqoXBBqoCV5BXneMo97S: 20 //Mining Reward<br>Balance of 15uQXdzyY8Bki98UQgeXcLtXaDhW4szBjp: 5</pre><h3>Get Final Blockchain</h3><h4>Node 2</h4><pre>$ go run main.go printblockchain</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*cnnOs58mgmWF7o-otVWRiA.png" /></figure><h3>Blockchain Simulation Link;</h3><p><a href="https://github.com/cnrdmrci/blockchain-app">GitHub - cnrdmrci/blockchain-app: Multiple Node Blockchain Simulation with Mining and Wallet Module</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4be7866c5ffc" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi Node Blockchain Simulation Part 6: Network (EN)]]></title>
            <link>https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-6-network-en-73f39ceff952?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/73f39ceff952</guid>
            <category><![CDATA[networking]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[blockchain-development]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Sun, 21 Sep 2025 15:48:38 GMT</pubDate>
            <atom:updated>2025-09-21T15:51:20.403Z</atom:updated>
            <content:encoded><![CDATA[<p><em>You can access the first article from the following link;</em></p><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-1-blocks-en-37c530a9c887">Multi Node Blockchain Simulation Part 1: Blocks (EN)</a></p><p>This article explains how the network is formed in the blockchain simulation project, why we use <strong>gRPC</strong>, and how nodes start a server and join the network. We’ll finish with small, focused Go snippets you can reuse.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*PUsmsTvUGHKhcZKY" /><figcaption>Photo by <a href="https://unsplash.com/@theshubhamdhage?utm_source=medium&amp;utm_medium=referral">Shubham Dhage</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is a blockchain network?</h3><p>A blockchain network is a group of independent nodes that exchange data (transactions and blocks) over a communication layer and agree on a single, append-only ledger. Each node keeps a local copy of the chain; when a new transaction or block appears, nodes validate it and propagate it to peers.</p><p>At a high level:</p><ul><li><strong>Transport</strong>: how nodes talk (e.g., gRPC over TCP).</li><li><strong>Gossip/Sync</strong>: how nodes share new transactions/blocks and catch up with longer chains.</li><li><strong>Validation</strong>: rules that determine whether data is accepted.</li><li><strong>Consensus</strong> (in full systems): a protocol (e.g., PoW, PoS) to finalize blocks.</li></ul><p>For a simulator, we still need realistic networking: nodes must be able to start, connect, broadcast, and synchronize reliably, even if consensus is simplified.</p><h3>How integrated networking into the blockchain simulator</h3><p>Our simulator runs multiple nodes as separate processes. Each node:</p><ul><li><strong>Listens</strong> on a TCP port (a gRPC server).</li><li><strong>Dials</strong> known peers (gRPC clients).</li><li><strong>Exchanges</strong> requests like “what’s your tip?”, “send me block X”, “accept this transaction”, and “here’s a new block”.</li></ul><p>Typical lifecycle:</p><ol><li><strong>Bootstrap</strong>: a node starts its server and loads an initial peer list (static config, CLI flags, or env).</li><li><strong>Discovery / Dial</strong>: it connects to peers and performs a simple handshake (e.g., fetch remote tip).</li><li><strong>Sync</strong>: it pulls missing blocks until heights match.</li><li><strong>Broadcast</strong>: new transactions and mined blocks are sent to peers, which validate and relay.</li></ol><p>This keeps the simulation faithful to real-world patterns (propagation + catch-up) while remaining compact and testable.</p><h3>What is gRPC?</h3><p>gRPC is a high-performance Remote Procedure Call framework built on <strong>HTTP/2</strong> and <strong>Protocol Buffers</strong>:</p><ul><li><strong>Binary, schema-driven messages</strong> (smaller/faster than JSON in most cases).</li><li><strong>Strong typing &amp; code generation</strong> across languages.</li><li><strong>Unary and streaming RPCs</strong> (server/client/bidirectional) for real-time flows.</li><li><strong>Built-in deadlines, status codes, and interceptors</strong> for reliability and observability.</li></ul><p>In short, you define a service in a .proto file, generate code, implement the server, and call it like a local function from clients.</p><h3>Why gRPC for a Blockchain Simulation?</h3><p>We describe the network API in a single .proto file (service + messages), generate Go stubs, then:</p><ul><li>Spin up a <strong>gRPC server</strong> on each node and register the service.</li><li>Use a <strong>gRPC client</strong> to dial peers and make RPC calls for sync and gossip.</li><li>Add practical pieces: connection backoff, timeouts, simple deduplication for broadcast.</li></ul><p><strong>Benefits:</strong></p><ul><li><strong>Simple contracts</strong>: the .proto is the source of truth.</li><li><strong>Speed &amp; size</strong>: protobuf messages are compact; HTTP/2 multiplexing helps.</li><li><strong>Streaming option</strong>: easy to push blocks as a stream if needed.</li><li><strong>Maintainability</strong>: adding a new endpoint is just a .proto change + regen.</li></ul><h3>How the Network Works in blockchain-app</h3><h4>1) Bootstrapping nodes</h4><p>You export a NODE_ID (e.g., 3000, 4000, 5000) and run CLI commands. Typical flow from the repo’s README:</p><ul><li>Create wallets on different nodes</li><li>Initialize the blockchain on one node</li><li>Start a <strong>Full Node</strong> (server) and optionally a <strong>Miner Node</strong> (server + mining loop)</li><li>Update and send funds between nodes; then re-query balances.</li></ul><p>The important networking part: <strong>startnode</strong> spins up a node process that <strong>listens</strong> (gRPC server) on a port derived from NODE_ID and <strong>connects</strong> (gRPC client) to peers.</p><h4>2) Roles and traffic</h4><ul><li><strong>Server</strong>: exposes RPC methods like “get me the latest block”, “submit transaction”, “announce block”.</li><li><strong>Client (peer)</strong>: dials other nodes, calls those RPCs to <strong>sync</strong> and <strong>broadcast</strong>.</li></ul><p>Typical message types:</p><ul><li><strong>Chain sync</strong>: fetch headers/blocks from a peer with longer height.</li><li><strong>Transaction broadcast</strong>: push a new tx to peers (they validate &amp; relay).</li><li><strong>Block propagation</strong>: miner publishes a new block → peers validate → update tip.</li></ul><h3>Next Article;</h3><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-7-build-simulation-en-4be7866c5ffc">Multi Node Blockchain Simulation Part 7: Build Simulation (EN)</a></p><h3>Blockchain Simulation Link;</h3><p><a href="https://github.com/cnrdmrci/blockchain-app">GitHub - cnrdmrci/blockchain-app: Multiple Node Blockchain Simulation with Mining and Wallet Module</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=73f39ceff952" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi Node Blockchain Simulation Part 5: Database (EN)]]></title>
            <link>https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-5-database-en-1feac91549f9?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/1feac91549f9</guid>
            <category><![CDATA[distributed-ledgers]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[blockchain-development]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Sun, 21 Sep 2025 15:46:38 GMT</pubDate>
            <atom:updated>2025-09-21T15:49:39.347Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>You can access the first article from the following link;</blockquote><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-1-blocks-en-37c530a9c887">Multi Node Blockchain Simulation Part 1: Blocks (EN)</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2-zroFAzTakxKp-c" /><figcaption>Photo by <a href="https://unsplash.com/@tofi?utm_source=medium&amp;utm_medium=referral">Tobias Fischer</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is Blockchain Database?</h3><p>A blockchain database is a decentralized and distributed digital ledger that records transactions across a network of computers. Unlike traditional databases which are typically managed by a central authority, blockchain databases are maintained by multiple nodes (computers) in a peer-to-peer network. This structure ensures transparency, security and immutability of the recorded data.</p><p>Here’s a more detailed breakdown of what a blockchain database is and how it works:</p><h4>- Decentralization</h4><p>Traditional databases are typically managed by a central authority (like a company, bank or organization) where all data is stored in one location or under the control of a single entity. In contrast, a blockchain database is <strong>decentralized</strong>, meaning no single entity controls the database. The data is stored across a network of nodes (computers) and each node holds a copy of the entire blockchain.</p><h4>- Distributed Ledger</h4><p>A blockchain is often referred to as a <strong>distributed ledger</strong>. Each participant (node) in the network maintains a complete copy of the database. Every transaction or data entry is recorded across all nodes, ensuring all copies of the database are synchronized and identical.</p><h4>- Immutable Data</h4><p>The <strong>immutability</strong> is a key feature of blockchain databases, ensuring that historical records are <strong>tamper-proof </strong>.Once data is recorded in a blockchain, it cannot be changed or deleted. Each block of data which contains a list of transactions is linked to the previous one, forming a chain.</p><h4>- Transparency</h4><p>A blockchain database is generally <strong>transparent</strong> and depending on the type of blockchain (public or private). In public blockchains (like Bitcoin and Ethereum), anyone can view the transactions. Although the transactions are anonymous, all participants can see the data, ensuring <strong>accountability</strong> and <strong>trust</strong>.</p><h4>- Security Through Cryptography</h4><p>Blockchain databases use <strong>cryptographic techniques</strong> to secure the data. Every block in the blockchain contains a <strong>hash</strong> of the previous block, a <strong>timestamp</strong> and transaction data. These hashes make it computationally difficult to alter any data in the blockchain without changing all subsequent blocks.</p><h4>- Consensus Mechanisms</h4><p>To update the blockchain database, participants must reach an agreement, called <strong>consensus</strong>. Different blockchains use different consensus mechanisms, including:</p><ul><li><strong>Proof of Work (PoW)</strong>: Requires nodes (miners) to solve complex mathematical problems to validate transactions (Bitcoin).</li><li><strong>Proof of Stake (PoS)</strong>: Participants are selected based on the amount of cryptocurrency they hold and are incentivized to validate transactions (Ethereum 2.0).</li></ul><h3>UTXO Caching for Faster Access</h3><p>In blockchain systems, especially in UTXO-based (Unspent Transaction Output) models like Bitcoin, efficiently accessing and managing UTXO data is crucial for performance. Every transaction in a UTXO blockchain spends unspent outputs and creates new ones, meaning that nodes need to track the status of UTXOs at all times. As the blockchain grows, the number of UTXOs increases, making direct access to this data more time-consuming.</p><p>To optimize performance and reduce latency, blockchain nodes often implement <strong>UTXO caching mechanisms</strong>. A UTXO cache temporarily stores frequently accessed UTXO data in memory, allowing the node to quickly retrieve it without needing to perform full database lookups. This significantly reduces the overhead in validating transactions since it avoids querying large on-disk databases for every UTXO.</p><h3>Use Cases of Blockchain Databases</h3><ul><li><strong>Cryptocurrencies</strong>: The primary use case is managing and recording cryptocurrency transactions.</li><li><strong>Supply Chain</strong>: Recording the journey of goods across multiple parties.</li><li><strong>Healthcare</strong>: Storing and sharing patient data securely across different organizations.</li><li><strong>Voting Systems</strong>: Offering tamper-proof records of votes cast during elections.</li><li><strong>Finance</strong>: Managing cross-border payments and improving financial transparency.</li></ul><h3>What is BadgerDB?</h3><p>BadgerDB is a high-performance, embeddable key-value database written in Go. Ideal for fast read and write operations. It uses an LSM (Log-Structured Merge) Tree structure, making it efficient for handling heavy workloads like blockchain, caching and event logging.</p><p>Key features;</p><ul><li><strong>Speed</strong>: Optimized for fast data access and high throughput.</li><li><strong>Concurrency</strong>: Supports multiple readers and writers simultaneously.</li><li><strong>SSD Optimization</strong>: Designed to work efficiently with SSDs.</li><li><strong>Embedded</strong>: Runs within the application, reducing latency.</li></ul><p>BadgerDB is often used in systems requiring quick data storage and retrieval, making it a good fit for performance-critical applications.</p><h3>How to Integrate BadgerDb in Golang?</h3><p>The following Go code demonstrates how to manage BadgerDB connections in a multi-node setup. Each node in the system is assigned its own instance of the Badger database, with databases stored in a map (dbs), where the key is the nodeID.</p><p>The OpenDB function initializes the database map if it’s not already created, sets up the database options and opens the database for a specific node. The connection is then stored in the dbs map.</p><p>The CloseDB function safely closes the database connection for a given nodeID, handling any potential errors and removing the database entry from the map.</p><p>This methods allow for efficient management of multiple BadgerDB instances, ensuring that each node operates independently with its own database connection.</p><pre>const (<br>   dbPath = &quot;./database/blocks_%s&quot;<br>)<br><br>var dbs map[string]*badger.DB<br><br>func initDbs() {<br>   if dbs == nil {<br>      dbs = make(map[string]*badger.DB)<br>   }<br>}<br><br>func OpenDB(nodeID string) {<br>   initDbs()<br>   path := fmt.Sprintf(dbPath, nodeID)<br>   opts := badger.DefaultOptions(path)<br>   opts.Logger = nil<br>   db, err := openDatabase(path, opts)<br>   handlers.HandleErrors(err)<br>   dbs[nodeID] = db<br>}<br><br>func CloseDB(nodeID string) {<br>   if db, exists := dbs[nodeID]; exists {<br>      if err := db.Close(); err != nil {<br>         handlers.HandleErrors(err)<br>      }<br>      delete(dbs, nodeID)<br>   }<br>}</pre><p>The following Go code defines three methods Get, Set, and Delete for interacting with a multi-node BadgerDB instance.</p><ul><li><strong>Get</strong>: Retrieves a value from the database for a given key and nodeID. If the database for the specified node is closed or doesn&#39;t exist, it triggers an error. Otherwise, it starts a read-only transaction to fetch the item and returns its value in bytes.</li><li><strong>Set</strong>: Stores data in the database using the provided key and nodeID. It opens a write transaction to insert the key-value pair and commits the transaction. Any errors during the process are handled.</li><li><strong>Delete</strong>: Removes a key-value pair from the database for a specified nodeID. A write transaction is opened, the key is deleted and the transaction is committed with error handling at each step.</li></ul><pre>func Get(key []byte, nodeID string) []byte {<br>   db, exists := dbs[nodeID]<br>   if !exists {<br>      handlers.HandleErrors(errors.New(&quot;db closed&quot;))<br>   }<br>   txn := db.NewTransaction(false)<br>   item, _ := txn.Get(key)<br>   if item == nil {<br>      return nil<br>   }<br>  <br>   byteItem, _ := item.ValueCopy(nil)<br>   return byteItem<br>}<br><br>func Set(key []byte, data []byte, nodeID string) {<br>   db, exists := dbs[nodeID]<br>   if !exists {<br>      handlers.HandleErrors(errors.New(&quot;db closed&quot;))<br>   }<br>   txn := db.NewTransaction(true)<br>   setErr := txn.Set(key, data)<br>   handlers.HandleErrors(setErr)<br>   commitErr := txn.Commit()<br>   handlers.HandleErrors(commitErr)<br>}<br><br>func Delete(key []byte, nodeID string) {<br>   db, exists := dbs[nodeID]<br>   if !exists {<br>      handlers.HandleErrors(errors.New(&quot;db closed&quot;))<br>   }<br>   txn := db.NewTransaction(true)<br>   setErr := txn.Delete(key)<br>   handlers.HandleErrors(setErr)<br>   commitErr := txn.Commit()<br>   handlers.HandleErrors(commitErr)<br>}</pre><h3>Next Article;</h3><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-6-network-en-73f39ceff952">Multi Node Blockchain Simulation Part 6: Network (EN)</a></p><h3>Blockchain Simulation Link;</h3><p><a href="https://github.com/cnrdmrci/blockchain-app">GitHub - cnrdmrci/blockchain-app: Multiple Node Blockchain Simulation with Mining and Wallet Module</a></p><h3>Badger Db Tutorial;</h3><p><a href="https://dgraph.io/docs/badger/get-started/">Get started -</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1feac91549f9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Multi Node Blockchain Simulation Part 4: Wallet (EN)]]></title>
            <link>https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-4-wallet-en-286d9e21cb54?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/286d9e21cb54</guid>
            <category><![CDATA[crypto-wallet]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Fri, 19 Sep 2025 21:17:29 GMT</pubDate>
            <atom:updated>2025-09-21T15:47:18.976Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>You can access the first article from the following link;</blockquote><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-1-blocks-en-37c530a9c887">Multi Node Blockchain Simulation Part 1: Blocks (EN)</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*lyg_tKvFg_KhwCwS" /><figcaption>Photo by <a href="https://unsplash.com/@onefifith?utm_source=medium&amp;utm_medium=referral">Shubham Dhage</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is Blockchain Wallet?</h3><p>A blockchain wallet is a digital tool that allows users to store and manage their cryptocurrencies securely. It doesn’t hold the actual coins or tokens but stores the cryptographic keys necessary to access and control these digital assets. The wallet consists of a private key and a public key. The private key is a secret code that allows users to sign transactions and prove ownership of their cryptocurrencies. The public key is used to generate wallet addresses for sending and receiving funds.</p><h4>Public and Private Keys;</h4><ul><li><strong>Public Key:</strong> The public key is derived from the private key and serves as an address that can be shared with others to receive funds. It’s similar to a bank account number. Anyone can send cryptocurrency to a public key.</li><li><strong>Private Key:</strong> The private key is a secret number that allows you to spend the cryptocurrency associated with the public key. It must be kept secure and confidential because anyone with access to the private key can control the funds.</li></ul><h4>Wallet Address;</h4><ul><li>A wallet address is a shortened form of the public key. It is used as a destination for sending cryptocurrencies. Wallet addresses are often represented as alphanumeric strings or QR codes to make transactions easier.</li></ul><h4>Types of Blockchain Wallets;</h4><ul><li><strong>Hot Wallets; </strong>are connected to the internet, making them convenient for frequent transactions but also more vulnerable to hacking. <br><strong>Example</strong>: Web wallets, mobile wallets and desktop wallets are common types of hot wallets.</li><li><strong>Cold Wallets;</strong> are not connected to the internet, providing enhanced security for long-term storage of cryptocurrency.<br><strong>Example:</strong> Hardware wallets and paper wallets are typical cold storage solutions.</li></ul><h3>How Blockchain Wallets Work</h3><p><strong>1-) Creating a Wallet: </strong>When a new wallet is created, a pair of cryptographic keys (public and private) is generated. The user receives a wallet address derived from the public key for receiving funds.</p><p><strong>2-) Receiving Cryptocurrency: </strong>To receive cryptocurrency, the sender needs the recipient’s wallet address. The transaction is broadcasted to the blockchain network and recorded in a block once it is confirmed by miners or validators.</p><p><strong>3-) Sending Cryptocurrency: </strong>To send cryptocurrency, the user must initiate a transaction using their wallet. The transaction is signed with the private key to prove ownership of the funds. This signed transaction is then broadcasted to the blockchain network for verification and inclusion in a block.</p><p><strong>4-) Checking Balance: </strong>Wallets use the public ledger of the blockchain to check the balance. The wallet queries the blockchain to aggregate the total funds controlled by the wallet’s address by summing all UTXOs (Unspent Transaction Outputs) associated with it.</p><h3>How to Create a Wallet with Golang?</h3><h4>Create Wallet</h4><p>The following code demonstrates how to generate an ECDSA key pair and create a new wallet:</p><ul><li>The createEcdsaKeyPair function generates an ECDSA private key using the P-256 elliptic curve and combines the X and Y coordinates of the public key into a single byte slice. It returns both the private key and the public key in byte form.</li><li>The createNewWallet function calls createEcdsaKeyPair to generate a new key pair and then creates a wallet object containing the private and public keys.</li></ul><p>This approach provides a secure mechanism for generating cryptographic key pairs for a blockchain wallet.</p><pre>func createEcdsaKeyPair() (ecdsa.PrivateKey, []byte) {<br>   curve := elliptic.P256()<br>  <br>   private, err := ecdsa.GenerateKey(curve, rand.Reader)<br>   if err != nil { log.Panic(err) }<br>  <br>   pub := append(private.PublicKey.X.Bytes(), private.PublicKey.Y.Bytes()...)<br>   return *private, pub<br>}<br><br>func createNewWallet() *Wallet {<br>   private, public := createEcdsaKeyPair()<br>   wallet := Wallet{private, public}<br>  <br>   return &amp;wallet<br>}</pre><h4>Get Wallet Address</h4><p>The following code demonstrates how to generate a blockchain wallet address:</p><ul><li>The GetAddress method on the Wallet struct derives the wallet address by first obtaining a public key hash (getPublicKeyHash). Then, it creates a versioned hash and adds a checksum (getChecksum) to ensure address integrity. Finally, it encodes the full hash using Base58, returning the wallet address as a string.</li><li>The getPublicKeyHash function creates a hash of the public key by applying SHA-256 followed by the RIPEMD-160 hashing algorithms.</li><li>The getChecksum function calculates the checksum by performing two rounds of SHA-256 on the input and returns the first few bytes of the result as the checksum.</li></ul><pre>func (w *Wallet) GetAddress() string {<br>   pubHash := getPublicKeyHash(w.PublicKey)<br>   versionedHash := append([]byte{version}, pubHash...)<br>   checksumHash := getChecksum(versionedHash)<br>   fullHash := append(versionedHash, checksumHash...)<br>   walletAddress := encoders.Base58Encode(fullHash)<br>  <br>   return walletAddress<br>}<br><br>func getPublicKeyHash(pubKey []byte) []byte {<br>   pubHash := sha256.Sum256(pubKey)<br>  <br>   hasher := ripemd160.New()<br>   _, err := hasher.Write(pubHash[:])<br>   if err != nil {<br>      handlers.HandleErrors(err)<br>   }<br>  <br>   return hasher.Sum(nil)<br>}<br><br>func getChecksum(payload []byte) []byte {<br>   firstHash := sha256.Sum256(payload)<br>   secondHash := sha256.Sum256(firstHash[:])<br>  <br>   return secondHash[:checksumLength]<br>}</pre><h4>Wallet File</h4><p>The following JSON represents the structure of a blockchain wallet:</p><ul><li>The Wallets object contains a unique wallet address (15uQXdzyYba7i98U34eXcLtd3DhW4szBjp) which maps to the wallet&#39;s private and public key information.</li><li>The PrivateKey section includes the private key&#39;s D value as well as the PublicKey object that contains the elliptic curve used (P-256) and the X and Y coordinates for the public key.</li><li>Additionally, the PublicKey field contains the encoded public key string in Base64 format.</li></ul><p>This structure provides all necessary cryptographic details for the wallet, including both the private and public key information.</p><pre>{<br>  &quot;Wallets&quot;: {<br>    &quot;15uQXdzyYba7i98U34eXcLtd3DhW4szBjp&quot;: {<br>      &quot;PrivateKey&quot;: {<br>        &quot;D&quot;: 43308974610626687114329129843934090295437234361016429727796355430323669631100,<br>        &quot;PublicKey&quot;: {<br>          &quot;Curve&quot;: &quot;P-256&quot;,<br>          &quot;X&quot;: 45674459444534617207466792759099758920851305209026028009869603191663263947506,<br>          &quot;Y&quot;: 37420010274307973455863514152595111536386343211958433633523469177312441750879<br>        }<br>      },<br>      &quot;PublicKey&quot;: &quot;ZPrT8wKJc+bRoO2OO7ys6Y5LSa6xZ7vbbfX88KpnGvJSkzmqMlgU+GZQsguvMt5wSLjOtwDj/Fx2uTw9p4bVXw==&quot;<br>    }<br>  }<br>}</pre><h3>Next Article;</h3><p><a href="https://medium.com/@caner.demirci/multi-node-blockchain-simulation-part-5-database-en-1feac91549f9">Multi Node Blockchain Simulation Part 5: Database (EN)</a></p><h3>Blockchain Simulation Link;</h3><p><a href="https://github.com/cnrdmrci/blockchain-app">GitHub - cnrdmrci/blockchain-app: Multiple Node Blockchain Simulation with Mining and Wallet Module</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=286d9e21cb54" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[MSSQL To PostgreSQL Data Replication With Debezium(CDC)]]></title>
            <link>https://medium.com/@caner.demirci/mssql-to-postgresql-data-replication-with-debezium-cdc-18cd171c44fa?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/18cd171c44fa</guid>
            <category><![CDATA[cdc]]></category>
            <category><![CDATA[change-data-capture]]></category>
            <category><![CDATA[docker]]></category>
            <category><![CDATA[debezium]]></category>
            <category><![CDATA[kafka]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Wed, 05 Feb 2025 12:11:52 GMT</pubDate>
            <atom:updated>2025-02-05T12:11:52.499Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>What is CDC (Change Data Capture)?</strong></h3><p>Change Data Capture (CDC) is a technique that tracks and captures changes (inserts, updates, deletes) in a database and makes them available for real-time data streaming or synchronization without complex queries.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*uGwUqNmTU-xura5S" /><figcaption>Photo by <a href="https://unsplash.com/@jannerboy62?utm_source=medium&amp;utm_medium=referral">Nick Fewings</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><p>CDC is a powerful tool for efficiently capturing data changes in real-time, making it ideal for modern data workflows. However, it requires careful implementation to ensure optimal performance and minimize potential issues.</p><h4>Advantages of CDC:</h4><ul><li>Real-Time Data Processing: CDC captures and streams data changes instantly.</li><li>Efficient Data Replication: Reduces the need for full database scans by only capturing changes, making data synchronization faster and less resource-intensive.</li><li>Minimizes Database Load: Unlike periodic batch queries, CDC reads changes from transaction logs, reducing the impact on database performance.</li><li>Improved Data Consistency: Ensures accurate and up-to-date data synchronization across multiple systems.</li></ul><h4>Disadvantages of CDC:</h4><ul><li>Complexity in Implementation: Setting up CDC requires database and infrastructure configuration which can be complex depending on the system.</li><li>Increased Storage and Processing Needs: Storing and processing change logs can consume additional resources, especially in high-transaction environments.</li><li>Latency Concerns: While CDC is near real-time, network delays or processing overhead can introduce slight delays in data propagation.</li><li>Compatibility Issues: Not all databases natively support CDC, requiring additional tools or workarounds.</li></ul><h3><strong>What is Debezium?</strong></h3><p>Debezium is an open-source distributed platform designed to capture and stream real-time changes from databases. It uses Change Data Capture (CDC) techniques to monitor and track data changes, such as inserts, updates and deletes, in databases and sends these changes to messaging systems like Kafka. Debezium makes it easier to integrate databases with other systems by capturing database changes and pushing them to stream processing tools.</p><h4>Advantages of Debezium:</h4><ul><li>Real-Time Data Processing: CDC captures and streams data changes instantly, enabling real-time data synchronization across systems.</li><li>Efficient Data Replication: Reduces the need for full database scans by only capturing changes, making data synchronization faster and less resource-intensive.</li><li>Supports Event-Driven Architectures: Enables microservices and event-driven applications by turning database changes into actionable events.</li><li>Improved Data Consistency: Ensures accurate and up-to-date data synchronization across multiple systems.</li><li>Better Scalability: CDC allows organizations to handle growing data volumes without frequent full data transfers.</li><li>Supports Multiple Databases: Debezium supports a variety of popular databases, including MySQL, PostgreSQL, MongoDB, and SQL Server.</li></ul><h4>Disadvantages of Debezium:</h4><ul><li>Complexity in Implementation: Setting up CDC requires database and infrastructure configuration, which can be complex depending on the system.</li><li>Increased Storage and Processing Needs: Storing and processing change logs can consume additional resources, especially in high-transaction environments.</li><li>Compatibility Issues: Not all databases natively support CDC, requiring additional tools or workarounds.</li><li>Latency Concerns: While CDC is near real-time, network delays or processing overhead can introduce slight delays in data propagation.</li></ul><h3>Create Docker Apps</h3><p>We can set up the necessary applications for the test environment in our local Docker setup using the following <strong>docker-compose</strong> file.</p><p>With this file, we will deploy <strong>MSSQL, PostgreSQL, Debezium, Kafka, and Kafka-UI</strong> applications;</p><pre>version: &#39;3.9&#39;<br><br>services:<br>  mssql:<br>    image: mcr.microsoft.com/mssql/server:2022-latest<br>    container_name: sql-server<br>    ports:<br>      - 1433:1433<br>    environment:<br>      SA_PASSWORD: &quot;*****&quot;<br>      ACCEPT_EULA: &quot;Y&quot;<br>      MSSQL_AGENT_ENABLED: true<br>    volumes:<br>      - mssql_data:/var/opt/mssql<br><br>  postgres:<br>    image: postgres:latest<br>    container_name: postgres<br>    environment:<br>      POSTGRES_USER: my_user<br>      POSTGRES_PASSWORD: *****<br>      POSTGRES_DB: my_database<br>    ports:<br>      - &quot;5432:5432&quot;<br>    volumes:<br>      - postgres_data:/var/lib/postgresql/data<br><br>  kafka:<br>    image: quay.io/debezium/kafka:latest<br>    ports:<br>      - 29092:29092<br>    environment:<br>      - CLUSTER_ID=ABC<br>      - BROKER_ID=1<br>      - KAFKA_CONTROLLER_QUORUM_VOTERS=1@kafka:9093<br>      - KAFKA_LISTENERS=INTERNAL://:9092,EXTERNAL://:29092,CONTROLLER://:9093<br>      - KAFKA_ADVERTISED_LISTENERS=INTERNAL://kafka:9092,EXTERNAL://localhost:29092<br>      - KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INTERNAL:PLAINTEXT,EXTERNAL:PLAINTEXT,CONTROLLER:PLAINTEXT<br>      - KAFKA_INTER_BROKER_LISTENER_NAME=INTERNAL<br>      - KAFKA_CONTROLLER_LISTENER_NAMES=CONTROLLER<br><br>  connect:<br>    image: quay.io/debezium/connect:latest<br>    ports:<br>      - 28083:8083<br>    environment:<br>      - BOOTSTRAP_SERVERS=kafka:9092<br>      - GROUP_ID=1<br>      - CONFIG_STORAGE_TOPIC=my_connect_configs<br>      - OFFSET_STORAGE_TOPIC=my_connect_offsets<br>      - STATUS_STORAGE_TOPIC=my_connect_statuses<br><br>  kafka-ui:<br>    image: provectuslabs/kafka-ui:latest<br>    container_name: kafka-ui<br>    depends_on:<br>      - kafka<br>    ports:<br>      - &quot;8080:8080&quot;<br>    environment:<br>      KAFKA_CLUSTERS_0_NAME: local<br>      KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:9092<br><br>volumes:<br>  mssql_data:<br>  postgres_data:</pre><h3>MSSQL Configurations</h3><p>Create TestDatabase and dbo.SampleTable;</p><pre>CREATE DATABASE TestDatabase;<br>GO<br><br>USE [TestDatabase];<br>GO<br><br>CREATE TABLE [SampleTable]<br>(<br> [Id]          [int] IDENTITY(1,1) NOT NULL primary key,<br> [Name]        [varchar](100) NULL,<br> [Value]       [varchar](100) NULL<br>)<br>GO</pre><p>Activate cdc for database and table;</p><pre>EXEC sys.sp_cdc_enable_db<br>GO<br><br>EXEC sys.sp_cdc_enable_table<br>@source_schema = N&#39;dbo&#39;,<br>@source_name   = N&#39;SampleTable&#39;,<br>@role_name     = NULL,<br>@supports_net_changes = 1<br>GO</pre><p>Check cdc status;</p><pre>SELECT name, is_cdc_enabled FROM sys.databases <br>WHERE name = &#39;TestDatabase&#39;<br>GO<br><br>select name,type,type_desc,is_tracked_by_cdc from sys.tables <br>WHERE name = &#39;SampleTable&#39;<br>GO</pre><p>cdc requires SQLServerAgent to be running;</p><pre>EXEC xp_servicecontrol &#39;QUERYSTATE&#39;, &#39;SQLServerAgent&#39;;<br><br>SELECT servicename, status_desc FROM sys.dm_server_services <br>WHERE servicename = &#39;SQL Server Agent (MSSQLSERVER)&#39;;</pre><p>Changes made in the last 1 day for dbo.SampleTable;</p><pre>DECLARE @begin_time_stamp BINARY(10);<br>DECLARE @end_time_stamp BINARY(10);<br>SELECT @begin_time_stamp = sys.fn_cdc_map_time_to_lsn(&#39;smallest greater than&#39;, GETDATE()-1);<br>SELECT @end_time_stamp = sys.fn_cdc_map_time_to_lsn(&#39;largest less than or equal&#39;, GETDATE());<br>SELECT count(*)<br>FROM [cdc].[fn_cdc_get_all_changes_dbo_SampleTable](@begin_time_stamp,@end_time_stamp,&#39;all&#39;)</pre><p>Change the polling interval of the capture job;</p><pre>EXEC sys.sp_cdc_help_jobs;<br>EXEC sys.sp_cdc_change_job @job_type = &#39;capture&#39;, @pollinginterval = 1<br>EXEC sys.sp_cdc_stop_job @job_type = &#39;capture&#39;<br>EXEC sys.sp_cdc_start_job @job_type = &#39;capture&#39;</pre><p>Add new data to dbo.SampleTable;</p><pre>INSERT INTO SampleTable (Name, Value)<br>VALUES <br>    (&#39;Name1&#39;, &#39;Value1&#39;),<br>    (&#39;Name2&#39;, &#39;Value2&#39;),<br>    (&#39;Name3&#39;, &#39;Value3&#39;);</pre><p>You can see the changes at;</p><pre>SELECT * FROM [cdc].[dbo_SampleTable_CT]</pre><h3>Create MSSQL Connector</h3><p>A <strong>connector</strong> is a software component that facilitates data movement between an external system and a data streaming platform like Kafka.</p><p>You can create a connector using the following configuration file:</p><pre>curl --location &#39;http://localhost:28083/connectors&#39; \<br>--header &#39;Content-Type: application/json&#39; \<br>--data &#39;{<br>  &quot;name&quot;: &quot;mssql-connector-v1&quot;,<br>  &quot;config&quot;: {<br>    &quot;connector.class&quot;: &quot;io.debezium.connector.sqlserver.SqlServerConnector&quot;,<br>    &quot;database.encrypt&quot;: &quot;false&quot;,<br>    &quot;database.hostname&quot;: &quot;mssql&quot;,<br>    &quot;database.names&quot;: &quot;TestDatabase&quot;,<br>    &quot;database.port&quot;: &quot;1433&quot;,<br>    &quot;database.user&quot;: &quot;SA&quot;,<br>    &quot;database.password&quot;: &quot;******&quot;,<br>    &quot;table.include.list&quot;: &quot;dbo.SampleTable&quot;,<br>    &quot;database.history.kafka.bootstrap.servers&quot;: &quot;kafka:9092&quot;,<br>    &quot;database.history.kafka.topic&quot;: &quot;schema-changes.inventory&quot;,<br>    &quot;include.schema.changes&quot;: &quot;false&quot;,<br>    &quot;topic.prefix&quot;: &quot;sample&quot;,<br>    &quot;tasks.max&quot;: &quot;1&quot;,<br>    &quot;key.converter&quot;: &quot;org.apache.kafka.connect.json.JsonConverter&quot;,<br>    &quot;key.converter.schemas.enable&quot;: &quot;true&quot;,<br>    &quot;schema.history.internal.consumer.bootstrap.servers&quot;: &quot;kafka:9092&quot;,<br>    &quot;schema.history.internal.kafka.bootstrap.servers&quot;: &quot;kafka:9092&quot;,<br>    &quot;schema.history.internal.kafka.topic&quot;: &quot;sample.debezium-schema-history&quot;,<br>    &quot;schema.history.internal.producer.bootstrap.servers&quot;: &quot;kafka:9092&quot;,<br>    &quot;snapshot.isolation.mode&quot;: &quot;read_committed&quot;,<br>    &quot;snapshot.mode&quot;: &quot;initial&quot;, //initial, always, schema_only<br>    &quot;time.precision.mode&quot;: &quot;connect&quot;,<br>    &quot;timestamp.handling.mode&quot;: &quot;string&quot;,<br>    &quot;tombstones.on.delete&quot;: &quot;false&quot;,<br>    &quot;value.converter&quot;: &quot;org.apache.kafka.connect.json.JsonConverter&quot;,<br>    &quot;value.converter.schemas.enable&quot;: &quot;true&quot;<br>  }<br>}&#39;</pre><h3>Kafka Ui</h3><blockquote><a href="http://localhost:8080/">http://localhost:8080/</a></blockquote><p>All records from dbo.SampleTable are streamed into the sample.TestDatabase.dbo.SampleTable topic in Kafka. When creating the sink connector, this topic will be used to insert data into the public.sample_table in PostgreSQL.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ljV5ffuAXvs28XnXejKs6w.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Q03YSsxmw4KZDlRO9QMmOw.png" /></figure><h3>PostgreSQL Configuration</h3><p>Create public.sample_table to insert records;</p><pre>CREATE TABLE public.sample_table<br>(<br>    id          bigint generated by default as identity primary key,<br>    name        varchar(100) NULL,<br>    value       varchar(100) NULL<br>);</pre><h3>Create PostgreSQL Sink Connector</h3><p>Sink connectors consume data from Kafka topics and write it to the destination system. They help automate data pipeline processes by managing the flow of information and ensuring data is consistently delivered to the target system without manual intervention.</p><p>You can create a sink connector using the following configuration file:</p><pre>curl --location &#39;http://localhost:28083/connectors&#39; \<br>--header &#39;Content-Type: application/json&#39; \<br>--data &#39;{<br>    &quot;name&quot;: &quot;postgres-sink-connector-v1&quot;,<br>    &quot;config&quot;: {<br>        &quot;auto.create&quot;: &quot;true&quot;,<br>        &quot;auto.evolve&quot;: &quot;true&quot;,<br>        &quot;bootstrap.servers&quot;: &quot;kafka:9092&quot;,<br>        &quot;connection.url&quot;: &quot;jdbc:postgresql://postgres:5432/my_database&quot;,<br>        &quot;connection.username&quot;: &quot;my_user&quot;,<br>        &quot;connection.password&quot;: &quot;******&quot;,<br>        &quot;connector.class&quot;: &quot;io.debezium.connector.jdbc.JdbcSinkConnector&quot;,<br>        &quot;consumer.override.auto.offset.reset&quot;: &quot;latest&quot;,<br>        &quot;consumer.override.group.id&quot;: &quot;sample.group.db.postgres&quot;,<br>        &quot;delete.enabled&quot;: &quot;true&quot;,<br>        &quot;errors.deadletterqueue.context.headers.enable&quot;: &quot;true&quot;,<br>        &quot;errors.deadletterqueue.topic.name&quot;: &quot;sample.debezium.deadletterqueue&quot;,<br>        &quot;errors.log.enable&quot;: &quot;true&quot;,<br>        &quot;errors.log.include.messages&quot;: &quot;true&quot;,<br>        &quot;insert.mode&quot;: &quot;upsert&quot;,<br>        &quot;key.converter&quot;: &quot;org.apache.kafka.connect.json.JsonConverter&quot;,<br>        &quot;key.converter.schemas.enable&quot;: &quot;true&quot;,<br>        &quot;primary.key.fields&quot;: &quot;id&quot;,<br>        &quot;primary.key.mode&quot;: &quot;record_key&quot;,<br>        &quot;quote.identifiers&quot;: &quot;false&quot;,<br>        &quot;collection.name.format&quot;: &quot;public.sample_table&quot;,<br>        &quot;tasks.max&quot;: &quot;1&quot;,<br>        &quot;topics&quot;: &quot;sample.TestDatabase.dbo.SampleTable&quot;,<br>        &quot;transforms&quot;: &quot;ExtractAfter,timestamp,route1,route2,keyRename&quot;,<br>        &quot;transforms.ExtractAfter.field&quot;: &quot;after&quot;,<br>        &quot;transforms.ExtractAfter.type&quot;: &quot;org.apache.kafka.connect.transforms.ExtractField$Value&quot;,<br>        &quot;transforms.keyRename.renames&quot;: &quot;Id:id&quot;,<br>        &quot;transforms.keyRename.type&quot;: &quot;org.apache.kafka.connect.transforms.ReplaceField$Key&quot;,<br>        &quot;transforms.route1.regex&quot;: &quot;^sample\\.TestDatabase\\.dbo\\.SampleTable$&quot;,<br>        &quot;transforms.route1.replacement&quot;: &quot;dbo.SampleTable&quot;,<br>        &quot;transforms.route1.type&quot;: &quot;org.apache.kafka.connect.transforms.RegexRouter&quot;,<br>        &quot;transforms.route2.renames&quot;: &quot;Id:id,Name:name,Value:value&quot;,<br>        &quot;transforms.route2.type&quot;: &quot;org.apache.kafka.connect.transforms.ReplaceField$Value&quot;,<br>        &quot;transforms.timestamp.field&quot;: &quot;DATE_TIME&quot;,<br>        &quot;transforms.timestamp.format&quot;: &quot;yyyy-MM-dd HH:mm:ss&quot;,<br>        &quot;transforms.timestamp.target.type&quot;: &quot;Timestamp&quot;,<br>        &quot;transforms.timestamp.type&quot;: &quot;org.apache.kafka.connect.transforms.TimestampConverter$Value&quot;,<br>        &quot;value.converter&quot;: &quot;org.apache.kafka.connect.json.JsonConverter&quot;,<br>        &quot;value.converter.schemas.enable&quot;: &quot;true&quot;<br>    }<br>}&#39;</pre><h3>Check Connectors</h3><pre>curl --location &#39;http://localhost:28083/connectors&#39;</pre><pre>[<br>    &quot;postgres-sink-connector-v1&quot;,<br>    &quot;mssql-connector-v1&quot;<br>]</pre><h3>Check PostgreSQL Table</h3><p>You can see the replicated records;</p><pre>select * from public.sample_table;</pre><p>You can also insert, update or delete any record in MSSQL, and the changes will be reflected in the public.sample_table.</p><h3>Summary</h3><p>In this article, we’ve explored the powerful features of Debezium and Change Data Capture (CDC) and how they facilitate real-time data integration. We also demonstrated how to set up a Docker Compose environment that includes Debezium, Kafka, Kafka-UI, MSSQL and PostgreSQL. By walking through the installation and setup process, we’ve shown how data flows between these components, keeping the systems synchronized and ensuring the information is always up-to-date.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=18cd171c44fa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Protecting System Integrity with the Anti-Corruption Layer (ACL) Pattern in Software Architecture]]></title>
            <link>https://medium.com/@caner.demirci/protecting-system-integrity-with-the-anti-corruption-layer-acl-pattern-in-software-architecture-5b57db6263d9?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b57db6263d9</guid>
            <category><![CDATA[acl]]></category>
            <category><![CDATA[anti-corruption-layer]]></category>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[software-architecture]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Tue, 01 Oct 2024 18:56:27 GMT</pubDate>
            <atom:updated>2024-10-01T18:56:27.881Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Anti-Corruption Layer (ACL)?</h3><p>The Anti-Corruption Layer is a software design pattern used to establish a boundary or a layer of isolation between your system and an external system that you need to interact with. Its main purpose is to prevent the external system’s design, domain model or technology constraints from adversely affecting or “corrupting” the internal structure and design of your system. Essentially, it allows you to integrate with legacy or third-party systems without compromising the integrity and cleanliness of your own architecture.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*vY04a_wDeb_R7jX1" /><figcaption>Photo by <a href="https://unsplash.com/@omilaev?utm_source=medium&amp;utm_medium=referral">Igor Omilaev</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Key Components of ACL</h3><ul><li><strong>Adapter or Translator</strong>: The core function of the ACL is to adapt or translate between the external system’s domain model and your system’s domain model. It ensures that both systems can communicate effectively without directly exposing internal complexities.</li><li><strong>Mapping Logic</strong>: This component handles the mapping between external and internal models. For example, if the external system follows different naming conventions, data structures or communication protocols, the mapping logic in the Anti-Corruption Layer ensures that the data is transformed and aligned with the internal system’s format.</li><li><strong>Isolation</strong>: The ACL ensures that your system remains isolated from the external system’s design flaws or inconsistencies. By encapsulating the integration code within the ACL, changes in the external system can be managed without affecting the core of your system.</li><li><strong>Error Handling and Validation</strong>: Since external systems can behave unpredictably or return unexpected results, the ACL also typically includes robust error handling and data validation mechanisms to prevent unexpected failures or corrupt data from entering your system.</li></ul><h3>When to Use ACL?</h3><p>The Anti-Corruption Layer is particularly useful in the following scenarios:</p><ul><li><strong>Integration with Legacy Systems</strong>: When your system interacts with legacy systems that use outdated domain models, old design patterns or even obsolete technologies, an Anti-Corruption Layer (ACL) serves as a safeguard. It acts as a buffer, transforming and adapting the legacy system’s complexities into a format compatible with your modern architecture. This ensures that your system remains up-to-date, clean, and efficient, without inheriting the technical debt or design flaws of the legacy system.</li><li><strong>Third-Party Services</strong>: Integrating with third-party services, especially those you do not control, can expose your system to risks of changes or inconsistencies in their domain models. An ACL helps to buffer your system from these external changes.</li><li><strong>Domain Model Differences</strong>: When two systems have vastly different domain models, integrating them directly could lead to confusion, inconsistencies and bugs. An ACL allows you to abstract away the differences and make integration smoother and more maintainable.</li><li><strong>Protecting from External Design Flaws</strong>: If the external system is poorly designed or overly complex, an ACL shields your system from adopting or being influenced by these design flaws, ensuring that your architecture remains clean and modular.</li></ul><h3>Example of ACL in Action</h3><p>Imagine that you are building a modern e-commerce system that needs to integrate with an old inventory management system. The legacy system has a complicated and outdated domain model, where product quantities are tracked using multiple inconsistent fields and the API is cumbersome to work with. Directly integrating this into your e-commerce system could lead to a chaotic domain model on your side, making future maintenance difficult.</p><p>Instead, you can implement an ACL. This layer would interact with the inventory system, translating its complex data structures into a clean, modern domain model that your e-commerce system understands. It might also handle inconsistencies, such as reconciling multiple fields into a single quantity value and provide graceful error handling in case the legacy system behaves unpredictably.</p><h3>Advantages of ACL</h3><ul><li><strong>Prevents System Corruption</strong>: By decoupling your system from external systems, an ACL ensures that your system’s architecture remains unaffected by the external domain model, poor design or complexities.</li><li><strong>Simplifies Integration</strong>: The ACL simplifies the integration process by translating between different domain models and technologies, making it easier to manage long-term maintenance and upgrades.</li><li><strong>Enhances Maintainability</strong>: By isolating external system dependencies, changes to the external system can be handled within the ACL without requiring major changes to your own system.</li></ul><h3>Disadvantages of ACL</h3><ul><li><strong>Increased Complexity</strong>: Introducing an ACL adds another layer of complexity to your architecture. You need to write and maintain the adapter or translation logic which can become cumbersome if the external system is highly complex.</li><li><strong>Performance Overhead</strong>: Depending on the complexity of the translation logic, an ACL can introduce performance overhead. Especially if it involves heavy data transformation or validation.</li><li><strong>Delayed Integration</strong>: Since the ACL introduces an additional layer, integrating with external systems can take more time to implement and test.</li></ul><h3>Conclusion</h3><p>The Anti-Corruption Layer is an essential design pattern when dealing with external systems that have complex or inconsistent domain models. It acts as a shield, protecting your system’s architecture from being polluted by external influences while still allowing effective communication. While it may introduce some additional complexity, the long-term benefits of maintainability, flexibility and isolation often outweigh the costs.</p><p>By implementing an ACL, you can ensure that your system remains clean, modern and scalable, even when integrating with legacy or poorly designed systems.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b57db6263d9" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Is PASETO better than JWT?]]></title>
            <link>https://medium.com/@caner.demirci/is-paseto-better-than-jwt-d613cdb86ddd?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/d613cdb86ddd</guid>
            <category><![CDATA[jwt-authentication]]></category>
            <category><![CDATA[paseto]]></category>
            <category><![CDATA[jwt-token]]></category>
            <category><![CDATA[jwt-auth]]></category>
            <category><![CDATA[jwt]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Tue, 24 Sep 2024 19:54:35 GMT</pubDate>
            <atom:updated>2024-09-24T19:54:35.677Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DZSCwM_DNJWuPvwx" /><figcaption>Photo by <a href="https://unsplash.com/@moneyphotos?utm_source=medium&amp;utm_medium=referral">rc.xyz NFT gallery</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>What is Jwt?</h3><p><strong>JWT (JSON Web Token)</strong> is a lightweight, self-contained token used to securely transmit information between two systems. It’s commonly used in web applications for authentication and authorization.</p><p>JWT consist of three parts and each parts which are Base64URL-encoded, separated by a dot (.)</p><h4>Header</h4><p>The header typically consists of two parts;</p><ul><li>The <strong>type</strong> of the token which is JWT.</li><li>The <strong>signing algorithm</strong> being used (HMAC SHA256 or RSA).</li></ul><pre>{<br>  &quot;alg&quot;: &quot;HS256&quot;,<br>  &quot;typ&quot;: &quot;JWT&quot;<br>}</pre><h4>Payload</h4><p>The payload contains the <strong>claims</strong> which are statements about an entity and additional metadata. Claims can be divided into three categories:</p><ul><li><strong>Registered claims</strong>: Standardized claims like iss (issuer), sub (subject), aud (audience), exp (expiration time) etc.</li><li><strong>Public claims</strong>: Custom claims created by users to share additional information.</li><li><strong>Private claims</strong>: Custom claims shared between systems that agree on their use.</li></ul><pre>{<br>  &quot;sub&quot;: &quot;1234567890&quot;,<br>  &quot;name&quot;: &quot;John Doe&quot;,<br>  &quot;admin&quot;: true<br>}</pre><h4>Signature</h4><p>The signature is used to verify the authenticity of the message. To create the signature, the encoded header, the encoded payload and a <strong>secret</strong> are combined and signed using the algorithm specified in the header.</p><p>Formula to create the signature:</p><pre>HMACSHA256(<br>    base64UrlEncode(header) + &quot;.&quot; + base64UrlEncode(payload),<br>    secret)</pre><h4>Here is a JWT example:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*aw1PzuTyE-ghqvTHm9IYmA.png" /></figure><h3>What is PASETO?</h3><p><strong>PASETO (Platform-Agnostic Security Tokens)</strong> is a new and improved security token format that was created to be safer and easier to use than JWT. While JWT is widely used, it has some security risks and is easy to misconfigure. PASETO was designed to avoid these problems by using strong security rules and making the process simpler for developers.</p><p>PASETO consist of four parts and each parts which are Base64URL-encoded, separated by a dot (.)</p><h4><strong>Version</strong></h4><p>Version tells us which version of PASETO is being used (like v1, v2).</p><h4><strong>Purpose</strong></h4><p>Purpose defines whether the token is encrypted or just signed.</p><ul><li><strong>Local</strong> tokens use symmetric encryption (one shared secret key).</li><li><strong>Public</strong> tokens use asymmetric signing (a public/private key pair).</li></ul><h4><strong>Payload</strong></h4><p>This is the data inside the token, such as user information. Depending on the purpose, it is either encrypted (so only certain people can read it) or signed (so its integrity can be verified).</p><h4><strong>Footer</strong> (optional)</h4><p>Extra information that can be added to the token, like metadata.</p><h4>Here is a PASETO local example:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2Np0GkgTzvdaPUa-1qtNjQ.png" /></figure><h4>Here is a PASETO public example:</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1ODXEViPpai9tmG1f8xCTA.png" /></figure><h3>Is PASETO better than JWT?</h3><p>JWT (JSON Web Token) and PASETO (Platform-Agnostic Security Tokens) serve similar purposes in authentication and information exchange but they differ significantly in security and integration aspects.</p><p>In terms of <strong>security</strong>, JWT can be vulnerable to common pitfalls like weak signing algorithms and poor secret management. This can lead to issues such as token forgery or unauthorized access if not implemented correctly. PASETO, on the other hand, is designed with security as a priority, enforcing strong cryptographic practices and minimizing implementation complexity. Its clear specifications help prevent common vulnerabilities, making it inherently more secure.</p><p>When it comes to <strong>integration</strong>, JWT has the advantage of being widely adopted, supported by numerous libraries and frameworks across different languages and platforms. This makes it easier to integrate into existing systems. PASETO is newer and less common which may result in compatibility challenges and a smaller ecosystem of tools. However, its design focuses on ease of use, allowing for straightforward integration once the necessary libraries are available.</p><p>In summary, while JWT is well-integrated and widely used, it requires careful implementation to ensure security. PASETO offers enhanced security features and a simpler design but faces challenges with adoption and available integration tools. The choice between them should consider the security needs and existing infrastructure of the project.</p><p>So, if you ask again that is PASETO better than JWT; its up to your security needs and existing project requirements…</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d613cdb86ddd" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mastering OpenTelemetry: A Practical Guide to Metrics, Logs and Tracing in .Net]]></title>
            <link>https://medium.com/@caner.demirci/mastering-opentelemetry-a-practical-guide-to-metrics-logs-and-tracing-in-net-23c3548ab805?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/23c3548ab805</guid>
            <category><![CDATA[grafana]]></category>
            <category><![CDATA[opentelemetry]]></category>
            <category><![CDATA[jaeger]]></category>
            <category><![CDATA[prometheus]]></category>
            <category><![CDATA[hexagonal-architecture]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Thu, 12 Sep 2024 09:55:05 GMT</pubDate>
            <atom:updated>2024-09-12T09:55:05.890Z</atom:updated>
            <content:encoded><![CDATA[<h3>Mastering OpenTelemetry: A Practical Guide to Metrics, Logs and Tracing in .Net</h3><h3>What is OpenTelemetry?</h3><p>OpenTelemetry is an open-source observability framework that provides standardized, vendor-neutral instrumentation for collecting, processing and exporting telemetry data. It is a project under the Cloud Native Computing Foundation (CNCF) and plays a vital role in modern distributed systems, microservices architectures and cloud-native applications.</p><p>OpenTelemetry (often abbreviated as OTel) is a collection of tools, APIs and SDKs that allow developers to instrument, generate, collect and export telemetry data. It unifies the three core elements of observability into one cohesive system; <strong>traces</strong>, <strong>metrics</strong> and <strong>logs.</strong></p><p>Before OpenTelemetry, developers often had to rely on vendor-specific SDKs or multiple libraries to instrument their applications for monitoring. OpenTelemetry solves this by offering a standardized and extensible framework that works with various backends like Jaeger, Prometheus, Zipkin, Loki and others.</p><p>The OpenTelemetry architecture comprises several components;</p><ul><li><strong><em>Instrumentation libraries</em></strong>: These libraries allow developers to add tracing, metrics and logging to their code.</li><li><strong><em>SDKs</em></strong>: The SDKs provide the implementation for OpenTelemetry and allow customization of data collection and export.</li><li><strong><em>Collector</em></strong>: The OpenTelemetry Collector is a standalone service that receives, processes and exports telemetry data. It can be deployed as an agent (running locally on the same host as your application) or as a centralized service.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*E_TLuCNt_A6VS91R" /><figcaption>Photo by <a href="https://unsplash.com/@razvan_mirel?utm_source=medium&amp;utm_medium=referral">Razvan Mirel</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Key Concepts in OpenTelemetry</h3><ul><li><strong><em>Traces</em></strong>: Traces represent operations or workflows within an application. A trace is composed of multiple spans where each span is a single unit of work with a start time, end time, and metadata. Tracing is essential for understanding how requests flow across services in a distributed system.</li><li><strong><em>Metrics</em></strong>: Metrics represent measurements of various system properties over time. Examples include CPU usage, request count and error rates. OpenTelemetry supports multiple metric types, including counters, gauges and histograms.</li><li><strong><em>Logs</em></strong>: Logs capture textual records about events occurring within your application, providing context and insights. With OpenTelemetry, logs can be enriched with trace and span data, offering a more comprehensive picture of what’s happening.</li></ul><h3>Tracing</h3><p>In .NET applications, tracing often involves specific concepts and terminologies that have their counterparts in OpenTelemetry. This section explains these .NET tracing concepts and their equivalents.</p><p><strong><em>Resource</em></strong>: is a collection of attributes that describe the entity generating telemetry data. This includes information such as the environment, service, host and other contextual details that help identify where and under what conditions the telemetry data was produced.</p><pre>public void AddOpenTelemetryRegistration(this WebApplicationBuilder builder)<br>{<br>    var resource = ResourceBuilder.CreateDefault().AddService(<br>            ApplicationConstants.ServiceName,<br>            ApplicationConstants.ServiceNamespace, <br>            ApplicationConstants.ServiceVersion<br>        );<br><br>    builder.Services.AddOpenTelemetry()<br>        .WithTracing(traces =&gt; traces<br>            .SetResourceBuilder(resource)<br>        );<br>}</pre><p><strong><em>Trace Provider ( ActivitySource )</em></strong>: ActivitySource is a class in .NET that is responsible for creating and managing Activity instances which represent individual operations or spans within a trace. It serves as the entry point for instrumenting code with tracing and is used to generate trace data that can be collected and analyzed.</p><pre>public static class ActivitySourceProvider<br>{<br>    public static ActivitySource Source = null;<br>}<br><br>public void AddOpenTelemetryRegistration(this WebApplicationBuilder builder)<br>{<br>    ActivitySourceProvider.Source = new ActivitySource(<br>        ApplicationConstants.ActivitySourceName);<br><br>    builder.Services.AddOpenTelemetry()<br>        .WithTracing(traces =&gt; traces<br>            .AddSource(ApplicationConstants.ActivitySourceName)<br>        );<br>}</pre><p><strong><em>Span ( Activity )</em></strong>: is the essential elements in the tracing model, capturing detailed information about each step or process in a distributed system.</p><p>A span context carries the trace ID, span ID and other essential information that is passed between services.</p><pre>using var activity = ActivitySourceProvider.Source.StartActivity(<br>            name: nameof(CreatePersonCommandHandler)<br>            //Add Span Kind<br>            kind: ActivityKind.Server, <br>          );<br><br><br>//Create Attribute<br>activity?.SetTag(&quot;key&quot;, &quot;value&quot;);<br><br>//Create Event<br>var eventTags = new ActivityTagsCollection();<br>eventTags.Add(&quot;key&quot;, &quot;value&quot;);<br>activity?.AddEvent(new ActivityEvent(&quot;user created.&quot;,tags:eventTags) );<br><br>//Set Span Status<br>activity.SetStatus(ActivityStatusCode.Error, ex.Message);</pre><p><strong><em>Span Components:</em></strong></p><ul><li><strong><em>Operation Name</em></strong>: A descriptive name for the operation that the span represents ( HttpPost /api/v1/persons )</li><li><strong><em>Start and End Time</em></strong>: Spans have a start time and an end time which helps measure the latency or duration of an operation.</li><li><strong><em>Parent Span ID</em></strong>: Spans can be nested, representing the hierarchical relationship of operations. A span can have a parent span, creating a tree-like structure of related operations.</li><li><strong><em>Attributes ( Tags )</em></strong>: Key-value pairs that provide additional context, such as status codes or relevant metadata.</li><li><strong><em>Events ( Logs )</em></strong>: Timestamped events that occur during the operation, often used for logging significant moments like errors or checkpoints.</li><li><strong><em>Span ( Activity ) Kinds</em></strong>; Internal, Server, Client, Producer, Consumer…</li><li><strong><em>Span ( Activity ) Status</em></strong>; Ok, Error, Unset…</li></ul><h4>Example of a Span in Action</h4><p>Consider a web application where a user makes a request to create a user. The trace for this request might contain several spans:</p><ul><li><strong>Span 1</strong>: “HttpPost /api/v1/persons” (Root span)</li><li><strong>Span 2</strong>: “Add user details to database” (Child span of Span 1)</li><li><strong>Span 3</strong>: “Send an event which user created” (Child span of Span 1)</li></ul><h3>Metrics</h3><p><strong>Metrics</strong> are numerical data points that represent the quantitative characteristics of a system over time. In the context of observability, metrics help monitor the health and performance of applications, services and infrastructure by measuring things like latency, request counts, error rates, CPU usage, memory consumption and more.</p><ul><li>Counter: Represents a value that only ever increases, like the number of HTTP requests or errors. For example, a counter could track how many users created on a website.</li><li>Gauge: Represents a value that can go up or down, like CPU usage or temperature. A gauge captures the current state, for example, the number of active users or memory utilization.</li><li>Histogram: Measures the distribution of values, such as request latency. It records data into buckets and provides insights into the spread and distribution of measurements, like tracking how long requests take.</li></ul><pre>public static class OpenTelemetryMetric<br>{<br>    public static readonly string UserCreatedEventCounterMeterName = <br>            &quot;UserCreatedEventCounter.Meter&quot;;<br><br>    private static readonly Meter meter = <br>            new Meter(UserCreatedEventCounterMeterName);<br>    <br>    public static Counter&lt;int&gt; UserCreatedEventCounter = <br>            meter.CreateCounter&lt;int&gt;(&quot;user.created.event.count&quot;);<br>}<br><br>public void AddOpenTelemetryRegistration(this WebApplicationBuilder builder)<br>{<br>    builder.Services.AddOpenTelemetry()<br>        .WithMetrics(metrics =&gt; metrics<br>            .AddMeter(OpenTelemetryMetric.UserCreatedEventCounterMeterName)<br>        );<br>}<br><br>using (var activity = ActivitySourceProvider.Source.StartActivity(<br>            name: nameof(CreatePersonCommandHandler)<br>            kind: ActivityKind.Server, <br>          ))<br>{<br>  OpenTelemetryMetric.UserCreatedEventCounter.Add(1,<br>            new KeyValuePair&lt;string, object?&gt;(&quot;event&quot;,&quot;add&quot;));<br>}</pre><h3>Logging</h3><p>Logging involves recording information during the execution of an application. Logs typically capture a snapshot of events, errors or informational messages that can help developers understand what’s happening inside an application. They provide a timeline of actions and context, such as the state of the application at a specific time, error details and user or system interactions.</p><p>Each log can carry trace and span context, linking logs to specific traces. This is crucial for understanding the broader context of what’s happening in your system.</p><p>The OpenTelemetry Collector can be configured to process logs. Filter them based on certain criteria and enrich them with additional data. The Collector exports logs to backends like Elasticsearch, Loki or other log storage solutions. From there, logs can be visualized, queried and analyzed using tools like Kibana or Grafana.</p><pre>public static void AddOpenTelemetryLoggingRegistration(this WebApplicationBuilder builder)<br>{<br>    var resource = ResourceBuilder.CreateDefault().AddService(<br>            ApplicationConstants.ServiceName,<br>            ApplicationConstants.ServiceNamespace, <br>            ApplicationConstants.ServiceVersion<br>        );<br>    <br>    builder.Logging.AddOpenTelemetry(options =&gt;<br>    {<br>        options.SetResourceBuilder(resource);<br>        options.AddOtlpExporter();<br>    <br>        options.IncludeScopes = true;<br>        options.IncludeFormattedMessage = true;<br>        options.ParseStateValues = true;<br>    });<br>}</pre><h3>Instrumentation</h3><p><strong>Instrumentations</strong> refer to libraries or code integrations that automatically generate telemetry data like traces, metrics and logs without requiring significant manual coding. In the context of OpenTelemetry, instrumentations are responsible for automatically capturing information from frameworks, libraries and applications so that you can monitor their performance and behavior.</p><pre>dotnet add package OpenTelemetry.Extensions.Hosting<br>dotnet add package OpenTelemetry.Instrumentation.AspNetCore<br>dotnet add package OpenTelemetry.Instrumentation.Runtime<br>dotnet add package OpenTelemetry.Instrumentation.Process<br>dotnet add package OpenTelemetry.Instrumentation.Http<br>dotnet add package OpenTelemetry.Instrumentation.EntityFrameworkCore<br>dotnet add package OpenTelemetry.Exporter.Prometheus.AspNetCore<br>dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol<br>dotnet add package OpenTelemetry.Exporter.Console</pre><h3>Distributed Tracing</h3><p>In a monolithic application, a single trace (or request) can be followed within the boundaries of a single process or application. However, in modern distributed systems where multiple microservices are involved, a single request might span across several services. Each service contributes to the overall processing of the request, making it challenging to understand performance bottlenecks or detect failures.</p><p>Distributed tracing addresses this challenge by recording data as the request moves across different services. It provides a complete view of the request lifecycle, from the initial user action to the final response, including any internal service-to-service calls.</p><p>Distributed tracing uses three primary concepts;</p><ul><li><strong><em>Trace</em>:</strong> A trace is the complete journey of a request across different services in a system. It represents the overall process that a user action or request triggers. It consists of multiple spans that together form the entire trace.</li><li><strong><em>Span</em>:</strong> A span is a single unit of work in a trace. Each span represents an operation, such as an API call or a function execution. Spans have important attributes like a unique span ID, a parent span ID (indicating which span triggered it), start and end timestamps and metadata (tags, events etc.)</li><li><strong><em>Context Propagation</em>:</strong> Context propagation allows trace information (like trace ID and span ID) to be passed across different services. When a request is passed from one service to another, the trace context is propagated, ensuring that all operations within a single trace are linked.</li></ul><h4>traceparent</h4><p>The traceparent header carries trace context information, such as the trace ID, span ID and trace flags, in a standardized format. It allows different services to share the same trace context, making it possible to link spans together into a single trace that represents the flow of a request through multiple microservices or distributed components.</p><p>The traceparent is part of the <a href="https://www.w3.org/TR/trace-context/">W3C Trace Context specification</a>, which defines how trace context information should be propagated between services.</p><p><strong><em>traceparent Header Structure;</em></strong></p><pre>traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01</pre><p>This example breaks down into four parts:</p><ul><li><strong>Version</strong>: 00<br>Indicates the version of the Trace Context specification.</li><li><strong>Trace ID</strong>: 4bf92f3577b34da6a3ce929d0e0e4736<br>A 16-byte unique identifier for the trace. All spans within the same trace share this trace ID.</li><li><strong>Span ID</strong>: 00f067aa0ba902b7<br>A unique 8-byte identifier for the individual span. Each span within a trace has its own unique span ID.</li><li><strong>Trace Flags</strong>: 01<br>A 1-byte flag used to indicate whether the trace should be sampled. In this example, 01 means the trace is sampled.</li></ul><h4>tracestate</h4><p>The tracestate header is a part of the <a href="https://www.w3.org/TR/trace-context/">W3C Trace Context specification</a>, designed to propagate additional tracing metadata that is specific to one or more vendors. While traceparent focuses on conveying standardized trace context data, tracestate allows for the inclusion of proprietary or vendor-specific data within a trace.</p><p>The tracestate header contains a list of key-value pairs, where each key represents a vendor and the value represents the vendor-specific trace data;</p><pre>tracestate: vendor1=value1,vendor2=value2</pre><ul><li><strong>Key:</strong> Identifies the vendor or system that owns the data.</li><li><strong>Value:</strong> The associated metadata that the vendor wants to pass along.</li></ul><pre>activity.SetBaggage(&quot;userId&quot;, request.UserId.ToString());<br>var userId = Activity.Current?.GetBaggageItem(&quot;userId&quot;);</pre><h3>Otel Collector</h3><p>The <strong>OpenTelemetry Collector</strong> is a key component in the OpenTelemetry ecosystem, designed to receive, process and export telemetry data (traces, metrics and logs) from various sources. Unlike traditional SDKs which are tied to specific languages or environments. The OpenTelemetry Collector is a vendor-agnostic, stand-alone service that can be deployed in different environments like on-premises, cloud or containerized setups.</p><p>The Collector allows you to filter, sample, aggregate or enrich telemetry data centrally before sending it to observability backends.</p><p>The Collector is typically broken down into three core sections:</p><ol><li><strong>Receivers:</strong> Receivers are the components that gather telemetry data from various sources. You can think of them as the inputs to the Collector.</li><li><strong>Processors:</strong> Once the data is received, it can be processed before being exported. Processing includes actions like batching, filtering or transforming the data.</li><li><strong>Exporters:</strong> Exporters are the components that send processed telemetry data to specific destinations like monitoring and observability tools.</li></ol><h4>Use Cases:</h4><ul><li><strong>Centralized Telemetry Pipeline:</strong> Collect and aggregate telemetry data across multiple services or environments.</li><li><strong>Data Transformation:</strong> Modify or enrich data before sending it to a backend.</li><li><strong>Multi-Destination Exporting:</strong> Send telemetry data to multiple backends simultaneously, like Prometheus for metrics and Jaeger for traces.</li><li><strong>Advanced Sampling and Filtering:</strong> Implement centralized sampling strategies, filtering unwanted data before export, and reducing overhead.</li></ul><h4>OpenTelemetry Collector Config File:</h4><pre>receivers:<br>  otlp:<br>    protocols:<br>      grpc:<br>        endpoint: 0.0.0.0:4317<br><br>exporters:<br>  otlp:<br>    endpoint: jaeger:4317<br>    tls:<br>      insecure: true<br>  prometheus:<br>    endpoint: &quot;0.0.0.0:9090&quot;<br>  loki:<br>    endpoint: &quot;http://loki:3100/loki/api/v1/push&quot;<br><br>processors:<br>  batch:<br>    timeout: 5s<br>  filter/ottl:<br>    error_mode: ignore<br>    logs:<br>      log_record:<br>        - &#39;instrumentation_scope.name != &quot;Microsoft.AspNetCore.Hosting.Diagnostics&quot;&#39;<br>        - &#39;instrumentation_scope.name != &quot;Microsoft.AspNetCore.Hosting.Diagnostics&quot;&#39;<br>        - &#39;instrumentation_scope.name != &quot;Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker&quot;&#39;<br>        - &#39;instrumentation_scope.name != &quot;Microsoft.Hosting.Lifetime&quot;&#39;<br><br>extensions:<br>  health_check:<br>    endpoint: &quot;0.0.0.0:13133&quot;<br><br>service:<br>  extensions: [health_check]<br>  telemetry:<br>    logs:<br>      level: info<br>  pipelines:<br>    traces:<br>      receivers: [otlp]<br>      processors: [batch]<br>      exporters: [otlp]<br>    metrics:<br>      receivers: [otlp]<br>      processors: [batch]<br>      exporters: [prometheus]<br>    logs:<br>      receivers: [otlp]<br>      processors: [batch]<br>      exporters: [loki]</pre><h3>Setting Up and Exploring Monitoring Tools</h3><p>I’ll walk through how to set up a monitoring and observability stack using Docker Compose with the OpenTelemetry Collector, Jaeger, Prometheus, Loki and Grafana. This setup enables seamless tracing, metrics collection and log aggregation for .NET application which I’ve written. We’ll configure the OpenTelemetry Collector to receive traces and metrics, forward them to Jaeger and Prometheus respectively and ship logs to Loki for centralized logging. With Grafana, you can visualize and correlate these observability signals, making it easier to diagnose issues in a distributed system. Let’s dive into how this can be achieved step-by-step.</p><pre>services:<br>  # OpenTelemetry Collector<br>  otel-collector:<br>    image: otel/opentelemetry-collector-contrib:0.107.0<br>    ports:<br>      - &quot;13133:13133&quot; # health_check extension<br>      - &quot;4317:4317&quot; # OTLP gRPC receiver<br>    volumes:<br>      - ./otel-collector-config.yaml:/etc/otel-collector-config.yaml<br>    command: [&quot;--config&quot;, &quot;/etc/otel-collector-config.yaml&quot;]<br>    depends_on:<br>      - jaeger<br>      - prometheus<br>      - loki<br>  <br>  # Jaeger<br>  jaeger:<br>    image: jaegertracing/all-in-one:1.60<br>    container_name: jaeger<br>    ports:<br>      - &quot;16686:16686&quot; # UI<br>      - &quot;14268&quot; # Collector HTTP<br>      - &quot;14250&quot; # Collector gRPC<br>      <br>  # Prometheus<br>  prometheus:<br>    image: prom/prometheus:latest<br>    container_name: prometheus<br>    volumes:<br>      - ./prometheus.yml:/etc/prometheus/prometheus.yml<br>    ports:<br>      - &quot;9090:9090&quot;<br>      <br>  # Grafana<br>  grafana:<br>    image: grafana/grafana:latest<br>    container_name: grafana<br>    ports:<br>      - &quot;3000:3000&quot;<br>    environment:<br>      - GF_SECURITY_ADMIN_PASSWORD=admin<br>    depends_on:<br>      - prometheus<br>      <br>  # Loki<br>  loki:<br>    image: grafana/loki:3.0.0<br>    container_name: loki<br>    ports:<br>      - &quot;3100:3100&quot;<br>    volumes:<br>      - ./loki-config.yaml:/etc/loki/loki-config.yaml<br>    command: [&quot;-config.file=/etc/loki/loki-config.yaml&quot;]</pre><h4>Create User with .Net App</h4><p>We will manage and search logs using Loki, inspect trace data through Jaeger, and monitor user creation metrics with Prometheus. The trace ID will be utilized to correlate and analyze the data, as illustrated in the screenshot below.</p><p><strong><em>trace-id</em></strong>: 3d595b9acc21ca83f6afcb97d8ec2050</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LLKQdfQ9B00NvkCudJP2fA.png" /></figure><h4>Prometheus</h4><p>Prometheus is an open-source monitoring and alerting toolkit designed for collecting and querying metrics from applications and services. It uses a time-series database to store metrics and supports powerful querying with PromQL.</p><p>In .Net app, we have integrated a custom metric for monitoring user creation events. Specifically, we use OpenTelemetry to emit a metric called user.created.event.count whenever a new user is created. This metric helps track the number of user creation events over time.</p><pre><br>public static class OpenTelemetryMetric<br>{<br>    public static readonly string UserCreatedEventCounterMeterName = <br>            &quot;UserCreatedEventCounter.Meter&quot;;<br>    private static readonly Meter meter = <br>            new Meter(UserCreatedEventCounterMeterName);<br>    <br>    public static Counter&lt;int&gt; UserCreatedEventCounter = <br>            meter.CreateCounter&lt;int&gt;(&quot;user.created.event.count&quot;);<br>}<br>public class CreatePersonCommandHandler<br>{<br>    public void Handle()<br>    {<br>        OpenTelemetryMetric.UserCreatedEventCounter.Add(1,<br>              new KeyValuePair&lt;string, object?&gt;(&quot;event&quot;,&quot;add&quot;));<br>    }<br>}<br></pre><p>The accompanying screenshot from Prometheus shows how this metric is visualized and monitored. Prometheus collects and stores these metrics, allowing us to analyze and query the data effectively. By setting up appropriate alerts and dashboards, you can gain insights into user activity and system performance.</p><p><strong><em>Url</em></strong>: <a href="http://localhost:9090/">http://localhost:9090/</a></p><p>Query: “rate(user_created_event_count_total[3m])”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*umKwDLGAOboQpgAlAPbdpA.png" /></figure><h4>Grafana</h4><p>You can visualize metrics in Grafana by connecting it to Prometheus. Grafana allows you to create dynamic dashboards and graphs to monitor user.created.event.count and other metrics, providing a powerful way to analyze and visualize your data.</p><p><strong><em>Url</em></strong>: <a href="http://localhost:3000/">http://localhost:3000/</a></p><p>Login Grafana ;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/702/1*hJ38TlqURLlGmOCt2JUOwQ.png" /></figure><p>Configure Prometheus as a Data Source in Grafana ;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*2i9Sud_QPOR6VtLDGLuwDA.png" /></figure><p>Create Dashboard in Grafana ;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*kqor171dhcsxNwbx3yGMyA.png" /></figure><h4>Loki</h4><p>Logs will be managed through Loki, which provides a scalable and efficient log aggregation system. In Grafana, you can set up Loki as a data source to query and visualize your logs. This integration allows for seamless log searching, filtering, and analysis directly within Grafana’s user-friendly interface.</p><p><strong><em>Url</em></strong>: <a href="http://localhost:3100/ready">http://localhost:3100/ready</a></p><p>Configure Loki as a Data Source in Grafana;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*y6ZUEclewXrR-8W6225iOA.png" /></figure><p>Searching Logs in Grafana Loki ;</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hGlRgCKlPUlqWununmDLnw.png" /></figure><h4>Jaeger</h4><p>Jaeger is a distributed tracing system that helps in monitoring and troubleshooting complex applications. It provides insights into how requests flow through your system by visualizing trace data. Using trace IDs, Jaeger allows you to track and analyze the lifecycle of specific requests across different services, helping identify performance bottlenecks and understand service interactions.</p><p><strong>Url</strong>; <a href="http://localhost:16686/trace/3d595b9acc21ca83f6afcb97d8ec2050">http://localhost:16686/trace/3d595b9acc21ca83f6afcb97d8ec2050</a></p><p>Search with trace-id ; 3d595b9acc21ca83f6afcb97d8ec2050</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*oEUThdD0BX0_wTuH-4c4MA.png" /></figure><h3>Source Code</h3><p><a href="https://github.com/cnrdmrci/HexagonalArchitecture">GitHub - cnrdmrci/HexagonalArchitecture: Hexagonal Architecture Sample</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=23c3548ab805" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Modular Monolithic Architecture: Domain-Centric Design]]></title>
            <link>https://medium.com/@caner.demirci/modular-monolithic-architecture-domain-centric-design-e082d79eb821?source=rss-ee1d4a1caf0e------2</link>
            <guid isPermaLink="false">https://medium.com/p/e082d79eb821</guid>
            <category><![CDATA[architecture]]></category>
            <category><![CDATA[modular-monolith]]></category>
            <dc:creator><![CDATA[Caner Demirci]]></dc:creator>
            <pubDate>Sat, 31 Aug 2024 17:12:52 GMT</pubDate>
            <atom:updated>2024-08-31T17:12:52.484Z</atom:updated>
            <content:encoded><![CDATA[<h3>What is Modular Monolithic Architecture?</h3><p><strong>Modular Monolithic Architecture</strong> is a design approach where a single, unified codebase is divided into well-defined, independent modules while remaining a single application. This architecture combines the simplicity of monolithic applications with the benefits of modular design.</p><p>Modular Monolithic Architecture combines the principles of modular design with the monolithic application style;</p><ul><li><strong>Monolithic</strong>: The entire application is deployed as a single unit which means all components share the same process and runtime.</li><li><strong>Modular</strong>: The application is divided into distinct modules or components, each responsible for a specific aspect of the application’s functionality. These modules interact with each other but are designed to be as independent as possible.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v6TiQs_iPxMVRg9p" /><figcaption>Photo by <a href="https://unsplash.com/@boliviainteligente?utm_source=medium&amp;utm_medium=referral">BoliviaInteligente</a> on <a href="https://unsplash.com/?utm_source=medium&amp;utm_medium=referral">Unsplash</a></figcaption></figure><h3>Advantages of Modular Monolithic</h3><ul><li><strong><em>Simplicity</em></strong>: Maintaining a single codebase simplifies deployment, testing and operations compared to microservices.</li><li><strong><em>Performance</em></strong>: Modules within the same process can communicate directly, which is faster than network calls between distributed services.</li><li><strong><em>Code Organization</em></strong>: Modular design improves code manageability and readability by separating concerns within a unified application.</li><li><strong><em>Easier Refactoring</em>: </strong>All modules reside in the same codebase, making it easier to refactor and update the application as a whole. End-to-end testing is more simple because everything runs in a single process.</li><li><strong><em>Reduced Overhead</em></strong>: Unlike microservices, there’s no need for complex communication protocols or data serialization between services.</li><li><strong><em>Single Deployment Pipeline</em>:</strong> A unified pipeline for building, testing and deploying the application reduces the complexity of the DevOps process.</li></ul><h3>Disadvantages of Modular Monolithic</h3><ul><li><strong><em>Scalability Limitations</em>: </strong>Scaling the application involves scaling the entire monolith which may not be as efficient as scaling individual microservices. If one module needs more resources, the entire application may require scaling, even if other modules do not.</li><li><strong><em>Complexity in Large Applications</em></strong>: As the application grows, the single codebase can become unmanageable and understanding more challenging.</li><li><strong><em>Deployment Challenges</em></strong>: If the monolithic application fails, the entire system goes down, potentially affecting all users. Deploying the entire application can be slower and riskier compared to deploying smaller, independent services.</li><li><strong><em>Team Coordination</em></strong>: In large teams, managing changes and resolving conflicts within a single codebase can become inefficient.</li></ul><h3>Communication Between Modules</h3><p>Communication between modules is crucial for maintaining cohesion and functionality across the application. Here are the two primary methods of communication between modules;</p><h4>Communication via Interface (Synchronous way)</h4><p>Modules often communicate with each other through well-defined interfaces. These interfaces are usually abstract classes or interfaces in programming languages that define the methods and properties that other modules can use.</p><h4>Communication via Bus (Asynchronous way)</h4><p>Communication via a bus involves using a central communication mechanism or event bus that modules use to send and receive messages or events. This approach can be useful for managing asynchronous communication and decoupling modules.</p><h3>Project Structure</h3><p>Below is an example of a project structure designed for a modular monolithic architecture using Go. It organizes the application into separate, well-defined modules while keeping the entire codebase unified and manageable:</p><pre>├── README.md<br>├── docker-compose.yml<br>├── go.mod<br>├── go.sum<br>├── build<br>│   └── ci<br>│       ├── docker<br>│       │   ├── user-api<br>│       │   │   └── Dockerfile<br>│       │   └── product-api<br>│       │       └── Dockerfile<br>│       └── kube<br>│           ├── user-api<br>│           │   └── deployment.yaml<br>│           └── product-api<br>│               └── deployment.yaml<br>├── cmd<br>│   ├── user-api<br>│   │   └── main.go<br>│   └── product-api<br>│       └── main.go<br>└── internal<br>    ├── app<br>    │   ├── user-api<br>    │   │   ├── create_user.go<br>    │   │   ├── create_user_test.go<br>    │   │   ├── get_user.go<br>    │   │   ├── get_user_test.go<br>    │   │   ├── handlers.go<br>    │   │   └── service.go<br>    │   └── product-api<br>    │       ├── create_user.go<br>    │       ├── create_user_test.go<br>    │       ├── get_user.go<br>    │       ├── get_user_test.go<br>    │       ├── handlers.go<br>    │       └── service.go<br>    └── pkg<br>        ├── auth<br>        │   └── authorize.go<br>        ├── config<br>        │   └── config.go<br>        ├── email<br>        │   └── sender.go<br>        ├── middleware<br>        │   ├── auth_middleware.go<br>        │   └── logging.go<br>        ├── models<br>        │   ├── errors.go<br>        │   ├── product.go<br>        │   └── user.go<br>        └── server<br>            └── server.go</pre><ul><li><strong><em>cmd</em></strong> : Contains the entry points for different modules.</li><li><strong><em>internal/app </em></strong>: Holds application-specific logic that should not be accessed outside the package.</li><li><strong><em>internal/pkg </em></strong>: Includes reusable components and utilities for all modules.</li><li><strong><em>build </em></strong>: Contains configuration and build scripts.</li></ul><h3>Monolithic vs Modular Monolithic vs Microservice</h3><ul><li><strong><em>Monolithic Architecture</em></strong>: All components are integrated into a single, cohesive application. This can simplify deployment but can lead to scalability issues and difficulties in managing large codebases.</li><li><strong><em>Modular Monolithic Architecture</em></strong>: The application is divided into distinct modules within a single codebase. Each module has its own functionality, improving maintainability and separation of concerns while keeping the deployment and management simpler compared to microservices.</li><li><strong><em>Microservice Architecture</em></strong>: The application is divided into smaller, self-contained services that can be deployed independently. Each service is responsible for a specific business function and communicates with other services over a network. This approach improves scalability and flexibility, but it can also add complexity in terms of managing multiple services and handling inter-service communication.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e082d79eb821" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>