<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Daniel Maina on Medium]]></title>
        <description><![CDATA[Stories by Daniel Maina on Medium]]></description>
        <link>https://medium.com/@majorwrites?source=rss-bc98b06ce1f0------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*l6SdAqhVWzB3PPbq</url>
            <title>Stories by Daniel Maina on Medium</title>
            <link>https://medium.com/@majorwrites?source=rss-bc98b06ce1f0------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 03:10:32 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@majorwrites/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Build a Blockchain From Scratch using Golang: Part 3 — Nodes and the Mempool]]></title>
            <link>https://medium.com/@majorwrites/build-a-blockchain-from-scratch-using-golang-part-3-nodes-and-the-mempool-eeb98cb4c1ae?source=rss-bc98b06ce1f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/eeb98cb4c1ae</guid>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[software-development]]></category>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[golang]]></category>
            <dc:creator><![CDATA[Daniel Maina]]></dc:creator>
            <pubDate>Wed, 12 Feb 2025 11:31:38 GMT</pubDate>
            <atom:updated>2025-02-17T08:29:18.975Z</atom:updated>
            <content:encoded><![CDATA[<h3>Build a Blockchain From Scratch using Golang: Part 3 — Nodes and the Mempool</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CcFbtnuprTFNq60HfctHdA.png" /><figcaption>Source: aminagroup.com</figcaption></figure><p>In <a href="https://medium.com/@majorwrites/build-a-blockchain-from-scratch-using-golang-part-2-ef0b3a5ec2f7">part 2</a> of the series, we calculated the Merkle root, verified blocks, and ensured the integrity of our blockchain by validating blocks before adding them to the chain. In this part, we will introduce the <strong>mempool</strong> and its significance.</p><p>Have you ever wondered how transactions get processed in crypto blockchains like Bitcoin/Ethereum before they are confirmed? The mempool is like a waiting room where transactions queue up before being added to a block. In this part, we’ll explore how it works.</p><p>But, before taking a deep dive into the mempool, it is essential to understand <strong>nodes</strong> first as they are where the mempool resides.</p><h3>Nodes in a Blockchain Network</h3><p>Nodes: the backbone of any blockchain network. They are computers that run the blockchain software and maintain a copy of the blockchain ledger. By storing this ledger, nodes can verify and validate each other, ensuring the network’s integrity without the need for a central authority. This decentralized architecture sets blockchains apart from traditional systems like banks, which have the final say over transactions.</p><h4>Key Functions of Nodes:</h4><ol><li><strong>Storage</strong>: Nodes maintain copies of the blockchain ledger.</li><li><strong>Verification</strong>: Nodes validate transactions and blocks.</li><li><strong>Communication</strong>: Nodes share information with one another keeping the network in sync.</li><li><strong>Consensus</strong>: Through communication, nodes ensure all copies of the blockchain are identical.</li><li><strong>Transaction management(mempool):</strong> nodes temporarily store unconfirmed transactions until they are added to the block.</li></ol><h4>Types of Nodes</h4><p>Blockchain nodes can generally be categorized into two main types: <strong>full nodes</strong> and <strong>light nodes</strong>.</p><h4>1. Full Nodes</h4><ul><li><strong>Function</strong>: Store the entire blockchain ledger, including all transactions and blocks from the beginning of the chain.</li><li><strong>Role</strong>: Provide a full copy of the blockchain to new nodes joining the network.</li><li><strong>Advantages</strong>: Offer maximum security and independence, as they verify all transactions and blocks directly.</li><li><strong>Requirements</strong>: Significant storage, processing power, and bandwidth are required.</li><li>They have a mempool.</li></ul><h4>2. Light Nodes</h4><ul><li><strong>Function</strong>: Store only block headers instead of the entire blockchain ledger.</li><li><strong>Role</strong>: Rely on full nodes to validate transactions, making them more lightweight.</li><li><strong>Advantages</strong>: Ideal for devices with limited storage or processing power, such as mobile phones.</li><li><strong>Limitations</strong>: Depend on full nodes for transaction verification and cannot independently verify the entire chain.</li><li>They <strong>do not</strong> have a mempool.</li></ul><p>There are other types of nodes like <strong>miner</strong> and <strong>validator nodes</strong>, etc. Read more about them <a href="https://builtin.com/blockchain/blockchain-node">here</a>.</p><blockquote><strong>Why Decentralization Matters</strong></blockquote><blockquote>The decentralized nature of blockchain ensures that no single entity controls the network. Instead, nodes work together to verify and add transactions, broadcast information, and maintain consensus. This design enhances security, transparency, and resilience, making blockchains highly robust compared to traditional centralized systems.</blockquote><h4>Node Communication</h4><p>Nodes in a blockchain network communicate using a peer-to-peer (P2P) architecture. Instead of relying on a central server, each node connects to multiple peers and shares information using the gossip protocol. This means transactions and blocks spread efficiently from node to node, ensuring the blockchain remains synchronized across the network. <strong>M<em>ore on this in the next part of the series.</em></strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*U2z2cgZHqaZZY-fImDadVg.jpeg" /><figcaption>Diagram showing nodes and their connected peers. Source: me 🙂</figcaption></figure><h3>Mempool</h3><p>Each full node has a mempool <em>(memory pool</em>), a temporary storage for transactions before they are added to the block. Think of it as a pending queue of transactions awaiting addition to a new block.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vqTgf8cWEpVcvfWUQyxPtg.jpeg" /><figcaption>Each node has its own mempool: Source: me 🙂</figcaption></figure><h4>How does the mempool work?</h4><p>Imagine a case where a user initiates a transaction, e.g., sends 10 BTC to person X from a crypto wallet.</p><ul><li>This transaction is sent to a node <strong>A,</strong> which the wallet is connected to and is stored in the mempool.</li><li>Node <strong>A</strong> performs validation checks on the transaction e.g. if the transaction has a valid signature, there&#39;s no case of double spend.</li><li>If valid, node <strong>A</strong> forwards the transaction to its peers, and it’s spread to the entire network,<em> i.e., transaction propagation using the gossip protocol.</em></li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ylMFGrG_ceHiUfcuQLZapw.jpeg" /><figcaption>Gossip Protocol in action for block propagation to peers: Source: me 🙂</figcaption></figure><p>So, we have transactions in the mempool. You might ask. What criteria do we use to pick transactions from the mempool first? 🤔</p><p>We can employ either of the following strategies:-</p><ul><li><strong>Transaction fee</strong> — Miners prefer transactions with higher fees to be prioritized. Miners want to maximize profits.</li><li><strong>Transaction size</strong> — transactions that take up less space and have higher fees are prioritized. Blocks have size limits, so miners prefer smaller transactions with higher fees.</li><li><strong>Transaction time in mempool</strong> — some nodes might prefer older transactions(that have spent more time in the mempool) to prevent indefinite delays.</li><li><strong>Replace-by-fee(RBF)</strong> —in Bitcoin, if a transaction is marked as RBF-enabled, a new version of the transaction with a higher fee can replace it.</li></ul><p>These are a few examples. Feel free to look up other possible strategies.</p><h4>Coding the Mempool Operations</h4><p>In this section, we’ll create the structure and methods that will manage the mempool e.g. adding transactions, removing one, and clearing the mempool among others. This will be important in future sections of this series when we introduce nodes that utilize these operations to manage their mempool.</p><p>In the root of the project, create a new folder, and name it network then add a new file called mempool.go</p><pre>// network/mempool.go<br>package network<br><br>import (<br> &quot;github.com/majorshift/safari-chain/crypto&quot;<br> &quot;github.com/majorshift/safari-chain/types&quot;<br> &quot;sync&quot;<br>)<br><br>// MemPool is the structure for the mempool<br>type MemPool struct {<br> allTransactions     *TxMap // Stores all transactions in the pool<br> pendingTransactions *TxMap // Stores only pending transactions<br> maxSize             int    // Maximum number of transactions in the pool<br>}<br><br>func NewMempool(maxLength int) *MemPool {<br> return &amp;MemPool{<br>  allTransactions:     NewTxMap(),<br>  pendingTransactions: NewTxMap(),<br>  maxSize:             maxLength,<br> }<br>}<br><br>// Add inserts a new transaction to the mempool<br>func (p *MemPool) Add(tx *crypto.Transaction) {<br> // prune the oldest transaction that is sitting in the allTransactions pool<br> if p.allTransactions.Count() == p.maxSize {<br>  oldest := p.allTransactions.First()<br>  p.allTransactions.Remove(oldest.Hash(crypto.TxHash{}))<br> }<br><br> // prevent duplicate inclusion of transactions to mempool<br> if !p.allTransactions.Contains(tx.Hash(crypto.TxHash{})) {<br>  p.allTransactions.Add(tx)<br>  p.pendingTransactions.Add(tx)<br> }<br>}<br><br>// Contains checks if a transaction already exists in the mempool<br>func (p *MemPool) Contains(hash crypto.Hash) bool {<br> return p.allTransactions.Contains(hash)<br>}<br><br>// GetPendingTx returns a slice of transactions that are in the pending pool<br>func (p *MemPool) GetPendingTx() []*crypto.Transaction {<br> return p.pendingTransactions.transactionList.Data<br>}<br><br>// ClearPendingList deletes all transactions in the pending list of the mempool<br>func (p *MemPool) ClearPendingList() {<br> p.pendingTransactions.Clear()<br>}<br><br>// PendingTxCount returns the number of transactions in the pending list<br>func (p *MemPool) PendingTxCount() int {<br> return p.pendingTransactions.Count()<br>}<br><br>// AllTxCount returns the number of transactions in ever handled by the mempool<br>func (p *MemPool) AllTxCount() int {<br> return p.allTransactions.Count()<br>}<br><br>// TxMap The TxMap struct acts as a thread-safe transaction storage<br>// with both fast lookup and ordered access capabilities.<br>// It efficiently manages transactions by combining a hashmap<br>// for quick retrieval and a list for ordered storage while ensuring<br>// safe concurrent access using a read-write mutex.<br>type TxMap struct {<br> // Lock for concurrent access<br> lock sync.RWMutex<br> // Quick lookup for transactions by hash<br> transactionsByHash map[crypto.Hash]*crypto.Transaction<br> // Ordered list of transactions<br> transactionList *types.List[*crypto.Transaction]<br>}<br><br>func NewTxMap() *TxMap {<br> return &amp;TxMap{<br>  transactionsByHash: make(map[crypto.Hash]*crypto.Transaction),<br>  transactionList:    types.NewList[*crypto.Transaction](),<br> }<br>}<br><br>// First returns the first transaction in the pool<br>func (t *TxMap) First() *crypto.Transaction {<br> t.lock.RLock()<br> defer t.lock.RUnlock()<br> first := t.transactionList.Get(0)<br> return t.transactionsByHash[first.Hash(crypto.TxHash{})]<br>}<br><br>// Get returns transaction in the pool matching hash<br>func (t *TxMap) Get(h crypto.Hash) *crypto.Transaction {<br> t.lock.RLock()<br> defer t.lock.RUnlock()<br> return t.transactionsByHash[h]<br>}<br><br>// Add inserts a new transaction to the pool<br>func (t *TxMap) Add(tx *crypto.Transaction) {<br> hash := tx.Hash(crypto.TxHash{})<br><br> t.lock.Lock()<br> defer t.lock.Unlock()<br> if _, ok := t.transactionsByHash[hash]; !ok {<br>  t.transactionsByHash[hash] = tx<br>  t.transactionList.Insert(tx)<br> }<br>}<br><br>// Remove deletes transaction matching hash from the pool<br>func (t *TxMap) Remove(h crypto.Hash) {<br> t.lock.Lock()<br> defer t.lock.Unlock()<br> t.transactionList.Remove(t.transactionsByHash[h])<br> delete(t.transactionsByHash, h)<br>}<br><br>// Count returns the number of transactions in the transactionsByHash map<br>func (t *TxMap) Count() int {<br> t.lock.RLock()<br> defer t.lock.RUnlock()<br> return len(t.transactionsByHash)<br>}<br><br>// Contains checks if a transaction matching hash is contained in the mempool of a node<br>func (t *TxMap) Contains(h crypto.Hash) bool {<br> t.lock.RLock()<br> defer t.lock.RUnlock()<br> _, ok := t.transactionsByHash[h]<br> return ok<br>}<br><br>// Clear removes all transactions in the transactionsByHash<br>func (t *TxMap) Clear() {<br> t.lock.Lock()<br> defer t.lock.Unlock()<br> t.transactionsByHash = make(map[crypto.Hash]*crypto.Transaction)<br> t.transactionList.Clear()<br>}</pre><p>We have 2 structs in this file; Mempool and TxMap . The TxMap stores transactions both in a key-value store for quick access i.e. the transactionByHash property and an ordered list i.e. transactionList .</p><p>At this point, you might notice an error in the transactionList property of TxMap because List does not exist. Worry not, we’re fixing that next. We want to maintain an ordered list of items(in this case transaction) in the order in which they arrive in the mempool and List will help us achieve this. The List struct has methods to manage items in the list.</p><p>At the root of the project, create another directory and name it types and in it add a new file list.go</p><pre>package types<br><br>import (<br>    &quot;fmt&quot;<br>    &quot;reflect&quot;<br>)<br><br>// List is a generic list implementation<br>// that stores elements of any type<br>type List[T any] struct {<br>    Data []T<br>}<br><br>func NewList[T any]() *List[T] {<br>    return &amp;List[T]{<br>       Data: []T{},<br>    }<br>}<br><br>// Get finds element by index<br>func (l *List[T]) Get(index int) T {<br>    if index &gt; len(l.Data)-1 {<br>       err := fmt.Sprintf(&quot;the given index (%d) is higher than the length (%d)&quot;, index, len(l.Data))<br>       panic(err)<br>    }<br>    return l.Data[index]<br>}<br><br>// Insert adds item to list<br>func (l *List[T]) Insert(v T) {<br>    l.Data = append(l.Data, v)<br>}<br><br>// Clear deletes all items in the list<br>func (l *List[T]) Clear() {<br>    l.Data = []T{}<br>}<br><br>// GetIndex will return the index of v. If v does not exist in the list<br>// -1 will be returned.<br>func (l *List[T]) GetIndex(v T) int {<br>    for i := 0; i &lt; l.Len(); i++ {<br>       if reflect.DeepEqual(v, l.Data[i]) {<br>          return i<br>       }<br>    }<br>    return -1<br>}<br><br>// Remove deletes a item from the list<br>func (l *List[T]) Remove(v T) {<br>    index := l.GetIndex(v)<br>    if index == -1 {<br>       return<br>    }<br>    l.Pop(index)<br>}<br><br>// Pop deletes item matching index<br>func (l *List[T]) Pop(index int) {<br>    l.Data = append(l.Data[:index], l.Data[index+1:]...)<br>}<br><br>// Contains checks if an item v exists in the list<br>func (l *List[T]) Contains(v T) bool {<br>    for i := 0; i &lt; len(l.Data); i++ {<br>       if reflect.DeepEqual(l.Data[i], v) {<br>          return true<br>       }<br>    }<br>    return false<br>}<br><br>// Last returns the last item in the list<br>func (l List[T]) Last() T {<br>    return l.Data[l.Len()-1]<br>}<br><br>// Len returns the size of the list<br>func (l *List[T]) Len() int {<br>    return len(l.Data)<br>}</pre><p>So far so good. We have mempool operations defined and ready to be plugged into a node. Before signing off, we’ll add tests. Add a new file in the network directory and name it mempoll_test.go</p><pre>package network<br><br>import (<br> &quot;github.com/majorshift/safari-chain/crypto&quot;<br> &quot;github.com/stretchr/testify/assert&quot;<br> &quot;testing&quot;<br>)<br><br>func TestTxPoolAdd(t *testing.T) {<br> p := NewMempool(1)<br> assert.Equal(t, 0, p.PendingTxCount())<br> assert.Equal(t, 0, p.AllTxCount())<br><br> tx := crypto.NewTxWithSignature([]byte(&quot;hello world&quot;))<br> p.Add(tx)<br><br> assert.Equal(t, 1, p.PendingTxCount())<br> assert.Equal(t, 1, p.AllTxCount())<br><br> txHash := tx.Hash(&amp;crypto.TxHash{})<br> assert.True(t, p.allTransactions.Contains(txHash))<br>}<br><br>func TestTxPool_Prune(t *testing.T) {<br> // maximum number of transactions in pool is 1<br> p := NewMempool(1)<br><br> tx1 := crypto.NewTxWithSignature([]byte(&quot;hello world&quot;))<br> p.Add(tx1)<br> assert.Equal(t, 1, p.AllTxCount())<br><br> txHash1 := tx1.Hash(&amp;crypto.TxHash{})<br> assert.True(t, p.allTransactions.Contains(txHash1))<br><br> // add another transaction. expect original tx to be pruned<br> tx2 := crypto.NewTxWithSignature([]byte(&quot;hello world 1&quot;))<br> p.Add(tx2)<br><br> assert.Equal(t, 1, p.AllTxCount())<br><br> // transaction count remains the at maximum<br> assert.Equal(t, 1, p.allTransactions.Count())<br>}<br><br>func TestTxPool_ClearPending(t *testing.T) {<br> p := NewMempool(1)<br><br> tx1 := crypto.NewTxWithSignature([]byte(&quot;hello world&quot;))<br> p.Add(tx1)<br> assert.Equal(t, 1, p.PendingTxCount())<br><br> // clear pending<br> p.ClearPendingList()<br> assert.Equal(t, 0, p.PendingTxCount())<br>}</pre><p>These are just a few tests. Feel free to add more. To challenge yourself, try adding tests for types/list.go.</p><p>Find the code changes for this part <a href="https://github.com/majorshift/safari-chain/commit/6d29c6fe42eb22a5ec967b013b552092969e1c2d">here</a>.</p><h3>Conclusion</h3><p>In this part, we have accomplished the following:-</p><ol><li>We’ve explained the function and types of nodes in a blockchain network and how they communicate.</li><li>We’ve demystified the mempool, its importance and how they function in full nodes.</li><li>We’ve coded the operations of the mempool.</li></ol><p>In part 4 of this series, we’ll create the <strong>RPC</strong>(Remote Procedure Call) protocol to facilitate communication between nodes in the network. This will handle the transmission of data between nodes such as transactions, blocks and other messages ensuring efficient and secure data exchange within the network. See you on the next one 😎</p><p>References:-</p><ul><li><a href="https://www.ledger.com/academy/what-is-a-node-and-why-should-i-operate-one">Crypto Nodes: What Are They And How Do They Work? | Ledger</a></li><li><a href="https://utimaco.com/service/knowledge-base/blockchain/what-are-the-types-of-nodes-in-blockchain">What are the Types of Nodes in Blockchain?</a></li></ul><p><a href="https://cointelegraph.com/learn/articles/what-is-the-bitcoin-mempool">https://cointelegraph.com/learn/articles/what-is-the-bitcoin-mempool</a></p><ul><li><a href="https://www.blocknative.com/blog/mempool-intro">What is the Mempool? - How Blockchain Transactions Work</a></li><li><a href="https://www.cyfrin.io/blog/what-is-a-blockchain-mempool-and-how-does-it-work">Comprehensive Guide to Blockchain Mempools: How does it work?</a></li><li><a href="https://www.ledger.com/academy/what-is-a-mempool">What Is a Mempool? | Ledger</a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=eeb98cb4c1ae" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build a Blockchain From Scratch using Golang: Part 2]]></title>
            <link>https://medium.com/@majorwrites/build-a-blockchain-from-scratch-using-golang-part-2-ef0b3a5ec2f7?source=rss-bc98b06ce1f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/ef0b3a5ec2f7</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Daniel Maina]]></dc:creator>
            <pubDate>Thu, 30 Jan 2025 17:43:11 GMT</pubDate>
            <atom:updated>2025-02-12T11:32:19.884Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4TD6RX8a1l0vEwaIgJvWwg.jpeg" /><figcaption>Source from unsplash.com</figcaption></figure><p>In <a href="https://medium.com/p/3a37b87fbfde">part 1</a> of the series, we laid the foundation of our blockchain by setting up hashing, transactions, and some block functionality. You can refer to the code from the previous post <a href="https://github.com/majorshift/safari-chain/commit/a19f0bc30ae0e2468a0bd239bee6ec5fb76d02f7">here</a>.</p><p>In this part, we will create the Merkle Root <em>(hash of transactions), </em>verify blocks, and add blocks to the blockchain.</p><blockquote><strong>NOTE: I try to put explanations of what’s going on in the code in comments. It’s much easier that way. Make sure to go through them.</strong></blockquote><h4>Merkle Root</h4><p>The Merkle Tree is a computer science concept used majorly in blockchain and cryptography to ensure data integrity and consistency. In blockchain, Merkle roots ensure that data passed between nodes on a peer-to-peer network are whole, undamaged, and unaltered.</p><p>The Merkle Root is the hash value of all transactions in a block. For example, if there are <strong>n</strong> transactions to be added to a block, the Merkle root is calculated with the following steps:-</p><ol><li>Compute the hash of each transaction.</li><li>Pair up the hashes and hash the pairs. If <strong>n</strong> is an odd number, duplicate the last one then hash both.</li><li>Repeat the pairing &amp; hashing process with the resulting hashes until one hash is left. This final hash is the Merkle Root.</li></ol><p><em>See the image below to visualize the process</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8Jj1rLDduUHv2Dq3rbabfQ.jpeg" /><figcaption>A pictorial explanation of how to calculate the Merkle Root</figcaption></figure><p>Now, we’ll implement this process in the block.go file:-</p><pre>// ComputeMerkleRoot calculates the merkle root of a given list of transactions<br>func ComputeMerkleRoot(transactions []*Transaction) Hash {<br>  if len(transactions) == 0 {<br>    return Hash{}<br>  }<br><br>  // Step 1: Create a list of hashes from the transactions<br>  var txHashes []Hash<br>  for _, tx := range transactions {<br>    txHashes = append(txHashes, tx.Hash(TxHash{}))<br>  }<br>  // calculate merkle root by repeatedly pairing hashes until one hash is left<br>  for len(txHashes) &gt; 1 {<br>    var nextLevel []Hash<br>    for i := 0; i &lt; len(txHashes); i += 2 {<br>      // If there&#39;s an odd number of hashes, duplicate the last one<br>      if i+1 &gt;= len(txHashes) {<br>        nextLevel = append(nextLevel, hashPair(txHashes[i], txHashes[i]))<br>      } else {<br>        nextLevel = append(nextLevel, hashPair(txHashes[i], txHashes[i+1]))<br>      }<br>    }<br>    txHashes = nextLevel<br>  }<br>  // this is the merkle root<br>  return txHashes[0]<br>}<br>// hashPair combines 2 hashes<br>// returns a hash of the paired hashes<br>func hashPair(hash1, hash2 Hash) Hash {<br>  combined := append(hash1[:], hash2[:]...)<br>  h := sha256.Sum256(combined)<br>  return h<br>}</pre><p>We calculate the Merkle root when initializing a new block and also during the block verification process<em>(coming later in this post)</em>.</p><p>Let’s update the NewBlock function in the block.go file and initialize the block with its Merkle root:-</p><pre>func NewBlock(h *Header, txs []*Transaction) *Block {<br>  // compute the merkle root and add it to block header<br>  merkleRoot := ComputeMerkleRoot(txs)<br>  h.MerkleRoot = merkleRoot<br>  <br>  return &amp;Block{<br>    Header:       h,<br>    Transactions: txs,<br>  }<br>}</pre><h4>Block Verification</h4><p>Block validation is essential to maintain security, consensus and integrity across the blockchain network. Without it, malicious actors can create malicious transactions, double-spend cryptocurrency and create fraudulent blocks undermining the system.</p><p>Before a block is added to the blockchain, we need to perform some crucial validation operations:-</p><ol><li>check that it is signed</li><li>verify that the block signature is valid</li><li>verify all transactions in the block</li><li>recalculate the Merkle Root and confirm that transactions are not tampered with.</li></ol><p>If the above checks are okay, the block gets the green light to be added to the chain. On that note, open crypto/block.go file and add a Verify method:-</p><pre>// Verify checks the validity of a block<br>func (b *Block) Verify() error {<br> // check that the header is signed<br> if b.Signature == nil {<br>  return errors.New(&quot;block header has no signature&quot;)<br> }<br><br> // verify that the signature is valid i.e. no mismatch<br> if !b.Signature.Verify(b.Validator, b.Header.ToBytes()) {<br>  return errors.New(&quot;block header has invalid signature&quot;)<br> }<br><br> // check the validity of all transaction signatures; all must be valid<br> for _, tx := range b.Transactions {<br>  if err := tx.Verify(); err != nil {<br>   return err<br>  }<br> }<br><br> // verify merkle root<br> merkleRoot := ComputeMerkleRoot(b.Transactions)<br> if b.Header.MerkleRoot != merkleRoot {<br>  return fmt.Errorf(&quot;merkle root does not match&quot;)<br> }<br><br> return nil<br>}</pre><p>At this point, we’ll add tests for block functionality. But first, let’s update the crypto/example.go file by adding a handy function to generate a signed block<em>(remember that functions in this file help minimize code repetition when creating examples for tests).</em></p><pre>// NewSignedBlockExample creates a new blocked that is signed by a validator<br>func NewSignedBlockExample(validator *PrivateKey, transactions []*Transaction, height uint32, prevBlockHash Hash) *Block {<br> header := &amp;Header{<br>  Version:       1,<br>  PrevBlockHash: prevBlockHash,<br>  Height:        height,<br>  Timestamp:     time.Now().UnixNano(),<br> }<br><br> b := NewBlock(header, transactions)<br> b.Sign(validator)<br><br> return b<br>}</pre><p>Open the crypto/block_test.go file and add the following tests.</p><pre>func TestVerifyBlock_Success(t *testing.T) {<br> // create a validator private key<br> validator, _ := GeneratePrivateKey()<br> // the new block in this case is the genesis block<br> // so no previous hash<br> prevBlockHash := Hash{}<br><br> var transactions []*Transaction<br> // create seven signed transactions adding them to the transactiosn list<br> for i := 0; i &lt; 7; i++ {<br>  // append the index to Hello, World to generate unique transactions<br>  transactions = append(transactions, NewTxWithSignature([]byte(&quot;Hello, World&quot;+strconv.Itoa(i))))<br> }<br><br> // create a signed block<br> b := NewSignedBlockExample(validator, transactions, 0, prevBlockHash)<br><br> // error is not because transaction is not signed<br> assert.Nil(t, b.Verify())<br>}<br><br>func TestVerifyBlock_FailsForUnsignedTransaction(t *testing.T) {<br> validator, _ := GeneratePrivateKey()<br> prevBlockHash := Hash{}<br><br> // create a transaction without a signature<br> unsignedTx := &amp;Transaction{Data: []byte(&quot;Hello, World&quot;)}<br> b := NewSignedBlockExample(validator, []*Transaction{unsignedTx}, 0, prevBlockHash)<br><br> // error is not because transaction is not signed<br> assert.NotNil(t, b.Verify())<br>}<br><br>func TestComputeMerkleRoot(t *testing.T) {<br> var transactions []*Transaction<br> for i := 0; i &lt; 7; i++ {<br>  // append the index to Hello, World to generate unique transactions<br>  transactions = append(transactions, NewTxWithSignature([]byte(&quot;Hello, World&quot;+strconv.Itoa(i))))<br> }<br><br> validator, _ := GeneratePrivateKey()<br> // create a new block: the merkle root is created during block creation<br> b := NewSignedBlockExample(validator, transactions, 0, Hash{})<br><br> // recalculate merkle root<br> originalMerkleRoot := ComputeMerkleRoot(transactions)<br> assert.Equal(t, b.MerkleRoot, originalMerkleRoot)<br><br> // alter the first transaction<br> b.Transactions[0] = NewTxWithSignature([]byte(&quot;Altered Transaction&quot;))<br><br> // recalculate merkle root: comparison to original merkle root<br> // should show a mismatch<br> alteredMerkleRoot := ComputeMerkleRoot(b.Transactions)<br> assert.NotEqual(t, originalMerkleRoot, alteredMerkleRoot)<br><br> // verification fails<br> assert.Error(t, b.Verify())<br>}</pre><p>At the project’s root directory, run the command make test to check if all the tests are passing. <em>Feel free to play around with the tests and alter the assertions to see what makes them pass/fail.</em></p><p>So far, we have added functionality for calculating the Merkle Root and verifying a block. You can reference the changes on the latest commit <a href="https://github.com/majorshift/safari-chain/commit/49f937abdcf31a16055e6b1e49d74ed29f3dddb8">here</a>.</p><h3>Chaining the Blocks</h3><p>At this point, we can successfully create blocks that currently exist individually. Now we need to connect them forming the “chain”.</p><p>In the crypto folder, create a file and name it blockchain.go and add the following:-</p><pre>package crypto<br><br>type Blockchain struct {<br>  lock      sync.RWMutex   // Mutex to manage concurrent read and write access  <br>  headers   []*Header      // Slice to store block headers instead of full blocks for efficiency  <br>  logger    *logrus.Logger // Logger to track blockchain activity and debugging  <br>  validator Validator      // Validator to verify block and transaction validity  <br>}</pre><p>We create a struct named Blockchain and add three properties to it:-</p><p><strong>lock</strong> — This is a read-write mutex (RWMutex) from Go’s sync package, used to manage concurrent access to the blockchain. Since multiple goroutines will interact with the blockchain, RWMutex ensures safe reads and writes.</p><p>It allows multiple goroutines to <strong>read data concurrently</strong> (e.g., retrieving the number of headers stored) while ensuring that <strong>writes are blocked</strong> during reads. Conversely, when a goroutine performs a <strong>write operation</strong> (such as adding a new block header), all other goroutines are temporarily blocked from reading or writing. This mechanism prevents data corruption and ensures consistency.</p><p><strong>headers </strong>— we store a list of block headers to help us track the blockchain height. We store the header instead of the block since it reduces memory &amp; storage usage since they are lightweight. Also, instead of checking all transactions in a block, nodes can <strong>validate only the header</strong> to verify block correctness.</p><p><strong>logger</strong> — to log events and actions occurring in the system, helping to monitor system behavior. We’ll be using <a href="https://github.com/sirupsen/logrus">logrus</a> <em>(there are others like </em><a href="https://github.com/rs/zerolog"><em>zerolog</em></a>,<em> etc)</em>. To install logrus, run go get github.com/sirupsen/logrus at the root of the project.</p><p><strong>validator </strong>— this will help validate the blockchain. We’ll create the Validator shortly.</p><p>In the crypto/blockchain.go file, add the code below the new Blockchain struct.</p><pre>// NewBlockchain initializes a new Blockchain and sets up the logger<br>// the genesis block is also passed and added to the blockchain<br>// using the addBlock method. A blockchain validator is also initialized<br>func NewBlockchain(log *logrus.Logger, genesis *Block) *Blockchain {<br>  bc := &amp;Blockchain{<br>    headers: []*Header{},<br>    logger:  log,<br>  }<br>  <br>  bc.validator = NewBlockValidator(bc)<br>  bc.addBlock(genesis)<br>  return bc<br>}<br><br><br>// HasBlock checks if a block of a given height exists in the blockchain<br>func (bc *Blockchain) HasBlock(height uint32) bool {<br> return height &lt;= bc.GetBlockchainHeight() // less than/equal to since height begins at 0<br>}<br><br>// GetHeaderByHeight returns header at given height<br>// or error is height is higher than the blockchain height<br>func (bc *Blockchain) GetHeaderByHeight(height uint32) (*Header, error) {<br> if height &gt; bc.GetBlockchainHeight() {<br>  return nil, fmt.Errorf(&quot;given height (%d) is greater than the blockchain height&quot;, height)<br> }<br><br> bc.lock.Lock()<br> defer bc.lock.Unlock()<br><br> return bc.headers[height], nil<br>}<br><br>// GetBlockchainHeight returns the height of the entire blockchain<br>// height is calculated similar to array indices hence<br>// height = (number of blocks in the blockchain) - 1<br>func (bc *Blockchain) GetBlockchainHeight() uint32 {<br> bc.lock.RLock()<br> defer bc.lock.RUnlock()<br><br> return uint32(len(bc.headers) - 1)<br>}<br><br>// addBlock a new block to the blockchain by appending it&#39;s<br>// header to the blockchain header list<br>func (bc *Blockchain) addBlock(b *Block) {<br> bc.lock.Lock()<br> defer bc.lock.Unlock()<br><br> bc.headers = append(bc.headers, b.Header)<br> bc.logger.WithFields(logrus.Fields{<br>  &quot;height&quot;:                 b.Header.Height,<br>  &quot;hash&quot;:                   b.Hash(BlockHash{}),<br>  &quot;number of transactions&quot;: len(b.Transactions),<br> }).Info(&quot;adding new block&quot;)<br>}</pre><p>In the above snippet, we initialize a new blockchain and add operations for managing the blockchain. You should notice two errors in the crypto/blockchain.go file and both are because we have not created the Validatorand its methods. We’ll fix that next 🧑🏽‍🏭</p><p>In the crypto directory, create a new file and name it validator.go and add the following:</p><pre>package crypto<br><br>import &quot;fmt&quot;<br><br>type Validator interface {<br> ValidateBlock(*Block) error<br>}<br><br>// BlockValidator implements the Validator interface<br>type BlockValidator struct {<br> bc *Blockchain<br>}<br><br>// NewBlockValidator initializes a new BlockValidator<br>func NewBlockValidator(bc *Blockchain) *BlockValidator {<br> return &amp;BlockValidator{bc: bc}<br>}<br><br>// ValidateBlock validates a new block before being added to the blockchain<br>// to ensure correctness<br>func (v *BlockValidator) ValidateBlock(b *Block) error {<br> // no error means that we are attempting to add a block at a height in the blockchain<br> // already occupied by an existing block<br> if _, err := v.bc.GetHeaderByHeight(b.Header.Height); err == nil {<br>  return fmt.Errorf(&quot;the blockchain already contains block of height: %d with hash %s&quot;, b.Header.Height, b.headerHash)<br> }<br><br> // the new block to be added must occupy the next available slot i.e. blockchain height + 1<br> // if the block height is greater than that, return error<br> if b.Header.Height &gt; v.bc.GetBlockchainHeight()+1 {<br>  return fmt.Errorf(&quot;the height of block with hash (%s) is too high&quot;, b.Hash(BlockHash{}))<br> }<br><br> // get header of the previous block in the blockchain<br> prevHeader, err := v.bc.GetHeaderByHeight(b.Header.Height - 1)<br> if err != nil {<br>  return err<br> }<br><br> // recalculate hash of previous block header<br> prevHash := BlockHash{}.Hash(prevHeader)<br><br> // the recalculated previous hash must match the PrevBlockHash of the new block<br> // being added to the chain, otherwise treated as a fraudulent case<br> // raise an error<br> if prevHash != b.Header.PrevBlockHash {<br>  return fmt.Errorf(&quot;hash mismatch. expected previous hash: %s actual hash: %s&quot;, prevHeader.PrevBlockHash, prevHash)<br> }<br><br> // verify the new block<br> if err := b.Verify(); err != nil {<br>  return err<br> }<br><br> return nil<br>}</pre><p>In the crypto/blockchain.go add a new method AddBlock note that it’s different from addBlock since it includes the newly added ValidateBlock method.</p><pre>// AddBlock adds a new block to the blockchain<br>func (bc *Blockchain) AddBlock(b *Block) error {<br> if err := bc.validator.ValidateBlock(b); err != nil {<br>  return err<br> }<br><br> // add block<br> bc.addBlock(b)<br><br> return nil<br>}</pre><p>At this point, we can initialize the blockchain with the genesis block and add subsequent blocks soundly ensured by the blockchain validation. The last thing we’ll do is add tests so create a new file in the crypto directory and name it blockchain_test.go</p><pre>package crypto<br><br>import (<br> &quot;github.com/sirupsen/logrus&quot;<br> &quot;github.com/stretchr/testify/assert&quot;<br> &quot;testing&quot;<br>)<br><br>// helper function to get previous block&#39;s hash<br>func getPrevBlockHash(t *testing.T, bc *Blockchain, height uint32) Hash {<br> prevHeader, err := bc.GetHeaderByHeight(height - 1)<br> assert.Nil(t, err)<br><br> return BlockHash{}.Hash(prevHeader)<br>}<br><br>// helper function initializes a new blockchain with a genesis block<br>func newBlockchainWithGenesisExample() *Blockchain {<br> tx := NewTxWithSignature([]byte(&quot;Hello, World&quot;))<br> prevBlockHash := Hash{}<br><br> validatorPrivateKey, _ := GeneratePrivateKey()<br> genesisBlock := NewSignedBlockExample(validatorPrivateKey, []*Transaction{tx}, 0, prevBlockHash)<br><br> log := logrus.New()<br> return NewBlockchain(log, genesisBlock)<br>}<br><br>func TestNewBlockchain_NewBlockAddedSuccessfully(t *testing.T) {<br> blockchain := newBlockchainWithGenesisExample()<br><br> // confirm blockchain has height of 0<br> assert.Equal(t, uint32(0), blockchain.GetBlockchainHeight())<br><br> // add another block<br><br> // block validator private key<br> validatorPrivateKey, _ := GeneratePrivateKey()<br> // get the block hash of the previous block in the blockchain<br> prevBlockHash := getPrevBlockHash(t, blockchain, 1)<br> tx1 := NewTxWithSignature([]byte(&quot;Hello, World&quot;))<br> // create a new block<br> block1 := NewSignedBlockExample(validatorPrivateKey, []*Transaction{tx1}, 1, prevBlockHash)<br> // add block1 to the blockchain<br> err := blockchain.AddBlock(block1)<br> assert.Nil(t, err)<br><br> // blockchain height should be updated to 1<br> assert.Equal(t, uint32(1), blockchain.GetBlockchainHeight())<br><br> header, err := blockchain.GetHeaderByHeight(1)<br> assert.Nil(t, err)<br><br> assert.Equal(t, block1.Header, header)<br>}<br><br>func TestNewBlockchain_AddBlockFailsWithInvalidPreviousBlockHash(t *testing.T) {<br> blockchain := newBlockchainWithGenesisExample()<br><br> // confirm blockchain has height of 0<br> assert.Equal(t, uint32(0), blockchain.GetBlockchainHeight())<br><br> // add another block<br><br> // block validator private key<br> validatorPrivateKey, _ := GeneratePrivateKey()<br> //set previous block hash to an invalid one<br> prevBlockHash := Hash{}<br> tx1 := NewTxWithSignature([]byte(&quot;Hello, World&quot;))<br> // create a new block<br> block1 := NewSignedBlockExample(validatorPrivateKey, []*Transaction{tx1}, 1, prevBlockHash)<br> // add block1 to the blockchain<br> err := blockchain.AddBlock(block1)<br> assert.NotNil(t, err)<br><br> // blockchain height remains as 0<br> assert.Equal(t, uint32(0), blockchain.GetBlockchainHeight())<br>}<br><br>func TestBlockTooHigh(t *testing.T) {<br> blockchain := newBlockchainWithGenesisExample()<br> assert.Equal(t, uint32(0), blockchain.GetBlockchainHeight())<br><br> // add block at a height higher than the next available slot<br> // block validator private key<br> validatorPrivateKey, _ := GeneratePrivateKey()<br> // get the block hash of the previous block in the blockchain<br> prevBlockHash := getPrevBlockHash(t, blockchain, 1)<br> tx1 := NewTxWithSignature([]byte(&quot;Hello, World&quot;))<br> // create a new block with height higher than the blockchain&#39;s height<br> block1 := NewSignedBlockExample(validatorPrivateKey, []*Transaction{tx1}, 2, prevBlockHash)<br><br> err := blockchain.AddBlock(block1)<br> // addition of block1 to blockchain fails<br> assert.Error(t, err)<br> // blockchain height remains at 0<br> assert.Equal(t, uint32(0), blockchain.GetBlockchainHeight())<br>}</pre><p>Remember to run, make test to check if the tests pass.</p><p>With these implementations, our blockchain is taking shape! We’ve successfully calculated the Merkle root, verified blocks, and ensured the integrity of our blockchain by validating blocks before adding them. The changes made for chaining the blocks and blockchain validation can be found <a href="https://github.com/majorshift/safari-chain/commit/deb2dff1fe001653eaff2bb94e3c9c7d209402d2">here</a>.</p><p>In <a href="https://medium.com/@majorwrites/build-a-blockchain-from-scratch-using-golang-part-3-nodes-and-the-mempool-eeb98cb4c1ae">part 3</a> of this series, we’ll focus on developing the mempool — the temporary holding area for transactions awaiting confirmation before being included in a block. This step is fundamental, as it sets the stage for constructing the network and ultimately bringing the nodes to life.</p><p>Have any questions or feedback? Drop a comment below! 🚀</p><p>See you on the next one</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=ef0b3a5ec2f7" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Learning the inner workings of Blockchain by building one from scratch: Part One]]></title>
            <link>https://medium.com/@majorwrites/learning-the-inner-workings-of-blockchain-by-building-one-from-scratch-part-one-3a37b87fbfde?source=rss-bc98b06ce1f0------2</link>
            <guid isPermaLink="false">https://medium.com/p/3a37b87fbfde</guid>
            <category><![CDATA[golang]]></category>
            <category><![CDATA[programming]]></category>
            <category><![CDATA[blockchain]]></category>
            <dc:creator><![CDATA[Daniel Maina]]></dc:creator>
            <pubDate>Sun, 26 Jan 2025 07:19:12 GMT</pubDate>
            <atom:updated>2025-02-24T07:51:01.211Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IZpg9hGihVMDWpNbIhVvnw.png" /><figcaption>Source: pixabay.com</figcaption></figure><p>I recently challenged myself to learn blockchain development and in the process built one from scratch<em>(still in progress)</em>. All this is to get a deeper understanding of the inner workings of blockchain protocols.</p><p>In this series, I’ll document what I’ve learned and take you along as we build a blockchain network from scratch. The code can be found on this <a href="https://github.com/majorshift/safari-chain">repo</a>.</p><p>The following is the typical process in blockchain systems and we’ll try to build all these components/processes into the blockchain:-</p><ol><li><strong>Transaction Initiation</strong>: A transaction is initiated by a user e.g. sending crypto to someone.</li><li><strong>Signing Transactions</strong>: The transaction is signed using the sender’s private key to ensure authenticity and prove that the transaction was authorized by the owner of the private key.</li><li><strong>Transaction Verification</strong>: Nodes in the network verify the signature using the sender’s public key. This step ensures the transaction has not been tampered with and was indeed created by the sender.</li><li><strong>Add transaction to block</strong>: Once verified, transactions are temporarily added to the blockchain’s mempool. Transactions are later picked from the mempool and added to a block.</li><li><strong>Hashing Transactions</strong>: Transactions in the block are hashed, often organized into a Merkle tree, to ensure data integrity and enable efficient verification.</li><li><strong>Adding to the Blockchain</strong>: The block, containing the hashed transactions, is added to the blockchain once it meets the network’s consensus criteria (e.g., proof-of-work or proof-of-stake).</li></ol><h4><strong>Setting expectations</strong></h4><p>In this post<em>(part one)</em>, we want to achieve <strong>transaction and block definitions, hashing, transaction validation, and creation</strong><em>(basic level for this part)</em><strong>.</strong></p><p>Broadcasting transactions to the network, consensus mechanism and other components will come in later parts of the series.</p><p><strong>Setup</strong></p><p>I’ll be using <strong>Golang(Go)</strong> but feel free to use any language you choose.</p><p>Go to your working directory and create the project folder. I’ll name mine safari-chain</p><pre>$ mkdir safari-chain # create the safari-chain project folder<br>$ cd safari-chain # get into the project folder<br>$ go mod init your-path # initialize project, set go version, prepare for dependency management etc<br>$ touch main.go # create a main.go file at the root of the project</pre><p>Initialize go mod init to your path. In my case, I am matching my GitHub project path i.e. github.com/majorshift/safari-chain</p><p>Add the following to the main.go file.</p><pre>package main<br><br>import &quot;fmt&quot;<br>func main() {<br>    fmt.Println(&quot;Hello World&quot;)<br>}</pre><p>Create a Makefile at the root of the project</p><pre>$ touch Makefile</pre><p>add the following commands to Makefile</p><pre>build:<br> @go build -o bin/safarichain<br>pak<br><br>run: build<br> @./bin/safarichain<br><br>test: # for running tests<br> @go test -v ./...</pre><p>The build command builds the project and specifies the name &amp; location of the output binary generated by the command.</p><p>The file path to the output binary i.e. bin/safarichainmust be the same as the :run build command.</p><p>At this point, you’re good to go; no pun intended</p><h3>Part 1: Laying the foundation</h3><p>A <strong>blockchain</strong> is a decentralized database/digital ledger that records transactions across a peer-to-peer network of computers ensuring that the data is secure, transparent, and immutable<em>(tamper-proof)</em>.</p><p>First, we need to know what forms a blockchain at a high level.</p><ol><li><strong>Transactions &amp; Blocks</strong></li><li><strong>Hashing and Cryptography</strong> — <em>ensure the blockchain is secure and tamper-proof.</em></li><li><strong>The peer-to-peer network</strong> — <em>no central entity that enforces rules in the network. All nodes are peers.</em></li><li><strong>Consensus mechanism</strong> — <em>the protocol that allows a blockchain network to agree on a single version of a digital ledger, or transaction history.</em></li></ol><p>We can have other components like state management, smart contracts(if applicable), incentive mechanisms, fork handling, etc.</p><h4>Blocks &amp; Transactions</h4><p>A <strong>transaction</strong> in a blockchain is an operation representing the transfer of data, assets<em>(crypto, tokens &amp; more) </em>between participants in the network. It is the basic unit of activity on a blockchain.</p><p>Think of a <strong>block</strong> as a container housing a group of transactions. The contents of transactions are hashed and the hash value is stored as part of the block<em>(merkle root or other variants)</em>. Each block contains the hash value of the previous block making it possible to access the previous block hence forming a “chain” and this is where we get the term <strong>blockchain</strong>.</p><p>If it helps, think of a blockchain as a Linked List data structure<em>(see image below)</em>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*nUxSU6OnpIdTapFAbhONWQ.jpeg" /></figure><p>We’ll start by defining the data structure for transactions and blocks. In the project folder create a directory and call it cryptoand three files within the new folder; block.go , keypair.go&amp;hash.go .</p><pre>$ mkdir crypto<br>$ touch crypto/{block.go, keypair.go, hash.go}</pre><h4>Use of Keypairs</h4><p>A <strong><em>key pair</em></strong> is a concept of <strong><em>public-key cryptography</em></strong> or <strong><em>asymmetric-key cryptography</em></strong><em>. </em>It is a set of two keys, a public and a private key, that are mathematically linked and used for encryption<em>(public)</em> and decryption<em>(private)</em>.</p><p>The public key is generated from the private key and is shareable while the private key <strong>must</strong> be kept secret. Read more about public key cryptography <a href="https://www.geeksforgeeks.org/blockchain-public-key-cryptography/"><em>here</em></a>.</p><p>In the keypair.go file we’ll define the structures of public &amp; private keys and the signature as well.</p><pre>// keypair.go file<br>package crypto<br><br>import (<br>  &quot;crypto/ed25519&quot;<br>)<br><br>type PrivateKey struct {<br> key ed25519.PrivateKey<br>}<br><br>type PublicKey struct {<br> Key ed25519.PublicKey<br>}<br><br>type Signature struct {<br> Value []byte<br>}</pre><blockquote>Note: we use <strong>ed25519</strong> algorithm for it’s security advantage compared to others like <strong>RSA</strong> and <strong>ECDSA</strong>. Each algorithm has it’s advantage compared to others so be sure to make the right trade off. See this <a href="https://vulnerx.com/ssh-key-algorithms/">post</a> for comparisons.</blockquote><p>We will define the structure of a transaction in transaction.go file</p><pre>package crypto<br><br>// Transaction structure<br>type Transaction struct {<br> Data      []byte<br> From      *PublicKey // public key of the one sending value/initiating transaction<br> Receiver  *PublicKey // public key of the one receiving value<br> Signature *Signature // signature verifying transaction&#39;s authenticity<br>}<br><br>func NewTransaction(from, receiver *PublicKey, data []byte) *Transaction {<br> return &amp;Transaction{<br>  From:     from,<br>  Receiver: receiver,<br>  Data:     data,<br> }<br>}</pre><p>In the block.go file, we’ll define the block and header structure</p><pre>// block.go<br>package crypto<br><br>// Header structure<br>type Header struct {<br> Version       uint32 // current block version<br> PrevBlockHash Hash   // hash of the previous block<br> MerkleRoot    Hash   // hash of transactions in the block<br> Timestamp     int64  // when the block was created<br> Height        uint32 // number of blocks in the blockchain - 1<br>}<br><br>// Block structure<br>type Block struct {<br> *Header<br> Transactions []*Transaction // a list of transactions within the block<br> Validator    *PublicKey     // public key of the validator that will add the block to the chain<br> Signature    *Signature     // signature verifying block&#39;s authenticity<br> headerHash   Hash           // cached hash value of the block&#39;s header; for quick access<br>}<br><br>// ToBytes converts Header to a byte slice<br>func (h *Header) ToBytes() []byte {<br> buf := &amp;bytes.Buffer{}<br> enc := gob.NewEncoder(buf)<br> enc.Encode(h)<br><br> return buf.Bytes()<br>}<br><br>func NewBlock(h *Header, txs []*Transaction) *Block {<br> return &amp;Block{<br>  Header:       h,<br>  Transactions: txs,<br> }<br>}</pre><p>and in the hash.go file, add this <em>(more to come later)</em>.</p><pre>// hash.go<br>package crypto<br><br>type Hash [32]uint8</pre><p>Before implementing transaction and block operations, we’ll first add essential keypair functionalities, such as signing with private keys, creating public keys, verifying signatures, etc. Update keypair.go file to the following:</p><pre>package crypto<br><br>import (<br> &quot;crypto/ed25519&quot;<br> &quot;crypto/rand&quot;<br> &quot;io&quot;<br>)<br><br>const (<br> privKeyLen   = 64 // length of the private key<br> seedLen      = 32<br> pubKeyLen    = 32 // length of the public key<br> signatureLen = 64<br>)<br><br>type PrivateKey struct {<br> key ed25519.PrivateKey<br>}<br><br>// ToBytes converts private key to byte array<br>func (p *PrivateKey) ToBytes() []byte {<br> return p.key<br>}<br><br>// Sign uses private key to create a signature<br>func (p *PrivateKey) Sign(msg []byte) *Signature {<br> return &amp;Signature{<br>  Value: ed25519.Sign(p.key, msg),<br> }<br>}<br><br>// GeneratePrivateKey creates a new private key<br>func GeneratePrivateKey() (*PrivateKey, error) {<br> seed := make([]byte, seedLen)<br> _, err := io.ReadFull(rand.Reader, seed)<br> if err != nil {<br>  return nil, err<br> }<br><br> return &amp;PrivateKey{<br>  key: ed25519.NewKeyFromSeed(seed),<br> }, nil<br>}<br><br>// PublicKey returns public key from the private key<br>func (p *PrivateKey) PublicKey() *PublicKey {<br> b := make([]byte, pubKeyLen)<br> copy(b, p.key[32:])<br><br> return &amp;PublicKey{<br>  Key: b,<br> }<br>}<br><br>type PublicKey struct {<br> Key ed25519.PublicKey<br>}<br><br>func (p *PublicKey) ToBytes() []byte {<br> return p.Key<br>}<br><br>type Signature struct {<br> Value []byte<br>}<br><br>// ToBytes returns the byte array value of a public key<br>func (s *Signature) ToBytes() []byte {<br> return s.Value<br>}<br><br>// BytesToSignature converts byte array to Signature type<br>func BytesToSignature(b []byte) *Signature {<br> if len(b) != signatureLen {<br>  panic(&quot;invalid signature length, must be 64&quot;)<br> }<br><br> return &amp;Signature{Value: b}<br>}<br><br>// Verify checks if the signature is valid<br>func (s *Signature) Verify(pubKey *PublicKey, msg []byte) bool {<br> return ed25519.Verify(pubKey.Key, msg, s.Value)<br>}</pre><p>Check the commented explanations in the code for more details of what’s going on.</p><p>The function NewTransaction in the transaction.go file initializes a new Transaction instance.</p><p>At this stage, we’ll handle the signing of blocks and transactions. The essence of signing is to verify the authenticity of actions in the blockchain ensuring that only authorized parties initiate transactions and preventing any tampering or fraudulent activity.</p><p>The user initiating the transaction has to sign it with a private key. Add the following to the transaction.go file.</p><pre>// Sign signs a transaction<br>func (tx *Transaction) Sign(privKey *PrivateKey) {<br> sig := privKey.Sign(tx.Data)<br><br> tx.Signature = sig<br>}</pre><p>For blocks, it is more efficient to sign the header instead of the whole block. Add this to the block.go file</p><pre>// Sign uses the private key to sign the block header<br>func (b *Block) Sign(privKey *PrivateKey) {<br>    sig := privKey.Sign(b.Header.Bytes())<br><br>    b.Validator = privKey.PublicKey()<br>    b.Signature = sig<br>}</pre><h4>Hashing</h4><p>This is a way of protecting sensitive information and ensuring data integrity. Remember the characteristic of blockchain as being tamper-proof? We use hashing to make this happen.</p><p>Every block in the blockchain is linked to the previous one by including the previous block’s hash in its block hence forming a chain. You might be curious why this is the case/how this works.</p><p>One of the properties of hash functions is that if you hash something say the string &quot;Hello, World&quot; right now, it will return the same hash result if you hash it again later on <em>(even a year from now)</em>. This is used in blockchain to achieve data consistency.</p><p><strong><em>How about the tamper-proof property?</em></strong> Let’s say that the input changes from &quot;Hello, World&quot; to &quot;Hello World <em>(notice the missing comma), </em>the hash result will be completely different.</p><p>The same analogy applies to blocks. Imagine a block <strong>A </strong>already in the blockchain. If you hash block <strong>A</strong> a hundred times, it will always return the same hash result. If another block <strong>B</strong> is added after <strong>A</strong> <em>(meaning it contains the block hash value of </em><strong><em>A</em></strong><em>), </em>when validating block <strong>B</strong> before being added to the chain, we recalculate the hash of block <strong>A</strong> and compare it to the previous block hash (of A) passed. If they don’t match, the creation of block B fails and it’s this is flagged in the network as fraudulent. This is what makes blockchains tamper-proof.</p><p>Now that we understand the essence of hashing, we’ll add functionality that will enable us to hash transactions &amp; blocks. Add the following the crypto/hash.go file.</p><pre>package crypto<br><br>import (<br> &quot;crypto/sha256&quot;<br> &quot;encoding/hex&quot;<br>)<br><br>const (<br> hashLen = 32<br>)<br><br>type Hash [32]uint8<br><br>// IsHashed checks if hash has been set<br>func (h Hash) IsHashed() bool {<br> for i := 0; i &lt; hashLen; i++ {<br>  if h[i] != 0 {<br>   return false<br>  }<br> }<br><br> return true<br>}<br><br>// ToByteArray converts hash to []byte<br>func (h Hash) ToByteArray() []byte {<br> b := make([]byte, hashLen)<br> for i := 0; i &lt; hashLen; i++ {<br>  b[i] = h[i]<br> }<br><br> return b<br>}<br><br>// ToString encodes hash to string<br>func (h Hash) ToString() string {<br> return hex.EncodeToString(h.ToByteArray())<br>}<br><br>// BytesToHash receives []byte and converts to Hash<br>func BytesToHash(b []byte) (Hash, error) {<br> if len(b) != hashLen {<br>  return Hash{}, fmt.Errorf(&quot;expected bytes of length 32, got %d&quot;, len(b))<br> }<br><br> var value [hashLen]uint8<br> for i := 0; i &lt; hashLen; i++ {<br>  value[i] = b[i]<br> }<br><br> return Hash(value), nil<br>}<br><br>// Hasher interface<br>type Hasher[T any] interface {<br> Hash(T) Hash<br>}<br><br>// BlockHash to implement the Hasher interface for blocks<br>type BlockHash struct{}<br><br>// Hash hashes the header of the block<br>func (BlockHash) Hash(header *Header) Hash {<br> h := sha256.Sum256(header.ToBytes())<br> return Hash(h)<br>}<br><br>// TxHash to implement the Hasher interface for transactions<br>type TxHash struct{}<br><br>// Hash hashes the Data property of a Transaction<br>func (TxHash) Hash(tx *Transaction) Hash {<br> return Hash(sha256.Sum256(tx.Data))<br>}</pre><p>Having done that, go back to the transaction.go file and add the snippet below to hash transactions:-</p><pre>// transaction.go<br><br>// Hash hashes transactions<br>func (tx *Transaction) Hash(hasher Hasher[*Transaction]) Hash {<br> return hasher.Hash(tx)<br>}</pre><p>Do the same for block.go</p><pre>// block.go<br><br>// Hash hashes a block header<br>func (b *Block) Hash(hasher Hasher[*Header]) Hash {<br> return hasher.Hash(b.Header)<br>}</pre><h4>Transaction Validation</h4><p>Validation has to take place before transactions are added to a block and the block is added to the blockchain.</p><p>For a transaction, it <strong>must</strong> be signed by the initiator’s<em>(eg user)</em> private key and the signature must be checked to be valid. So in the transaction.go file, add this snippet.</p><pre>// Verify checks the validity of the transaction signature<br>func (tx *Transaction) Verify() error {<br> // transaction must be signed<br> if tx.Signature == nil {<br>  return fmt.Errorf(&quot;transaction has no signature&quot;)<br> }<br> // if signature is set, it must be valid and not tampered with<br> if !tx.Signature.Verify(tx.From, tx.Data) {<br>  return fmt.Errorf(&quot;transaction has invalid signature&quot;)<br> }<br><br> return nil<br>}</pre><p>At this point, we have a basic understanding of blockchain, have implemented asymmetric cryptography i.e. keypairs, can create, handle hashing, and signing of transactions and blocks. So far so good.</p><p>Since we’re building this project the TDD way, we’ll now add tests. First, we’ll install a go package to help with assertions. At the root of the project, run the following command:</p><pre>$ go get github.com/stretchr/testify</pre><p>Next, we’ll start by testing our transactions. Create a new file within the crypto folder and call it transaction_test.go then add the code below to the new test file.</p><pre>package crypto<br><br>import (<br> &quot;testing&quot;<br><br> &quot;github.com/stretchr/testify/assert&quot;<br>)<br><br>func TestSignTransaction(t *testing.T) {<br> // private key of sender<br> privKeyFrom, err := GeneratePrivateKey()<br> assert.NoError(t, err)<br> // private key of receiver<br> privKeyReceiver, err := GeneratePrivateKey()<br> assert.NoError(t, err)<br><br> // public key of sender<br> pubKeyFrom := privKeyFrom.PublicKey()<br> // public key of receiver<br> pubKeyReceiver := privKeyReceiver.PublicKey()<br><br> // data to be sent - in the future data will be more elaborate<br> data := []byte(&quot;Hello, World&quot;)<br><br> // create new transaction<br> tx := NewTransaction(pubKeyFrom, pubKeyReceiver, data)<br><br> // transaction is not signed yet, signature property should be nil<br> assert.Nil(t, tx.Signature)<br> // verifying at this stage fails since the transaction is not signed<br> assert.Error(t, tx.Verify()) // error is: transaction has no signature<br><br> // Sign transaction<br> tx.Sign(privKeyFrom)<br> assert.True(t, tx.Signature.Verify(pubKeyFrom, data))<br>}<br><br>func TestVerifyTransaction(t *testing.T) {<br> privKeyFrom, err := GeneratePrivateKey()<br> assert.NoError(t, err)<br><br> privKeyReceiver, err := GeneratePrivateKey()<br> assert.NoError(t, err)<br><br> pubKeyFrom := privKeyFrom.PublicKey()<br> pubKeyReceiver := privKeyReceiver.PublicKey()<br><br> data := []byte(&quot;Hello, World&quot;)<br> tx := NewTransaction(pubKeyFrom, pubKeyReceiver, data)<br><br> // sign transaction with appropriate private key(sender&#39;s)<br> tx.Sign(privKeyFrom)<br> // no error when verifying<br> assert.NoError(t, tx.Verify())<br><br> // should sign with sender&#39;s private key(the From property of Transaction)<br> // here we sign with the wrong private key (receiver&#39;s private key)<br> tx.Sign(privKeyReceiver)<br><br> // verification should fail with the error: transaction has invalid signature<br> assert.Error(t, tx.Verify())<br>}</pre><p>Remember to read the explanations in the comments.</p><p>To run the tests, run the following command in the root directory of the project.</p><pre>$ make test</pre><p>The command above runs <strong>all</strong> tests in the project but you can run them individually as you’d normally do on your editor.</p><p>To simplify testing, we’ll add a file called examples.gothe cryptodirectory with functions to create reusable Block and Transaction examples, avoiding the need to create them from scratch each time. After creating the example file, add the following code to it:-</p><pre>package crypto<br><br>import &quot;time&quot;<br><br>// NewTxWithSignature returns a signed transaction<br>func NewTxWithSignature(data []byte) *Transaction {<br> fromPrivKey, _ := GeneratePrivateKey()<br> receiverPrivKey, _ := GeneratePrivateKey()<br> tx := NewTransaction(fromPrivKey.PublicKey(), receiverPrivKey.PublicKey(), data)<br> tx.Sign(fromPrivKey)<br><br> return tx<br>}<br><br>// ExampleBlock returns a block<br>func ExampleBlock(height uint32, prevBlockHash Hash) *Block {<br> tx := NewTxWithSignature([]byte(&quot;hello world&quot;))<br> header := &amp;Header{<br>  Version:       1,<br>  PrevBlockHash: prevBlockHash,<br>  Height:        height,<br>  Timestamp:     time.Now().UnixNano(),<br> }<br><br> return NewBlock(header, []*Transaction{tx})<br>}</pre><p>Next, we’ll test the block functionality.</p><p>In the crypto directory, create a file, and name it block_test.go and add the following:</p><pre>package crypto<br><br>import (<br> &quot;github.com/stretchr/testify/assert&quot;<br> &quot;testing&quot;<br>)<br><br>func TestSignBlock(t *testing.T) {<br> // create private key<br> privKey, _ := GeneratePrivateKey()<br><br> // the prevHash in this case will be nil since the block being created<br> // is the genesis block<br> prevBlockHash := Hash{}<br> b := ExampleBlock(1, prevBlockHash)<br><br> // sign block<br> // the private key used to sign the block belongs to the validator<br> b.Sign(privKey)<br><br> // after signing, the validator property f the block is set to<br> // its public key<br> assert.Equal(t, b.Validator, privKey.PublicKey())<br> assert.NotNil(t, b.Signature)<br>}</pre><h3>Conclusion</h3><p>We’ve come to the end of part 1 of the series 😃. I know, it&#39;s a long one but it had since we’re laying the groundwork. So far, we’ve accomplished the following:-</p><ol><li>defined what a blockchain is</li><li>defined the structures for blocks, headers, and transactions</li><li>implemented hashing</li><li>signed blocks and transactions</li><li>verified transactions to validate them (an essential step before adding to blocks)</li></ol><p>I encourage you to research widely on these concepts to deepen your understanding.</p><p>In <a href="https://medium.com/@majorwrites/build-a-blockchain-from-scratch-using-golang-part-2-ef0b3a5ec2f7?source=friends_link&amp;sk=216b115d02a35d7e4e67f0fe6bbe9d4a">part 2</a> of the series, we’ll implement block verification, add transactions to the block, and calculate the Merkle Root of the transactions.</p><p>The changes from this post can be found <a href="https://github.com/majorshift/safari-chain/commit/a19f0bc30ae0e2468a0bd239bee6ec5fb76d02f7">here</a>.</p><p>There’s a lot more to cover and I hope to see you on the next one 🫡</p><h4>Special Thanks</h4><p>This series is inspired by <a href="https://www.youtube.com/playlist?list=PL0xRBLFXXsP6-hxQmCDcl_BHJMm0mhxx7">this playlist</a> by Anthony GG, who, in my opinion, creates some of the best hands-on tech content on YouTube. I expanded on what I learned by doing extensive research, enhancing the explanations with illustrations, and making several modifications to the code. Be sure to check out his content!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3a37b87fbfde" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>