Context:

I’m planning this series of short explanatory articles as a way to write down knowledge on various technical/non-technical subjects I’m currently interested in, whilst allowing me to share thoughts that may be helpful to other individuals studying those same topics.

This first part will cover some portions of the ethereum P2P layer, seen through the prism of the Geth codebase (https://github.com/ethereum/go-ethereum), as we’ll try to demistify it in order to obtain a more higher-view of these critical protocols.

1. Understanding the ethereum P2P communication layer:

Considering the ethereum network as a living organism, a simple analogy would be to think of the different nodes composing its complex structure as cells : each devoted to multiple/different tasks (archiving state trace, relaying data to other peers, validating…), those actors achieve to maintain an incentive-led and truly-decentralized infrastructure.

Let’s directly dig down to the Node object, as it is described in Geth :

The fields that are here attracting our attention are :

• config
• server

The config (*Config object) struct contains, among other elements, the instance name of the node, its identifier, the data directory path, and more specifically the node P2P networking configuration.

The p2p networking config object holds different parameters, that are set in order for the node p2p server to maintain specific options as long as it is running :

Nothing realy fancy here, some simple specifications related to peers connectivity (max peers allowed, static nodes (aka maintained peers that are directly reconnected following disconnections), trusted nodes, peers database, etc…), but what interest us for the moment is the BootStrapNodes field (v5 refers to the latest version of the node discovery protocol, we’ll soon be back at it).

BootStrapping Nodes are nodes maintained by the Ethereum Foundation, that are only taking part in the network discovery protocol. When initializing your client, it will automaticaly handshake then establish a connection to those nodes, using them as anchors points that will then allow its peers discovery.

As you can observe, they are directly hardcoded in the Geth params :

Obtaining initial peers is a great start, but as previously discussed bootnodes are just a way to bootstrap your peers discovery, as they do not relay blockchain data.

How to reach other parts of the network, in order to communicate with different participants and synchronize your client data ? Requesting relative peers infos from the connected bootnodes, then handshaking them in order to establish a connection would appear as the most logical path to follow, but then what ? Do we have to follow this procedure in a pseudo-random fashion, picking closest peers and storing them locally ?

It is in fact not the case, thanks to the Ethereum Discovery Protocol, aka the native implementation of a Kademlia-like DHT.

2. Node Discovery protocol :

Kademlia is a distributed hash-table model theorized by Petar Maymounkov and David Mazières, in their initial research paper : https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf.

As a quick summary of the eth kademlia implementation :

• Every node within the network has a cryptographic identity (a key on the secp256k1 elliptic curve) : those keys are used to compute the “distance” between two nodes, applying the bitwise XOR operator on the keccak256 of both keys, taken as the number. The “distance” should not be considered as “physical” distance : it is only a value allowing nodes to be compared, and indexed as table entries.

• The peers detection process is called Recursive Lookup. The initiator (aka searching peer) starts by picking the α (client-hardcoded concurrency factor, in geth it is 3) closest peers from the target it knows, and sends them a FindNode packet, asking them to return their 16 closests-peers (neighbors).

• The initiated Lookup loop will then send peer requests to yet unreached peers among the previously discovered ones, until it reaches the initially-specified k amount of closest peers, while keep adding them to its local table.

For each node to store all of the network participants informations would be an inneficient, slow and costly process, same goes for a global and centralized infrastructure.

The Kademlia indexing structure theorized in order to solve this scalability issue is, as we previously mentioned, called a “Table” : each node within the network maintain its own version, regularly verifying neighboors liveness / requesting node records to assert versions validity.

In order to acquire a deeper understanding of Kademlia, I’d advise you to read this great article from Kevin Leffew :

A Brief Overview of Kademlia, and its use in various decentralized platforms

While we’re breaking down those steps in a linear fashion to ease their understanding, let’s keep in mind that some parts are in fact running concurrently. It is for example the case each time a connection between two peers is established : a Status message is sent from both clients, and following receipts validation an Ethereum Session is opened between them, allowing the transmission of other messages.

This exchanges facilitating protocol is called the Ethereum Wire Protocol.

3. The Ethereum Wire Protocol / Snapshot Protocol

Once an Ethereum Session is established between two peers, three high-level tasks can be performed within it :

(Edit : Post Merge, the block propagation is now done through the Consensus layer. As we’re today only focused on the EL, I’ll describe the pre-Merge propagation as it is still used by some Geth forks.)

Block propagation : two-step process allowing block data distribution within the network.

At the moment a NewBlock announcement is received from a peer, the client verifies the block header validity, applying multiple checks (including gas limit, gas used within the block, timestamp, block height, etc…) and exclusions (in POS, block nonce is always null, and there is no uncle blocks anymore), comparing received values to the Beacon engine data (consensus client). Following this validity check, the block is imported, and all the transactions it contains are executed, allowing the client to compute the chain post state (but we’ll keep that for an upcoming episode of our series ;) .

It will then relay the block to a small fraction of its peers (this amount usually corresponds to the square root of the total number of peers he’s connected to), having completed its duty as link.

Transaction exchange : nodes within the network must propagate pending transactions in order to relay them to validators, wich will then pick them for inclusion within the blockchain.

Following an Ethereum session opening, both peers must synchronize their transaction pool. A transaction pool is a local structure keeping track of a set of pending transactions the node has knowledge of, and that are waiting for inclusion : it can, depending on different client limits / implementations, contain up to many txs (e.g. thousands).

A “NewPooledTransactionHahes” mesage is sent from both ends, containing all pooled txs from both nodes. They then apply multiple filters for each of those transactions , allowing them to discard invalid or already knows txs from the tx pool.

With the validity checks being :

Chain synchronization: nodes participating in the eth protocol are expected to have knowledge of the complete chain of blocks, from the genesis block to latest. It is obtained by downloading it from other peers, following the sending of a Status message that includes the hash of the latest block they have knowledge of, and their TTD (total difficulty).

The client with the lowest TTD will then continuously download block headers from the other peer, using the “GetBlockHeaders” message.

Till recently, State synchronization was realised directly through the eth protocol , as a part of the Chain synchronization module. This task is now performed by the Ethereum Snapshot Protocol, running alongside eth and retrieving continuous segments of accounts from the Ethereum state trie, or segments of storage slots from a particular storage trie.

That concludes the first part of our Geth series. As I wanted to maintain a short format for the p2p topic, I tried my best not to dig too deep into the client tech layer in order to keep this accessible : the upcoming episode will focus on the Geth implementation of the EVM, explaining more in-depth concepts in a detailled way.

Don’t hesitate to contact me on Twitter (https://twitter.com/Coltcivers_Alt), for any feedback or questions !