We recently added two exciting new features to evm.storage: transaction traces and bytecode-generated storage layouts. Today, we’re announcing another: our new ABI component, enabling interaction with contracts.

To explore how it works, let’s use the same example Alanna used in her post announcing transaction traces: a user provides ~13 USDC and 0.01 ETH to the Uniswap V2: USDC liquidity pool. Here’s the transaction page. Let’s explore more about this interaction with the new ABI component.

ABI objects tab

Navigating to the receiving contract, UniswapV2Router02, the first thing you’ll notice is that our storage component only serves a raw view, and all of the values in the slots are zero. This is true for many router contracts: their storage is simply unused — instead, they route interactions to other contracts where the state is updated accordingly. But router contracts are still very interesting as their ABIs show all of the possible interactions a user can pursue.

You can see our new ABI component at the bottom of the contract page. The main/default tab is called ABI objects, and it contains the following:

1. Getter functions with values resolved per the user-defined block state of the contract page
2. Read-only functions
3. State-modifying functions
4. Events
5. Constructor
6. Fallback and receive
7. Custom errors

These are all derived from the contract’s ABI. We currently only offer this component for verified contracts, but will add support for unverified contracts soon.

One of the functions you’ll find in the state-modifying section is addLiquidityETH, which was the method called in the sample transaction.

Function-specific tabs

If you want to interact with a contract via the functions, you can click into any of the read-only or state-modifying functions to open an input form. Let’s look further into addLiquidityETH.

Interacting with complex contracts via forms is a bit painful when there are many arguments. We’ve done our best to make it as easy as possible with the UI, showing the label and type for each argument and offering validation. But, realistically, you’re probably going to want to cheat with our Generate Template feature. Here I’ve entered the transaction hash of the sample transaction to prepopulate all of the fields from it. I could have also used the Generate Template feature without supplying a transaction hash, and it would have used the most recent transaction for which addLiquidityETH was the method.

Transaction simulation

You may have noticed that below the arguments we have simulation inputs. These were also populated for us by the Generate Template feature to replicate the conditions of the original transaction.

You can change any of the function or simulation inputs to simulate counterfactual transactions. A few things to try:

1. Change gas from 218883 to 160000 and simulate. You’ll see that the simulated transaction reverts. The original transaction used 161109 gas, so this runs out of gas.
2. Change to to a different address and simulate. If the sender had wanted, they could have redirected the LP token (Uniswap V2) to a different wallet.
3. Change amountTokenDesired to a very large number. The simulated transaction still succeeds and has the same outcomes, because it was still able to honor amountTokenMin.

Most importantly, for any valid simulation, you can click View Traces to see a full transaction page for the simulated transaction, with all the features we have for actual transactions. You can even share the URL for most simulated transactions with others — here’s one I created.

We shared some simple examples above, but we’re more excited to see how you use it. Some ideas:

1. Before you’re executing a transaction, simulate it to make sure you’re happy with the outcomes, e.g., get an estimate of slippage in a swap transaction.
2. Take more control of your blockchain activity by constructing transactions yourself, relying less on dApps.
3. Take a transaction that reverts and simulate it with different inputs to work out how to get it to succeed.
4. Use it for security analysis: simulate potential attack vectors to make sure your contract isn’t at risk.

What’s next?

We already have some ideas to further improve contract interactions:

1. Unverified contract coverage: We’re working on making it easier for you to interact with unverified contracts.
2. State overrides: Perform a simulation with counterfactual contract state.
3. Generating templates from internal transactions: You’ll find that we don’t have templates available right now for some functions that are only used in internal transactions, particularly read-only functions. With a bit of work, we can supply templates for these from internal transactions.
4. Executing transactions: We want you to be able to connect your wallet and execute a transaction directly from evm.storage.

We also plan on offering simulation as a service/SDK for users who want to simulate large volumes of transactions programmatically. If you’re interested in this, please reach out.

As always, if you have any questions or feedback, get in touch via our Telegram group.

Announcing contract interaction on evm.storage was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

Working on evm.storage, we often stumble upon unique and interesting contract designs. If it can be done, someone has probably tried to do it. We think that’s a wonderful feature of decentralized blockchains, even if it sometimes causes us implementation headaches. In this post, we’ll share a few surprises that we’ve encountered:

· A “big” contract
· Many proxies → one implementation
· One proxy → many implementations
· Overlapping layouts
· Deeply nested variables

A “big” contract

XENCrypto’s interesting primarily because it’s big. Our data shows it has over 190MM storage writes! You’d think that this would suggest the token had many holders, but we find only 303K keys in _balances and 54K in _allowances have even been written.

So why is it so big? There’s a hint in how they define the project: “XEN Crypto is a social mining project… XEN is categorized as a Proof-of-Participation (PoP) cryptocurrency.”

Diving into the data, we see over 10MM keys (!) used in the userMints mapping. Each value in that mapping is a struct XENCrypto.MintInfo that requires 6 slots, so we have 60MM slots associated in total. Some of those userMints contain members whose values are zero (an example).

What’s the mechanism driving this high volume of mints? Free mints! Except for gas, mind you. Anyone with a wallet can submit a transaction to start minting Xen. The user defines a term in days and can claim their minted tokens at the end of the term. The amount a user receives is increasing in the number of days and decreasing in the number of other users minting over the same period.

For completeness, we also see almost 20K keys used in userStakes and a further 5K used in userBurns.

CoinGecko reports that XENCrypto at one point was using up almost half of Ethereum block space, with its resulting gas price impact contributing significantly to Ether becoming a deflationary currency! Apparently now it’s on Base! Despite all of this, XENCrypto posed no issues for our infrastructure.

Incidentally, if you liked the data about the number of keys in a mapping, join our Telegram group and yell at us to put mapping key counts in the product. We haven’t prioritized it yet — I beg our engineering team for it every cycle but nobody listens to me.

Many proxies → one implementation

Ambisafe is a blockchain infra provider, and their work includes smart-contract wallets: 0x0b…d0 is an unverified proxy contract with unverified implementation 0xc3…5c. Here’s what’s remarkable: we detect that 0xc3…5c is the implementation address for over 545K proxy contracts! And there’s another Ambisafe-deployed implementation contract, 0x07…16, that has 540K associated proxy contracts.

Without knowing much about their implementation, at a high level it makes a lot of sense to allow users to deploy smart-contract wallets that get all of their logic via DELEGATECALLs into a shared implementation contract.

This Ambisafe pattern caused us minor hardship in building evm.storage. We identify proxy-implementation pairs through DELEGATECALLs. We define the implementation contract’s relevant range by the first and last encountered DELEGATECALLs. We chose this approach because it doesn’t require contracts to be verified or assume that an implementation contract’s address is written in a specific slot within the proxy’s storage (related: ERC-1967).

When we started working on proxy coverage, we were mostly thinking about it as an approach to upgradeability. In that setting, each implementation is typically associated with a single proxy, and a proxy may be associated with a handful of implementations as the developer updates the logic across several versions.

But another common pattern is using proxies for gas optimization. If you want to execute the same logic across many contracts, it’s efficient to deploy that logic in a single implementation contract and then DELEGATECALL to it from many proxies, as we see with Ambisafe. This broke our initial approach to tracking pairs as our processes bogged down in constantly loading large lists from DB to memory, processing duplicate checks, and writing them back. Switching to a linked list structure to maintain the relationships allowed us to better scale across these many-to-one relationships.

One proxy → many implementations

Ambisafe had the implementation with the most proxies. What’s the proxy with the most implementations? Our data suggests it’s this MEV Bot, with 418 associated implementation contracts (e.g.). Note that I’m just counting distinct DELEGATECALLees — these likely don’t represent standard proxy-implementation relationships. Thirteen of them had implementation ranges of less than ten blocks.

[Edit: banteg pointed out that this contract could be some variant of the ds-proxy family, popularized by MakerDAO. We did some static analysis to confirm. It’s not a direct copy as it was compiled with a newer solc version (0.8.9) and contains some functions not found in the original:

{
    "1cff79cd": "execute(address,bytes)",
    "1f6a1eb9": "execute(bytes,bytes)",
    "60c7d295": "cache()",
    "65fae35e": "rely(address)",
    "78e111f6": "Unresolved",
    "948f5076": "setCache(address)",
    "97645e37": "Unresolved",
    "9c52a7f1": "deny(address)",
    "a90e8731": "Unresolved",
    "bf353dbb": "wards(address)",
    "c9892a5f": "Unresolved"
}

But it definitely appears to be some variant. Thanks!]

Right now there’s not much more that we can say given that the contracts are unverified, but we’re working on generating storage layouts directly from bytecode, which will provide more visibility. Here’s a teaser:

One thing we do see is that the relevant ranges for the implementation contracts are overlapping, meaning that the proxy is delegating logic to multiple implementations at once. This logic also features in ERC-2535: Diamonds, Multi-Facet Proxy. The idea is that a given proxy contract (a “diamond”) may use logic in multiple implementation contracts (“facets”), each of which is responsible for a separable part of the logic and set of slots on the proxy’s storage.

We currently only index proxy-implementation pairs based on their block ranges, i.e., the duration from the first block in which we observe a DELEGATECALL to the last. To provide complete coverage for Diamonds, we’ll also need to record the relevant slots for each. We’re planning this work but haven’t started.

Overlapping layouts

One common pattern we see with non-empty proxy layouts is that the proxy’s layout is identical to the implementation’s (e.g., PositionToken). We’re thinking of adding some visual affordance to make it clear which layout each entry in the unified storage map (and table) is coming from. Right now it might look like duplicate data in these cases.

A similar pattern is to have one layout contain explicit gap variables in the slots governed by the other contract’s layout, as we see with this Spice Finance proxy. Another “feature” we discovered is that a single contract can assign the same name to different variables so long as they’re in different scopes, so you can have many of these identical gap variables.

We’ve also found some more creative usage. EternalStorageProxy puts an address for _upgradeabilityOwner in the 0x00…00 slot even though its implementation uses the same slot for a mapping. This overloading works because the value actually used in the slot of the mapping is meaningless; typically it’s left as zeroes, but there’s no reason it can’t serve another purpose. They do a similar trick in slots 0x00…03 through 0x00…05, except in these cases the proxy and implementation agree that it’s a mapping but disagree on what mapping it is. Of course, this could cause collisions if any key is used in both mappings.

Deeply nested variables

This is not apocrypha, per se, as we frequently come across examples like these, but we love deeply nested variables! Check out allowance on Uniswap’s Permit 2: a triple-nested mapping with a three-member struct at the end of the rainbow! Gorgeous!

Cheers to the smlXL colleagues (and @banteg) who helped with this post. Curses upon those that keep refusing to prioritize mapping key counts.

Ethereum apocrypha was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

What happened?

(If you actually don’t yet know what happened to SVB, read Matt Levine’s piece.)

Driving down to SVB’s headquarters in Santa Clara on Friday morning, I couldn’t rid my mind of that South Park meme. Our CEO, Dor, had asked me to go to a branch to see if they were able to write cashier’s checks or had info on our pending transfers. We’re a small startup. A big chunk of our operating capital was in SVB. Dor had already tried at a branch in New York, to little avail.

We both knew the trip was futile. The California Department of Financial Protection and Innovation had already closed SVB and appointed the Federal Deposit Insurance Corporation (FDIC) as receiver. The FDIC’s press release had plenty of ambiguity, but it was clear that the bank was closed.

I wanted to try for three reasons: I like my job and my colleagues, and I wanted to do something. I don’t trust America: it wouldn’t really have surprised me if there was some special back door for well connected depositors to get their funds out of a closed bank. What better place to find it than at the HQ, in Silicon Valley? And I went out of morbid curiosity–what would it look like?

The answer: not much. When I arrived, there were about a dozen customers like myself, hoping, against their better judgment, that they might find some assistance. There were a few photographers and journalists. The lights were on inside but the doors were locked, a lone person at the front desk mostly avoiding eye contact. The FDIC press release was taped to the door.

The absurdist highlight was the arrival of the Brink’s truck. We all took photos, media and customers alike. As the driver went in, someone asked him if he was bringing money in or taking it out.

I chatted with a few friendly founders. We exchanged commiserations. But that was about the extent of it. And, yes, Justin Sullivan of Getty Images took a photo of me, shown above, that ended up in stories for NPR and CNBC. I was initially mortified by this — I never consented! — but it is pretty funny, and I presume he’s not required to get consent, was just doing his job. My mom said I look like SBF — that’s the worst part.

What is to be done?

The section title is an intentional albeit imprecise (perhaps asinine) literary reference. I’ll lay out my utopian ideas à la Chernyshevsky, although you’ll see my mood is better represented by Doestoevsky’s Underground Man’s sense of disaffection and futility.

FDIC insurance should be uncapped

Let’s start with the basics of banking: Stan had it coming. Stan didn’t object when the banker put the $100 check his grandmother had given him into a money market mutual fund. He didn’t even object when the banker reinvested the earnings into foreign currency accounts with compounding interest! Stan pursued a risky investment strategy and it didn’t work. It’s gone! If we gave Stan his $100 back, he’d be incentivized to pursue risky strategies in the future, pocketing the rewards when it goes well and getting bailed out when it doesn’t. That’s moral hazard!

But Stan’s experiences are meaningfully different than those of SVB depositors. SVB depositors have lost access to funds sitting in checking accounts designated for operational purposes like payroll. These were deposits, not investments. (Ironically, one founder I met at SVB said he believed customers with money in SVB money market funds would be fine because SVB was only facilitating those products and the money was held by other banks.) Why must depositors be exposed to this risk?

Let’s try the moral hazard argument: uninsured depositors must face losses when a bank fails because this incentivizes depositors to use healthier banks that are less likely to fail, thereby reducing systemic risk. As Bill Ackman points out, this puts the onus of evaluating banks’ creditworthiness on each individual depositor. The best demonstration that this doesn’t work is what’s just happened with SVB. We had a lot of money there because we perceived it as safe relative to alternatives. Apparently Roku, Roblox, and many others felt similarly.

You could argue depositors were careless for concentrating funds in few accounts and in sizes beyond the FDIC insurance cap of $250K. Fortunately for us, Dor spread out smlXL’s operational funding enough that we have no immediate operational concerns. But Garry Tan estimates that 30% of SVB depositors will fail to make payroll in the next 30 days. I suspect that’s inflated, and it’s easy to laugh at silly startups suffering from irresponsible financial management (see below), but were they crazy to consider their deposits safe? When was the last time uninsured depositors actually lost funds? No, genuinely, I’m asking you. I spent 30 minutes Googling it. ChatGPT tells me it was when depositors of Guaranty Bank in Wisconsin lost about a million in 2017, but the FDIC says those were converted into deposits at First-Citizens. In this disorienting week, the only solace I take is that ChatGPT is still lying confidently to me.

To the point: remove the $250K limit on FDIC insurance. It’s actually free to do this in two important ways:

First, we’re going to commit to regulating banks in the future properly such that these failures never happen. Right?

Second, I’m here to announce that I’m partnering with my technical cofounder Giannis Antetokounmpo to launch a new startup focused on checking account abstraction–you thought this only mattered for Ethereum?! You’ll log in to Giannis Bank and see a checking account with millions in funding. Underneath the hood we’ll have that parceled up into as many FDIC-insured $250K chunks held at actual banks as required. We’ll do it. We’ll build a fully FDIC-insured synthetic checking account with no limits. If you don’t want me extracting rent from this nonsense, and I swear on my tech worker identity that I will find ways to unreasonably enrich myself, let’s just acknowledge that the $250K limit applied on an account level is arbitrary and capricious. That’s legalese for dumb.

Once we’re not holding depositors responsible for bank creditworthiness, who should take that responsibility? I don’t know… Maybe the OCC or the FDIC? What’s Gary Gensler up to nowadays? Is he busy?

No bailouts, unless *I* really need one

We’ll never have this problem again now that we fixed the FDIC. But what about the bailout that Tan, Ackman, and others are advocating? More precisely, they want SVB depositors to be made whole, quickly, and they’re quite pointedly arguing this would be the government acting on behalf of depositors to make them whole and avoid contagion, not a bank bailout. Rough day for SVB equity holders — nobody’s standing up for them!

You have to be really careful with positionality when making appeals to “fairness.” Sure, I agree on the surface with Tan that it’s unfair that some startups will fail based solely on the bank in which they deposited their capital, a random cull imposed on a generation of innovators. But, and I’m sorry in advance for being flippant, there’s part of me that responds as follows:

Boohoo for them. Silicon Valley is already random. What’s a little more? Who’s to say whether this is actually good or bad for overall innovation? Isn’t it also true that making depositors whole would constitute a massive transfer to the same VCs that contributed to the bank run by telling their portfolio companies to get out early? Was that fair? To what degree is this profiteering via scaremongering?

This is my visceral response even though I would benefit from depositors being made whole. I am them. Boohoo for me. I can’t imagine how the rest of America must feel about this prospect of the FDIC going out of its way to serve the generally very privileged customers of Silicon Valley Bank.

I am sincerely sorry if you’re reading this and you’re worried about whether you’ll make payroll or receive your paycheck next cycle. I don’t mean to trivialize or mock the very real consequences — I’m trying to reflect on perceptions.

However you may feel about what constitutes fair in this context, the bottom line is that as uninsured depositors, we’re not legally entitled to much at all beyond relatively preferred status in the long line of SVB’s creditors.

So what should happen now?

Let’s see if the FDIC can work out how to make depositors materially whole, preferably soon, without too much special treatment. Unlike Stan’s case, it’s not actually gone. We presume it’s not all gone. Right?

First, end the bank run, as they did overnight Thu/Fri. Perhaps they should’ve acted more quickly — $42 billion escaped amid rumors on Thursday — but that also may be more the discretion of the California Department of Financial Protection and Innovation than the FDIC. It’s also an incredibly important decision and one that you’d want to analyze very carefully. I find it hard to fault the timing. I presume they can’t claw back withdrawals made on Thursday? If they can, they should — those withdrawals are precisely as legitimate as those that failed to go through on Friday.

Next, fulfill your obligations to insured depositors. If the FDIC loses credibility, ruh roh. The FDIC promised in the press release that “all insured depositors will have full access to their insured deposits no later than Monday morning.” Thinking about the logistics of this, I find it remarkable.

Finally, how can they do right by uninsured depositors and other creditors? Their toll-free hotline is active right now, over the weekend, and those damn bureaucrats are diligently working away at making sure they have correct records for uninsured depositors. The obvious next step is as follows: “Uninsured depositors will receive a receivership certificate for the remaining amount of their uninsured funds.” But the more important part is in the preceding sentence: “The FDIC will pay uninsured depositors an advance dividend within the next week.” They know depositors need to access these funds in the near term and they’re issuing an advance to try to accommodate. Depending on the details, it’s this advance, presumably but not necessarily calculated as a percentage of the uninsured deposit, that will mitigate fears about startups not making payroll.

Three things I’ll be looking out for in the weeks and months ahead, assuming no buyer is found over the weekend:

1. To what degree will next week’s advance make uninsured depositors whole? 80% would have drastically different outcomes than 20%.
2. To what degree and on what timeline will uninsured depositors be made whole eventually, via the receivership certificates?
3. How will the private sector accommodate startups with liquidity problems, perhaps by allowing them to trade or borrow against receivership certificates?

And, yeah, if smlXL’s operations ever come under immediate threat because of this nonsense, you’d better believe I’ll be crying to anybody who will listen for a rescue. I like my job. IT’S NOT FAIR!

Stuff you put at the bottom of a blog post

These are my own views, not those of my employer, smlXL. I didn’t want to comment too specifically about what this means for smlXL. Our CEO, Dor, has been transparent throughout. You can read more about how he experienced this (it’s pretty funny!) and what it means for us in his LinkedIn post. I also appreciate that he’s cool with me posting this — he even helped me by sharing some links.

I didn’t really edit this. I’m tired. If you see errors or have advice on how to make the post better, please email me at shaneauerbach@gmail.com. I’ll even acknowledge you here if you want.

I’m not sure if I’m allowed to use that image of me, taken by Justin/Getty, at the top. I’m going to ask for forgiveness rather than permission here. It is my likeness, right? At the very least it’d be a hilarious lawsuit! Anyway, Justin/Getty, if you want me to take it down, please pop me a cease and desist. I’ll chuck that in my “Fond Memories of the End Times” scrapbook and take the photo off this post. 🙂

Updates

1. 3/12 #1: Astute readers have pointed out that removing the FDIC cap would not be free because the two reasons I gave for why it would be free above are counterfactual. Good point! And it’s also true that you, noble depositor, would likely have to pay for the removal of the cap in some form, given that FDIC insurance is funded by the banks whose depositors are protected. Anybody have any idea how much it would cost and how it would be passed through? In any case, depositors perceive the checking account as a risk-free store of value. If providing said product is a little more costly than a store of value that’s risk-free almost all of the time, so be it. Let’s surface that cost. If that drives consumers away from banks to stablecoins and mattresses, then I’d argue that’s a reflection on banking as a modern technology.
2. 3/12 #2: Guys in my fantasy football league have pointed out that Giannis Bank already exists in some form, with examples like IntraFi and Max. I’m very disappointed by this — Giannis promised to do the due diligence on this one.
3. 3/13: We were bailed out, but don’t call it a bailout! But, shit, yeah, it’s a relief to hear that Dor has access to all the funds once more.

We start with a description of the Ethereum Virtual Machine and the state and non-state data it maintains. Then we cover challenges including interpreting compiled bytecode, observing contract-to-contract calls, and more.

Understanding the Ethereum Virtual Machine (EVM)
 — What is the EVM?
 — How is EVM state maintained?
 — What non-state EVM data exists?

Challenges in using Ethereum data
 — On-chain contract code is compiled bytecode
 — Contract source code is often private
 — Contracts call other contracts (and observability is poor)
 — Contract code is no longer immutable
 — Contract storage can be hard to explore
 — If it just happened, it might not have happened

Conclusion

Understanding the Ethereum Virtual Machine (EVM)

What is the EVM?

The Ethereum Virtual Machine (EVM) is sometimes referred to as a World Computer. Thousands of nodes across the world execute the same code to maintain the same state.

How is EVM state maintained?

Figure 1 illustrates EVM state management and data structures. We start from an arbitrary State n-1 in which an instance of the EVM has three types of state data:

1. Account balances (in ETH) and nonces. Figure 1 shows two addresses (0xa5…37 and 0x93…af), each with 5 ETH. Each address also has a nonce, not shown, which represents a count of its transactions.
2. Contract code. Figure 1 shows pseudocode for a simple contract, deployed at 0x93…af, that maintains a storage variable x and increments it each time the account receives a transfer of at least 1 ETH.
3. Contract storage. Figure 1 shows contract storage for 0x93…af as a dictionary of key-value pairs, including entry x : 1.

Let’s walk through an example. Account 0xa5…37 sends 1 ETH to the contract, 0x93…af, and the submitted/pending transaction sits in the “mempool” until it is included in a block. The example transaction costs 0.1 ETH in gas. Part of this is burned, per EIP1559. After a proposer creates Block n, validators verify that it is legitimate. The new block propagates across the network and receiving nodes then run the code triggered by the included transactions to update their state to State n. Because 1 ETH was received, x is incremented in storage from 1 to 2 and Event is emitted in the log of the receipt associated with the transaction. Going from state n to n+1, only 0.5 ETH is sent. The contract code runs but neither increments x nor emits the event because the if condition is not met.

We recommend Diving into Ethereum’s world state by Timothy McCallum and Getting Deep Into Ethereum: How Data Is Stored In Ethereum? by vasa.

What non-state EVM data exists?

Blocks contain transactions that cause state to change. It can be intuitive to think of a block like a diff used in git software, in that it records a set of changes. That being said, it’s a somewhat incomplete diff: a block only records the transactions that initiate the state changes, not the impact of executions on state, which may involve many internal transactions between contracts.

EVM instances generate a receipt for each processed transaction. While receipts are off-chain and stored on each node, they are verifiable: a hash of each receipt is included in the block. The receipt, in turn, contains logs / events, which are data emitted by smart contracts called in transactions. In the example contract code, emit Event would create a log with the data designated by the smart contract.

We recommend Ethereum.org’s EVM documentation; Ethereum, Smart Contracts, and the World Computer by ConsenSys; and the About the EVM page at evm.codes.

Challenges in using Ethereum data

Ethereum is a public blockchain in the sense that anyone can run a node by running geth or an alternative client. New nodes catch up to current state by receiving blocks on the network and processing them in order.

One might expect that it would be easy to access and interpret all data on a public blockchain like Ethereum. Alas, it isn’t. Let’s explore how these distributed systems are complex.

On-chain contract code is compiled bytecode

Smart contract developers write readable code, often with rich commenting and specification like Natspec, in languages like Solidity (JS-like) and Vyper (Pythonic). There are also lower-level options like Yul(+).

However, the smart contract code stored on the blockchain and executed on each node is compiled EVM bytecode, often represented in hexadecimal. While it’s executable, it can be difficult to interpret and/or decompile to understand the workings of a smart contract.

Contract source code is often private

A smart contract developer can choose to make their Solidity/Vyper/Yul contracts public by submitting them to services like Etherscan, which compile the submissions and check that the resulting bytecode matches what’s in the EVM state. But many developers will keep their smart contract source code private either because they see limited advantage in making it public or because they fear the source code could expose possible security issues or exploits.

Another contract source code object that may or may not be public is the Application Binary Interface (ABI), which tells other services how they may call a contract and how to interpret its events / logs. Without an ABI, it can be difficult to call a contract or interpret its logs. You may be able to guess at least a partial ABI for a contract: Identify its general purpose and assume its functions and events are the same as those for standard contract templates (e.g. ERC20, ERC721) for that purpose. Or you can use a function signature database like sig.eth.samczsun or 4byte.directory. It may also be possible to infer parts of the ABI by analyzing disassembled bytecode (e.g. whatsabi) if the bytecode was generated by a standard compiler. Here’s an example of a contract written in Yul that was exploited for ~$1MM by an attacker who reverse engineered the decompiled assembly to find an unused approve function without standard checks. In summary, you can call functions and interpret logs as long as you can work out the parts of the ABI that correspond to those functions and events.

Contracts call other contracts (and observability is poor)

Every smart contract computation on the EVM is kicked off by a signed transaction. Currently, only externally owned accounts (EoAs) can sign transactions. But a smart contract called by a transaction may call other contracts, which can in turn call others. These contract-to-contract calls are often referred to as internal transactions. Just about any moderately complex smart contract system will involve internal transactions.

One common use case for internal transactions is using proxies to enable upgradeability. If you want to allow your users to do some complex thing, but you also want to be able to upgrade / update / fix how that complex thing is executed, you can have users interact with a proxy contract that calls an implementation contract with the address written in the proxy’s storage. When you are ready to upgrade the implementation logic, deploy the upgraded implementation contract and then call the proxy to reset its stored pointer to the new contract. Figure 2 illustrates the idea.

There are a few variants of these designs: One can use a proxy purely as an updateable registry that forwards the user to another contract. An alternative approach is to use the DELEGATECALL pattern, in which code from the called contract is executed on the calling contract’s storage, allowing for updated functionality on the same state storage.

Critically, these internal transactions are not recorded explicitly in blocks. Recovering them requires using a debug_traceTransaction method or instrumenting a custom EVM client. The latter approach is employed by services like Etherscan and Tenderly.

We recommend DelegateCall: Calling Another Contract Function in Solidity by zeroFruit, and Accessing contract internal transactions and Instrumenting EVM on Ethereum Stack Exchange.

Contract code is no longer immutable

Ethereum’s original CREATE code deploys a contract to an address based upon a hash that includes the deployer address and their nonce, the count of transactions initiated by the account. Funds can be sent to a contract before or after it is deployed. Contract code and storage can be deleted, and the balance sent to a designated party if the SELFDESTRUCT instruction is called, but nothing can be redeployed at that address with CREATE because the original deployer’s nonce would have increased.

The Constantinople upgrade introduced CREATE2 to allow developers to deploy a contract at a deterministic address, so long as that address doesn’t already have deployed code. It is like CREATE in that it includes the deployer’s address in a hash for the deployment address, but different in that it replaces the deployer’s nonce with the initialization code of the contract and a user-selected seed (“salt”). CREATE contracts will never collide with CREATE2 contracts because of a domain separation: CREATE2 puts a 0xFF as the first byte input to the hash.

CREATE2 also enables “metamorphic contracts,” where the code at a given address can be changed. Figure 3 and the instructions below illustrate the logic:

1. Create a smart contract factory, i.e. a contract that creates and deploys contracts, to serve as your deployer for the metamorphic contract. Give that factory the ability to deploy a self-destructible metamorphic contract with CREATE2. The initialization code of the metamorphic contract created by the factory should instruct it to copy the code of an implementation contract whose address is specified in the factory’s storage.
2. Create implementation contract v1 with the code you want in the metamorphic contract. Put the address of implementation contract v1 in the factory’s storage. Use the smart contract factory to deploy the metamorphic contract, which will call the factory in its init to find the address of the implementation contract whose code it will copy with EXTCODECOPY.
3. When you wish to upgrade implementation, i.e. metamorphose the metamorphic contract: a) deploy the new logic in implementation contract v2, b) call SELFDESTRUCT in the metamorphic contract, c) replace the pointer from implementation contract v1 to v2 in the factory’s storage, d) redeploy the metamorphic contract with the factory. Because the deployer, initialization code, and salt remain unchanged, the new code from implementation contract v2 will be deployed at the same address as was used previously, achieving the metamorphosis.

CREATE2 also has significant security implications: a user may be confident with the impact of their transaction with a certain contract address only to find that the code at the address has been updated. There is also some risk of this with proxy structures, but proxy structures are clearer / more detectable in that the user can, at least in theory, see them pointing to new upgrades.

You cannot even feel particularly safe with a contract that was created with CREATE. Call that Contract C and suppose it was created using CREATE by Contract B, which was in turn created with CREATE2 by Contract A. The owner of A can update B with SELFDESTRUCT and CREATE2, resetting its nonce in the process. B’s nonce can then be incremented up to where it was when C was created, and then C can be upgraded with SELFDESTRUCT and CREATE called by B. With these “cascaded resurrections,” you’re at some risk of unexpected contract upgrades if you’re interacting with any contract that is anywhere downstream of a contract that was created with CREATE2, which isn’t easy to detect.

These and other considerations motivate EIP-4758, a current proposal to deactivate SELFDESTRUCT by changing it to SENDALL, thereby allowing for recovery of funds without deleting code or storage. It is a candidate for inclusion in the upcoming Shanghai fork and there’s some fascinating discussion on the Ethereum Magicians forum.

We recommend A Tool for Detecting Metamorphic Smart Contracts, by Michael Blau; Not all Code are Create2 Equal by Michael Fröwis and Rainer Böhme, The Promise and Peril of Metamorphic Contracts by 0age, A Postmortem on the Parity Multi-Sig Library Self-Destruct by Parity Technologies, and Parity multisig hack resulting in 153,037 stolen ETH by Parity Hack Trace.

Contract storage can be hard to explore

Contract storage is a dictionary of key-value pairs. Slots are generally assigned to variables in the order that they appear in the contract code, with some nuance around dynamically-sized arrays, mappings, and type nesting. Even if you know the source code of the contract, you often can’t retrieve a complete version of the contract’s state by digging in storage unless you know exactly what you’re looking for. To interpret storage slots in proxy contracts, you need to know the source code of the implementation contract at the time of the storage change you’re interested in. If you don’t know the source code, you’re really going to be wandering blindly. A variable’s representation in storage depends on both its type and the high-level language’s compiler. For Solidity:

• Fixed-size, single-valued variables: If the first two declared variables in a contract are single-valued and their sizes sum to greater than 32 bytes, they can be retrieved via RPC call to any Ethereum node with eth_getStorageAt(contract_address, 0x…0) and eth_getStorageAt(contract_address, 0x…1), respectively. Two variables with total size less than or equal to 32 bytes would be packed together in the first storage slot.
• Dynamically sized, single-valued variables: string and bytes are not capped to the 256 bits of a single storage slot. If the length of the dynamically sized variable is less than or equal to 31 bytes, its value is packed together with its length in the storage slot: the value is stored in the higher-order bytes and length * 2 is stored in the lowest-order byte. If the variable is 32 bytes or longer, then only length * 2 + 1 is stored in the lowest-order byte of the slot, and the value itself is stored starting at keccak256(storage_slot) and spanning as many storage slots as required. The * 2 and * 2 + 1 operations on the length are designed such that we know an even value represents a short variable that can be retrieved from the higher-order bytes of the same slot, whereas an odd value suggests that the variable’s value is stored starting at keccak256(storage_slot).
• Multi-valued fixed-sized variables: Suppose the third declared variable is uint256[2]. Its two values can be found with keys 0x…2 and 0x…3, respectively.
• Dynamically-sized arrays: If the next declared variable is a dynamically-sized array, the size of the array is stored at 0x…4 and then the index zero element of that array is stored at the hash of that slot, i.e. keccak256(0x…4).The index k element of that array is stored at keccak256(0x…4) + (k * elementSize).
• Mappings: If the next declared variable is a mapping, it is assigned to 0x…5, but the storage slot is left empty. Instead, for a key x defined in the mapping, its value is stored at slot keccak256(x,0x…5). Note that this means that one cannot iterate over mappings in contract storage without knowing all of the keys that exist in the mapping. For example, you can’t determine all of an ERC20’s token holders by looking at the balances (mapping(address, uint256)) in the contract storage because you need to know the token holder’s address to find their balance.
• Combinations of types: Where types are nested within each other recursively, the location of a value is found by recursively applying the structures above.

We recommend Understanding Ethereum Smart Contract Storage by Steve Marx; Decoding the Storage of an Ethereum Contract by Inuka Gunawardana; and Data Representation in Solidity by Harry Altman.

If it just happened, it might not have happened

Forks and reorgs mean that recent blocks that appear to be part of a consensus at a given point in time may end up being discarded, potentially along with some of the transactions and updates to state therein. Figure 4 shows a fork in which two competing valid blocks emerge (Block n: A and Block n: B), only to be resolved when an additional block emerges on one of the two chains. In this case, transactions in Block n: A that weren’t also included in Block n: B would end up being dropped even after some nodes who viewed Block n: A as the consensus may have represented the transaction to the user as successful.

Forks and reorgs come in varying degrees of severity. A Proof of Work consensus process may run into predictable ambiguity where 2+ valid competing blocks propagate simultaneously across the network. These may persist several blocks, but will resolve without intervention as one chain grows larger than the other. Sometimes longer forks can emerge if connectivity in the network is degraded.

Consensus can fail altogether where nodes in a network may be running different clients or different versions of the same client. While they should all be following the same consensus rules, a failure in consensus can occur if one client, or a version of a client, deviates. One example of this was a hard fork created by a geth update in late 2020.

There are also intentional hard forks, often to update consensus protocols. Ethereum.org maintains a history of these. A particularly controversial example came from the first Ethereum DAO launching in 2016, only for an assailant to discover vulnerabilities in its smart contract and siphon off $55M. The Ethereum community was split on how to respond: They could simply roll back the state of Ethereum to where it was before the abuse occurred, thereby making victims whole and depriving the assailant of their gains. Many did take this approach, but others felt that such coordinated actions against the stated consensus protocol violated the decentralization at the core of Ethereum’s philosophy. Thus emerged a hard fork, which persists to this day, between the community that reverted and those that refused to revert, where the chain resulting from the latter is referred to as Ethereum Classic.

Ethereum’s move from Proof of Work to Proof of Stake substantially reduced the frequency and size of reorgs. Before the merge, one of our nodes running on the edge tended to experience reorgs about once per hour, on average. Most included just a single block, but there were longer reorgs with some frequency. The percentage of blocks in the consensus chain that explicitly reference reorged blocks is called the ommer/uncle rate: it ran a little over 5% pre-merge. Post-merge that rate is zero mechanically as blocks don’t reference ommers. In the few post-merge months that we’ve been running a node on the edge, we’ve experienced roughly two reorgs daily. Each of the post-merge reorgs we’ve witnessed has involved a single block. We recommend Investigating why reorgs happen in proof of stake Ethereum by Sam Lewis for more on this.

Conclusion

Ethereum is a public computer with state that results from the collective action of its users. While every user with access to a node theoretically has access to that state, in practice there are many challenges to consuming that data.

Thank you to Yoav Weiss, Michael Zhu, Dan Boneh, and the smlXL team members who offered advice and feedback for this post.

Respond to this post if you have any questions.

We’re hiring!

How Does Ethereum Manage Data? was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.