Building applications and services that rely on real-time data, particularly on-chain, is challenging. It requires several specialist skill sets and is resource- and time-intensive. It’s especially hard to build performant apps that are on par with users’ expectations. This is true for both smaller teams and market-leading enterprises. And it is only getting more complex with rollups, sharding, DA layers, temporary storage and more. At this stage, chains and client teams are focused on writes and scalability, throughput, and decentralization. These sometimes come at the expense of readability and accessibility. If you’ve ever tried to break down a complex monolith into a distributed service, you will recognize this common tradeoff.

We saw this as a challenge and an opportunity. We set out to build an EVM client specifically optimized for reads–we plan on open-sourcing it by EOY–and a streaming, storage and computation platform around it to make it easier to develop modern event-driven applications and pipelines. sim allows developers to harness the power of blockchains by easily implementing complex real-time applications and data flows that runs alongside chain execution. It’s the fastest way to build performant applications and pipelines that rely on on-chain data.

Here are some examples of what people do with sim today (all implemented/running in the wild by our alpha customers):

1. You’re building an application (dApp) that has a real-time activity feed. You need to track and instrument multiple contracts and protocols, but you want to avoid writing brittle indexing logic, spending weeks or months building a backend for data transformations and middleware to join data for APIs.
2. You’re a fund/trader/searcher looking into various DeFi protocols to understand prices and activity. You want to track and instrument thousands of contracts/pools/tokens (verified or unverified) without requiring a ton of ecosystem-specific work. No problem: use our Lambda, or Patch, to get all the data you need, even if it’s not instrumented in the contract itself. All running at the tip and backfilled, available for you to use with our hosted solution or streaming to your own data infra.
3. You’re doing on-chain (and cross-chain) monitoring and auditing and need to scan entire blockchains for specific events. For example, you want to monitor and analyze storage writes to look for entries containing a specific address.

One example we’re excited to share is sim Explorer (fka evm.storage).

We’ve been eating our own dogfood by building sim Explorer on sim. It’s the fastest, most in-depth, developer-focused block explorer and transaction simulator available, offering unparalleled storage tracking capabilities. It’s loved and used by thousands of developers and completely powered by sim. Want to see for yourself? Check out the sim canvas and APIs powering the homepage.

These are just a few examples and we are excited to see what you build with it.

Another way to understand sim is to consider what it augments and replaces: node providers or running your own custom node, indexers, fork/shadow instrumentations, building/operating ETL and ingestion pipelines to your database, and even building and deploying the APIs and webhooks for your services/products/users. If you so choose, it’s all fully hosted, scaled and maintained for you. Have trust issues? (🙋, I can relate) All good! You can use sim to instrument on-chain data and write it to your own infrastructure, or hold until the EOY release of the OSS engine. sim is built for and around builders. We’ll help you reduce costs by replacing multiple expensive vendors you’re already using, but that’s not our main goal. Our mission is to help you do things that were once impossible and get better and faster products to market faster. If you’re doing anything with blockchain data, you’ll want to give it a try.

Over the next month we’ll have constant updates and sim examples, walkthroughs and technical information about how sim works. We intend to become a community where builders can share and build upon each others’ ideas and data.

To get started, visit sim.io. Read the docs at docs.sim.io. Say hello on Telegram and follow us on Twitter/x .

sim: a canvas for your ideas was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

As I was looking into what I wanted to do next, at the beginning of 2021, I was talking to the smartest people I know, getting their perspectives on different spaces, problems, and opportunities; the most polarizing subject was blockchains. Naturally, this made me want to delve deeper, learn more, and develop my own point of view. What eventually drew me in, overwhelming my skepticism, was the realization that blockchains enable public computing: software and infrastructure in the public right of way, managed by the public (they are also a giant Marketplace, but we will keep that to another post). I started to dedicate my time to blockchains in June 2021 and founded smlXL (pronounced /small-X-L/ or /s-m-l-x-l/) in October.

So much of our life depends on computing, and decentralized computing has shifted paradigms. The last decade has reinforced the urgency of moving away from centralized, obfuscated, and excessively abstracted data. It’s manifested in corruption and theft of user funds at FTX; censorship of social media, including silencing government critics, as well as allegations of election fraud; and even inflation, where the US Federal Reserve has difficulty measuring money supply due to complex financial instruments.

Blockchains promise transparency and accessibility, but these benefits come at a price. As distributed systems grow more complex and opaque over time, accepting these inherent challenges is necessary to realize blockchains’ potential.

As we explored blockchains and built tools in the EVM ecosystem, our mission became even clearer: to make public computers more transparent, useful, and accessible.

Paul Krugman famously underestimated the internet’s impact by projecting it would equate to that of fax machines. As is always the case for paradigm-shifting technologies, there were many skeptics of what Google was up to in the early days. We take a cue from Google’s original ethos: just as there was immense value in understanding each website’s context and je ne sais quoi, moving past merely indexing sitemaps and keywords, there will be immense value in comprehensively understanding blockchains.

Understanding every contract and account opens up endless opportunities, Here’s what the future holds:

• Search: Imagine smart contracts as APIs, a collection of smart contracts as an application, and transaction histories as a complete record of a user’s interactions with these applications. We’ll help users find the right applications and applications find the right users.
• Fulfilling intents: You can “remix” and make complete applications on the fly by composing transactions that interact with a combination of contracts. A simple application of this is routing, where a user wishes to perform some operation optimally (e.g., exchanging or borrowing assets at the most favorable rates), but does not care what applications are used.
• User-friendly interfaces: Right now, sending transactions to blockchains and even reading state from them is clunky, let alone generating aggregated reports. It won’t be. We have been running a prototype for our blockchain LLM interface in the ChatGPT plugin store for a couple of months, and it will soon be integrated into our existing products.

Comprehensive semantics unlock endless opportunities, just as Google showed with the web.

What do we need for the new fax machine? We’re building proprietary techniques to leverage blockchain’s determinism and transparency for rigorous analysis of EVM bytecode and contracts. Combining static and dynamic analysis, symbolic execution, instrumentation, and simulation purpose-built for blockchains, we can provide a complete and rich understanding of accounts, contracts, and their rich relationships.

In April 2022, after months of R&D and prototyping to validate our proposed approach, we started to execute our vision: our instrumented Ethereum Virtual Machine (iEVM) is a proprietary EVM client implementation geared towards simulating and instrumenting blockchains.

Our iEVM has capabilities that are not available through any other EVM client. It can run in a distributed manner across multiple machines, inject custom instrumentation contracts that have special access to other contracts and state and run custom transactions in a sandboxed environment, trigger simulations based on certain conditions/hooks, and emit data that is otherwise not emitted by a normal EVM client. It also executes orders of magnitude more transactions/blocks per second than the fastest EVM chain. This means we can run high-speed simulations and do additional off-chain computations in near-real time while keeping up with the chains as they progress. We’ve paired this with minimal latency and a powerful data-streaming engine so you can extract/emit data off the iEVM and its probes, transform it, and load it to your data stores, services, and applications.

A second area we’ve invested in is our static and dynamic analysis tools; these help us classify and test contracts, generate storage layouts, identify and track key storage variables, and much more.

To continue the Google Search analogy, you can think about what we’ve built as analogous to Google’s Web crawler, their simulation/headless browser rendering capabilities, and their compute infrastructure to process the data. While Google took a “black box” approach to understanding the internet, the internet at that time was mostly static. We’re working with dynamic programs. Our approach differs from other players in the space who have defaulted to indexing logs and events and relying on manual curation. This is similar to how now obsolete search engines indexed sitemaps, tags, and human-curated taxonomies and directories alone.

These capabilities will not only enable offerings for search, routing, and data products, but they’ll introduce a new paradigm where developers can pick and choose what to offload to off-chain computations, enabling them to ship more complex applications faster and cheaper. Consider the hybrid approach: a developer builds services that maintain critical state on-chain and interact seamlessly with other contracts. But they perform related computations that don’t require the security of decentralization cheaply in an iEVM simulation.

This could be as simple as near-costless off-chain logging: 880MM logs were emitted on Ethereum alone over the last year, accounting for ~4.4% of total gas cost and ~$75MM. Developers are already trying to reduce log emissions to cut costs, but this degrades observability in products that overly rely on logs. With our infra, developers have their cake and eat it too by having as much logging as they could want at a fraction of the cost, given that they don’t have to be emitted on every Ethereum node. Observability is so central to the value of public computing–we can’t coordinate actions without a shared understanding of both history and current state.

But off-chain logging is just the tip of the iceberg. Currently, developers build their smart contracts optimizing for cost, composability, and security, but not how they’d want to consume the data in their application and services, or for ease of third-party analysis and observability. Our solution addresses that, letting developers save time and resources and create products that would be otherwise cost-prohibitive! Developers can build out entire software backends on our iEVM instances, combining real-time and historical data from multiple blockchains and off-chain sources and running complex derivations and aggregations, all of it streaming at the tip. Parts of this platform are already live in production, powering evm.storage.

Our work also enables off-chain zero-knowledge (ZK) proofs that a certain state exists across a given set of contracts at a particular block. We expect these capabilities to be increasingly important as chains further integrate zero-knowledge techniques to increase privacy and efficiency.

We launched our first product, evm.codes, last year. It’s an interactive guide for EVM opcodes, and it’s established itself as one of the go-to resources for every EVM developer.

Our latest product, evm.storage, is a first step towards blockchain search and exploration. It’s also an early showcase of our unique infrastructure capabilities; no other product provides deep insight into contract storage. With evm.storage, we are able to decode and track every Solidity contract in real-time.

While we have operated mostly in stealth, we have expanded through word of mouth and have made headway in building products that target our initial personas: developers and blockchain prosumers, and the response has been overwhelmingly positive (and no, we don’t have a token.)

Today, between evm.codes and evm.storage, we see about ~15k monthly active users (MAUs). Electric Capital, a notable crypto VC, estimated in a recent report that there were 21k monthly active developers in June 2023.

While we’re thrilled with the progress, we’re more excited about what’s to come. Today, we’re announcing two major improvements to evm.storage:

1. We’re expanding our coverage to unverified Solidity contracts. Until now, the public’s ability to understand a contract’s state has been at the discretion of the developer. If they don’t submit their source code to a third party, off-chain provider, there’s not much that users can see. These also aren’t edge cases–many critical contracts used in high-volume trusted applications are unverified. In the public interest, we’re also open-sourcing (under AGPL license), a critical static analysis tool that we used to generate storage layouts for unverified contracts. (read announcement)
2. We’re launching transaction pages. If it happened on-chain, we know about it, and we’re striving to present it in a way that’s rich, intuitive, and useful. We’ll be adding traces and an invocation flow in the near future.

We will continue to expand evm.storage and introduce new products building upon the unique capabilities of our iEVM, streaming data platform, and program analysis tools.

Our roadmap:

1. We’ll extend evm.storage coverage to all primary blockchain entities (contracts, accounts, transactions, and blocks) across primary EVM chains and testnets.
2. We’ll enable users to interact with contracts, simulate transactions, and explore simulated traces.
3. We’ll introduce (sic), our natural language interface to blockchains, enabling users to more easily explore, interact and engage with blockchains…
4. Introduce our paid self-serve offerings, opening our infrastructure to developers, enabling them to leverage our iEVM, off-chain computation engine, program analysis suite, and blockchain data products.

We couldn’t have done this alone. When we started this journey, we raised $13.4 million in seed funding from some incredible investors–a16z crypto, Greylock, and an amazing battery of angels–and we want to thank them for enabling us to spend the time and resources to build something we believe will be truly valuable.

We are always looking for people who are passionate about our mission and products. If you want to make public computers more transparent and useful, check out our site (launched today!) or join our TG group.

Dor & the smlXL team

Making public computers more transparent, accessible, and useful was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

As a company, we aim to make public computers (blockchains) more transparent, useful, and accessible. You can’t understand a machine without understanding its moving parts. For Ethereum, these are its smart contracts.

We can’t fully understand what a contract really does without inspecting its state, i.e., its storage slot. With evm.storage we’ve made strides in representing storage slots and their occupying variables for verified contracts. However, our understanding will never be complete if it’s limited to verified contracts. The industry has yet to solve the problem of examining and making accessible the state of unverified contracts.

While we’ve tracked over 59MM contract deployments on Ethereum, we’re only able to present ~5MM of these with verified source code-based layouts. Of those 5MM, we see only about 500K distinct code hashes. A tremendous amount of blockchain activity runs through unverified contracts.

Today, we are excited to announce that evm.storage now provides bytecode-generated layouts for unverified Solidity contracts. This allows anyone to view and track the storage of these previously opaque contracts.

We are also open-sourcing the code (under AGPL license) for a key component: our Storage Layout Extractor tool. The tool symbolically executes a contract’s bytecode to infer its storage layout. While we’re early in the tool’s lifecycle, we are quite happy with the initial results. In preliminary benchmarks, we were able to compute layouts for more than 120k contracts in under 110ms on average. At times the resulting layouts are even more accurate than those generated by solc with source code! You can read more about this in our technical deep dive.

When users land on an unverified contract in evm.storage, they will be presented with the bytecode-generated layout of that contract. It will look and feel similar to the experience for verified contracts, only without variable labels–these can’t be inferred from bytecode, and while we’re creative souls, we figured we shouldn’t make them up! A good example is Opensea’s Openstore contract, which is unverified, you can see the generated layout and storage history for mappings/variables. Raw layouts are also still available.

One evm.storage feature we love is the ability to see the historical state of slots, variables, mapping keys, etc. This is also now supported for unverified solidity contracts, but works a bit differently. You’ll be able to immediately browse a bytecode-generated layout (unless we have a bug… and, if so, please report it!). But we’ll only add this contract to our storage history processing queue upon the first user visit. The data will be made available gradually as data for this contract starts streaming from our infrastructure. While the data is backfilling, you will see a banner at the top of the contract page. This “on-demand storage tracking” setup metrics helps us prioritize unverified contracts that our users are interested in (or a ploy to make you visit our site twice, thereby further inflating our sky-high usage.) We track all verified contracts by default.

The backstory and details

Given our mission, we have to support unverified contracts. At the same time, we believe that a vast majority of our users are currently interested in interacting primarily with verified Solidity contracts. So, it both had to be done and was hard to prioritize. We feel similarly about Vyper/Huff/ASM, for what it’s worth (and we will get it done). Early on, we scoped out the work required to support unverified contracts, but it lingered in our backlog.

Having experience in writing analysis tools that depend on the contract’s bytecode rather than its source code, we initially proposed a static analysis-based solution, including (roughly):

• Generating a Control Flow Graph (CFG) from the contract’s bytecode.
• Scanning the basic blocks for storage-related opcodes (SLOAD/SSTORE).
• Tracing forward and backward from the storage operations to find out the subject storage slot and its data type.

As time went by, it remained hard to justify taking on this work given the opportunity cost. We also anticipated this work would have a lot of “gotchas” and edge cases. So we started looking at ways we could tackle this while minimizing the impact on our top priorities. We kept our eyes open for the right people to take this on as a project, only to stumble upon Marcin and Ara from Reilabs. They both had experience with compilers and VMs and were into crypto. It seemed like a great match, so we contacted Reilabs and discussed embedding Ara or Marcin in a small “tiger-team.” They were excited, and Ara had availability so we got to work, picking up from where we left off. After a few conversations with Ara, we decided that it would be beneficial to take an even more ambitious approach:

• Disassemble the contract’s bytecode to a stream of EVM instructions (opcodes).
• Symbolically execute the above stream on a custom EVM implementation that explores all reachable execution paths, i.e., take both branches in a JUMPI instruction.
• For each value observed throughout execution, have the VM build a symbolic value that represents the operations performed to that particular piece of “data.”
• Pass these symbolic values (trees, really) through a type-inference process and finally build a storage layout from the results.

While this approach might not be suitable for other architectures that make use of registers and execute huge programs, we were confident it would work fairly well for a special architecture like the EVM as it is a simple stack machine, and the smart contracts that it executes have a strict size limit due to the network rules.

To read more about our benchmarks and technical details, please check out our technical deep-dive post and the repo, and please join us in our TG group if you have any questions.

Announcing bytecode-generated storage layouts on evm.storage was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

At smlXL, our mission is to make public computers more useful, accessible and transparent. We believe that blockchains are important infrastructure. As society’s reliance on computing grows, having computers and software that can be operated, maintained, run and accessed by the public is increasingly important.

Today we’re sharing evm.storage, our latest contribution to making the EVM and its data more transparent, starting at its foundation: storage. Though blockchains remain complex systems, evm.storage is our first step toward comprehensible public infrastructure. We have a full roadmap for evm.storage and it is the first of several upcoming products, all powered by our instrumented EVM (iEVM), simulation and program analysis suite.

We hope you explore evm.storage and provide feedback on how we can further advance our mission and what areas you’d like visibility into. Progress requires collaboration; join us! And if you have any project that does not seem possible/feasible by any other blockchain data provider or not feasible to do on-chain, please reach out to us. We might be able to help — we are starting to onboard select customers.

About evm.storage

evm.storage is designed help you dive deep into the storage of any contract on the Ethereum blockchain:

1. Explore contract storage and state at any block height.
2. View variables as they’re defined in the contract and values presented according to their type for verified contracts, with support for solc versions as low as 0.3.6.
3. Explore deep, complex storage structures like nested mappings, arrays, and structs.
4. Review the storage history for variables that occupy a single storage slot, and some others (e.g. entries in a mapping), For instance, you’ll be able to see all changes to your balance for a given token, including corresponding transaction hashes.

Our Alpha release has some caveats:

1. Limited functionality for unverified contracts: The tool currently works best for verified contracts. For unverified contracts, we offer a raw presentation treating each slot as a variable. For now this includes unverified contracts even when their bytecode matches a verified contract, as well as those deployed by a verified factory.
2. Limited support for contracts with Yul/inline assembly: At present, storage slots that are accessed exclusively via inline assembly will not be included in our storage map/table. This will be fixed shortly.
3. Lots of moving parts: We will update things regularly and favor velocity of iteration over reliability. We’re still treating it as an alpha. We are also currently running on our staging machines, as we await our servers in our new DC.

We invite you to report bugs and offer other feedback via our bug report feature, or by joining our Telegram group. We’re also planning to open-source the frontend so that users can contribute directly.

Our Roadmap

We are actively working on these features for evm.storage:

1. Storage layouts for unverified contracts: We are currently showing raw storage slots for unverified contracts. We’re working on enriching coverage through both static and dynamic analysis.
2. Transaction tracer and view: Transaction view and detailed transaction traces, including rich information around SHA3/storage operations
3. Richer information on contracts: We’re planning to enhance the contract summary with richer information that’s stored on-chain like balance, ABI, constants/immutables, and more.
4. Contract interaction and transaction simulation: We want to empower users to interact with contracts and simulate transactions.
5. Other EVM chains: Avalanche is next in line with Arbitrum currently planned after.
6. Filtering, sorting and search: We’ll augment the experience with features that you’re excited about, such as search/sort/filter features for tables.

Shoutouts:

We would like to thank Yoav Weiss, Dan Boneh, Arjun Rao, Kevin Britz, Cece Z., Patricio Palladino, Sam Ragsdale and maurelian.eth for their early impressions and feedback.

Introducing evm.storage — Increasing transparency and accessibility for EVM contracts was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

If you know me or worked with me, you probably heard me say that I am notoriously bad at naming. It is one of my least fav things, and I mostly opt for functional names, so I did what our team normally does when we face a hard problem, search for the experts in their fields and learn from them. We sought the help of renowned naming experts, and these experts came up with a bunch of great names, evaluation criteria, and goals, which led us to our three main naming goals: our name had to be memorable, it had to be engaging, and it had to be easy to find (there are other considerations like trademarks, domains etc…).

Ultimately, the names proposed didn’t feel like “us”, they felt very fungible. Coincidentally, as I was trying to make the call between keeping the name Comity Labs or going with a new name, this old architecture book popped up.

Eighteen years ago, my roommate had this massive and distinct book at home. After checking it out, despite it NOT being an easy read at all, this book inspired me to read other more “digestible” architecture publications. These publications ranged from topics like public spaces and how people use them, how to think about materials, how to finish and use the properties of the material to your benefit instead of fighting against it, and the differences between a place and a space — topics that I am still interested in to this day. So, after discussing it with the team, and the agency (they ended up loving the name too), smlXL was born. Pronounced: /s-m-l-x-l/ OR /small-X-L/ OR however you’d like to like to.

Some reasons we like it:

• It is far more playful and exciting compared to a “typical startup name” that an agency would come up with; it is important to have something memorable/curious.
• We find the “conflict” between small<>XL fun
• I lived in Germany for several years and always found it cute when you say something is “BIG-small” for very small.
• As an early-stage startup, we wanted a name that gives us space to explore and is not too suggestive of one product or space, we see this name as our company name, not our product name.
• We are starting small but have big aspirations
• We know we wanted to build different products that work across lifecycles and various types of customers.
• The “sizes” (s,m,l,XL) are a “collection” we are a group of people working on data/compute running on a collection of nodes/different people.
• We are opinionated about our products and solutions, but we still want you to make it your own, so choose your own “fit”
• We find it unique, ambiguous, and fun, and we hope you do too!

So yeah, this is everything you might ever want to know about our name, if you ever need a name for your startup, we might have a few unused professionally sourced names for you. :)

Dor

smlXL?!? was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.

We know there isn’t much online about us — we plan on changing that soon. We are happy to share some of the beliefs that inspire our work. We believe that computing is key to our lives. This is both hard to refute and increasingly true with each day. Having computers and software that are operated by the public and in the public right of way is incredibly and increasingly important.

However you may feel about the early implementations and applications of blockchains, blockchains are critical to enabling public computers.

The features of blockchains that allow them to operate as public computers also make them complex distributed systems, which are notoriously hard to make transparent, operate, and understand. As usual, you can’t have your cake and eat it too.

So, where do we come in? We are out to make public computers and blockchains more useful, accessible, and transparent. We’ve started by documenting and creating resources specific to EVM opcodes with evm.codes and plan to use this blog to share some of our research, learnings, and core technologies behind our products. We are kicking off this blog with four posts: How Does Ethereum Manage Data, Transactions on Ethereum, EVM Contracts Construction, and Meta-Transactions.

Enjoy, and stay tuned for more!

All of us @ smlXL

P.S We are hiring

Welcome to Our Blog was originally published in smlXL on Medium, where people are continuing the conversation by highlighting and responding to this story.