For the uninitiated: mining pools are software providers who enable many mining machines to pool together their mining power and share rewards. Mining pools are essential in PoW mining on two levels: first, because earnings for individual miners are highly volatile, and second, because setting up the software infrastructure around mining is increasingly complex. By pooling resources, individual miners can lower variance and have a more predictable business.

But with this power comes great responsibility, and mining pools hold a lot of power. This is because mining pools ultimately decide which blocks get worked on by their miners and which transactions are included in those blocks. Mining pools decide on what MEV gets extracted and who gets to extract it, they vote on the gas limit, and they take part in major political battles. That’s why it’s essential to Ethereum culture that the barrier to entry for mining pools be as low as possible, to maximize decentralization.

When MiningDAO set out to build our own independent pool, we were surprised to find that it was incredibly challenging! There’s very little open and publicly shared info on how to run a competitive mining pool, and a lot of the open-source software is out of date. So we figured: let’s fix that by releasing an open-source, step-by-step guide.

Building a pool consists of two parts: (1) setting up a full node client with good peer-to-peer networking and fast processing speed, and (2) connecting the full node to pool software that manages hashrate and distributes workload across all the miners. Here, we’ll cover both.

This guide comes from our first-hand experience building the MiningDAO.io pool, and outlines how we brought our uncle rates from 10%-14% down to approximately 4%-5%, on par or better than some top-10 pools.

1) Set up Ethereum full node client

Running a mining pool requires running an Ethereum full node client. This client will be responsible for receiving new blocks and pending transactions, as well as producing its own blocks and broadcasting them to other nodes. This section covers how to properly set up one’s full node client.

1.1) Server hardware requirements

Running a fully synced node requires fairly good hardware. We recommend at least 32GB RAM, and at least 2TB SSD storage (syncing the chain with HDD will take forever).

Bandwidth is important as well. It is preferable to co-locate as close as possible to other nodes, to receive new blocks as soon as possible. We recommend cloud-hosted dedicated machines on services popular with other pools: OVH and Hetzner in Europe, Alibaba and AWS in Asia.

1.2) Geth or OpenEthereum? Geth!

The next decision to make is which Ethereum client to use. The most popular and well-tested choices are Geth and OpenEthereum (née Parity). Geth leads on protocol development and is always up-to-date.

For comparison, we did some small-scale experimentation with Parity-2.7.2 (latest stable branch before the OpenEthereum refactoring) and OpenEthereum, but both had poor results with block import times and block production times, leading to unacceptably high uncle rates. We welcome anyone to perform a more thorough A/B test and reach out to us with more data, but at the current stage we simply recommend Geth.

Here are the flags we use:

geth --datadir=/ssd/gethdata --syncmode=fast --cache=21000
 --maxpeers=250 --txpool.globalslots=1000
 --http --http.api=eth
 --miner.etherbase='0xADDRESS' --mine --miner.threads=0
 --miner.extradata='MiningDAO'
 --miner.notify='http://127.0.0.1:8107' &>> ~/geth-log.txt

Here --cache=21000 means to allocate 21GB for in-RAM state storage (the most Geth can handle), and the remaining flags will be explained below.

More importantly, the modifications to the Geth code we will describe below can be found here as a repo to download, or here as a patch to apply.

1.3) Minimize frequency of empty blocks

Two things destroy value for miners: mining uncle blocks, and mining empty blocks. In fact, the two are almost equally bad: uncle block rewards are 1.75 ETH, and empty block rewards are 2 ETH, with no transaction fee surplus in both cases. For comparison, full blocks with transaction fees typically bring 3-4 ETH in total rewards, and sometimes a lot more. So why do mining pools sometimes produce empty blocks, and how can one minimize their frequency?

When another pool mines a fresh block (say at height N), any other blocks at height N are likely to become uncles. So whenever a new block is found, Geth instantly switches the miners' jobs to mine an empty block at height N+1. This empty block does not have transaction fees, but that is still better than mining blocks destined to become uncles. Subsequently, geth constructs a "real" block at height N+1, and switches the miners' jobs once again. Constructing such a "real" block takes time (0.1-0.3 seconds), hence the two-step process. But in that interim 0.1-0.3 seconds-long period miners are working on an empty block.

It might be tempting to collect all the pending transactions to maximize fees, but getting greedy with --txpool.globalslots substantially increases the amount of processing Geth has to do to construct a "real" block (up to 1 second and more). We recommend values no larger than 1000 or 2000.

For more details on this, check out https://github.com/ethereum/go-ethereum/issues/21899

1.4) Minimize frequency of uncle blocks

With empty blocks out of the way, we can get started on the hard part. To minimize uncle rates, two things are key:

1. when other pools produce new blocks, learn about it as quickly as possible
2. when your pool produces a new block, propagate it as widely as possible (so others start mining on top of it)

The first step to good p2p is, as explained earlier, running your full node on a cloud server with good bandwidth next to other nodes.

Second, good bandwidth allows the node to handle more direct peers, thus reducing the number of p2p hops necessary to receive new information. The Geth flag for the number of peers is --maxpeers.

Below we will explain a few more nuanced and powerful tricks to maximize block import speed and block propagation speed.

1.4.1) Use bloXroute

bloXroute is a service dedicated to improving connectivity between miners and lowering their uncle rates. Most pools are connected to bloXroute, and even major established pools report significant improvements from using bloXroute. Measurements performed by KeeperDAO further confirm the massive advantage bloXroute holds over comparable services.

Our experiments showed significant improvements as well. On a freshly-synced node with default peer settings, approximately 90% of all new blocks come from bloXroute first (and only 10% from all other peers). Even after our node has been fine-tuned to connect to top peers, still 40%-60% of new blocks come from bloXroute first.

After following the bloXroute setup tutorial, don’t forget to add the bloXroute node into the “trusted peers” set for your Geth, we will need that later. Trusted peers are pre-set nodes that Geth will always connect to, irrespective of the random peer initialization process. Trusted peers also do not count against the connections limit. Adding the bloXroute gateway to trusted peers ensures Geth will not accidentally drop that connection.

We further recommend connecting to Taichi Network. Taichi is a block propagation network developed by Sparkpool. Connecting to it can be done by adding the Taichi endpoints to the same trusted peers file.

1.4.2) Propagate your blocks aggressively

Whenever Geth successfully mines a new block, it sends the block to propagate across the network. By default, Geth only propagates it to a random subset of size sqrt(n_peers), who then propagate the block to some of their peers, and so on. This mechanism is not ideal even if all peers were equally useful, but it is especially detrimental when some peers are more powerful than others and these peers end up not being included in the subset.

In particular, the first thing to do upon mining a new block is to send it to bloXroute, so that it will be forwarded to all the other participating mining pools. If the bloXroute gateway doesn’t end up in that random sqrt(n_peers) subset, your chances of getting uncled go way up!

Next, you’d want to send the block to the highest-quality peers, and then to all the remaining peers.

We have open-sourced the following Geth patch and recommend applying it to your client. It propagates all newly mined blocks to all trusted nodes (including bloXroute), and then to all remaining peers.

1.4.3) Cultivate the most well-connected peers

Vanilla Geth is designed for maximum decentralization and a flat network structure. This choice works well for hobbyists and supports a robust ecosystem of thousands of nodes. However, as we’ve already seen in the previous section, these defaults do not work as well for nodes that perform critical responsibilities and have significant dollar costs associated with failures.

In reality, not all peers are equally useful. Some have slow connections and will neither supply new blocks nor help your blocks propagate. Others, especially the nodes of other mining pools, will produce a constant stream of new block data. Following advice from Sparkpool, we tweaked our Geth to log which peers were the first to send us a new block. Collecting that data for several months allowed us to figure out the best peers to always keep connected to (via the “static”/”trusted” node settings in Geth). Here is a Python script we used to process that data and convert it into a trusted_nodes.json list that Geth can ingest.

Because MiningDAO is present in each geozone (North America, Europe, Asia), we have data-mined the lists of top peers for each geozone. Unfortunately, we cannot share the node IPs publicly to avoid the risk that these nodes will be DDoSed. Can probably share privately on serious inquiries with good justification. Also happy to share our own nodes in each geo for other pools to connect!

2) Set up mining pool software

Once the full node is set up properly, the next step is to set up the mining pool software itself. This software will be responsible for handling connections from all the individual miner rigs, keeping track of worker shares, and managing payouts.

2.1) Pick pool software

We briefly analyzed the following 4 options: miningcore, open-ethereum-pool, NOMP (node open mining portal), and MPOS (mining portal open source). We later learned about Flexpool Solo but did not experiment with it.

We had a great experience with Miningcore for two reasons. First, it keeps all past data on disk in an SQL database, unlike open-ethereum-pool, which keeps data only in RAM via Redis. Disk storage offers stability against reboots and ability to analyze historical data. Second, we enjoyed the highly readable, object-oriented code of Miningcore.

Ultimately, MiningDAO ended up implementing our own pool software, written in Go for speed and modeled after Miningcore. We expect to open-source our implementation soon, but in the meantime we recommend using Miningcore.

2.2) Fix pool software latency

A major hiccup we encountered while using Miningcore is the way it processes work updates. By default, Miningcore pings the full node RPC every 0.5 seconds to see if the latest job has changed. This setup may work for other cryptocurrencies with longer block times (it would be a negligible problem for Bitcoin with 10-minute blocks), but for Ethereum such a setup leads to unacceptably high uncle rates.

For context, there is an easy way to calculate the increase in uncle rates from any processing delay. Block times are Poisson-distributed, which means that no matter how long it has been since the last mined block, the probability of finding the next block in the next second (or millisecond or whatever) is always the same. For example, Ethereum targets 13-second block times, which means the probability of a block being mined in the next second is always 1 sec / 13 sec ~= 7.7%. So if your mining pool has a 0.1 sec delay anywhere in the pipeline for any reason, it will have 0.1 sec / 13 sec ~= 0.77% extra uncle blocks as a result of that delay. The uncle blocks will come from that 0.1 sec period of time that your miners are working on an outdated job.

Back to Miningcore. Using the formula above, a 0.5 second delay in updating the miners’ job will lead to 0.5 sec / 13 sec ~= 4% extra uncle blocks (absolute, not relative percentage). Naturally, such a high rate of unforced errors is unacceptable. We have experimented extensively with lowering the update frequency from 0.5 seconds down to 50 milliseconds and below, but found that setting rather unreliable: the updates were still significantly delayed.

A better solution is to make use of Geth’s notifyWork feature, so that Geth proactively sends job updates to the mining pool software as soon as they appear. We patched Miningcore to support this option, and released the modification. After transitioning to notifyWork, we found the communication delays between Geth and Miningcore to become practically negligible, and thus our uncle rates significantly improved.

Conclusion

Hopefully this post proves useful and leads to more people running Ethereum mining pools or solo-mining, helping keep Ethereum open and decentralized. To summarize, we started with vanilla Geth on default settings and vanilla Miningcore implementation. This default setup produced uncles at a rate of approximately 10%-14%. Progressively applying the modifications outlined here brought our uncle rate down to 4%-5%, comparable or better than some existing top-10 pools (the Etherscan uncle rate is a bit higher because we sometimes experiment in prod).

Our Geth modifications can be found here as a repo and here as a patch. Our Miningcore modifications can be found here, and a corresponding pool config file can be found here.

If you have further ideas on how to improve this setup, please send us a pull request or an email!

Miningcore speedup developed by Alexander Melnikov. Many thanks for the suggestions and ideas that came from Alex Obadia (Flashbots), Eyal Markovich and Shen Chen (bloXroute), Xin Xu and Dr. Yang Ze (Sparkpool), Chris (Flexpool) and Haseeb Qureshi (Dragonfly Capital).

How to build an Ethereum mining pool was originally published in Dragonfly Research on Medium, where people are continuing the conversation by highlighting and responding to this story.

The protocol allows anyone to propose entire blocks to be mined, with a reward attached for mining them. Mining pools can select the blocks with highest rewards, all without ever knowing the actual contents of the proposed blocks. We expect MiningDAO to be the most profitable choice for mining pools due to enhanced opportunities for earning MEV. Compared to existing systems, MiningDAO minimizes reliance on centralized servers and assumptions of honest miner behavior.

To showcase MiningDAO, we have built a mining pool that follows the protocol at https://miningdao.io/

Why MiningDAO?

Algorithmic arbitrage on decentralized finance protocols, better known as “miner-extractable value” (MEV) is a large and growing industry. It started small, at approximately $1M/year in 2017 (on Bancor and EtherDelta), and grew to be ~$500M/year by current estimates.

There are multiple actors in the space that play different roles in MEV. First, there are independent arbitrageurs and algo-trading funds, who aim to maximize their profits within the fixed rules of the game. Second, there are mining pools: they are responsible for forming all Ethereum blocks, and thus make the ultimate decisions on how transactions are included. Until recently, mining pools simply followed the default rules implemented in geth and Parity. Third, there are companies trying to change the way the game is played: BloXroute offers private transactions that avoid public-by-default mempools, Flashbots offers more profitable transaction sets for mining pools, KeeperDAO offers a decentralized “hiding game”. Our approach is largely inspired by SmartPool.

By default, we’d expect most of the MEV to go to miners via arbitrageurs competitively bidding away all the profit. Nevertheless, several mining pools kickstarted attempts to speed up this transition, and capture even greater MEV profits. Ethermine (tied for largest Ethereum pool with Sparkpool) runs its own in-house frontrunning bots. ArcherDAO partnered with several smaller pools (2miners, Ezil, etc) to build and run in-house bots for them, splitting the profits 50/50. These MEV battles, while fascinating to watch, lead to wasted resources and potential consensus instability: consider Salmonella, a clever attack that stole ~120 ETH from Ethermine’s in-house front-running bot, or Ethermine’s revenge, that selectively used transactions from an orphaned block to exploit a Flashbots user. These problems demand a generalized and robust solution.

MiningDAO is building a decentralized and transparent protocol for block formation, where the same rules apply to everyone. We propose an open-source, modular infrastructure, with no secretive lock-in components, where all miners and all block producers participate on equal footing. Anyone with an Ethereum address can propose the next block to be mined (via a block sealhash), and attach a bounty for successfully mining it. The mining pools would then mine on the highest-bounty proposal, brute-forcing nonces while remaining oblivious to the actual contents of the block. Other than putting everyone on equal footing in relation to MEV, this design greatly increases Ethereum’s censorship resistance.

How MiningDAO works

To simplify, the protocol works like a DEX orderbook for blocks, so if you’re familiar with 0x, it should be easy to visualize what’s going on.

1. Block producers deposit ETH into a “DEX” smart contract, locked with some time delay.
2. To propose a block, a block producer signs an off-chain commitment, containing two variables: (a) block sealhash, and (b) proposed block reward.
3. At any given moment, miners look for the highest-reward block proposal at the current block height, and bruteforce to find a nonce that would seal that block.
4. Whenever such a nonce is found, it is immediately forwarded to the block producer, as well as posted to the smart contract, with the corresponding block reward taken out of the block producer’s deposit.

Initially, our protocol relies on a centralized coordinator to ensure all nonces are correct and timely (to avoid mining long-stale shares).

Why go beyond Flashbots

Our functionality has substantial similarities to Flashbots, as well as substantial differences, so it’s worth explaining the benefits of our approach.

First, we want to minimize centralization risk in Ethereum block formation. The status quo used to be that top 5 pools produced ~65% of all Ethereum blocks. That’s already risky, but at least no single pool ever got close to 50% hashrate that would let it dominate the network. As it stands, Flashbots routes ~80% of the hashrate, substantially increasing the risk that it can dictate the future Ethereum trajectory (e.g. by routing transactions via a proprietary proxy). We believe alternative coordination mechanisms, focusing on minimalism and decentralization, would benefit the robustness and censorship-resistance of Ethereum.

Second, we believe our technology is a substantially better design. MiningDAO is the first protocol to offer truly private transactions that do not rely on an honesty assumptions by miners. The protocol can work end-to-end without revealing any of the transactions to miners, up until the point a block is mined. Further, our block auction is open-bid, which is substantially more convenient for arbitrageurs: they know in advance whether they’re winning and can always bid up, instead of having to build complicated pricing models based on past history.

Ultimately, we see a parallel here with the evolution of cryptocurrency trading, as it progressed from centralized exchanges running on AWS, to decentralized exchanges running on smart contracts. Our mission is to facilitate a similar evolution for block production on Ethereum.

Roadmap towards full decentralization

The V1 that we launched today is completely permissionless and private for block producers, but not trustless (it requires a centralized coordinator to ensure block producers cannot be exploited by miners). For pools that would like to join, the system is currently whitelist-based to protect against double-claiming work. Below is our plan to further decentralization.

The trustless V2 design will incorporate the following primitives:

1. SmartPool-style share validation to verify submitted nonces (We intend to log all shares on L2 with merkle root logged in L1)
2. Gas savings: validating PoW on every block is prohibitively costly (~3M gas), but an optimistic challenge-response validation can obviate this cost (i.e. only run the ~3M gas verification in case of disputes)
3. ChainLink “first seen” oracle for pending transactions to punish timing-based griefing attacks

How to participate

We are building an open, permissionless system, where anyone can participate on equal footing.

As such, we are providing the following endpoints for participants:

• For miners: we have launched our own MiningDAO pool, which automatically mines the best available block template. The pool takes zero fees. MiningDAO runs multiple failover block producers (both vanilla and MEV-extracting), so that mining in the MiningDAO pool is guaranteed to yield more ETH than any non-MEV-extracting pool. Here are the instructions for how to point your rigs to MiningDAO pool.
• For mining pools: we welcome all mining pools to supplement their income via MiningDAO block templates. If you have any questions about how to integrate MiningDAO with your hashrate, please send us an email at contact@miningdao.io.
• For arbitrageurs: we have two endpoints available. For easy onboarding, we welcome Flashbots-style transaction bundles. For more sophisticated or composite strategies, we welcome block producers to bid on block templates directly.

Learn more

• Follow us on Twitter for key updates
• Join on Discord and Telegram for discussions
• Explore & contribute on Github

Introducing MiningDAO was originally published in Mining DAO on Medium, where people are continuing the conversation by highlighting and responding to this story.

The problem is inherent to Mimblewimble, and I don’t believe there’s a way to fix it. This means Mimblewimble should no longer be considered a viable alternative to Zcash or Monero when it comes to privacy.

In the last two years, Mimblewimble has grown in popularity as an up-and-coming, lightweight privacy protocol. Mimblewimble was invented in 2016 by a pseudonymous hacker known as Tom Elvis Jedusor, who dropped a text description of the protocol into an IRC chat and then disappeared. Since then, Mimblewimble was most famously implemented in the “fair launched” privacy coin Grin, the VC-backed projects Tari and BEAM, and is even being considered for integration into Litecoin.

Several researchers have hypothesized a possible privacy weakness in Mimblewimble [1][2][3][4]. My contribution is to demonstrate the precise way to perform an attack, prove its viability on a live network, and measure its efficacy. In live testing on Grin, I was able to unmask the flow of transactions with a 96% success rate. Therefore, it’s now clear that Mimblewimble should not be relied upon for robust privacy.

Here is a more technical deep-dive into this attack, complete with open-source code to reproduce it, data collected, and a technical FAQ. What follows in this article will be a high-level, intuitive explanation of linkability, how the attack works, and what it means for privacy tech.

What is linkability?

It’s important to understand what this attack means and what it doesn’t mean.

This attack does not let us determine the amounts that people are getting paid. Mimblewimble successfully obfuscates payment amounts using vanilla elliptic curve cryptography (Pedersen commitments). What this attack does let us do is determine who paid who. In other words, it lets us link transactions together and determine the flow of payments.

It might not be obvious why that’s a big deal.

Say Coinbase knows that a certain address belongs to a Venezuelan individual named Daniel. You, an American user, try to cash out on Coinbase. But after deobfuscating the transaction graph, Coinbase determines that you received money from Daniel, though they don’t know how much you received. Due to OFAC restrictions on Venezuela, Coinbase shuts down your account. (Naturally, exchanges will know a lot about the transaction graph because they have KYC information on users who cash out into fiat.)

Or say an authoritarian government knows that a certain address belongs to a political dissident. You send that dissident a small donation. Later, when you send your Mimblewimble transfer to the local exchange, that exchange shares your transaction data with the government. Since the government can see the entire transaction graph, they now know that you supported a political dissident.

These attacks would not be possible in Zcash. This is because Zcash is unlinkable — or in other words, every Zcash shielded transaction has a large anonymity set. The anonymity set is essentially the set of transactions that your transaction is indistinguishable from. Think of it like blending into the crowd: the larger the anonymity set, the larger “crowd” your transaction is mixed into.

In Zcash, each shielded transaction’s anonymity set includes all shielded coins. This is the maximum possible anonymity from an information theoretic perspective.

In Monero, each transaction’s anonymity set is the set of all (believable) decoy transactions. While the Monero client lets you specify the size of the decoy set, the current default is 11. (Monero has its own issues around securely sampling decoys, but I believe it mostly works, give or take.)

It was originally believed that Mimblewimble’s anonymity set includes all the transactions in the same block and looks like this:

But in reality, it looks like this:

This whittles down Mimblewimble’s anonymity set to a single address.

To be clear, this is not an indictment of Grin. I have great respect for the Grin community and core developers, who have all been tremendously helpful in answering my questions.

Grin still affords a stronger privacy model than Bitcoin or other non-privacy coins, since amounts are safely encrypted. But Mimblewimble provides a strictly weaker privacy model than Zcash or Monero. This makes it insufficient for many real-world privacy use cases.

A high-level overview of the attack

So how exactly can you trace the transaction graph in Mimblewimble?

I noted that despite encrypting the payment amounts, Mimblewimble still leaves a linkable transaction graph. But the protocol designers were aware of this, so Mimblewimble uses two chief techniques to combat linkability: the first is full-block cut-through aggregation, and the second is Dandelion.

The idea behind block cut-through is that as transactions are accumulated within a block, those transactions get aggregated into one “super-transaction”. That “super-transaction” basically looks like one giant CoinJoin — essentially, all of the inputs and outputs are tossed into one giant bucket, with no easy way to determine who paid who within that bucket. There’s just a bunch of inputs going to a bunch of outputs, with amounts obscured.

Sounds solid, right? Only one problem: this CoinJoin has to be built up one transaction at a time. Because transactions are continually being created and broadcasted from separate places, if you run a sniffer node that picks up all transactions before cut-through aggregation is finished, it’s trivial to unwind the CoinJoin. Any sniffer node can just observe the network and take note of the original transactions before they get aggregated. It’s pretty straightforward if you just archive all messages you observe in the P2P network.

Wait — really? Is that all there is to it?

Well, there’s another line of defense that the Grin team put up: Dandelion protocol. Dandelion is a networking technique developed by researchers at CMU that attempts to obfuscate the originator of a transaction.

Normally, in a cryptocurrency like Bitcoin, the originator of a transaction just shouts their transaction to all of their peers, and it quickly cascades through the P2P network. But in Dandelion protocol, every transaction broadcast starts with a secret game of telephone. The originator will whisper their transaction to just one peer, who whispers it to one other peer, and so on in a chain. After a random number of hops, the final peer will shout out the transaction just as in Bitcoin. But this peer is so far removed from the originator, for any observer it’s impossible to tell who the chain started with.

This works great for obfuscating the IP of the transactor. But Dandelion has a second function within Grin: it just so happens to defeat sniffer archive nodes. Because every transaction starts off in a Dandelion chain, whenever two transactions cross in their Dandelion chains, they’ll get aggregated early. If this happens, then by the time the transaction is broadcast for everyone to observe, a sniffer node cannot disaggregate them. They’ve already been CoinJoined.

Against a sniffer node, this is Grin’s principal defense against linkability. But even this defense can be defeated.

By default, each Grin node connects to 8 other peers. But by jacking up the number of peers, I can connect my sniffer node to every other node in the network. Assuming I stay alive long enough, eventually almost every node will connect to me, making me a supernode.

Once I am a supernode, there’s a high probability that I will be on the Dandelion path for any transaction. Therefore, the only way that I cannot catch a transaction before it is aggregated is if two transactions both intersect in their Dandelion path before I see either of them. If I see either transaction before they’re aggregated, I can use simple set subtraction to disaggregate them.

In my attack, I was able to link 96% of all transactions while only connecting to 200 peers out of the total 3000 peers in Grin’s network. But if I wanted to spend a bit more money, I could easily connect to 3000 nodes to disaggregate almost all transactions. I also don’t have to do this as a single supernode; the same attack works by launching 3000 separate nodes with unique IPs, each only connected to one peer. As long as I’m sniffing all the transaction data and dumping it into a central master database, the attack works just the same.

So is Mimblewimble salvageable?

It depends. I believe that Grin, as currently envisaged, has no clear path to unlinkability. Simply ratcheting up the Dandelion factor can be easily combated by a motivated attacker, as I discuss in the technical writeup.

But linkability aside, Mimblewimble still has unique and valuable properties! It allows cut-through aggregation, which is an effective compaction technique for full nodes, and efficiently hides transaction amounts. If you want strong privacy, you can always combine Mimblewimble with another protocol that obscures the transaction graph, such as in Ethereum 9¾ (which combines Mimblewimble with a Zerocash-style commitment-nullifier scheme).

But it’s clear that Mimblewimble on its own is not strong enough to confer robust privacy.

Bitcoin is now 11 years old, but cryptocurrencies are still in their infancy. It wasn’t that long ago that devastating bugs were uncovered in both Zcash and Monero. This is to be expected — most of the interesting technologies are still in the realm of basic science.

But this is how science always advances: we propose new theories, and then repeatedly knock them down, until what’s left standing has stood the test of time.

Thanks to Haseeb Qureshi for major help in putting together this write-up and for the anonymity set illustrations. Additional thanks to Oleg Ostroumov, Elena Nadolinksi, Mohamed Fouda, Lucas Ryan and Nader Al-Naji for reviewing drafts of this post. And a huge thanks to Jake Stutzman (NEAR Protocol) for the Dandelion and block aggregation illustrations.

For more of my writing, follow me on Twitter at https://twitter.com/IvanBogatyy.

Disclosure: Dragonfly Capital abides by a no-trade policy for the assets listed in this report for 3 days from the date of publication.

Breaking Mimblewimble’s Privacy Model was originally published in Dragonfly Research on Medium, where people are continuing the conversation by highlighting and responding to this story.

By now everyone has heard of CryptoKitties, a cute game that went viral to the point of overloading the whole Ethereum network. This is the inside story behind how we made $107K investing in CryptoKitties and briefly set the record for the largest sale ever (currently second-largest). Later, we made ~$8K running an automated arbitrage bot. While playing the speculation game at the height of the mania was exciting, the bot was fairly technically involved and will be interesting to people who want to learn blockchain engineering in general.

As I was toying with my Ethereum smart contracts on a regular Saturday night (December 2), something was clearly off: transactions were stuck and took much longer than usual to confirm. A quick investigation showed there were 10–20 times more pending transactions than usual, and many of them were going to the same mysterious address, 0x06012..66d. This is how I first learned of the now-famous CryptoKitties game.

Investment thesis

Next morning I called Oleg Ostroumov, a close buddy who runs a top crypto hedge fund in Russia to discuss this unusual phenomenon. While the CryptoKitties game was merely a curiosity for me, Oleg immediately spotted a lucrative investment opportunity. First, we were clearly among the earlier entrants. Second, the game looked like it had all the precursors of going viral, similar to Pokemon Go, and was in the early stage of a hockey stick explosion.

Now the question was, which Kitties should we buy? The marketplace offered 4 ways to sort Kitties: cheapest first, most expensive first, newest first, oldest first (note: the website had been redesigned since). The first 3 options are clearly transient: you can always put a cheaper, more expensive, or a newer Kitty on the market. Oldest, however, is like diamonds: forever. Thus we decided to buy single-digit Founder Cats, despite their already hefty price tags: somebody just snatched them at 25 ETH and re-listed the lineup at 50 ETH ($25K), with Founder Cat #1 trading even higher at 150 ETH.

Oleg bought Founder Cats #4, #6, #8 at 50 ETH each ($25K at the time), and I put up $5K for a 20% equity stake in Founder Cat #4. We were pretty optimistic about the mania and re-listed the Kitties at 250 ETH, 5x the purchase price.

As a side-note, we wasted about an hour because of posting a low gasPrice, and had to reinstall MetaMask (at the time it did not have the “resend transaction with increased gasPrice” feature — it was added right after). Lesson learned: if you’re making a $25K purchase, do not try to save $20 on fees.

Right after hanging up on the call I considered the chances that this was going to be my personal record yet for the most ridiculous way to waste $5K. Probably pretty high.

The subtle art of exiting the top

Over the next days (December 2–5), the game entered the viral explosion phase. Given the addictiveness, it was pretty hard to focus on anything else. Clearly, the craze was not going to last: the game mechanics were way simpler than say Pokemon Go, with just two functions available, either buy or breed them to produce new ones. We assumed it could last for a couple weeks tops, so it was important to formulate a rational exit strategy in advance and stick to it.

We used three metrics. First, Google Trends for “CryptoKitties” showed how frequently people Googled information about the game, which is a pretty good leading indicator of interest. Second, GasGuzzlers shows the top gas-consuming smart contracts on the network and the share of gas they consume (for CryptoKitties it was ~4% when we entered and ~20% at peak popularity). Third, we watched the market conditions and the prices of recent sales on a very nicely built app, https://kittysales.herokuapp.com/.

On December 4–5, several less prestigious Kitties (#35, #78, #87) sold for almost 200 ETH, so we were confident our listing price was well-adjusted. However, December 5–6 was the top on every metric, and by the end of December 6 the popularity was clearly trending downhill. Observing this, we had an emergency call on December 6 trying to figure out what to do with the Kitties, none of which sold at our listed 250 ETH price yet. I wanted to re-list the Kitty at a lower price of 100 ETH, whereas Oleg was steadfast on sticking to 250 ETH. Given our disagreement about the value of the Kitty, we did the obvious arbitrage: I sold my 20% stake to him at a 100 ETH “Kitty valuation”, essentially doubling my money on the spot. Little did I know I’d only be happy about this deal for about 20 minutes…

I kept watching the marketplace out of curiosity. Now, 20 minutes after Oleg bought me out, I saw our Founder Cat #4 sold at the original 250 ETH price. I was so confident it was Oleg generating game activity for his merchandise, I barely bothered to ask him. When I did, however, I learned the full power of HODLing: it was a genuine purchase, we had no idea who was the buyer, and he’d just set the world record for selling a collectible on the blockchain.

Automatic arbitrage

Having had such a lucky start to the game, we discussed what else we could do with it. Peak hype was clearly over and ridiculous prices for the rarest Kitties were unattainable, but the long tail remained: people still wanted to play, and to get some kind of Kitty for themselves.

So we decided to try running an automatic arbitrage bot on the long tail of cheaper Kitties. Like any reseller strategy, it essentially consisted of the following three steps:

• Determine clusters of Kitties that should be priced roughly the same. The parameters we looked at were Rarity, Cooldown and Generation (only Gen 0 and Gen 1 were expensive enough to bother). Kitties with same values across these 3 parameters were deemed equivalent (ignoring cuter artwork or Cattribute variations).
• Buy underpriced items within each class. In our case, we only bought the cheapest Kitty across every parameter set.
• Resell the acquired item for more. In our case, we wanted to avoid a race to the bottom with other potential resellers, so we listed each Kitty at slightly below the second-cheapest price (similar to Google’s second-price auction).

After exploring several blockchain libraries, we found refactoring my previous Bancor front-running code to work best. The strategy was: watch for transactions that put Kitties on sale, buy out immediately if it matches the criteria. This instant reaction mechanism is pretty similar to front-running except it’s more like post-running: one cannot put the “buy” transaction before the “list on the auction” transaction, so it needs to happen immediately after. On a more technical level, there are several ways to implement this:

• Whenever you see a pending “listing” transaction, try to buy out the Kitty within the same block. You could either send the “buy” transaction with a slightly lower price (e.g. if the “list” transaction pays 25 gwei for gas, you can pay 24.999 gwei), or, alternatively, pay precisely the same. In the former case, you risk being just below the block cutoff threshold, and then your transaction would stay “pending” for hours if the activity spikes. In the latter case, you risk miners putting your “buy” transaction in front of the “list” transaction (since gasPrice is the same, the order can be arbitrary and depends on miner software), which would result in our transaction failing.
• Wait until the “listing” transaction is confirmed, and send out the “buy” transaction right after with a very high gasPrice, ensuring confirmation within the next block.

You wouldn’t expect this cute game to be a particularly dense arena, but we had competitors who employed each of those options trying to snatch the Kitties! And since each of those options had its own benefits, we had to implement all of the strategies to have the highest chance (this can be done either the naive way, by running the 3 versions from 3 accounts, or by re-using the nonce within the same account, but further technical details are beyond this story).

Another possibility we considered was the following: assume you want to buy a Kitty, and its price is slowly going down as determined by the linear time auction. Instead of buying it right away, one can wait until another contender appears and front-run their purchase, achieving lowest possible price. As much fun as it would have been to be such evil masterminds, the market was crashing too fast to warrant spending that much attention on any single big-ticket Kitty.

Aftermath

More than a month after the peak hype subsided, we received a pleasant surprise: somebody bought out Kitty #6 as well. 29 ETH was below the initial purchase price, but decent money nevertheless (plus Ethereum doubled in price at the time).

The remaining Kitty #8 gets Oleg’s undivided care and adoration. The Kitty has its own laptop stickers, t-shirts and even a custom-made silver pin. I hear it is about to get its own Instagram channel too.

In the end, it was a very hands-on lesson on speculative manias. I undershot and Oleg overshot, but both of us almost precisely doubled our money, which was a pretty amazing outcome. Yes, there is absolutely no guaranteed intrinsic value or price floor for imaginary Kitties. Nevertheless, one can still play this game and even win. And of course, it was practically impossible to predict where the top of the bubble would be, although we came pretty close.

As for my ~$5000 profit from this adventure, I decided to donate it to MCCME’s Summer Informatics School, an exceptional coding bootcamp in Russia where I studied as a high-schooler, as well as the Freedom of the Press Foundation. Here I doubled my money again thanks to Google’s generous Gift Match Program.

Thanks to Oleg Ostroumov, Alina Aseeva, Aashna Mehra, Haseeb Qureshi and David Smooke for reading drafts of the story. For more, follow me on Twitter at https://twitter.com/IvanBogatyy

This post is a deep-dive into programmatically trading on the Ethereum / Bancor exchange and exploiting a game-theoretic security flaw in Bancor, a high-profile smart contract on the Ethereum blockchain. The full code can be found at https://github.com/bogatyy/bancor. We collaborated with the Bancor team to make sure the current exploit is protected against, although for a little while there would still be a chance to make some beer money for educational purposes.

Imagine trying to hack Bank of America — except you can read all of their code in advance, all of their transactions are public, and if you steal the money it’s irreversible. Sounds like a paranoid worst-case scenario? Well, this is exactly the setup Ethereum smart contract developers have to deal with every day. Bitcoin and the blockchain technology unlocked tremendous possibilities in international payments, and the Ethereum further magnified it by allowing to manage these payments through programs called smart contracts. However, smart contracts also give hackers a much easier setup for attacks.

Front-running is one such attack. The term originated in the stock market, back in the days when trades were executed on paper, carried by hand between the trading desks. A broker would receive an order from a client to buy a certain stock, but then place a buy order for themselves in front. That way the broker benefits from the price increase at the expense of their client. Naturally, the practice is unfair and was outlawed.

On the blockchain, the problem becomes a lot more severe. First, all the transactions are broadcast publicly. More importantly, blockchain participants across the world are not bound by the same relationship as a broker and their client, so attackers can exploit their knowledge of a pending transaction with impunity.

Several months ago, researchers at Cornell uncovered that Bancor, an ICO that spectacularly raised over $150M in funding over a few minutes, was vulnerable to front-running. They pointed out that miners would be able to front-run any transactions on Bancor, since miners are free to re-order transactions within a block they’ve mined. While the Bancor team gave a thoughtful response, up until very recently, there has not been any progress on fixing the issue (more on that later).

Our research goes a step further. In fact, we show that it is both possible and practical to front-run Bancor as a non-miner. Which means you don’t need to the lucky miner who happens to mine the block with a Bancor trade to profit from front-running. You simply need to be a regular user monitoring the blockchain to perform this attack.

Surprisingly, the vulnerability does not seem to have been exploited so far (front-running is readily identifiable on the blockchain), so in this post we’ll examine exactly how one implements such an attack. Turns out, all it takes is about 150 lines of Python to get a working front-running algorithm. We also ran simulations to determine how much money one could make from front-running consistently (spoiler: an attacker could have had a ~117% ROI on the money they invested into the attack over July and August, chipping away from other Bancor users). Finally, I executed the attack against a single trade, making~$150 net of all fees, after which I returned the money to the person I front-ran and stopped the program.

Now, I know that relinquishing a working trading strategy would be a cardinal sin to any trader, but as it turns out, I am more curious than greedy. Implementing and countering attacks is not only a fascinating game, but also the cornerstone of advancing cryptographic security. Most importantly, I believe in the long-term impact of the blockchain ecosystem, and for the blockchain economy to fully develop, vulnerabilities like this need to be understood and protected against.

So let’s dig in.

Background

Bancor is a protocol for trading and pricing Ethereum ERC-20 tokens, as well as the eponymous token, abbreviated as BNT, with the current market capitalization of approximately 180 million dollars. The core problem Bancor solves is as follows: normally, for a trade to happen, there has to be a buyer and a seller, having opposing desires (to buy and to sell) at the same moment in time. This limitation may be fine for large publicly traded stocks, but for long-tail crypto tokens this can create a serious inconvenience.

Bancor solves this problem by allowing anyone to trade against a public smart contract, which offers an automatically calculated token price following a precise formula. Essentially, Bancor is fulfilling the role of market-makers in traditional finance. The smart contract has an Ethereum reserve, and as more people buy the token, reserves grow and the price goes up. Consequently, when people sell, the contract adjusts the price to go down, so that the reserve is never depleted entirely. Unlike most other exchanges, where trades are managed off-chain, with Bancor every order is a self-contained Ethereum transaction (money + data).

Unfortunately, the current setup contains a flaw, allowing anyone to front-run large transactions and make guaranteed profit. Let us expand on what makes the attack possible. When somebody broadcasts a transaction on the Ethereum network, it becomes available to other nodes almost immediately as a pending transaction and is added to the common queue, but it is not confirmed until the block confirmation hash is found by some miner (thus confirming all the transactions in that block), which tends to happen once every ~20 seconds. Further, up until the block is confirmed, the order of the pending transactions is up for grabs, and miners basically sort transactions by how much they’re paid per gas (that is, per unit of computation they’ll have to perform).

This discrepancy creates an attack vector: any user running a full-node Ethereum client can spot a pending transaction and insert their own transaction in front of it by paying more per gas. If you see a large BUY is about to happen, you know the BNT price will increase (following their deterministic formula), so if you buy in before that transaction you get an instant appreciation of your tokens and a guaranteed return on your investment. Similarly, if somebody sent out a pending SELL, an attacker can sell their tokens in front.

Given we will be implementing our attack using Ethereum client API, now would be a good time to take a step back and give a general overview of how the Ethereum distributed applications (or DApps for short) landscape looks like. At a high level, implementing DApps is fairly similar to regular Web applications. The backend is a smart contract, running on the Ethereum blockchain, typically implemented in Solidity and then deployed to the network. Then there is client software, which interacts with the backend by sending transactions.

Just like in the regular world, smart contracts (the backends) receive most of the attention, with many high-profile smart contracts and several high-quality developer guides appearing recently (personally, what I’ve found most useful was fiddling with the examples from Solidity official docs, as well as great intro guides by Hudson and Karl). However, on the front-end side, I currently do not know of any non-trivial client-side applications (an example of such an application would be a decentralized poker client, where the majority of compute has to happen off-chain, on the players’ machines). Right now, the only way for users to interact with smart contracts is to send transactions manually, either by running their own full node (for example, by using the geth client), or by relying on third-party web services (like MyEtherWallet). Clearly, this would have to change: the current situation is about as convenient as manually sending POST requests through Telnet every time you wanted to browse the Web.

Easy Mode: high-frequency trading by hand

The simplest way to confirm the vulnerability is by hand. We will not need any tools except a Web browser and a wallet with some Ether.

First, separate your Ether between two wallets equally (it will not work from a single wallet). Second, go to MyEtherWallet and, following Bancor purchase instructions, set up two BUY transactions (do not click Send yet!). You should prepare two equivalent transactions from both wallets, but make sure gas price on the first wallet is lower than on the second.

Now, when it’s set up, click “Send” on the first wallet (with the lower gas price), and then “Send” on the second wallet, with the higher gas price. If you did everything right, the transaction that was submitted second would actually be processed first, and get more BNT per same deposit!

Transaction 1 (transaction from wallet 1, submitted first, fulfilled second)

BNT tokens received: 11.014424733254973428

Transaction 2 (transaction from wallet 2, fulfilled first, should get better price)

BNT tokens received: 11.014423186864343663

Notice the letdown: while the front-running transaction should’ve gotten a better price, it actually got a slightly worse one. Upon careful investigation, I realized this is because the rather arcane implementation of the Bancor formula can deviate quite significantly from the theoretical formula, especially for smaller amounts (specifically, error in 4th digit for 0.1 ETH or roughly $20 transactions). After querying the BancorFormula contract for a bit, I learned that even for 1 ETH the precision is not good enough, but if we’re willing to make 10 ETH-sized transactions or larger, we’d actually observe a better price as predicted. The Bancor team recently updated the formula and mentioned it is a lot more accurate. Ultimately, I decided to skip re-doing this experiment with larger transaction sizes or the new formula in favor of the actual front-run.

Hard Mode: trading automatically

Now, unless we are ready to sit in front of a computer all day and hit refresh on etherscan.io, the process needs to be automated. Luckily, most Ethereum clients provide a JSON RPC to interact with the blockchain and automate away the low-level details of interacting with the blockchain.

You just need to run a full node client and send API requests to localhost:8545.

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository -y ppa:ethereum/ethereum
$ sudo apt-get update
$ sudo apt-get install ethereum
$ geth --rpc

Here is an example curl request that looks up a transaction by hash (in this case, a huge BUY order on Bancor):

Now let’s send the same request using Python:

If you got the same output as from curl, congratulations! The hardest part of learning to programmatically interact with the blockchain is already over.

Now implementing the front-running trader becomes a matter of putting together a few common API requests (pseudocode below for brevity). If the goal is to be maximally efficient, it is better to avoid cashing out in between transactions: front-running can be done in both directions, and it would be the most profitable to only sell once we see a pending sell, and only buy once we see a pending buy:

In my case I only wanted to prove that the idea works (and yields non-negligible amounts of money), so I did the front-run once and sold immediately. Better yet, in this case the “loser” of the trade is easy to pin-point (the only person losing money is the owner of the trade being front-ran), so it was easy to return that money too. More details to be found in the full code on my GitHub.

The results of the front-run (transactions in reverse chronological order):

We made 0.477 ETH, or a ~0.5% return in less than a minute! I calculated the amounts (my own principal and the threshold for front-running) so that the profit would be at least $100, which I deemed convincing enough for the purposes of my post. Noteworthy, just a few days later there was a whopping 5856 ETH trade purchasing Bancor, which would have yielded an approximately 9% return, or about $3000 given the same 100 ETH principal!

The next part will explain where do these numbers come from and how to calculate the return from front-running a given trade.

Simulations & ROI evaluation

Let us describe the two core assumptions behind the Bancor pricing system, and the exact formula that is derived from those assumptions. First, Bancor maintains a constant ratio between the traded token market capitalization and total reserve token value. As of right now, the only traded token is BNT, and the only reserve token is ETH. Assume there is approximately 70K ETH in reserves (which is roughly the case) and the reserve ratio is 10%. This means the whole market cap of the system is implied to be 700K ETH, and the price per BNT is determined from this total market cap, divided by total BNT supply. If somebody buys BNT with ETH, the whole ETH amount is added to the reserves, thus pushing the price up (there was more value added to the reserves than tokens issued), increasing the value of everybody else’s tokens. Conversely, selling BNT reduces the reserve and thus the price per token (this way, the reserve never gets depleted, you just get less and less of it per token).

The second assumption is that a large trade should be equivalent to making a set of smaller trades of the same size (so to determine the final price, one would have to calculate an integral of Price d(Size)). Turns out these two assumptions are sufficient to uniquely define the behavior of the system in all cases. For those with a strong mathematical inclination, there is proof available. Here are the resulting formulas:

For our practical purposes though, the exact formulas are not really necessary (and as we have learned, the actual implementation is not very precise anyway). Approximating the pricing formula with the linear part of its Taylor’s series, we get:

NEW_PRICE ~= OLD_PRICE * (1 + DEPOSIT / RESERVE_BALANCE)

Interestingly, the RESERVE_RATIO is not a part of the approximation. I’ll skip the full derivations here, but basically, it cancels out in the numerator and the denominator, so the only thing that ends up mattering is RESERVE_BALANCE = TOTAL_MONEY * RESERVE_RATIO. This pricing formula is a good approximation as long as the deposit is much smaller than the total reserve. So if BNT has 70K ETH in reserves, and somebody invests 700 ETH, the price goes up by 700 / 70K = 1%. If somebody withdraws 700 ETH out of the system, the price drops 1%. Thus in both cases front-running those transactions would give us instant 1% ROI. Note that our theoretical approximation matched practice quite well: a 350 ETH trade yielded a 0.477% return, against approximately 0.5% predicted.

Based on these calculations, I wrote some code to evaluate how much money could have been made front-running Bancor in July and August. Assuming we leave small transactions alone and only go after big ones (> 100 ETH) so that gas prices don’t really matter, we get:

$ python simulation.py
...
ROI for front-running all transaction >= 100 ETH:
July 88.7%
August 28.6%
With a principal of 100 ETH, that would make you $35190

Not bad: an attacker can more than double the money they invested into the attack over a couple months, chipping away from other Bancor users. In practice, the profitability threshold is smaller than 100 ETH (the total fees one would need to beat come out to a few dollars), though it also depends on the principal invested by the attacker.

Front-running other Bancor-exchangeable tokens

While we did show that it was possible, BNT itself is relatively hard to front-run because of the large reserves: its price changes very little between transactions. But for any smaller token the 1 / RESERVE_BALANCE fraction would skyrocket an attacker’s profits and rob honest investors very fast. To prove that, I have deployed my own token following the Bancor Protocol (address, code) and made instant 2X profit front-running a large transaction.

Our FBT (short for Front-Runnable Bancor Token) was initialized to have a total supply of 1M wei (where wei is the smallest possible unit in Ethereum, equal to 10^-18 ETH ), and the total initial supply of 2M tokens. Given the 10% reserve rate ratio, the implied market capitalization of our token becomes 10M wei , and the price per token 10M / 2M = 5 wei . Now what happens when the front-runner makes a deposit before a large “honest” BUY, but sells after?

TX1, front-run:1000 wei gives 199 FBT , in line with the price we calculated

TX2: very large buy, increasing the price roughly 2X

TX3: attacker withdraws 199 FBT for 1910 wei, doubling their money

Since the intended use of Bancor is to serve the lesser known tokens (which may not have enough demand and liquidity for regular exchanges), these tokens would naturally have smaller reserves, making this vulnerability especially dangerous. In an extreme case where the reserve is very low and an attacker has a lot of money, they can extract more value from an honest investor’s deposit than the investor would get themselves!

Further, all of this is possible as a mere full node (that is, a person with a decent laptop). Miners, and especially miner pools, are in a privileged position and can do an order of magnitude more damage. Full-node attackers have to broadcast their transactions and thus risk their principal, whereas miners can mine blocks with their own front-run included, but never reveal it publicly unless they do mine the block. That way, they can profit at no risk or cost to themselves. Further, they can rearrange transactions within a block in whatever way they want, arbitrarily creating winners and losers out of other participants.

Ethics

Like any new technology, this situation raises very interesting ethical questions. Is the strategy we discussed “hacking”? Is it high-frequency trading? Is it simply an ability to make informed decisions based on public information faster than other investors? Ultimately, is this kind of trading “bad”?

Personally, I took an easy way out and made the decision that lets me sleep the most soundly: returning the money after proving the point and stopping the program. Nevertheless, I would not have blamed someone in a similar spot if they decided to do otherwise.

Solutions

Over the past month, Haseeb Qureshi and I discussed several solutions with the Bancor team, making sure the vulnerability is contained. Yesterday, the team released a fix that handles most of the risk in practice. Long-term, theoretically robust solutions are fairly complex, and while analyzing those can be a whole different post, I want to briefly mention the options available.

TL;DR don’t send large transactions and use Bancor Web3 UI that sets a minReturn for you.

One partial solution is to set a minReturn on trades, basically canceling your order if you realize someone squeezed in front of you. This does prevent attackers from making guaranteed money instantly, but raises the question: what is a buyer supposed to do next? The order just revealed their intention to buy, and presumably they still want their Bancor, so they’ll place another buy order sometime in the near future, which will eventually raise the Bancor price and, on average, will profit the attacker just the same.

Yesterday, the Bancor team released a Web3 interface that implements the minReturn solution. Longer-term, and assuming perfectly intelligent adversaries, this might lead to some curious Nash equilibria (e.g. front-runners might block trades unless they are allowed a small profit margin, though a lower one than if they were front-running entirely naive users), but right now, this should solve most of the practical risk.

The Bancor team also suggested setting a universal maxGasPrice, to make sure non-miner front-runners cannot bid higher. This would fully protect the users from non-miner attacks (at a cost of lower liquidity during network congestion), although the original front-runs by miners would not be affected. This fix will be out soon too.

More robust solutions would involve a version of the commit-reveal scheme, one of the go-to instruments in a cryptographer’s toolbox. Submarine sends by Cornell researchers proposes the most beautiful and general solution that I know of. However, given the specifics of the Bancor protocol, a much smaller solution could suffice: do a commit-reveal with a penalty for non-revealing in Bancor tokens themselves. Basically, whenever someone commits a hash of a trade and doesn’t reveal, a percentage of both their Bancor and ERC-20 Ethereum tokens is burned (note that it works irrespective of the direction of the trade, thus revealing no information). If designed with careful attention to detail (for example, making sure that reveals are only accepted for commits in a previous block), this can fully solve the front-running problem, including front-running by miners.

Of course, even the simplified scheme is so complicated that almost no one would want to perform it by hand in MyEtherWallet. As the ecosystem grows, more sophisticated client programs (like the auto-trader we just implemented, or the commit-reveal client) would have to take that role on behalf of the users. Again: people aren’t sending their own Telnet requests every time they want to browse the Web, so why should the crypto world be any different?

Acknowledgements

This project started in collaboration with Haseeb Qureshi and Preethi Kasireddy at the IC3 Ethereum bootcamp under guidance from Ari Juels, Iddo Bentov and Phil Daian. The post was rewritten considerably with massive help from Haseeb and Nader Al-Naji, who is about to take over the world with Basis.

A disclaimer just in case: this is a personal project and it does not represent my employer or anyone else’s opinions. Since I worked on it in my spare weekends, the timeline ended up being very protracted, and I hope the attentive readers will forgive some small discrepancies (the Ethereum price may oscillate across the post, and some Bancor contract addresses had to be updated).