Stories by Memo Akten on Medium

The Unreasonable Ecological Cost of #CryptoArt (Part 3)

Memo Akten — Mon, 25 Jan 2021 12:05:29 GMT

This is Part 3 of a N-part series, accompanying the website http://cryptoart.wtf

In Part 1, I present my motivations, findings, and my conclusions as a result of these findings.

In Part 2, I present the underlying data and methodology that gave rise to these findings, along with more statistics. It’s more aimed at those who wish to dig deeper into the numbers, and/or technically interrogate my claims.

In Part 3 (this article), I unpack some of the myths behind the NFT market.

NFTs are a scam

NFTs are a scam in that blockchain as a technology does not provide the benefits regarding digital rare collectibles that it is often sold as providing — in particular, as a technology it does not solve the issues plaguing ‘real world’ markets, which it is touted as solving:
- It does not provide scarcity or authenticity for assets linked to NFTs
- It does not guarantee secondary sales royalties
- Using a decentralized, distributed database does not decentralize power, or lead to more evenly distributed wealth.

In other words, many problems that plague more traditional ‘real world’ markets, still plague NFT markets. Except, a transaction on a PoW based blockchain is literally hundreds of thousands of times worse for the environment than a non-crypto transaction.

**The blockchain does not provide scarcity or authenticity for NFT related content**

An NFT is simply a a long hexadecimal (base16) number (tokenId), such as: 0x41A322b28D0fF354040e2CbC676F0320d8c8850d/XXXXX

This is what you own when you buy an NFT. What is permanently etched into the records of the blockchain, is that you own this number. And it is true that the blockchain guarantees uniqueness of this number, and the history of the records of who owns, and has previously owned — i.e. the provenance of — this number cannot be tampered with (currently).

Also written on the blockchain is an association between this tokenId and some metadata which further links to a URL which contains the ‘original’ file.

However, this association is non-exclusive.

Not only is there nothing within the blockchain as a technology which can stop anybody from downloading this artwork, and re-selling it as their own. But the blockchain as a technology does not even prevent anybody (whether it’s the artist themselves, or somebody else) from creating new tokens that link straight to the exact same URL¹. There is nothing within the blockchain as a technology that provides exclusive access to the contents or URL of a work.

In other words, (a crypto-analogy if you will) I could sell someone the Brooklyn Bridge, and guarantee its authenticity by giving them a one of a kind, unique certificate. While it is true that there is indeed only one of that particular certificate (token) in the world, which they hold in their hands, I can still go and create other certificates (tokens) that link to the exact same object — the Brooklyn Bridge — and sell them on to others.

Of course to do this would be immoral, if not illegal. But that does not prevent it from happening in the ‘Real’ Art Market, and it won’t prevent it from happening in the CryptoArt / NFT Market¹.

The blockchain does not even provide artificial scarcity within the rules of its own imaginary game. It simply does not provide any scarcity or authenticity for NFT related content at all.

All that you really ‘own’ when you buy an NFT, is the hexadecimal tokenId.

The blockchain does not guarantee secondary market income

It is not uncommon for artists to request royalties from future sales of their work. There are established mechanisms in the worlds of music, film and literature which try to ensure that artists receive royalties (these are of course plagued with issues of their own). But in the fine-art world, no such established mechanisms exist (and this topic was brilliantly tackled by Caleb Larson in his 2009 piece “A Tool to Deceive and Slaughter” [19]).

It was proposed that the blockchain could address this and automate secondary sales royalties. However this is not the case. By switching platforms, collectors can resell NFTs without triggering the functionalities implemented in the original smart-contract — which is typically bound to the platform. The collector can then choose not to respect the original Terms & Conditions, and avoid paying the ‘spoiled artists’ [20].

https://twitter.com/satman78704554/status/1339912522514780163

Of course to do so would be immoral, if not illegal. But again my point is, this happens in the ‘Real’ Art Market, and it happens in the CryptoArt / NFT Market. The blockchain as a technology does not prevent this from happening⁵.

The myths of decentralization

When people speak about the blockchain and NFTs, at some point they will inevitably speak about the wonders of decentralization, as an ‘alternative’ to The Old Centralized Ways of The Past, and an antidote to the horrors that this beget.

The blockchain is essentially a database, a list of records. And it is true that instead of this list of records being stored in one single place (or a few key places, all owned by the same entity), the database is instead duplicated — potentially in its entirety — to anybody who wishes to become part of the network. No one body can control the data, or the flow of data. No one body has the power to alter the records to their own favor. Anybody can download the entire history of the database — or alternatively, they may simply download enough of the data to verify each and every transaction or block. It is true that this is one of the greatest affordances that the blockchain offers.

And in principle, this does sound wonderful. What the ingeniously idiotic¹ ² approach of the PoW blockchain has managed to do (apart from exploit humanity’s greed and gluttonous craving for games of value and ownership, at the expense of the planet), is to provide a statistically and game-theoretically [23] sound solution to the problem of distributing a database among non-trusting agents.

And while a decentralized database does have its merits, it is important to remember that decentralizing the database, does not decentralize power.

While the underlying Technology concerning the blockchain might indeed appear to be decentralized, when we consider The Real World of Technology [10] — which takes into account the social, cultural and political context, structures and values in which Technology is designed, developed and deployed — it is impossible not to notice a trend towards re-centralization. Who would have thought ¯\_(ツ)_/¯

‘Mining’ and adding new blocks to the blockchain has reached stages where it requires ludicrous amounts of compute power. So around the world, enormous data-center-like mining-farms (not to be confused with mining-pools) are popping up, containing thousands of computers [11, 12, 13], dedicated to sitting around doing nothing but generating random numbers all day to mine blocks for the blockchain, and reap the rewards for their owners².

This is further exacerbated by the fact that dedicated, improved hardware is continuously being developed specifically to mine for blocks. These ASICs as they are called, are so much more efficient at mining, that as new ASICs are released, from a competitive standpoint it becomes non-profitable to mine without them [14, 15]. Furthermore, the production and distribution of this hardware is very much centralized. (In fact 52% of ASICs produced stay in China [41], which is perhaps one of the reasons why 65% of Bitcoin is mined in China [24]. The second reason being the cheap cost of electricity).

In other words, the protocol of the blockchain might be decentralized, but the hardware required to mine it is most definitely not.

Furthermore, Crypto is often put forward as an anarcho-cypherpunk decentralized elixir to The Old Centralized Ways of The Past.

There seems to be this delusion that cryptocurrencies will replace banks, and everything will be great.

The Financial Systems of The Old, like Banks, do not accumulate power because they centralize records. They accumulate power because they centralize assets. When we complain about The Banks and the Financial Systems of The Old, we don’t complain that they deviously and maliciously manipulate the records that they hold. That is not the problem that needs addressing. We complain that too much wealth is concentrated within too few bodies, and such concentration of wealth, brings an immense concentration and imbalance of power.

One of the largest NFT websites NiftyGateway is owned by The Winklevoss Bros (of Facebook fame), now crypto-billionaires [16]. Aside from the NFT platform, The Bros also own Gemini, one of the leading cryptocurrency exchanges. The Bros also claim to own 1% of all Bitcoin in existence [17].

Furthermore, many exchanges such as Coinbase, Binance, Kraken, Poloniex, Gemini — and even NiftyGateway — don’t even record the activities on the blockchain. They’re custodial exchanges. You give them your assets, and all activity then takes place on their centralized servers. This comes with the advantage of much faster performance and cheaper transactions, but without the security, transparency, immutability and provenance offered by the blockchain (this is not an accusation to say that they are not secure or transparent etc. But if they are, it’s not because of the blockchain, but their own technologies). They are literally banks.

In case anybody was still clinging onto this crypto-anarcho-punk dream:

Even if crypto does replace banks, it’s not going to replace bankers.

Decentralizing the database does not decentralize the ownership of assets, or mitigate the centralization of power.

Crypto — like gold, money or art (and unlike water, food, medicine and fuel) — has no implicit value due to its usefulness. Its value comes instead purely from speculation and hyped-up demand. As a result, whenever its value is threatened, stakeholders — those with large lots, with a lot to lose if it is devalued — will inevitably do anything they can to ensure that the value of their lot is not threatened. This includes making up all kinds of myths. Of course crypto-billionaire Mr Winklevoss himself plays into this narrative, identifying as a ‘cryptopunk’, while selling dreams of money and art in space.

https://twitter.com/cameron/status/1350141884795129858

Crypto is selling an ideology, but delivering the opposite. In the Real World of Technology [10], the crypto-space is exactly replicating the power-dynamics of the non-crypto-space (except it’s frying the planet orders of magnitude faster in the process).

The Dream

Some might point to folks who are winning really big.

This is how the beguiling seduction of the American Dream works. A select few folks win, and they win big — real big. They become the poster child for success — the symbol that demonstrates that it does indeed work, the dream is real. We see an example of it working, and we know that that could be us. It could be me. It could be you.

To those who do win — who previously were not winning under the Old System — the New System indeed does feel like a Democratization Of Success.

But success is still not evenly distributed.

Crypto-Artist total income distribution

(including both primary sales and 10% of secondary sales,
more income analytics in Part 2)

Of the 633 artists on SuperRare:

The top 0.1% of artists earned 8% of total income ($1.5M of $19M)
The top 1% earned 21% of total income ($4M of $19M)
The top 10% earned 57% of total income ($11M of $19M)
The top 20% earned 75% of total income ($14.5M of $19M)
The bottom 40% earned 2% of total income ($390K of 19M)
The bottom 20% earned 0.2% of total income ($40K of 19M)

https://docs.google.com/spreadsheets/d/1WsQYjFObDToCUeIe2QFvk-Xg4m3dhrNJt4zoMWRIJDo/edit?usp=sharing

Though it wasn’t designed to apply at this scale, we can try looking at a metric of income inequality such as The Palma Ratio (PR) — a ratio of the top 10%’s share of income, compared to the bottom 40% [25]. (lower PR implies more equally distributed income, higher PR implies more inequality)

The US’s PR is relatively high within the ‘Western’ world at 1.85. Japan and Denmark are more egalitarian at around 0.9. Countries like South Africa are amongst the worst with a PR of 7. The entire world’s PR — i.e. the income gap between the richest people in the world, compared to the poorest people in the world is 32.

SuperRare has a Palma Ratio of 29. (And SuperRare is a very tightly curated platform. If we were to measure across all NFT artists on open platforms as well, the ratio would undoubtedly be even higher).

Using a distributed database does not help distribute success.

Don’t get me wrong, I’m incredibly appreciative for any kinds of sales at all — even if it is ‘just’ 225 USD per NFT (the median) — since digital works typically are very difficult sell.

But why does this have to involve contributing to frying the planet in the process?

In conclusion, all of the problems that plague non-blockchain based markets still plague NFT markets. The blockchain as a technology does nothing to remedy any of this, and replicates the issues exactly — except it contributes to the global warming of the planet by many orders of magnitude higher when compared to other payment transaction technologies. In fact, it wouldn’t bother me so much that the NFT market is based on these myths, if the current underlying technology was not so idiotically (and deliberately²) wasteful.

Footnotes

Technically, one could attempt to store the data itself on-chain so-to-speak, by breaking it into chunks. However, ETH was not designed for such storage and this is a terrible idea for many reasons. With digital assets being many MBs in size it would probably take hours — if not days — to ‘mine’, costing tens — if not hundreds — of thousands of dollars in transaction fees. And since the blockchain history is immutable by design, it would quickly inflate to unmanageable proportions (imagine having to download all of YouTube just to be able to watch one video). For this reason, large data (anything beyond a few KB) is stored off-chain — usually linked to via the hash of the data. One could implement some kind of mechanism to ensure that off-chain NFT content is not being linked to by multiple tokens, or even checking if the same content is not being re-uploaded with visually imperceptible changes to bypass such automated security measures. However, this then becomes a property not of the blockchain, but some other tools which could equally be deployed on other, non-blockchain networks as long as there is some open-standard agreed on by all platforms involved. And if this open-standard were to be developed, it does not need to be deployed on a PoW blockchain. In other words, the blockchain itself does not provide scarcity or authenticity for NFT related content.
Similar to the above note, mechanisms could be developed to ensure cross-platform guarantees of secondary sales. However, again this would require the cooperation of all platforms involved to develop a cross-platform open-standard, and this then becomes not a property of the blockchain, but the agreement between the platforms, which does not require the blockchain to function.

References

The Unreasonable Ecological Cost of #CryptoArt (Part 2: Extras)

Memo Akten — Thu, 31 Dec 2020 01:37:19 GMT

The Unreasonable Ecological Cost of #CryptoArt (Part 2)

Update — Sep 2022

As of 15th September, after many years of delays, Ethereum Mainnet has finally switched to Proof-of-Stake in an event known as The Merge. This uses a completely different algorithm to the one discussed below (Proof-of-Work). As a result, moving forward, the numbers below are no longer relevant, and current Ethereum energy consumption is thought to be reduced by ~99.95%.

Update — Dec 2021

There is now more research on the emissions of
1. The Ethereum network as a whole,
2. NFT platforms on the Ethereum network, and
3. Per-Transaction footprint / responsibility allocation.
conducted by Kyle McDonald.

The above resources currently contain the most up-to-date figures.

Kyle has used a completely different approach (bottom-up) compared to the method my calculations are based on (top-down, as used by digiconomist). Kyle also uses a Fee-based accounting model as opposed to the Gas-based accounting model that I use (note that Gas and Fee are highly correlated, and at times NFT-related tx fees are disproportionately higher than the Gas that they use, see links #2 and #3 above). Despite these differences in methodology, the carbon footprint of an NFT comes out very similar with both approaches. Using Kyle’s calculations yields an average carbon footprint for a single-edition NFT averaged across all NFT platforms on Ethereum (rounded to closest 100Kg) as:

100 KgCO2 to mint (1–2 hour flight, e.g. London-Frankfurt)
200+ KgCO2 for a sale with a few bids (3 hour flight, e.g. London-Rome)
500+ KgCO2 for more bids and more sales (5+ hour flight, NYC-LA)

Note that these figures are very similar to those calculated in the article below. This is because, (until very recently) digiconomist’s top-down estimates (which is what my calculations are based on) were very close to Kyle’s bottom-up estimates. It’s worth noting that very recently, digiconomist’s estimates have increased significantly higher than Kyle’s estimates. This is most likely due to the differences in the bottom-up and top-down approaches to handling events such as the rapid increases in the value of ETH leading to miners seeking to use older (and less efficient) GPUs, and the banning of mining in China causing miners to relocate to other countries (such as mostly coal-powered Kazakhstan). In reality, most likely digiconomist is now over-estimating, while Kyle is most likely under-estimating. This means that it’s very probable that the current true figures are considerably higher than those mentioned in this article.

This is Part 2 of a N-part series, accompanying the website http://cryptoart.wtf

Part 0 (26th Feb): A short summary of the main article has been published in the online publication Flash Art under the title “Toward a New Ecology of Crypto Art: A Hybrid Manifesto”. I suggest this as a starting point.

Part 1: I present my motivations, findings, and my conclusions as a result of these findings.

Part 2 (this article): I present the underlying data and methodology that gave rise to these findings, along with more statistics. It’s more aimed at those who wish to dig deeper into the numbers, and/or technically interrogate my claims.

Part 3 (25th Feb): I — with contributions from many others — have put together some information with regards to more sustainable NFTs.

[Data: Sources | NFTs (anonymized)| Artists (anonymized)]

Additional notes

These figures are calculated from blockchain data and existing research. Are they “accurate”? They are as accurate as they can be based on the current data and research, which does have many unknowns. Furthermore, the true figures are continually changing. In that respect these figures are best thought of as representing the true scale (e.g. hundreds of Kg C02, tonnes of CO2, tens of tonnes of CO2 etc. i.e. orders of magnitude). The accuracy also depends on the acceptable error margin. One of the most authoritative critiques of the method used by [1] comes from Jonathan Koomey [16], who finds the model too “simplistic”. For an analyst who works with clients where lives are literally at stake if errors are made [19], this is completely understandable. For the purposes of our discussion however, even if the true carbon footprint of a multi-edition NFT is 50 tonnes CO2 instead of 100 tonnes, it really should not affect this conversation — as it is not going to be in reality, 50 grams, or even 50 Kg. Furthermore, the “Bevand” method — which is recommended by Koomey — is used by [15] to estimate the energy consumption of Bitcoin, and is currently producing 40% higher estimates than the method used by [1, 23], rendering it unlikely that the Ethereum estimates will be significantly (e.g. an order of magnitude) over-estimating.

There is a common fallacy along the lines of “The same energy will be consumed whether a block is empty or contains your transaction, thus a transaction has no impact on the energy consumption of a block being mined, or on the environment”. This is based on a gross misunderstanding of what a carbon footprint is. This statement is analogous to claiming “when a 500 tonne airplane flies from NYC to LA, it consumes the same amount of fuel whether I ride it or not (because the weight of a person is negligible compared to the weight of the plane). Thus my flying has no impact on the environment”. It is true that one person deciding to fly (or not) does not have an immediate effect on emissions. However, there is a footprint associated with a seat on a plane. Furthermore, the market demand for flying, affects how the aviation industry schedules flights. In fact, if all of a sudden, lots more people wanted to fly, the aviation industry would not be able to all of a sudden schedule more flights overnight. However, the mining industry can respond much quicker to increases in demand, compared to the aviation industry. Miners even usually have older mining equipment which are no longer profitable to mine with (i.e. the electricity costs of running the equipment is greater than the financial rewards). But if demand goes up on the network, transaction fees also go up, and mining becomes more profitable to the extent that miners can respond quite quickly by turning on such older equipment [13]. More detail on calculating the footprint of individual transactions can be found in [22].

Mining is not primarily powered by renewables. One report produced by a crypto-investment agency [21] claims that (Bitcoin) mining is 75% powered by renewables. However, other independent (and arguably more thorough, as they include more regions and the changing seasons and miner migration) studies put the figure much lower at 29%–39% [13]. The data that I use to calculate emissions, includes usage of low-carbon energy sources weighted by hashrate of regions, with a global weighted average of ~40% lower-carbon energy production (0.58 KgCO2/kWh) — inline with this research. This is based on data from [7, 22], and the limitations of the accuracy of this approach is detailed in those links. (On a side note, using renewables is not a solution if it displaces others from using that renewable source [12]).

Mining is not primarily powered by surplus hydro-electric power in China. Emphasis mine: “There is a notion that electricity surplus in some APAC areas, such as the province of Sichuan in China, gives hashers who relocate their operations there during the rainy season [May-Oct/Nov] a competitive advantage in minimising their running costs. However, survey data demonstrates that this seasonal advantage appears to be offset by less affordable electricity prices throughout the rest of the year when hashers migrate back to other provinces, such as Xinjiang or Inner Mongolia in China. […] Coal-based mining is principally adopted in regions such as the Chinese provinces of Xinjiang and Inner Mongolia, and in Kazakhstan, whereas hydroelectric energy is mainly generated in South-Western regions of China (Sichuan and Yunnan). China’s oversupply of hydroelectric energy during the rainy season has often been used as evidence in claims that a vast majority of mining is powered by environment-friendly power sources. While it is true that the Chinese government’s strategy to ensure energy self-sufficiency has led to the development of massive hydropower capacity, the same strategy has driven public investments in the construction of large-scale coal mines. Like hydroelectric power plants, these coal power plants often generate surpluses. It should not come as a surprise then that a significant share of hashers in the region equally report using both hydropower and coal energy to power their operations”. p24, p27 [13].

China’s mining distribution (for BTC) in Sep 2020. Note Sichuan (Hydro) is leading. [14]

China’s mining distribution (for BTC) in Apr 2020. Note Xinjiang (Coal) is leading. [14]

Introduction

So far I’ve chosen to focus most of my analysis (and the website http://cryptoart.wtf) on SuperRare — which is just one of many CryptoArt NFT platforms, and by no means the worst offender when it comes to carbon footprint. E.g. SuperRare offers ‘one-off’ NFTs. In Part 1, I also share some figures from NiftyGateway (which I calculate in the same way, measuring Ethereum Gas). NiftyGateway doesn’t track bids and sales on-chain. However, they do allow editions of 100s, which incur footprints orders of magnitude higher than those mentioned below.

On SuperRare I’ve analyzed:

17812 NFTs
79977 transactions relating to these NFTs
633 CryptoArtists

(I don’t know the total number of NFTs, transactions or artists on SuperRare, but I believe this is very close to almost all of them).

Background: how is the footprint calculated?

Different Ethereum transactions require different amounts of Ethereum Gas (not to be confused by ‘real world gas’), based on the complexity of the actions carried out by the smart-contract as a result of the transaction [17]. Blocks also have a Gas limit (currently 12.5 million). This limits the amount of computation that can be required by all of the transactions in a block, effectively also limiting the number of transactions in a block. “In general, the more complex the transactions are, the fewer you can fit within one block” [18]. Thus the amount of Gas required by a transaction is representative of how much of a block it it is taking up. E.g. a simple transaction (~21K Gas) will take up 21000/~12500000 => ~0.17% of a block. Whereas an operation such as ‘minting’ an NFT (~260K Gas) will take up 260000/~12500000 => ~2% of a block. Since the energy required and footprint of mining a block is independent of its contents and number of transactions, the Gas required by a transaction is representative of the portion of a block’s footprint it will incur.

This is explained in more detail in [22] and a discussion can be found here.
(Kyle McDonald has been doing more research on “Per-Transaction footprint responsibility models”, see Update 27.04.2021 at the top of this page).

Footprint per unit of Gas

From the total amount of Ethereum Gas and energy consumed by the Ethereum network over a period [1, 2, 3], we can calculate the energy footprint per unit of Gas. Then we can calculate the footprint of a particular transaction³, from the amount of Gas that it used. Likewise for each NFT, by adding up the footprints of each transaction. Carbon footprint is calculated in a similar manner, looking at emissions averaged across regions where mining is most common [7]¹.

These give[20] ¹ ³ :
Energy footprint per unit of Gas = 0.0005454743 kWh
Carbon footprint per unit of Gas = 0.0003182308 KgCO2

Update Mar 03 2021: The figures above are averages across the period Dec 2020 — Jan 2021, for when this study was conducted. Here are annual averages:

2017 (20 May–31 Dec): 0.0008427156 kWh, 0.0004916419 KgCO2
2018 (01 Jan–31 Dec) : 0.0011741214 kWh, 0.0006849847 KgCO2
2019 (01 Jan–31 Dec) : 0.0005121816 kWh, 0.0002988077 KgCO2
2020 (01 Jan–31 Dec) : 0.0003735465 kWh, 0.0002179278 KgCO2
2021 (01 Jan–01 Mar ): 0.0006310654 kWh, 0.0003681648 KgCO2

Footprint per transaction type

The footprint of a single average ETH transaction is estimated to be 35 kWh [1]. However, this is not the energy consumption of a typical transaction relating to a NFT.

Different kinds of NFT transactions have different complexities, and thus different Gas requirements. While the values fluctuate, on SuperRare they are typically in the range:

Minting: 260K Gas => 142 kWh, 83 KgCO2
Bids: 75K Gas => 41 kWh, 24 KgCO2
Cancel Bid: 22K Gas = > 12 kWh, 7 KgCO2
Sale: 160K Gas => 87 kWh, 51 KgCO2
Transfers: 95K Gas => 52 kWh, 30 KgCO2

Calculating footprint of NFTs

For each NFT, I search the Ethereum blockchain for all of the transactions that the NFT has been involved in. Using the footprints per unit of Gas that I mentioned above, I can calculate the footprints relating to the NFT (or artist)³.

I believe this provides a lower-bound for the energy consumption of that NFT being tracked on the Ethereum network.

From what I can tell, SuperRare uses (at least) three smart-contracts, two for managing NFT ownership and transfers (v1 and v2), and the other for bids. They might also be using other contracts for doing other things which I have not come across yet. This would boost the numbers even higher.

Moreover, the contracts dealing with ownership is open-source, so querying it to get ownership and transfer related transactions is relatively trivial. The contract dealing with bids however, is closed-source. And without a so-called ABI (Application Binary Interface), I cannot programmatically query the contract for any bid transactions that involve a specific NFT. So instead I have to resort to more brute-force methods of search. And these are likely to miss transactions that are related to a particular NFT. Given that a transaction cannot use negative gas, any transactions that I do miss, would have pushed the numbers that I calculate even higher.

For this reason, I see the figures that I give as a lower-bound.

NiftyGateway however, is a bit different. They use one smart-contract per ‘collection’, and they don’t log bids or transfers on the blockchain unless the NFT is withdrawn from the platform. Minting does take place on the blockchain, and since it is the most expensive of the transactions, NFTs on NiftyGateway still have a high footprint. Especially when editions are taken into account, because each edition minted is an additional mint transaction (in large batches, the Gas per mint can drop from 260K to 210K, but for editions of 100s, this saving is then overriden by orders of magnitude).

Footprint per average NFT

79,977 transactions on SuperRare used 12,040,321,070 Gas, giving a total of 6,567,686 kWh (6.5 GWh) and 3,831,601 KgCO2 (3831 tonnes CO2).

Across 17812 NFTs, this gives an average of:
4.5 transactions, 369 kWh, 215 KgCO2 per NFT
http://cryptoart.wtf/#NFTs=1

(More statistics in the following sections)

Footprint per average transaction relating to a NFT

12,040,321,070 Gas across 79,977 transactions gives:
150,547 Gas => 82 kWh / 48 KgCO2 per transaction (on SuperRare).
http://cryptoart.wtf/#transactions=1

Note that this is more than double the 35 kWh of an average ETH transaction.

And I would again like to put out a reminder, that these figures relating to energy consumption and CO2 emissions, are only due to the tracking of bids and sales of the NFT on the blockchain. They do not include the production of the works, or even storage of the work online. Just the records of who owns what.

NFT Statistics

The data for this section can be found here, and more up-to-date data can be downloaded from http://cryptoart.wtf/#list=nfts

http://cryptoart.wtf/#list=nfts

KgCO2 is computed from kWh, which is computed from Gas. So these are linearly correlated. Number of transactions however, is only indirectly correlated. As mentioned above, some transactions (such as minting) are more Gassy compared to others, such as bidding. So while NFTs with more transactions are more likely to have a higher footprint, there can be exceptions (e.g. a NFT with 5 sales and transfers could be higher than a NFT with no sales and transfers but more bids).

Energy consumption per NFT

Carbon emissions per NFT

NFT Age (time since being minted)

SuperRare’s first NFT was minted 33 months ago.
50% of NFTs were minted in the last 9 months
20% of NFTs were minted in the last 3 months
6% of NFTs were minted in the last month.

Number of transactions per NFT

1 in 9 NFTs (11%) have no transactions in addition to being minted
half (48%) have 3 or fewer transactions
3 in 4 (73%) have 5 or fewer transactions
1 in 14 (7%) have 10 or more
0.5% have 20 or more
0.1% have 30 or more
1 NFT has more than 50 transactions

Artist Statistics

The data for this section can be found here , and more up-to-date data can be downloaded from http://cryptoart.wtf/#list=artists

http://cryptoart.wtf/#list=artists

Number of NFTs minted per artist

1 in 17 artists (6%) have minted only 1 NFT
1 in 3 (31%) have 5 or fewer NFTs
half (55%) have 10 or more
1 in 3 (36%) have 20 or more
1 in 6 (16%) have 50 or more
1 in 14 (7%) have 100 or more
1% have 200 or more
2 artists have 300 or more NFTs

Energy consumption per artist

Carbon emissions per artist

Artist Age (time since joining platform)

1 artist joined over 2.5 years ago
1 in 7 (14%) joined over 2 years ago
2 in 3 (65%) joined in the last year
half (45%) joined in the last 6 months
1 in 4 (27%) joined in the last 3 months
1 in 11 (9%) artists joined in the last month

Income and revenue distributions

NFT primary sales

The average sale price is 0.868 ETH ($975)
1 in 3 NFTs (37%) have not sold at all
Half of the NFTs sold for less than $225 (i.e. median)
4 out of 5 NFTs (78%) sold for less than $1000
1.5% sold for more than $10K
1 NFT sold for more than $100K

NFT secondary sales (10% goes to the artist)

90% have not resold (i.e. zero secondary sales royalties)
99% generated less than $5K (less than $500 for the artist)
3 NFTs (0.02%) generated more than $100K (more than $10K for the artist)

Total revenue per NFT (including primary and 10% secondary sales)

Artist primary sales

The average primary sales income is 24 ETH ($30K)
1 in 7 artists (14%) have made no primary sales at all
Half of the artists made less than $9.5K (i.e. median)
1% made more than $250K
1 artist made more than $1M

Artist secondary sales (10% goes to the artist)

Half of the artists (56%) did not receive any secondary sales royalties.
1 in 3 (38%) generated less than $1K (i.e. less than $100 per artist).
3 artists (0.5%) generated more than $250K (i.e. more than $25K per artist)

Total income per artist (including primary and 10% secondary sales)

Though it wasn’t designed to apply at this scale, we can try looking at a metric of income inequality such as The Palma Ratio (PR) — a ratio of the top 10%’s share of income, compared to the bottom 40% (lower PR implies more equally distributed income, higher PR implies more inequality) [10].

The US’s PR is relatively high within the ‘Western’ world at 1.85. Japan and Denmark are more egalitarian at around 0.9. Countries like South Africa are amongst the worst with a PR of 7.The entire world’s PR — i.e. the income gap between the richest people in the world, compared to the poorest people in the world is 32.

SuperRare has a Palma Ratio of 29.

Final thoughts

See Part 1 for conclusions, I won’t repeat them here.

Footnotes

The KgCO2 calculation has a larger margin of error compared to kWh. First, it inherits the error of kWh. Second, naturally some modes of energy production (e.g. coal, oil, gas) release more greenhouse gases compared to others (e.g. hydro, wind, solar, nuclear) [8]. Perhaps unsurprisingly, there are heated disagreements over whether the energy fueling the massive mining farms around the world are predominantly greener than average, or worse. The current estimate with regards to percentage of total hashing energy coming from renewables is 39% [11]. However, even with renewables, unless the source is an off-grid, or otherwise underutilized plant, one must also take displaced renewables into account [12]. Footnote 5. has more information.
It’s important to note, that while there are some very useful tools out there to calculate kWh and KgCO2 estimations for ETH wallets [9], these only look at the transactions that a ETH wallet was directly involved in (e.g. transfer of ETH funds). However, minting, sales and bids on an NFT are not necessarily linked to the authors’ ETH account depending on the platform used, so these tools will miss them. Instead, to calculate the footprint of an NFT, one must look at the smart-contract(s) involved. (Note that there may be more than one smart-contract involved for a single NFT, as it is on SuperRare: one contract for minting and transfers, another contract for bids).
It might be more accurate to compute the energy consumption per unit of Gas at the time of the transaction, by using historical figures. However I don’t do this, and I use recent figures. The reason for this is because I’m less interested in the footprint of past NFTs, and I’m more interested in the footprint of current NFTs.
Due to sheer volume, simple, typical, daily, activities also add up. A single email is thought to produce in order of ~1–10g of CO2. But just the CO2 emissions of spam emails were estimated to be around 3 billion KWh per year according to a report in 2009 [15].
(moved to main body)

References

The Unreasonable Ecological Cost of #CryptoArt

Memo Akten — Mon, 14 Dec 2020 13:36:23 GMT

The Unreasonable Ecological Cost of #CryptoArt (Part 1)

Update — Sep 2022

Update — Dec 2021

The above resources currently contain the most up-to-date figures.

In under half a year, one artist’s multi-edition NFTs have a footprint of 260 MWh, 160 tonnes of CO2 emissions.

Updates

23.01.2021 Anonymized all data. For verification purposes only, details available upon request
10.02.2021 Moved some footnotes (e.g. ETH2) to main body
26.02.2021 A much shorter summary of this article has been published in the online publication Flash Art under the title “Toward a New Ecology of Crypto Art: A Hybrid Manifesto”

This post has attracted a lot of criticism for being too “one-sided”. This is absolutely true, it is intentionally so. The reason for this is that there are already hundreds of articles covering many other aspects of NFTs or CryptoArt, but not a single article or study looking at it from an ecological perspective. My mistake was that I did not make it clear from the beginning that this was my position. So now I’d like to make it very clear that what follows is not a general “all-sides-of-the-story” article, but focuses only on the CryptoArt / NFT Market from one perspective, which I believe has to be — but yet has not been — included as part of the conversation. Please refer to other articles for a broader picture regarding NFTs or CryptoArt.

There is of course a larger discussion around the ecological impact of PoW blockchains in general (e.g. Bitcoin, Ethereum). This ‘discussion’ is severely polarized between the pro-crypto who believe “mining is mostly green and without a doubt a net-positive for the climate and the world”, vs anti-(PoW) crypto who believe it to be a waste of resources. For a reasonable pro (Bitcoin) position please see [43], and a reasonable anti (PoW) position please see [60].

I hope that this article starts a productive conversation, and we do not readily jump on and accept the new commercial platforms handed to us by Silicon Valley-style move-fast-break-things accelerationism, and instead we strive for better, built on the values that we hope to carry into the future.

Introduction

This is Part 1 of a N-part series, accompanying the website http://cryptoart.wtf

Part 0 (26th Feb): A short summary of this article has been published in the online publication Flash Art under the title “Toward a New Ecology of Crypto Art: A Hybrid Manifesto”. I suggest this as a starting point.

Part 1 (this article): I present my motivations, findings, and my conclusions as a result of these findings.

Part 2: I present the underlying data and methodology that gave rise to these findings, along with more statistics. It’s more aimed at those who wish to dig deeper into the numbers, and/or technically interrogate my claims.

Part 3 (25th Feb): I —with contributions from many others — have put together some information with regards to more sustainable NFTs.

[Data: Sources | NFTs (anonymized)| Artists (anonymized)]

What is this? What’s going on?

The website http://cryptoart.wtf estimates the carbon footprint of CryptoArt NFTs as a result of blockchain (specifically Proof-of-Work, Ethereum) based transactions relating to the NFT. These figures do not include the production or storage of the works, the computers directly engaging in the act (e.g. the seller or buyers’ computers), or even the website itself, but is simply for the act of using the PoW Ethereum blockchain to keep track of sales and activity.

What? Why?

CryptoArt — using the blockchain to track sales and bids of (mostly digital) artworks — is booming.

Many people (myself included, until I started this investigation) are seemingly unaware how ludicrously energy consuming and ecologically destructive some blockchain based activities are — orders of magnitude worse when compared to other activities typically carried out by individuals.

Bitcoin, Ethereum, and some other cryptocurrencies use a consensus algorithm known as Proof-of-Work (PoW)¹ ². This allows information to be stored in a distributed, decentralized manner across many nodes in a network (e.g. The Internet), while remaining secure, reliable and dispute-free. (There are alternatives to PoW which are far more efficient, but PoW is currently the most widely used ¹).

Ingeniously idiotic, by design² the PoW algorithm is very compute intensive [1, 2, 3, 40]. The original cryptocurrency Bitcoin (BTC), is estimated to have annual energy consumption in the range 80–120 TWh [3, 40] which is about ~0.45% of the world’s entire electricity production.

In this article however, I will not focus on Bitcoin, but Ethereum.

Due to its ability to execute custom code in a decentralized manner via ‘smart-contracts’, Ethereum is currently a very popular blockchain in the world of CryptoArt, and more generally speaking, Non-Fungible Tokens (NFT).

NFTs are custom tokens that can be created (i.e. ‘minted’) on the blockchain, and then traded. NFTs can be minted as unique one-offs, or they can be limited edition runs (e.g. of a few dozen, hundreds, or thousands). And linking NFTs to other assets (physical or digital), seemingly introduces a scarcity to those assets that might otherwise be unattainable.

This alleged scarcity, combined with the speculative crypto-hype-bubble-startup-craze — has fueled the CryptoArt market, as well as a booming general NFT Rare Collectibles market [5].

After a slow but steady growth over the last few years, this year — perhaps accelerated by Covid19 — CryptoArt has exploded, with artists from many generations announcing their soon-to-be first NFT drop [6].

Truth be told, I am incredibly excited by the idea of purely digital platforms generating sustainable income for digital artists. This is why I started investigating a few months ago. I knew that Proof-of-Work blockchains were “bad for the environment”, but it turns out there is absolutely no information whatsoever regarding how this translates to NFTs as they are currently implemented. Hence I started this research.

When I began sharing this information in September 2020, it was mostly ignored, but at least one cryptoartist — Joanie Lemercier — became concerned and spent several months trying to get more information from the platform he was working with. In his own words: “Joanie Lemercier tried to have this conversation with NiftyGateway for months, to understand the impact of his release, but couldn’t get any information about this. He only got the classic Silicon Valley response: ‘we will check with the team and get back to you’. Lemercier is trying hard to reduce his studio’s electricity consumption every year, and it turns out his NG release consumed in 10 seconds more than his entire studio over one and a half years”.

This lack of transparency is unacceptable. Providing even ball-park figures, or simple advice regarding ecological impact should be compulsory. It was clear a more radical approach was needed to get people’s attention, to start talking about this, so I wrote this article and made the website cryptoart.wtf.

I was also approached by another cryptoartist asking about their footprint. They were horrified to hear it was in order of a hundred tonnes of CO2. (To put this into perspective, across Europe and the US the annual per capita carbon footprint of all industry, trade, and imported goods is on the order of ten tons of CO2 [42]).

The problem: the ecological cost

While not as bad as Bitcoin, a single Ethereum (ETH) transaction is estimated to have a footprint on average of around 35 kWh [4].

This in itself, is ludicrously high. To put that into perspective, this is roughly equivalent to an EU resident’s electric power consumption for 4 days [8].

And this is for simply a single ETH transaction. This is for an act which takes a fraction of a second from the point of view of the person engaging in the act. A single click of a mouse sets off a chain reaction and sends a signal to mining farms around the world [11, 12], which go on to have a footprint of 35 kWh for an ‘average’ transaction (details in Part 2), with emissions of close to 20 KgCO2 for that single mouse click³, due to the underlying PoW algorithm. (Whereas for example, an average email is estimated to have a footprint of a few grams of CO2 [33], and watching one hour of Netflix is estimated at around 36 grams CO2 [22]. An ETH transaction is thousands times more costly than other internet activities that individuals typically engage in).

And this is an average ETH transaction. Unfortunately, it turns out that NFT sales and related transactions on a PoW blockchain such as Ethereum, are a lot worse.

Depending on the complexity of its requirements, an ETH transaction might have a higher or lower footprint (details in Part 2). 35 kWh might be the average across all transactions that are happening on the ETH network, many of which consist of a simple value exchange. Transactions relating to NFTs however, have more complexities involved.

In fact, I have analyzed ~80000 transactions relating to ~18000 NFTs on the CryptoArt NFT market site SuperRare (which is just one of many CryptoArt NFT platforms).

The footprint of a single transaction relating to a NFT on SuperRare averaged across all 80000 transactions (including minting, bidding, sales, transfers etc) is 82 kWh, with emissions of 48 KgCO2 (details in Part 2).

Note that this is more than double that of an average ETH transaction.

Unfortunately, it gets worse.

A single NFT can involve many transactions. These include minting, bidding, cancelling, sales and transfer of ownership. If we were to break down the footprint by transaction type, we get (details in Part 2):

Minting: 142 kWh, 83 KgCO2
Bids: 41 kWh, 24 KgCO2
Cancel Bid: 12 kWh, 7 KgCO2
Sale: 87 kWh, 51 KgCO2
Transfer of ownership: 52 kWh, 30 KgCO2

This generally pushes the footprint of a single NFT into hundreds of kWh, and hundreds of KgCO2 emissions, and often higher.

In fact, of the ~18000 CryptoArt NFTs that I analyzed, the average NFT has a footprint of around 340 kWh, 211 KgCO2 (details in Part 2).

This single NFT’s footprint is equivalent to a EU resident’s total electric power consumption for more than a month, with emissions equivalent to driving for 1000Km, or flying for 2 hours.

http://cryptoart.wtf/#list=nfts

And I would like to again point out, that this energy consumption is simply for the act of using a Proof-of-Work (PoW) based blockchain — that was designed deliberately² to be so compute intensive— to keep track of bids and sales. It excludes the energy consumption of the actual production of the work. It excludes the energy consumption related to storing the work online. It excludes the energy consumption of the website itself, or the computers of those involved in minting, bidding or selling.

Accumulating a few of these NFTs easily pushes the footprint into MWh, with tonnes of CO2 emissions.

2 out of 3 (67%) of the 633 artists on SuperRare have NFTs with a footprint of more than 1 tonne CO2. And this is despite half of the artists joining the platform in only the last 6 months.

1 in 5 (18%) of the artists’ NFTs have a footprint greater than 10 tonnes of CO2. To put that into perspective, that is equivalent to 12 transatlantic flights, or a EU resident’s total electric power consumption for 5 years.

The average footprint of an artist’s NFTs on SuperRare is 10 MWh, 6 tCO2 while the median is 3 MWh, 2 tCO2. (The average time an artist has been on SuperRare is 11 months, the median is 7 months. Details in Part 2).

http://cryptoart.wtf/#list=artists

The footprint from all NFTs on SuperRare is 6 GWh, 3.8 Mt CO2.
(This is just for keeping tracking of bids and sales remember, this excludes the hosting of the actual website, the production of the work, the artists’ computers, the buyers computers etc).

http://cryptoart.wtf/#list=nfts#total

Unfortunately, it gets worse.

The above is based on an extensive analysis that I’ve carried out over the past few months on the CryptoArt NFT marketplace website SuperRare. SuperRare — as the name might imply — deals with one-off NFTs. However, other NFT websites allow editions.

Editions are a way that allows creators to get more ‘bang’ for their buck. Quite literally — as they can sell the same work more than once, and that ‘bang’ carries out into the environment, as 10x, 100x more tonnes of CO2.

NiftyGateway is another NFT marketplace site on the PoW blockchain Ethereum. Unlike SuperRare, NiftyGateway does not track all activity on the blockchain. They mint on the blockchain, and then track bids and sales on their own servers. Unfortunately, minting is the most ecologically expensive action, 3–4 times more costly than bidding for example (details in Part 2). For this reason, the PoW blockchain related eco-costs are still very high— especially when combined with editions.

One artist for example, released a handful of artworks, each editioned in the hundreds, totaling over 800 editions. In less than 3 months, thanks to the PoW algorithm used by the Ethereum blockchain, these 800+ editions collectively have a footprint in order of 140 MWh with 86 tonnes of CO2 emissions, equivalent to a EU resident’s total electric power consumption for 40 years, with carbon emissions equal to 100+ transatlantic flights.

In a few months, one artist’s multi-edition NFTs have a footprint of 140 MWh, 86 tonnes of CO2 emissions.

Another artist who has been on the platform slightly longer, has created dozens of NFTs with over 1500 editions in total. In under half a year, thanks to the PoW algorithm, these NFTs have a footprint in order of 260 MWh with 160 tonnes of CO2. That’s a quarter of a GWh, in under half a year.

In under half a year, one artist’s multi-edition NFTs have a footprint of 260 MWh, 160 tonnes of CO2 emissions.

It’s also important to remember, that the amount an NFT is sold for, i.e. it’s financial cost, does not affect its ecological cost. A transaction’s carbon footprint will be the same whether it sells for one dollar, or a million dollars (details in Part 2).

An incredibly painful to observe strategy that I’m routinely witnessing, is to price NFTs really cheap (e.g. in the 10s of $ range) and edition them in the many hundreds.

For example, one NFT (not by one of the above artists) was priced in the range of 10s of $ per edition, with many hundreds of editions. In 2 months, this single artwork has a footprint of approximately 74 MWh, with 46 tonnes of CO2 — equivalent to a EU resident’s total electric power consumption for 22 years , or the carbon emissions of over 50 transatlantic flights — in 2 months.

At this point I should be very clear that I am absolutely not blaming the artists. I have no doubt in my mind that they have no clue this is happening. There is no information out there regarding this. I’ve had to spend months to dig out and put this information together. And that is the aim of this article — to make it clear that this is happening.

Final thoughts

In this article, I focused on CryptoArt, because as a computational artist this is a topic that is dominating my spheres right now. However, all of what I say above, applies not just to CryptoArt, but PoW based NFTs in general (in fact most of it applies to PoW cryptocurrencies in general).

And interest in NFTs is growing rapidly, to the extent that the trend now is to NFT everything. Tokenizing and selling screenshots of tweets on the blockchain is a thing [28].

I want to express caution once again, against pointing to the artists. They are not the problem. I believe they have no idea that this is happening. The community that has emerged around CryptoArt is absolutely more supportive of digital arts in comparison to previous markets. By this, I am very encouraged and hopeful.

And I would love to see this community move to more sustainable platforms which do not have footprints of MWhs — even 100s of MWhs when editioned — per artwork.

Artists should be able to release 100s of digital artworks. There is absolutely no reason that releasing 100s of digital artworks should have footprints of 100s of MWh.

Right now there is absolutely zero information available with regards to the carbon footprint of NFTs. Today, one can look up the carbon footprint of flying, and then decide whether to fly or not. The same can be said of eating beef, or buying clothes. One can look up the carbon footprint of an email [33], or watching an hour of Netflix [22], and act accordingly. Are these figures accurate? Not necessarily. But if research says that an hour of Netflix is 36g, it’s unlikely to be in reality 10 tonnes. It might be 20g, 50g, 100g. Maybe even 200g. But not 10 tonnes. Such research is a start. I welcome others to look at my methodology, see the flaws, and put out new studies. And then we can make informed decisions.

Given the urgency of the current climate crisis, we are already faced with the immense challenge of having to change our existing habits. And changing habits which are deeply entrenched into every aspect of our lives and culture, and the systems that we depend upon, is really hard. Yet that’s what we must do. Given this, is it really wise to adopt new systems and habits that are as ecologically devastating as this?

To me it seems like lunacy.

When designing and developing new systems, platforms and habits, design and develop them compliant with the standards and expectations of the time!

130 kWh, 80 KgCO2 for a single mouse click to mint a single-edition NFT is ridiculous and unethical.

340 KWh, 210 KgCO2 just to record a handful of bids and sales on a single-edition NFT is outrageous and unethical.

100s of MWh, 100s of tonnes of CO2 in a few months, just to keep track of a few dozen editioned NFTs is absolutely deranged.

Of course even these figures — and anything an individual does — are a drop in the ocean of global emissions (~50 billion tonnes CO2e [44]). Of course we have to completely decarbonize our energy production (which must probably include nextgen nuclear [45]), and invent new, negative emission carbon capture and storage technologies [46]. But at best these are going to take decades to be invented, tested and deployed on a global scale. In the meantime, our emissions are not only not decreasing, they are increasing (excluding a small lockdown related decrease in 2020 [44]).

We need a radical shift in mindset.

Research into new technologies (including blockchain) is essential. But this has to follow a conscious, less extractivist model. I welcome the fundamental research happening in this space, but there is no doubt the recent rapid growth of the NFT market exemplifies Silicon Valley’s move-fast-break-things accelerationism. If this were not the case, more ethically aligned platforms might have included ecological statistics on their site, alongside sales and ROI statistics. They might have provided advice and caution regarding immense ecological impact. As Christina Akopova, cofounder of cryptoart platform Pixeos — which is not on Ethereum, but on the DPoS based EOS blockchain — notes “If you don’t buy the #cryptoart you like cause it’s not on ETH, you are basically an ETH tokens collector, not an art supporter.” [47]

If profit is going to be the primary motivation for this growth and research, then one of our strategies must be to make it more profitable for businesses to act more responsibly. Unfortunately, this kind of social pressure remains essential for corporate social responsibility [48, 49]. Reducing one’s own emissions is not just about reducing one’s own emissions. It also sends a signal to the market demonstrating that there is a demand for more sustainable and transparent businesses, and gives confidence to investors, developers and collectors to invest accordingly [61].

Alternatives to PoW have been deployed or in the works for many years [26, 35, 36, 50, 51, 52 , 53, 54 , 55, 56, 57]. In fact, as I write this, Ethereum has just launched a test network for ETH2 aka Serenity using a Proof-of-Stake (PoS) consensus algorithm [27] which is orders of magnitude more computationally efficient (this is also essential for scalability and business, as we are already seeing skyrocketing Gas prices). While there is no clear date for when ETH2 will be rolled out for mainstream use, the best case estimate is that it will be in the next year or two. While this will (hopefully) resolve the energy problem, it does not address the larger issue. Given the constraints imposed upon us by Global Warming — we have a severe problem if we cannot discuss the issues of today because we hope they will be resolved “in a year or two”, and if we obliviously allow and even encourage businesses and platforms to operate in this manner. Because if we don’t hold businesses and markets accountable, as artists, and as citizens, then who will?

I’ve been working on this project — the underlying research, the data analysis, and the website http://cryptoart.wtf ⁵— for months. I have seen probably thousands of CryptoArt NFTs. I’m blown away by how much absolutely awesome work there is on here. Much of this work does not fit into the typical, established Art Venues. These artists and works absolutely need a home, and digital artists absolutely should be able to earn a living making work that they love.

If we are building new systems, we should build them aligned with the values that we hope to carry into the future. I hope that this article can start a conversation and we can avoid the move-fast-break-things accelerationist mistakes of the past, and instead plan our way forward in a sustainable manner.

Additional notes

These figures are calculated from blockchain data and existing research (see Part 2 for details). Are they “accurate”? They are as accurate as they can be based on the current data and research, which does have many unknowns. Furthermore, the true figures are continually changing. In that respect these figures are best thought of as representing the true scale (e.g. hundreds of Kg C02, tonnes of CO2, tens of tonnes of CO2 etc. i.e. orders of magnitude). The accuracy also depends on the acceptable error margin. One of the most authoritative critiques of the method used by [4] comes from Jonathan Koomey [62], who finds the model too “simplistic”. For an analyst who works with clients where lives are literally at stake if errors are made [63], this is completely understandable. For the purposes of our discussion however, even if the true carbon footprint of a multi-edition NFT is 50 tonnes CO2 instead of 100 tonnes, it really should not affect this conversation — as it is not going to be in reality, 50 grams, or even 50 Kg. Furthermore, the “Bevand” method — which is recommended by Koomey — is used by [39] to estimate the energy consumption of Bitcoin, and is currently producing 40% higher estimates than the method used by [3, 4], rendering it unlikely that the Ethereum estimates will be significantly (e.g. an order of magnitude) over-estimating.

There is a common fallacy along the lines of “The same energy will be consumed whether a block is empty or contains your transaction, thus a transaction has no impact on the energy consumption of a block being mined, or on the environment”. This is based on a gross misunderstanding of what a carbon footprint is (see Part 2 for details). This statement is analogous to claiming “when a 500 tonne airplane flies from NYC to LA, it consumes the same amount of fuel whether I ride it or not (because the weight of a person is negligible compared to the weight of the plane). Thus my flying has no impact on the environment”. It is true that one person deciding to fly (or not) does not have an immediate effect on emissions. However, there is a footprint associated with a seat on a plane. Furthermore, the market demand for flying, affects how the aviation industry schedules flights. In fact, if all of a sudden, lots more people wanted to fly, the aviation industry would not be able to all of a sudden schedule more flights overnight. However, the mining industry can respond much quicker to increases in demand, compared to the aviation industry. Miners even usually have older mining equipment which are no longer profitable to mine with (i.e. the electricity costs of running the equipment is greater than the financial rewards). But if demand goes up on the network, transaction fees also go up, and mining becomes more profitable to the extent that miners can respond quite quickly by turning on such older equipment [41]. More detail on calculating the footprint of individual transactions can be found in [67].

Mining is not primarily powered by renewables. One report produced by a crypto-investment agency [64] claims that (Bitcoin) mining is 75% powered by renewables. However, other independent (and arguably more thorough, as they include more regions and the changing seasons and miner migration) studies put the figure much lower at 29%–39% [41]. The data that I use to calculate emissions, includes usage of low-carbon energy sources weighted by hashrate of regions, with a global weighted average of ~40% low-carbon energy production (0.58 KgCO2/kWh) — inline with this research. This is based on data from [66, 67], and the limitations of the accuracy of this approach is detailed in those links. (On a side note, using renewables is not a solution if it displaces others from using that renewable source [65]).

China’s mining distribution (for BTC) in Sep 2020. Note Sichuan (Hydro) is leading. [42]

China’s mining distribution (for BTC) in Apr 2020. Note Xinjiang (Coal) is leading. [42]

Acknowledgements

Thank you to everybody on this twitter thread and this twitter thread for the feedback and help. Even the negative comments were very helpful. And a special thanks to Hanna Radek, Primavera De Filippi, Kyle McDonald, Joanie Lemercier and Alan Warburton for feedback, thoughts, help with the design and general guidance.

Stay safe y’all.

Footnotes

(Info on ETH2 moved to main body under “Final Thoughts”)
Proof-of-Work (PoW) is designed to be computationally inefficient. Its security comes from the fact that it requires so much computation to write to the blockchain (i.e. add ‘blocks’). The logic is essentially: if it’s stupendously compute-heavy and difficult to write to the blockchain, then it can’t be done frequently enough to pose a security threat. You might see articles refer to this task as ‘solving a mathematical puzzle’. This is a vast romanticization. In reality, to write to a PoW blockchain (and reap a reward of the associated cryptocurrency, e.g. BTC or ETH etc.), one has to generate a random number and hope that it matches the criteria set by the protocol. If it doesn’t match, generate another random number. For this reason, endless arrays of computers are sitting around in giant data-center like mining farms around the world, doing nothing but generating random numbers all day every day, in the hopes of rewarding their owners with reaping the rewards [11, 12, 13]. Furthermore, as more people (or more compute power) get involved, the more likely it is statistically for somebody to stumble upon a random number that satisfies the criteria — and thus quicker. This could compromise the security of the network, so the protocol dynamically adjusts the criteria to become more difficult such that new blocks are always written at roughly a fixed frequency (every ~10 minutes for Bitcoin,~10–20 seconds for Ethereum). As a result, as more people (or computer power) get involved with a PoW blockchain, the more difficult it must become in order to remain secure. In ~10 years, since the introduction of BTC in 2009, its difficulty has increased 18 trillion times. That’s 18000000000000 times [18].
A similar argument could be made for using the Internet in general. In that, every email that we send, every tweet that we tweet, sets off a chain reaction involving routers and servers and data-centers around the world. The Internet definitely does consume a lot of energy, estimated to be in order of 200 TWh/year just for the data-centers [21]. This is primarily due to the immense volume of activity taking place on the Internet. When looked at from a per transaction point of view, blockchain based activities are ludicrously many many many orders of magnitude higher than other online activities. For example, an average email is thought to have emissions of around 1–10 gCO2 [33] (though now it’s thought to be lower). That’s 40000 times lower than an average single-edition NFT. Watching an hour of Netflix is estimated to be around 36 gCO2 [22]. That’s 6000 times lower than an average single-edition NFT.
(moved to main body)
http://cryptoart.wtf only reads from the blockchain (e.g. “Who bought this NFT?” etc.), so it does not incur the ludicrous eco-costs that I discuss in this article. Mining (i.e. writing to the blockchain) is the procedure that is so compute intensive² (e.g. “X created a new NFT”, “Y placed a bid on this NFT”, “Z bought this NFT ” etc.).
(moved to main body)
(moved to main body)

References

Attributions

The icons on the website are licensed under a Creative Commons license from The Noun Project: Car by ✦ Shmidt Sergey ✦, House by Liyuza Anizar Habiby, Kettle by Cuby Design, Swimming by SBTS, Laptop by Simon Child, PC by Jugalbandi, Face by BomSymbols, landscape by ProSymbols, texting by ProSymbols, Artificial Intelligence by Symbolon, Server by Iga, Gear by priyanka, greenhouse gas by Wichai Wi, Energy by Larea, Video by icons, person by Valerie Lamm, Arrow Direction Down by Gunali Rezqi Mauludi, Math by Llisole, link by Peter Carleton, text by ProSymbols, Info by NAS.

Building predictive models of people and their thoughts.

Memo Akten — Wed, 21 Oct 2020 11:38:33 GMT

Introduction and Background

UPDATE: These days most people know what a Large Language Model is. If you do too, you can skip the intro and go straight to the ‘Making stuff up’ section.

There’s a lot of press and hype around GPT-3 [1] right now.

GPT-3 is what’s known as a Language Model (people call it ‘Artificial Intelligence’ but remember, that’s the name of an academic discipline, that doesn’t mean GPT is itself An Artificial Intelligent Entity [2]).

GPT-3 is a ‘Deep’ (very very large) Artificial Neural Network [3] trained to produce text. You ask it a question, or give it a ‘prompt’ or ‘seed’ as it’s called, and the model generates a response [4]. It can answer questions about almost anything, write stories, poems, jokes and almost any kind of text [6].

GPT-3 hasn’t been released to the public yet and I don’t have access to it. But I have been playing with GPT-2 for over a year (and GPT-1 before that, and many earlier language models going back many years). GPT-2 is the previous model by the same company — OpenAI — which was released last year.

Before I continue, it’s worth noting that the size of an artificial neural network is given by the number of connections in the neural network aka weights aka parameters. This is also a rough proxy to the power or capabilities of the model (i.e. how good it is at doing whatever it’s supposed to be doing). GPT-3 has 175 billion parameters. I don’t have the words to express how humungous that is. For comparison, GPT-2 (released just last year) has 1.5-billion-parameters, and GPT-1 (released the year before — 2018) has 117 million parameters. That’s in order of 100x increase per year!

And here’s another crazy point worth mentioning. GPT-3 (175B parameters) has just been announced. When GPT-2 (1.5B parameters) was first announced last year, the creators of the model — OpenAI — deemed the model too dangerous to release to the public!!

“Due to our concerns about malicious applications of the technology, we are not releasing the trained model. As an experiment in responsible disclosure, we are instead releasing a much smaller model⁠ for researchers to experiment with, as well as a technical paper⁠.”

Over the course of last year, OpenAI first released a 10x smaller model at 124M parameters [7], then 355M parameters, 774M parameters [8], and eventually the full 1.5B parameter model in November 2019 [9].

I have been playing with GPT-2 quite a bit, and I have played with all of the models. I can say without a shadow of a doubt, going from 124M to 1.5B is an absolutely massive difference in quality. I can’t even imagine the difference jumping to GPT-3’s 175B parameters will make!

With GPT-2, I’ve written stories, poems, essays. I seed the system with a thought, the model generates some text, I read that text, and inspired by what I read, I ask the system new questions, and get back a new paragraph. It feels very collaborative, like a meandering conversation with an alien intelligence.

But the work that I’ve been making with GPT-2 isn’t the topic of this article. I will write about that separately.

UPDATE: see for example Distributed Consciousness [13], which I wrote using both GPT-2 and GPT-3.

Making stuff up

The topic of this post is something else. A major concern.

During my conversations with GPT-2, the model would often quote people. Due to the nature of my conversations (topics on philosophy, metaphysics, epistemology, science, religion etc), it would quote people like Plato, Aristotle, Richard Dawkins etc.

The model would say things like “The evolutionary biologist Richard Dawkins writes” followed by some text — an apparent quote — in “inverted commas”.

And these quotes were very realistic sounding. Really like something that Plato, or Dawkins might actually say. And I would search the web for these quotes. I searched Google, Google Scholar for academic writings, Google Books in case the quotes were from books. I have nearly all of Richard Dawkins books as e-books. So I searched all of them too for these alledged quotes.

But they don’t exist.

It seems GPT-2 is not really quoting Plato or Richard Dawkins etc.

GPT-2 is making these quotes up.

But it has learnt a model of what Richard Dawkins thinks. At least on these particular subjects. And is making up plausible sounding quotes.

Predictive models of people

Because Richard Dawkins has written a lot of books and articles (usually about religion or evolution), GPT-2 has learnt Dawkin’s thoughts on these topics. So I can ask the model “What does Richard Dawkins think about a particular topic XYZ”, and if Dawkins has written about that topic, the model will make something up that is quite likely closely aligned with what Dawkins actually thinks. (If Dawkins hasn’t written about that particular topic, the model will just make something up at random, which is a whole other can of worms).

It’s important to understand that the model no longer has — or needs — access to the original material, the original books or articles. It is a probabilistic predictive language model where the statistical ‘knowledge’ is stored in the weights of the neural network.

While this is potentially exciting for learning and education (I’d love to have interactive conversations with a virtual Einstein or Feynman!), in the current climate that we’re in, I find this terrifying.

I can download a model, just a few Gigabytes, and without access to the original context (books, texts, conversations etc.), I can ask this model what certain people think on a particular topic.

I don’t have a lot of public writing on the internet. So GPT-2 doesn’t know anything about me. But I’ve written probably millions of words in emails and private messages.

I think it’s very unlikely that public language models — like GPT-2, GPT-3, or GPT-N — will be trained on private data (though never say never, the overton window of what is acceptable in surveillance captialism is constantly shifting >_< ).

But I do think it’s very likely that private language models are being trained on private data.

Private Individual Models

These are models that tech giants like Google or Facebook or Microsoft could be training on your private emails and messages to build a Private Individual Model of you. (I don’t know if they are, but I would be very surprised if they weren’t. And if they aren’t right now, I’m quite sure they will be in the near future).

These models will never (hopefully!) be made public. But their existence — even potential existence — is still a nightmare.

Because in a similar manner to how I can query GPT-2 on Richard Dawkins’s thoughts on particular topics, if someone has access to these Private Individual Models, they could similarly ask: What does this person think about a particular topic XYZ? What does this person think about gay marriage? or trans rights? or abortion? or the resistance fighters of a particular country?

Of course this could also be flipped. If someone has access to a massive ensemble of Private Individual Models, they could ask: find me people who think a particular way about this particular topic. Who supports gay marriage? Who is pro-choice? Who supports the resistance fighters?

Crucially, they can do this without direct access to any of the actual content, emails, messages, DMs. Once the Private Individual Model is built from that content, then forever, this model contains this knowledge about you. And with every new message, every new email, the model of you becomes more and more comprehensive.

And when Google or Facebook or Microsoft say that they would never share your private messages or emails with anyone, does this also mean that they would never give specific individuals of power or authority access to these Private Individual Models? What about to an authoritarian government? What about producing reports for a 3rd party, like “give me a list of people who are likely to support a specific cause”.

Aren’t they already building models of us?

By now we already know very well that models are being built of us [11].

Every social media post, comment, like, or any other interaction, is a data point for ‘psychographic analytics’, which allows those running the models to micro-target messaging in such a way as to influence our behaviour, whether it be buying some product, or voting in a particular way.

But these social-media-interaction-based models are based on relatively little data, and have relatively limited effectiveness.

With these new massive language models, the trajectory that we are on seems even bleaker. The new GPT-style architectures (non-ironically called Transformers [5]) have immensely more power compared to psychographic analytics-style models. Furthermore, as a result of the immense power of these architectures, they can be trained on immensely larger datasets, like all of our private emails and messages, to produce very powerful predictive language models of us.

And it gets worse.

End-to-end encryption to save the day?

WhatsApp is said to use end-to-end encryption. This means, technically, even Facebook’s own servers don’t have access to the content of your messages.

This is great for privacy, right?

But potentially Facebook’s servers don’t even need access to the content of your messages.

The client-side app (i.e. the app you run on your phone to decrypt and read the messages) could locally run your messages through an encoder to extract abstract representations of your ‘thoughts’ [10].

These abstract representations (technically just a sequence aka ‘vector’ of numbers), mean nothing to human minds, and might as well be a bunch of random numbers. But they are exactly what the model needs to be trained on, to build a predictive private model of you.

Conclusion

As a reminder, I have only been playing with GPT-2, a 1.5B parameter model. I’ve seen the difference between the 124M parameter model and the 1.5B parameter model, and it’s huge. The 1.5B parameter model is very noticably more believable, and is arguably a lot closer to and representative of the text that it was trained on. I haven’t had the opportunity to play with GPT-3 yet, and I can’t even imagine the difference jumping to 175B parameters will make.

I don’t know for sure if tech companies are currently building Private Individual Models of us, but I would be very surprised if they weren’t. And if they aren’t already, with the trajectory that we’re on, I’m pretty sure that they will be in the near future. And such models could easily be used to reveal a person’s opinions on various sensitive topics, or queried to find people who hold certain views, all without access to the original content, therefore bypassing existing privacy frameworks and entering into legal grey areas, raising severe privacy and abuse risks.

These new transformer-based massive language models are like a search engine of the future. You don’t search for what has happened, or what has been said. You search for what could happen, what could be said.

Side note

One of the big open problems in AI (in Natural Language Processing — NLP), is Question Answering (QA).

For example, I say to you: “Alice has a book. She gives the book to Bob. Bob goes to the kitchen. Alice goes to the garden”.

And then I ask you “Where is the book?”

You will most likely say “The book is in the kitchen, with Bob”.

To be able to answer this correctly, you need to understand this text. You need to understand the concept of ‘having’. You need to understand that ‘She’ refers to Alice. You need to understand what ‘to give’ means. You need to understand what ‘to go’ means. You need to understand the concept of being somewhere.

And this is a very very simple example. But even this has proven to be a very difficult problem for AI / NLP systems, and it’s only in the last few years that significant progress has been made. And even now, after 3 or 4 sentences the systems still fail, trying to do Multi-Hop Reasoning as it’s called.

Here is another example (known as a Winograd Schema)

Take the sentence: “Bob tried to put the book in his bag, but he couldn’t, because it was too big”.

If I ask you, what does ‘it’ refer to? (i.e. what was too big?)

You can’t find the answer to this grammatically or syntactically, you need to understand the concepts of big and small, and ‘to put inside’.

Because if I had instead said: “Bob tried to put the book in his bag, but he couldn’t, because it was too small”.

This sentence is grammatically, and structurally identical to the previous one. But the answer is different.

In the first case, we understand that ‘it’ refers to the book. The book was too big to fit in his bag.

But in the second case, we understand that ‘it’ refers to the bag. The bag was too small for the book to fit in.

Solving these kinds of problems has been one of the holy grails of AI. This is a very active area of research. But of course not just for one sentence, but for entire paragraphs, for entire essays, books, thousands, millions of emails.

(There’s also a version of Question Answering called Visual Question Answering — VQA. Which involves building systems whch try to answer questions about what’s happening in images or videos. That’s also one of the holy grails of AI. I wonder why).

Because AI — at least as we have it today — is a surveillance technology born of a surveillance economy [12].

[1] “OpenAI’s new language generator GPT-3 is shockingly good — and completely mindless”.
https://www.technologyreview.com/2020/07/20/1005454/openai-machine-learning-language-generator-gpt-3-nlp/

[2] My rant on “Artificial Intelligence as the name of a research field vs a characteristic of being”.
https://memoakten.medium.com/artificial-intelligence-as-the-name-of-a-research-field-vs-a-characteristic-of-being-920cb5f3361e

[3] My tiny introducton to artificial neural networks I wrote to explain Deepdream (2015).
https://memoakten.medium.com/background-info-for-deepdream-is-blowing-my-mind-1983fb7420d9

[4] For a nice description of predictive text models, see Andrei Karpathy’s blog post. But keep in mind, Karpathy’s post is based on a radically different architecture — known as Recurrent Neural Networks. The new architecture underlying GPT models — known as Transformers [5] — are completely different and proving to be way more powerful. https://karpathy.github.io/2015/05/21/rnn-effectiveness/.

[5] The paper that introduced the ‘Transformer’ architecture underlying the GPTs and models like Google’s BERT. “Attention Is All You Need”
https://arxiv.org/abs/1706.03762

[6] https://gwern.net/gpt-3

[7] https://openai.com/index/better-language-models/

[8] https://openai.com/index/gpt-2-6-month-follow-up/

[9] https://openai.com/index/gpt-2-1-5b-release/

[10] My long (~1 hour read) article explaining high dimensional latent spaces and representations. The last sections ‘10. Word Embeddings’, ‘12. Thought Vectors’ and ‘13. Neural Story Teller’ are most relevant here. Though the sections before lay the ground work.
https://medium.com/artists-and-machine-intelligence/a-journey-through-multiple-dimensions-and-transformations-in-space-the-final-frontier-d8435d81ca51

[11] https://www.amnesty.org/en/latest/news/2019/07/the-great-hack-facebook-cambridge-analytica/

[12] My rant on the real reasons behind the rise of Deep Learning / Artificial Intelligence
https://memoakten.medium.com/retune-2016-part-1-the-dawn-of-deep-learning-672b5490f5a2

[13] https://www.memo.tv/works/distributed-consciousness/

Science & Religion. Why I’m so obsessed. #1

Memo Akten — Sat, 18 Jan 2020 05:52:08 GMT

Trigger Warning: This post mentions #FlatEarth #YoungEarth #Creationism and similar ‘theories’.

“Reducible Complexity”, 2013. https://www.memo.tv/portfolio/reducible-complexity/

MEMO M̸e̵h̶m̸e̸t̶ ̴S̴e̷l̴i̷m̸ ̵A̸k̶t̴e̷n̶ on Twitter

Looking at all the work I've ever done, I can conclude my work is about 2 things: 1) Waves 2) God. Luckily quantum mechanics unifies the two

It’s no secret that I’m obsessed with religions and gods, and a lot of my work and writing is theological in a sense (or at times, related to the tensions between science and religion).

I’d like to share just one of the many reasons why I’m obsessed with religions. I’m sure in the future I’ll write about some of the others.

https://medium.com/media/e573c7cfce4c575e99fb8973a45d2615/href

We are a fascinating species, with quite remarkable achievements. Even these words that I’m typing right now, are an archaic, ancient technology, that allow me to carve up my rich, high dimensional, continuous inner world, and butcher them into linear, one dimensional, discretized strings, enabling this simple (albiet less accurate) communication. And I won’t go into the incredible, invisible layers upon layers upon Layers of Abstraction (new project WIP ;) and complexities that enable this string to be received by my computer via switches in my keyboard, stored in my computer’s memory, displayed on my screen, encoded for transport, beamed wirelessly to my router, sent around the world in little packets, received by your computer, reassembled, decoded and displayed on your screen. I’m guessing there is not a single person on the planet (or at least very few), who knows and understands the full process on every level from all of the software components and protocols, to hardware implementations, electronic, electrical and low level molecular, physical and quantum phenomena involved. (A little teaser for an upcoming project ;)

And this is something that we all do hundreds of times a day, we don’t even consider it an incredible feat of science and engineering. We take it for granted. Perhaps because it is so ordinary now, and isn’t spectacular.

What is a spectacular feat of science and engineering? Really spectacular?

Of course there is almost a limitless supply of examples one could give. One that I often like to use, is the fact that we managed to land a little flying robot on a comet. (When I say ‘we’, I personally was not involved of course. I’m referring to ‘we’ as a species. To be specific, it was the wonderful folks at the European Space Agency ;)

https://en.wikipedia.org/wiki/Philae_(spacecraft)#/media/File:Rosetta_and_Philae_(crop).jpg

This little robot flew through space for over ten years. Think back to what you were doing ten years ago. That’s how long this thing was hurtling through space, catapulting itself from planet to planet (mostly back and forth between Earth and Mars), using the planets’ gravitional pull to gather momentum, and slingshot around, like tarzan jumping from vine to vine. Until ten years later (in 2014) it rendezvoused with its target, a tiny little lump of rock and ice by the name of Comet 67P/Churyumov–Gerasimenko.

https://commons.wikimedia.org/wiki/File:Comet_67P_on_19_September_2014_NavCam_mosaic.jpg

As if entry into a stable orbit of a little comet, travelling at speeds up to 135,000 km/h (84,000 mph) after ten years of hurtling through space and catapulting around planets isn’t enough, this little robot launched an even littler robot to go and land on the surface, and send information back to Earth. (NB. As it happens, it unfortunately bounced into a crevice so didn’t receive maximal sunlight, shortening it’s life on the comet).

One could give countless more examples of such incredible, mind-blowing, spectacular manifestations of human ingenuity, victories of science and engineering in the domestication of the fundamental laws of nature.

And we also have members of the same species who declare (to this day), that the Earth is flat, that there are no comets, or planets, or stars out there. Apparently there is no ‘outer space’, even gravity doesn’t exist. They believe that everything we are shown with regards to anything related to space, is Computer Generated Graphics (just like those movies, Star Wars, Gravity etc).

And the levels that some of these people go to, to try and reconcile their own observations — such as a solar or lunar eclipse — with their model of the universe (*ahem* sorry, world), is nothing short of astounding.

https://medium.com/media/16d2b5ea14b1bf99a928da71e5aa9a0b/href https://medium.com/media/f2b472267b8f116703f6275513bf2b53/href

The committment and dedication to experimentation, reasoning, fact checking, hypothesizing and debunking (constained by the requirement to accomodate the orginal belief) to me is more entertaining than, and demonstrates more creativity than anything I’ve seen come out of Hollywood in recent years. (Yes I have spent endless nights in the rabbit-holes of Flat Earth Youtube, and this guy is one of my favourites).

Of course before this very recent explosion of Flat Earthers, we had (and still have) Young Earth Creationists. They promote ideas that not Gravity and all of Physics is wrong (as proposed in Flat Earth), but Evolution, and all of Biology is wrong. Young Earth Creationists believe that the universe is only 6000 years old (this varies depending on the particular school of Creationism), and they’ll goto incredible lengths to ‘scientifically’ prove it (and ‘disprove’ any competing theories, such as, you know, mainstream science). They of course don’t believe in evolution, dinosaurs didn’t exist, and “God put their bones in the ground to test our faith” (Yes I’ve also spent endless nights in the rabbit-holes of Creationist Youtube).

And the fact that these Flat Earthers, and Creationists, then use one of the grandest, and most ingenious of human creations —The Internet, a global network of networks of digital computers — to spread these theories is just the most tragic irony.

But what I find the most remarkable, and (one of) the reasons why I’m so obsessed with these beliefs, is because it’s the same brain that gives rise to them, as does invent and create the flying robots that land on comets. It’s the same basic physical structure, the same design, the same biological principles, that gives rise to such extreme opposites (or perhaps they’re not such opposites?). Simply changing some (yet not fully determined) parameters (internal and/or external) can make the difference between a rocket scientist and a gravity denier; a genetic engineer and a banana evangelist (see video below for context).

In a sense, the existence of such diversity in belief, faith, rationality, reasoning and commitment; is a testament to the incredible flexibility, adaptability and malleability of the clump of neurons and chemicals that we call the human brain.

Note: It’s worth mentioning, while I half-jokingly refer to these beliefs as fascinating and even entertaining, I am in fact very concerned about the impact such mindsets have when it comes to policy and decisions affecting our collective (human+non-human) future. So concerned in fact, that I can’t just obliviously laugh them off. Instead I feel the need to spend endless nights in the rabbit-holes where they are born, incubated and spread, trying to understand how a brain so almost identical to mine, can result in a mind so orthogonal. And while I’ve picked rather extreme and easy targets here, consisting of what we’d like to think of as small fringe groups (though even such fringe groups will push hard if they are not pushed hard back), countless less fringe, but equally (if not more) dangerous examples can be given.

https://medium.com/media/075ff9fabb9c0ccb016eb1bf00652958/href https://medium.com/media/92cdfa105539570b5ed957e0756d45f1/href https://medium.com/media/c06a3769f7ff5e69209aff470ea9a971/href https://medium.com/media/bcfa30013e10d999efaade154977bec5/href

http://www.memo.tv/portfolio/simple-harmonic-motion/

http://www.memo.tv/portfolio/reducible-complexity/

http://www.memo.tv/portfolio/keeper-of-our-collective-consciousness/

https://medium.com/@memoakten/all-watched-over-by-machines-of-loving-grace-8c2464aa6fda

https://medium.com/@memoakten/consciousness-is-evolutions-solution-to-dealing-with-big-data-b013b8faebad#.shalvs18a

https://medium.com/@memoakten/science-is-the-new-religion-is-the-new-religion-60e4db4649aa

https://medium.com/@memoakten/a-semi-mathematical-theory-of-human-to-human-communication-738f4a5d7f6b

https://medium.com/artists-and-machine-intelligence/a-digital-god-for-a-digital-culture-resonate-2016-15ea413432d1#.h2cy5wpvo

“Artificial Intelligence” as the name of a research field vs a characteristic of being.

Memo Akten — Mon, 22 Jul 2019 19:03:48 GMT

You haven’t built / you’re not collaborating with “An Artificial Intelligence”.

https://medium.com/media/33b9976ac38c8b975e97715e4ae71115/href

This is a quick brain dump that I can point to when needed (rather often).

Just one of the many reasons I’m not fond of the term Artificial Intelligence, is because it’s worded in such a way that it creates confusion between the research field, and a characteristic of being.

I’ll try to use Biology as an analogue. While this is not a one-to-one correspondence, I hope it will be useful.

Life (noun) / alive (adj.) — like intelligence (noun) / intelligent (adj.) — is a category invented by humans. Unlike intelligence, we have managed to formulate slightly more rigorous definitions of what life is. However, particularly at the boundaries, we find that these definitions often fail. Viruses are a famous example that fall outside the traditional definition of life, while James Lovelock’s Gaia hypothesis [1] argues that Planet Earth satisfies the conditions required to call the planet itself a living organism. Especially now that we are looking for signs of life in space, we find that we may need to rethink our definitions, to be more inclusive of other forms of life [2] [3].

But aside from the definition of life itself, I’d like to draw attention to the term Biology.

Biology is the name of the field that studies life and living systems. This includes studying the ingredients required for systems to exhibit living behaviour, and to be considered alive, and how those components integrate and connect with each other. Amino Acids are by no definition considered to be alive. They’re organic molecules that act as building blocks for proteins, which in turn we believe to be an important component for what we call Life (related: discovery of self-replicating proteins [4]!). If I were to say “Amino Acids are studied in (or as part of) biology”, it should be quite clear to everybody reading or listening, that I am in no way claiming Amino Acids themselves to be alive. Furthermore, if you incorporate Amino Acids in a system, it doesn’t automatically make that system alive.

Artificial Intelligence is the name of the field that studies intelligence and intelligent systems. This includes studying the ingredients required for systems to exhibit intelligent behaviour, and to be considered intelligent, and how those components integrate and connect with each other. Computer vision, speech recognition, learning, even simple search and path-finding algorithms are studied (among many other things) as part of the field of Artificial Intelligence, as they are characteristics and behaviours which we have observed in intelligent systems (such as ourselves), and we believe would be beneficial to include when designing intelligent agents. Interestingly, if I were to say “X (e.g. a computer vision, speech recognition, learning, search or path-finding algorithm), is studied in (or as part of) artificial intelligence”, I’ve observed many people to misinterpret this as a claim that this automatically makes the system intelligent. This is a misunderstanding of the usage of the term Artificial Intelligence. If you incorporate X in a system (e.g. in your software), it doesn’t automatically make that system intelligent.

It definitely doesn’t make it *An* Artificial Intelligence.

(yes, even if you use neural networks or other forms of machine learning).

[1] https://en.wikipedia.org/wiki/Gaia_hypothesis

[2] https://www.astrobio.net/extreme-life/what-is-life/

[3] https://www.nasa.gov/vision/universe/starsgalaxies/life's_working_definition.html

[4] https://www.sciencealert.com/amyloid-protein-self-replication-abiogenesis-contrasts-rna-world

A semi-mathematical theory of human to human communication

Memo Akten — Tue, 23 Oct 2018 21:12:50 GMT

Part of a series of articles I’ll be posting over the next few days & weeks regarding thoughts from the past few years I’m only just getting round to writing about.

I believe many of the situations we face are generally of vastly complex nature and high dimensions, such that it is impossible for us to be able to see any situation in its entirety. All that we are able to perceive at any given time, is perhaps a tiny slice from a very specific perspective. And the best we can hope for, is to attempt to widen that slice by a tiny amount, and maybe even occasionally slightly shift our perspective. Unfortunately, contrary to popular belief, it’s not always being exposed to new ‘facts’ that helps us shift perspective — or at least, it’s definitely not enough. Instead we seem to need some kind of mental gymnastics to navigate to a new location inside our head, and gain these new view points. These mental gymnastics can often benefit from external nudging, and mental frameworks that act as structures around which we can build our thoughts. Traditionally this has been aided by natural language, evolved over thousands of years — but this isn’t always sufficient; and art for example is also critical to this end. I believe borrowing from other, more recent languages — such as computation — might also contribute to these mental frameworks.

In the spirit of the subject matter, doesn’t have to be read in any particular order:

Let’s assume for the sake of argument, that there is some kind of objective reality out there. This is of course, unreachable by us humans. The best we can hope to achieve, is to coat this objective reality in some kind of impenetrable shell of subjective human experience, and that is what we can try to perceive and learn about.

And then when it comes to communicating with each other about our experiences or ‘objective observations’, our default go to technology is natural language. Right now I’m using English, at times I’ll also often use Turkish. And already, just from two natural languages, I can see the differences in the way words have been assigned to the continuous landscape of human experience. Because language is discrete, it’s categorical. We take this immensely rich, high dimensional landscape of subjective reality, of meaning; and we literally butcher it. We carve it up into sections that we label with these inventions, technological artifacts that we call ‘words’. Surprisingly, this seems to work relatively well. Somehow we are able to communicate, exchange ideas, build culture and amass history. But I’m convinced it doesn’t work as well as we’d like to think. Perhaps related is the fact that a lot of this history that we’ve amassed consists of how we’ve destroyed other cultures, and why we thought it was appropriate to do so. And I’m even convinced that a lot of the social and political polarization which is on the rise today, is fueled by a failure in communication, perhaps more often than we might realize.

A mathematical theory of communication.

Some of you might be familiar with the above diagram. It is of course taken from the seminal 1948 paper in which Claude Shannon proposes what we now call ‘Information Theory’, one of the pillars upon which computer science — and in fact our entire digital civilization — is built upon. (The irony of talking about human-to-human communication, using a language invented for the human understanding of machine-to-machine communication, is not lost on me).

I don’t know about you, but I don’t think that I think in words. In my head, all of my thoughts feel to be more like amorphous clouds of ideas and concepts and connections and emotions and pictures and sounds and feelings and other things which I can’t really articulate. And they’re definitely not linear, but floating in a high dimensional space. And then, I squish those thoughts through what seems to me to be a kind of meat grinder; and out of the transmitter — i.e. my mouth — comes these words, linear, like a string of sausages. I find it really quite shocking how these words, just these flimsy little vibrations in the air, are somehow able to capture at least some of what I’m thinking. And once these words are out of my head and in the world, they are subjected to all kinds of noise and manipulation. I think we live in a time where we are unfortunately quite aware of the potential dangers that lurk in distorting words, whether it’s #fakenews or #alternativefacts etc. But I’m actually not going to dwell on this component too much, because I’d like to focus on other potential points of failure. So I’m going to assume for now, the situation where the words leaving my brain (i.e. that I’m typing on this page), are guaranteed to be reaching you without any interference (i.e. nobody hacks this website or the traffic reaching your computer). Because then we can agree that these sausages — my words — without any disturbance, do indeed reach your ears, or your eyes, or fingers, depending on mode of transmission. And then your brain explodes these words inside your head. And gives them meaning again.

To the universe, these words are meaningless, just arbitrary vibrations in the air, noise with a particular distribution. But to you, they somehow mean something. Interestingly though, I don’t know what exactly they mean to you. Even though we speak a common language, i.e. English; even though we share a vocabulary, and we (think we) share a dictionary, I don’t know how you’re unpacking what I’m saying. Because you only have access to these sausages to try and infer what I’m really thinking, you don’t have access to the thoughts inside my head. And that’s really a massive gap to fill, so you’re bringing everything you’ve ever seen, or heard or read to this decoding process. And unless I’ve read everything you’ve ever read, heard everything you’ve ever heard, lived everything you’ve ever lived; I am incapable of imagining what you’re taking out of what I’m saying right now. This is the difficulty of evaluating the performance of the listener’s (receiver) decoder.

And this is assuming that I have done a perfect job of squishing my thoughts through that meat grinder, to create a perfect string of sausages that truly represent what I’m really thinking inside my head. And that is highly unlikely. It’s a lot more likely, that these sausages are actually a very basic, crude, lossy representation of what I’m really thinking. And what really sucks is, while you only have these sausages to try and infer what I’m really thinking, when I look at those exact same sausages that came out of my mouth, I see the full picture! because I have access to the original thoughts that gave rise to them. So it can be quite difficult for me to see how you might ‘misunderstand’ me, to see how confusing my choice of words might be. This is the difficulty of evaluating the performance one’s own encoder.

So I think it’s good to assume the following, especially before resorting to quarreling:

When you say something, it is quite probable that the words you chose might not actually be conveying exactly what you intended to convey — either not conveying what you wanted to convey, and/or conveying something that you did not want to convey (poor encoding).
When a listener hears you say something, it is quite probable that they will unpack what you said in a way that is different to what you intended (poor decoding).
When somebody else says something, they might not be aware that it is quite probable that that the words they chose might not actually be conveying exactly what they intended to convey (poor encoding)
When you hear somebody else say something, they might not be aware that it is quite probable that you might unpack what they said in a way that is different to what they intended (poor decoding).

I’m not suggesting it’s impossible to communicate precisely with this clunky technology that we call natural language. Arguably, a poet is somebody who has mastered the art of navigating the rich, continuous, high dimensional space of subjective reality; armed only with these crude, low resolution, labels. A poet might not naively pick the first words that appear closest to the points of meaning that they seek. Instead, they are able to find very specific configurations of words, such that when these particular words are used together in a very particular order, they are able to transport you to very precise locations in that rich landscape of meaning, which might be otherwise inaccessible through more simplistic use of the same language*.

And this is one of the things that art tries to do too — through non verbal, alternative modes of communication — to take you to peculiar, difficult to reach locations in that rich, high dimensional space of meaning and subjective experience, often found only in the cracks between words.

*NB. This reminds me of when I first saw the classic PC DOS demo ATOM.exe (back in the age of dinosaurs), playing ‘analog’ audio through the 1-bit PC Speaker. It blew my mind, how could it be possible to get such sounds out of a speaker capable only of ‘on’ or ‘off’. And that’s how I found out about the clever hack known as pulse-width modulation, which gives you finer apparent control, resolution and precision than the system seems to be capable of by design.

High dimensional thinking

Memo Akten — Sun, 21 Oct 2018 07:34:18 GMT

Part of a series of articles I’ll be posting over the next few days & weeks regarding thoughts from the past few years I’m only just getting round to writing about.

In the spirit of the subject matter, doesn’t have to be read in any particular order:

(Conversations with Timothy Morton and Kenric Mcdowell were particularly influential in shaping these thoughts)

https://medium.com/media/c894e739bc4aad96fae2040649848db5/href

“A hyperobject is a name I invented for something that is so vastly distributed in time and space, relative to the observer, that we might not think it’s even an object at all. It’s good to have a word for things that are now only too thinkable, if not totally visible — global warming, radiation, the biosphere… They are huge, but they can be defeated and dismantled. Hyperobjects are very good to think with… we can figure out how to work with them and transform society in order to accommodate their reality in ways that are beneficial to humans and nonhumans alike.” — Timothy Morton. “Hyperobjects and creativity”

Morton chose the term hyper-object drawing inspiration from (one of my favorite musical artists) Björk’s wonderful song “Hyperballad” (with a music video directed by one of my favorite popular video directors Michel Gondry) — an “exploded version of a love song … evoking something that included her, but that was bigger than her, but that wasn’t more than her … something physically bigger, but ontologically smaller…”. The hyper prefix is incredibly appropriate, in multiple perspectives (in a rather meta-eloquently way).

https://medium.com/media/09690a4434d7643a23f759fdc306ff76/href

In mathematics — and particularly computational sciences (e.g. physics, biology, data science etc) the prefix ‘hyper’ is often used to refer to ‘high dimensional’. E.g. a hypercube is the generalization of a three-dimensional cube to any number of dimensions (typically four dimensions if not stated otherwise). Similarly, a 10D hypersphere is like a sphere, not in 3 dimensions, but in 10 dimensions (i.e. defined as the loci of points in ten dimensions equidistant to a single, specific point which we would call the ‘center’ of the hypersphere; while we’d call that distance, the ‘radius’).

Building an intuition for high dimensional spaces — or even grasping what the phrase ‘high dimensional space’ itself actually means — is initially tricky for all of us, even though implicitly we are able to mentally navigate high dimensional spaces, despite not being explicitly aware of it. I wrote a very lengthy post on this here, but for now let me give a very brief example.

Imagine I were to dump a bunch of toys onto a table, toys of animals. And I ask you to organize these animals side by side, in a straight line. “Organize them? But by which characteristic? Which feature?” you might ask, and that would be a very good question. Let’s say that we agree you are to organize the animals by ‘weight’. So you lay them out left to right: cockroach, tarantula, goldfish,crow, hedgehog, cat, fox, dog, horse, elephant, whale.

Animals sorted left to right increasing in ‘weight’ (images not to scale!)

Now I ask, instead of organizing the animals in a single straight line, feel free to use the entire surface of the table. Instead of one dimension, organize the animals in two dimensions. We agree to keep one of the axes as is, representing ‘weight’, and for the second axis we decide on ‘how common it is to domesticate this animal or keep as a pet’ (from here on I’ll refer to this as ‘petness’). Without changing the animals’ horizontal position, you slide them up and down across the table surface based on this ‘petness’ feature. ‘Horse, goldfish, cat, dog’ are quite near the top; ‘cockroach, fox, whale’ are at or very near the bottom; ‘tarantula, hedgehog, crow, elephant’ are maybe somewhere in between.

Animals sorted left to right increasing in ‘weight’, and bottom to top increasing in ‘petness’ (subjective opinion, may not be accurate!)

An interesting observation we can make is, this has established some new relationships. If we were to look at ‘cockroach’ and ‘whale’, on the ‘weight’ (i.e. horizontal) axis, they are on opposite ends of the table, really far apart. However, they have something in common now, the distance between them is very short along this new (vertical, i.e. ‘petness’) axes. They are both very unlikely to be kept as pets. Likewise with ‘horse’ and ‘goldfish’, on the ‘weight’ (i.e. horizontal) axis they are relatively far apart (not as far as ‘cockroach’ and ‘whale’ though), but very close on the ‘petness’ (i.e.vertical) axis (both quite likely to be pets or domesticated).

We have created a two dimensional plot, a two dimensional object which captures the relationships between all of our animals’ and their ‘weight’ and ‘petness’ characteristics.

But let’s not stop there, let’s add a third axis. We agree that this new 3rd dimension will represent ‘hairiness’. We attach a piece of string to each toy animal such that we can individually adjust their height from the table. The higher the animals is off the table, the hairier it is. (Or if you’re trying to visualize this from the 2D plot image above, you can think of pulling the hairier animals out of the screen).

‘Tarantula, cat, fox, dog’ etc are all nearer the ‘hairy’ end (lifted high off the table, or pulled far from the screen); ‘horse, hedgehog’ are perhaps somewhere in between; ‘cockroach, crow, elephant, goldfish, whale’ nearer the bottom.

Again we have established more new relationships. ‘Goldfish’ and ‘elephant’ were far apart on both ‘weight’ and ‘petness’ axes previously, but now along this new ‘hairiness’ dimension they are a little bit closer. ‘Hedgehog’ and ‘crow’ were quite close on both previous dimensions, now they will be further apart along this new dimension.

Ultimately the point of all of this is not only to see if things are close or far, but to be able to represent the relationships between many entities across many features in a single structure. Our animals hanging in this 3D space (with the axes representing ‘weight’, ‘petness’ and ‘hairiness’) represents a 3D plot. We can throw in thousands more animals: e.g. bat, chicken, worm, slug, octopus, eel, iguana etc. and try to visualize roughly where each animal would land in relation to the other animals on the various axes. The key takeaway is, we can think of the entire collection of animals in space as a single object. Each animal is a single data point in 3D space, and the collection of these data points carves out a 3D object. This object may look like a point cloud, it may look like a sheet of paper folded intricately and floating in space; we may refer to it as a manifold; but whatever it looks like, in its shape is encoded information regarding the entire dataset, and how everything relates to everything else across those chosen features.

It should be noted of course, three features are really not enough to capture the complexity of how these animals relate to each other or to us. So let’s add more features, let’s add a new axis! However, we have run out of physical, *spatial* axes, i.e. real world directions across which to distribute our animals. But we can still virtually organize our animals across each of these new features individually, and we just label each feature axis with a number. E.g. let the 4th dimension represent ‘natural habitat’, let the 5th dimension represent ‘average lifespan’, 6th dimension ‘intelligence’, 7th dimension ‘curiosity’, 8th ‘how dangerous they are’, 9th ‘how tasty’, 10th ‘cuteness’ etc. We won’t be able to spatially visualize all of these features simultaneously, but that shouldn’t stop us from adding them individually.

Let’s add a thousand, a million, a billion features (and corresponding axes). Each animal is now a data point in this billion dimensional feature space. Lets add a million animals. Now the entire collection of a million animals — the set of a million points — carves out a shape in this billion dimensional space. This is not an object anymore, it is a billion dimensional hyperobject, and in its shape it encodes all of the complex relationships between a million objects, across a billion features. Unfortunately we cannot visualize it (yet), but there it is, as real as each animal itself, as real as the simplified prototype 3D version hanging above our table with only three features which we can easily visualize, except this one has a million points in a billion dimensions.

But here’s where it gets interesting. It may seem like a hopelessly pointless, lost cause to even attempt to be able to do anything remotely useful with such an insanely high dimensional, complex geometry such as this hyperobject. But it turns out we’ve been able to develop so many tools and methodologies to manage exactly this. There are plenty of methods for decomposing, analyzing and processing shape, finding correlations, extracting principle components, effectively projecting or reducing to lower dimensions, visualizing etc. Each method does have its own weaknesses (they all incur some kind of loss after all, to manage to compress the information to a reduced complexity that our primate minds can comprehend), but they also provide unique insights and perspectives. And especially used collectively, they can allow us to navigate, parse and try to understand this otherwise immensely ungraspable structure.

I’m not suggesting that we can solve global warming and the sixth mass extinction by applying a principle component analysis or t-distributed stochastic neighbour embedding on every feature of every component that is related to climate change. All that I’m suggesting is that the monstrous enormousness of a complex situation can tend to be rather overwhelming and result in two rather unproductive outcomes.

In cases where the negative impact of the problem seems to be less immediately personal (e.g. global warming, pollution of the oceans, ecosystems etc.), one might tend to feel an apparent impossibility of having any chance to start talking about and tackle the issues, which in turn encourages submission to an allegedly inevitable fate, sinking into a black hole of despair from which no light rays of hope can escape (more on this later in a separate post).

On the flip side, many high dimensional complex problems tend be vastly over-simplified, especially when the negative impacts of the problem are more immediate, personal, and instant action is desired (e.g. issues around politics, social trends and movements etc.). This is most probably in an effort to reduce the unmanageablely complex problems into simpler dimensions and actionable scale (and perhaps isn’t helped by the bite-size attention spans we’re developing in the age of the social media instant sharing attention economy).

But these oversimplifications can lead to unnecessarily severe polarization and social divides. Sometimes things just aren’t as simple as ‘right’ vs ‘wrong’, or ‘guilty’ vs ‘innocent’ (and obviously not ‘left’ vs ‘right’). You might rightly point out that these examples aren’t actually relating to lack of ‘dimensions’, but is instead an issue of ‘binary’ vs ‘non-binary or continuous’. So let’s ditch the binary, and put these on a so-called ‘spectrum’. However, in my opinion even a continuous gradient is still unfortunately so vastly insufficient and incapable of capturing the complexity of many of the situations we find ourselves in. Because a ‘spectrum’ is by default (at least by most people’s understanding and treatment of the word) a one dimensional continuous space. Placing things ‘on a spectrum’ is equivalent to trying to visualize the relationships between them purely by placing them side by side on a table. Instead what might be more fruitful, is to be thinking about them in high dimensions, where things might be far apart and ‘opposite ends of the spectrum’ along one axis, but really closely related ‘and on the same side’ along another axis. ‘Right vs wrong’, ‘blame’, ‘political leaning’ etc. are too complex to be limited to a single dimension — even if non-binary and continuous. They deserve to be treated as distributions in high dimensional space (e.g. the same can be said about gender identity, see this study by Blaise Agüera y Arcas).

Of course when it comes to taking action, we are faced with the difficult task of eventually committing to a binary (or categorical) decision of ‘take action A’ vs ‘take action B’ etc. But if at least we can keep our thoughts and decision making processes, and conversations and debates with each other, in these high dimensional distributions, then hopefully we can maybe come out with less hate towards each other.

If we lack the language or mental frameworks to discuss or think about these complexities, high dimensional relationships, and effective strategies for dimensionality reduction; I fear we may oversimplify to the extent where we are no longer acting with sound reasoning and we’re unnecessarily creating enemies amongst ourselves due to miscommunications. And this isn’t just something that ‘they’ do, but ‘we’ do too.

To be continued…

Illustrations from openclipart.org:

Optimising for beauty - the erasure of heterogeneity and an appeal for a Bayesian world view

Memo Akten — Tue, 25 Sep 2018 07:46:00 GMT

Part of a series of articles I’ll be posting over the next few days & weeks regarding thoughts from the past few years I’m only just getting round to writing about.

In the spirit of the subject matter, doesn’t have to be read in any particular order:

https://medium.com/media/ac0f1f82dc5789db415d9dfccf05fa2a/href

Short version

An artificial neural network dreams up new faces whilst it’s training on a well-known dataset of thousands of celebrities. Every face seen here is fictional, imagined by the neural network based on what it’s seeing and learning. But what is it learning?

The politics of this dataset, who’s in it, who isn’t in it, how it’s collected, how it’s used and the consequences of this, is in itself a crucial topic, but not the subject of this work.

Indeed the dataset may not be representative of the diversity of the wider population. However, the learning algorithm itself provides a further layer of homogenisation. In these images produced by the neural network, whatever diversity was present in the dataset is mostly lost. All variety in shape, detail, texture, individuality and blemishes are all erased. They are smoothed out with the most common attributes dominating the results.

The network is learning an idealised sense of hyper-real beauty, a race of ‘perfect’, homogeneous specimens.

This is not a behaviour that is explicitly programmed in, it is an inherent property of the learning algorithm used to train the neural network. And while there are recent algorithms which specialise in generating more photo-realistic images (especially faces), the algorithm used here is one of the most widespread algorithms used in Machine Learning and Statistical Inference — Maximum Likelihood Estimation (MLE).

However, MLE is unable to deal with uncertainty. It’s unable to deal with the possibility that a less likely, a less common hypothesis might actually be valid.

MLE is binary. It has no room for alternative hypotheses. Instead, the hypothesis with the highest apparent likelihood is assumed to be unequivocally true.

MLE commits to this dominant truth, and everything else is irrelevant, incorrect and ignored. Any variations, outliers, or blemishes are erased; they’re blurred and bent to conform to this one dominant truth, this binary world view.

Perhaps not unlike the increasingly polarising and binary discourse in our increasingly divided societies.

Long version

The politics of this dataset, who’s in it, who isn’t in it, how it’s collected, how it’s used and the consequences of this, is in itself a crucial topic, but not the subject of this work.

When I first started seeing these kinds of results, I was fascinated by how everything was so blurry and smoothed out. Indeed the dataset may not be representative of the diversity of the wider population. However, the learning algorithm itself provides a further layer of homogenisation. In these images produced by the neural network, whatever diversity was present in the dataset is mostly lost. All variety in shape, detail, texture, individuality and blemishes are all erased. They are smoothed out with the most common attributes dominating the results.

The network is learning an idealised sense of hyper-real beauty, a race of ‘perfect’, homogeneous specimens.

That means — quite intuitively speaking — given a set of observations (i.e. data points), out of all possible hypotheses, find the hypothesis that has the maximum likelihood of giving rise to those observations. Or in fewer words: given some data, find the hypothesis which is most likely to have produced that data.

It makes a lot of sense right?

But it has some shortcomings.

Imagine we find a coin on the street. And we’d like to know whether it’s a fair coin, or weighted (i.e. ‘biased’) towards one side, possibly favoring heads or tails. We have no way of determining this by physically examining the coin itself. So instead, we decide to conduct an experiment. We flip the coin ten times. And let’s say we get 7 heads and 3 tails.

A maximum likelihood approach would lead one to conclude that the most likely hypothesis that can give rise to these observations, is that the coin is biased in favor of heads. In fact, such a ‘frequentist’ might conclude that the coin is biased 7:3 in favor of heads (with a 26.7% probability of throwing 7 heads). An extreme frequentist might even commit to that hypothesis, unable to consider the possibility that actually there are many other alternative hypotheses, one of which — though less likely — might actually be the correct one. E.g. it’s very possible that actually the coin is indeed a fair coin, and simply by chance we threw 7 heads (for a fair coin, the probability of this is 11.7%, not negligible at all). Or maybe, the coin is biased only 6:4 in favor of heads (probability of throwing 7 heads is then 21.5%, still very likely). In fact the coin might have any kind of bias, even in favor of tails. In this case it is very unlikely that we would have thrown 7 heads, but it’s not impossible.

Some of you might be critical of this example, citing that 10 throws is far too small a sample size to infer the fairness of the coin. “Throw it 100 times, or 1000 times” you might say. Heck, let’s throw it a million times.

If we were to get 700,000 heads out of 1 million throws, then can we be sure that the coin is biased 7:3 in favor of heads?

For practical reasons, we might be inclined to assume so. We have indeed quite radically increased our confidence in that particular hypothesis. But again, that does not eradicate the possibility that another hypothesis might actually be the correct one.

Evidence is not proof. It is merely evidence. I.e. a set of observations that increase or decrease the likelihood of — and our confidence in — some hypotheses relative to others.

MLE is unable to deal with uncertainty. It’s unable to deal with the possibility that a less likely, a less common hypothesis might actually be valid.

MLE is binary. It has no room for alternative hypotheses. Instead, the hypothesis with the highest apparent likelihood is assumed to be unequivocally true.

So are there alternatives?

At the opposite end of the spectrum, one could maintain a distribution of beliefs over all possible hypotheses, with varying levels of confidence (or uncertainty) assigned to each hypothesis. These levels of confidence might range from “I’m really very certain of this” (e.g. the sun will rise again tomorrow from the east) to “I’m really very certain this is wrong” (e.g. I will let go of this book and it will stay floating in the air) — and everywhere in between. Applying this form of thinking to the coin example, we would not assume absolute truth in any single hypothesis regarding how the coin is biased. Instead, we would calculate likelihoods for every possible scenario, based on our observations. If need be, we can also incorporate a prior belief into this distribution. E.g. since most coins are not biased, we’d be inclined to assume that this coin too is not biased, but nevertheless, as we make observations and gather new evidence, we will update this distribution, and adjust our confidences and uncertainties for every possible value of coin bias.

And most critically, when making decisions or predictions, we don’t act assuming a single hypothesis to be true, but instead we consider every possible outcome for every possible hypothesis of coin bias, weighted by the likelihood of each hypothesis. (Those interested in this line of thinking can look up Bayesian logic or statistics).

But, as I’ve mentioned in my other works and texts, my main interest is not necessarily these algorithms themselves. In a real world situation, no fool would settle on a maximum likelihood solution with only 10 samples (one would hope!).

My main interest is in using machines that learn as a reflection on ourselves, and how we navigate our world, how we learn and 'understand', and ultimately how we make decisions and take actions.

I’m concerned we’re losing the ability to safely consider and parse multiple views within our social groups, let alone within our own minds. I’m concerned we’re losing the ability to recognize and navigate the complexities of situations which may require acknowledging — or even understanding — the existence of lines of reasoning that may lead to different views than our own; maybe even radically different, opposing views. I’m concerned that ignoring these opposing lines of reasoning, and pretending that they don’t exist, might be more damaging in the long run; compared to acknowledging their existence and trying to identify at what point(s) our lines of reasoning are diverging, and how, and why, and trying to tackle these more specific issues at point; whether they are due to what we consider to be incorrect premises, flawed logic, differing priorities or desires.

I’m concerned we’re becoming more and more polarized and divided on so many different points on so many different topics. Views, opinions and discourse in general seems to be becoming more binary, with no room to consider multiple or opposing views. I’m concerned we want to ignore the messy complexities of our ugly, entangled world; to erase the blemishes; and commit unequivocally to what seems (to us) to be the most likely truth — the one absolute truth, in a simple, black and white world of unquestionable rights and wrongs.

Or maybe that’s just my over-simplified view of it.

Acknowledgements

Made using the ‘CelebA’ dataset and Variational Autoencoders

Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang, “Deep learning face attributes in the wild,” Proceedings of the IEEE International Conference on Computer Vision 2015 Inter, 3730–3738 (2015).

Diederik P Kingma and Max Welling, “Auto-Encoding Variational Bayes,” arXiv preprint arXiv:1312.6114, (2013).

Deep meditations: a meaningful exploration of inner self, a controlled navigation of latent space.

Memo Akten — Mon, 30 Jul 2018 22:44:38 GMT

It’s all about the Journey.

This is an article accompanying the paper with the same name which I’ll present at the 2nd Workshop on Machine Learning for Creativity and Design at the 32nd Conference on Neural Information Processing Systems (NeurIPS) 2018, where I discuss some of the underlying practical and technical methods (not so much the conceptual or artistic motivations) used in the creation of my “Deep Meditations” series of works. This article includes most of the information from the paper, expanded with some additional information.

paper: https://nips2018creativity.github.io/doc/Deep_Meditations.pdf

code: https://github.com/memo/py-msa-kdenlive

https://medium.com/media/5b9661df4710b8f7833c9b9f7314a192/href

Abstract

We introduce a method which allows users to creatively explore and navigate the vast latent spaces of deep generative models such as Generative Adversarial Networks. Specifically, we focus on enabling users to discover and design interesting trajectories in these high dimensional spaces, to construct stories, and produce time-based media such as videos, where the most crucial aspect is providing meaningful control over narrative. Our goal is to encourage and aid the use of deep generative models as a medium for creative expression and story telling with meaningful human control. Our method is analogous to traditional video production pipelines in that we use a conventional non-linear video editor with proxy clips, and conform with arrays of latent space vectors.

Background

‘Generative’ Models

One of the recent major advances we’ve seen in the so-called field of ‘Artificial Intelligence’ (AI) is in ‘generative models’. A nice summary (if slightly out of date due to the insane speed at which the field is moving) can be found on the openai blog.

First I’d like to take a slight diversion to underline the difference between two different definitions of ‘generative’: i) In the arts or other creative fields — I’ll refer to this as generative content. vs ii) In the context of Machine Learning (ML), or more broadly speaking, in statistics — I’ll refer to this as a (statistical) generative model.

In the arts, creative fields, computer graphics, computational design etc. ‘generative content’ refers to content made working with rule-based or (semi or fully) autonomous systems (i.e. which can also be thought of as governed by rules). From a computational perspective we might quickly think of early generative art pioneers such as Manfred Mohr or Vera Molnar; or the types of works typically made with creative programming toolkits such as openframeworks, processing, vvvv, houdini, touchdesigner, and many many more. However, ‘generative’ can also be ‘analog’ (yet still algorithmic), e.g. Lillian Schwartz, Sol LeWitt, Steve Reich, Terry Riley are classic oft-quoted examples. (Personally I prefer the term ‘procedural’ or ‘algorithmic’ over ‘generative’ in this context).

In ML/statistics however, a ‘generative model’ refers to a system which captures the distribution of some observations (e.g. ‘training data’), in the hope that ideally it is able to model the underlying dynamic processes that gave rise to those observations. So when we sample from this distribution, we are effectively executing the function which models that dynamic process; thus, this generates new data that should hopefully be structurally similar to the observations. This also means that when we manipulate parameters of the generative model, we are manipulating parameters of the dynamic process, and thus generating varying outputs that are consistent with the world from which the training data came from. (This is of course, an ideal case scenario. It’s important to remember that “All models are wrong, but some are useful”, and “…the practical question is, how wrong do they have to be to not be useful.”- George E. Box.)

These are two very different, quite unrelated definitions of ‘generative’. However, we can loosely relate them by noting that sampling from ‘statistical generative models’ can almost always be considered ‘generative content’ from the ‘generative art and design’ point of view. But ‘generative content’ is not by default ‘generative’ from a statistical point of view. Even using ML does not guarantee statistical generative-ness. E.g. Deepdream does not use a statistical generative model or process (there is no distribution of data, it’s just gradient ascent), but it is generative content.

My use of the term ‘generative’ in this article is the ML/statistics definition (and most likely, any paper or article you read on generative models in ML, particularly if written by someone with a background in statistics or ML, will also be using this definition. Whereas any paper or article you read written by someone with a background in art, computational design, computer graphics etc will most likely be using the ‘generative content’ aka ‘procedural’ definition).

‘Deep’ Learning (in this context)

Some of the major advances that have been happening recently in generative models is in deep generative models, i.e. using Deep Artificial Neural Networks, i.e. using Deep Learning (DL) — the current dominant paradigm in AI research.

Sometimes people (quite rightly) point out weaknesses of DL. A common one is that these systems often require a lot (e.g. thousands, hundreds of thousands, even millions) of examples of training data. While this is absolutely true, I find it difficult to see this as a weakness of DL per se. Rather, I see this as a weakness of the state of the field of AI research in general. Because according to my understanding, this is exactly the whole point of DL: to extract meaningful information from a vast sea of humanly-unmanagable big data. (Of course it’s no coincidence — in our current times of extreme surveillance capitalism — that the particular sub-field of AI which is seeing the most progress, and financial investment, is the particular field which deals with extracting information from big data. mini, short, long versions).

Referring to DL’s dependency on big data as a weakness, seems analogous to referring to the big size and heaviness of a sledgehammer as its weakness.

A sledgehammer. Big and heavy.

Whereas perhaps that is exactly what defines the sledgehammer and what it’s for. But luckily, we also have hammers of all sizes, suitable for different jobs.

Screenshot of Andy Dufresne (played by Tim Robbins) using his rock hammer in The Shawshank Redemption (source: Castle Rock Entertainment. https://small-change.uq.edu.au/blog/2016/03/hammer-shaped-university)

In fact we have all kinds of tools that aren’t even hammers.

https://www.machines4u.com.au/mag/100-years-ago-today-antique-woodworking-tools/

It’s undoubtedly important to note that right now DL is a big heavy weapon best suited for dealing with vast amounts of data, for smashing open massive boulders into millions of pieces, to see what’s hidden inside. And for optimum results, it’s best combined with other tools specializing in different functions for different purposes.

Control, Relinquishing control, Meaningful control

(This section was too epic, it will become its own post eventually, but ultimately a section on it belongs here).

Introduction

So the progress we have been seeing in the past few years that I’m referring to, has been in deep generative models, i.e. generative neural networks capable of training on vast datasets, to learn hierarchies of representations. And in this particular paper, I’ll be focusing on image based models: deep generative models that are able to learn to produce high resolution, realistic (or not) and novel images (though the techniques mentioned here can be adapted to other domains, in fact the audio in all of these videos — including “When I’m stuck with a day that’s grey” - was also created by a similar generative model, that will be the subject of an upcoming article).

The goal of these generative models is to learn the distribution of the training data such that we can sample from the distribution to generate new images. We refer to this distribution as the latent space of the model. It is usually high dimensional, though much lower than the dimensions of the raw medium. E.g. When dealing with full colour (e.g. RGB) images with 1024x1024 pixel resolution, we are dealing with roughly 3 million dimensions (i.e. features — 1024x1024x3), whereas we might use a latent space of only 512 dimensions (a compression ratio of 6144:1!).

We denote vectors (i.e. points, coordinates) in this latent space with z, and any point in the latent space can be decoded into pixel space via a special decoder function. These days it’s quite common for this decoder to be a deep Convolutional Neural Network (CNN), that e.g. takes a 512 dimensional z vector and outputs a ~3 million dimensional vector (i.e. a 1024x1024 RGB image). So in short, any point in this space can be decoded to a unique image. (A much longer introductory article on high dimensional latent spaces can be found here).

As mentioned in the openai blog linked above, there are a number of architectures and methods which are very good at this task. Since that post in 2016, more recent developments include Progressive Growing of GANs, Glow and Large Scale GAN (aka BigGAN). The research and techniques I discuss below are agnostic of the architecture and training method, I focus purely on how to explore and construct meaningful journeys in latent space — no matter how the space was constructed.

Depending on our goals, there are many different ways in which we could explore such a latent space. At one end of the spectrum, we could use a fully automated search whereby points of interest are found algorithmically based on various heuristics, such as novelty. At the other end of the spectrum, we could provide a target image and retrieve a corresponding z vector. Some architectures — e.g. Variational Auto-Encoders (VAE) or Glow — have an encoder which does exactly this, while other architectures — e.g. vanilla Generative Adversarial Networks (GAN) — do not. However it is still possible either by manually adding an encoder, such as in the case of a VAE/GAN, or using gradient based optimization techniques to search for a corresponding latent vector. Other techniques for exploring latent spaces include systematic visualizations of various interpolations between key points in space. A very recent example uses the metaphor of breeding, by mixing different images together (in latent space) to find new images.

Our paper is not an attempt to replace any of these methods for discovering interesting points in latent space. Rather, we can incorporate them into our process to enable users to discover and design interesting trajectories in latent space, to construct stories, and produce sequences with meaningful control over narrative. Our main contribution is a workflow based on familiar tools such as traditional non linear video editing software, operating on proxy video clips, and conforming the edit with arrays of latent space vectors.

Challenges

We face a number of challenges to overcome in crafting trajectories in such latent spaces:

The space is very large and high dimensional (e.g. 128, 512, 2048 etc.).
It is not distributed ‘evenly’, or as one might expect or desire. I.e. if we were to sample uniformly across the space, we might end up with many more images of one type over another (e.g. in the case of our test model, flowers seem to occupy a large portion of the space, probably due to a higher representation in the training data).
As a result of the uneven distribution, interpolating from latent vector z_A to z_B at a constant speed might produce images changing with a perceptually variable speed.
It is very difficult to anticipate trajectories in high dimensions. E.g. interpolating from z_A to z_B might pass through points z_X and z_Y, which may be undesirable.
The mass of the distribution is concentrated in the shell of a hypersphere.
The latent space changes with subsequent training iterations.

We discuss these all in more detail below.

Method

Background

It is common practice in video editing and post-production pipelines to perform an offline edit on a set of proxy clips (e.g. small, easily manageable, low quality video) and later conform the edit by (mostly automatically) applying the edit to online material (e.g. high quality video) — (NB this has nothing to do with being online or offline with regards to the internet :), but is based on much older terminology related to automated machines operating on reels of footage. Read more).

This conforming process is usually performed by transferring information regarding the edit to an online system (i.e. with access to the full quality rushes) using a file such as an Edit Decision List (EDL) which contains reel and timecode information as to where each video clip can be found in order to perform the final cut.

Summary

Our method is analogous to this workflow. Our offline clips are the video outputs from a generative model for given z-sequences. Our online clips are numpy arrays of corresponding z-sequences.

In other words, we define (z-sequence, video) pairs. A z-sequence is a numpy array (saved to disk) containing a sequence of z-vectors, i.e. a trajectory in latent space. A video is a normal QuickTime file where each frame is the image output from the model decoding the corresponding z vector from the corresponding z-sequence.

A high level summary of the process is as follows:

We can edit the videos in a conventional Non Linear Video Editor (NLE) such as Adobe Premiere, Apple Final Cut, Avid Media Composer or Kdenlive (my weapon of choice, discussed more in the appendix).
Run a custom script to conform the edit with the corresponding numpy arrays containing the z-sequences (i.e. apply the video edit from the NLE, onto the corresponding numpy arrays).
Feed the resulting conformed z-sequence into the model for final output.

It’s best to perform the edit on keyframes, i.e. temporally sparse destination points in latent space, so that after the conform, we can interpolate between them for smooth, continuous output.

https://medium.com/media/25940ac98481414ecfa466a7d3c5a36a/href

Process

Below is an example process in more detail. This is merely a suggestion, as many different workflows could work, however this is the process I used to create the videos on this page.

In the following context the phrase ‘render a z-sequence’ refers to i) saving the z-sequence to disk as a numpy array, and ii) decoding the z-sequence with many snapshots of the model (from 28 different training iterations, spaced 1000 iterations apart) and saving out a video where the output of each snapshot is tiled into a grid (e.g. 7x4) and labelled with the corresponding training iteration. Rendering multiple snapshots in a grid on a single frame in this way gives us an overview of how the latent space has evolved across training iterations, and allows us to easily see and select the most aesthetically desirable snapshot(s) (we go deeper into the motivations behind this in the Snapshots across time Appendix).

Take many (e.g. hundreds or thousands of) unbiased (i.e. totally ‘random’) samples in latent space and render. This produces a video (and corresponding z-sequence) where each frame is an entirely different ‘random’ image. This gives us an idea of what the model has learnt, and how it is distributed. It also gives us an idea of how the distribution changes across subsequent training iterations, and which snapshots provide more aesthetically desirable images.
Edit the video in a NLE to remove undesirable (i.e. ‘bad’) images or to bias (or de-bias) the distribution (e.g. remove some frames containing flowers if there are too many, or duplicate frames containing bacteria if there’s not enough of them etc.)
Run the script to conform the edit with the original z-sequence and re-render. This produces a new video (and corresponding z-sequence) where each frame is still an entirely different ‘random’ image, but which has hopefully a desired distribution (i.e. no ‘bad’ images, and a desirable balance between different images).
Repeat steps 2–3 until we happy with the distribution (one or two rounds is usually enough). Optionally apply varying amounts of noise in z to explore neighborhoods of selected frames (e.g. to look for and include more images of bacteria, with more variation).
Load the final edited z-sequence (with desired distribution) and render many (e.g. tens or hundreds of) short journeys interpolating between two or three random (or hand picked) z (selected from the z-sequence). This produces tens or hundreds of short videos (and corresponding z-sequences) that contain smooth, slow interpolations between two or three keyframes where the keyframes are chosen from our preferred distribution. This gives us an idea on how the model transitions between selected images. E.g. The shortest path from a mountain to a face might have to go through buildings, which might not be desirable, but inserting a flower in between might avoid the buildings and look nicer — both aesthetically and conceptually.
Repeat step 5, honing in on journeys which seem promising, optionally applying varying amounts of noise in z to explore neighborhoods of selected frames and journeys.

The above steps produce an arsenal of short snippets of video clips.

https://medium.com/media/9d830e0a615130a3696e6447c2a500a9/href https://medium.com/media/6d22bde7824a75c57fea74ebb5c917bc/href https://medium.com/media/6ccc76a2eeb5cf5848f82d22af373863/href https://medium.com/media/4f5cc38e33bc096063e7760523c939f0/href

These can then be further edited and joined in a NLE, and then conformed with the corresponding z-sequences to produce a final video. We have produced many hours worth of carefully constructed stories using this method.

https://medium.com/media/5e911c1af80443bd1421bccdb0bea77b/href

Appendix

Model architecture and data

We applied the approach mentioned in this paper on a number of different models and architectures, however the primary test case we refer to (and from which we also show the results) is a GAN (specifically, a Progressively Grown GAN) trained on over 100,000 images scraped from the photo sharing website flickr. The dataset is very diverse and includes images tagged with: art, cosmos, everything, faith, flower, god, landscape, life, love, micro, macro, bacteria, mountains, nature, nebula, galaxy, ritual, sky, underwater, marinelife, waves, ocean, worship and more. We include three thousand images from each category and train the network with no classification labels. Given such a diverse dataset without any labels, the network is forced to try and organize its distribution based purely on aesthetics, without any semantic information. Thus in this high dimensional latent space we find directions allowing us to seamlessly morph from swarms of bacteria to clouds of nebula, oceanic waves to mountains, flowers to sunsets, blood cells to technical illustrations etc. Most interestingly, we can perform these transformations across categories while maintaining overall composition and form.

Video editing and conforming the edit

Example of an editing z-sequenes in Kdenlive

I use the opensource Non Linear Video Editor Kdenlive on Ubuntu. Unfortunately this editor lacks support for exporting the industry standard EDL. However, Kdenlive’s native project file format is XML based. This allows us to write a python based parser to load the project file, inspect the edits, retrieve the corresponding numpy z-sequences and conform by performing the same edits on them and exporting a new z-sequence. At this point the conform is very simple and only supports basic operations such as trimming, cutting and joining, and does not include cross-fades or other more advanced features or transitions. However, to implement such additional features is relatively trivial and left as future work (e.g. a cross-fade between two images in the NLE can be thought of as an interpolation between the two corresponding points in latent space).

Interpolation

We use generative models with high dimensional (512D) multivariate Gaussian distributed latent spaces. Because these distributions are concentrated around the surface of a hypersphere, when we wish to interpolate between points in this space, we have to make sure that our trajectory stays within the distribution. A common solution is to use spherical instead of linear interpolation. However this produces visibly noticeable discontinuities in the movement of the output images due to sudden changes in speed and direction. The images below are two different z trajectories, i.e. journeys in latent space, created by interpolating between a number of arbitrary keyframes. In both images, a single pixel wide vertical slice represents a single z vector, and time flows left to right.

Fig C1. z sequence using spherical interpolation

Fig C2. z sequence using physical interpolation

Figure C1 visualizes the results of spherical interpolation. We can see notch-like vertical artifacts that happen when the interpolation reaches its destination and we set a new target, creating a sudden change in speed and direction. To remedy this we introduce a simple physics based dynamical system, the results of which can be seen in Figure C2. In the high dimensional latent space we create a particle connected to both the surface of the hypersphere and the next destination point with damped springs. This ensures that the particle stays close to the distribution, but also moves without discontinuities at keyframes.

A method I’ve been using more recently is borrowing from differential geometry and Riemannian manifolds. This involves projecting an offset (e.g. velocity) vector onto the tangent space of the manifold (i.e. hypersphere in the case of a normally distributed space), and from there transforming onto the manifold itself. This will be a post (and/or paper) in itself in the near future.

Snapshots across time

As the network trains, the latent space changes with each training iteration, to hopefully represent the data more efficiently and accurately. However a noticeable change across these iterations also includes transformations and shifts in space. E.g. what may be an area in latent space dedicated to `mountains’ at iteration 70K, might become `flowers’ at iteration 80K, while `mountains’ slide over to what used to be `clouds’ (this is a bit of an exaggerated oversimplification). To investigate the effects of these transformations, we render the same z-sequence decoding from a number of different snapshots across subsequent training iterations (e.g. the last 28 snapshots spaced 1000 iterations apart), and we tile the outputs in a grid (e.g. 7x4) when saving a video. An example video can be seen below.

https://medium.com/media/49b583306fe5a66a559eff90af1c2a34/href

Here, every tile within a frame is the same z-vector decoded from a different snapshot in time (i.e. training iteration). We can see in many cases the images are relatively similar with slight variations. In other cases there are more radical shifts, where earlier snapshots are hinting at generating one type of image while later snapshots are producing another for the same z-vector. Fascinatingly, even while semantically the images might be radically different, sometimes the overall form and composition is similar. E.g. in the images below, the semantic content of the tiles change as training continues, however overall shape is maintained. I hope to write a more in depth analysis of this in the near future.

When editing our videos in the NLE, we edit these videos containing the outputs from multiple tiled snapshots. This gives us an overview of the aesthetic qualities from the different training iterations, and allows us to choose the most aesthetically desirable snapshot(s) to use for our final output.

Conclusion

Many aspects of this process can be improved; from theoretical, computational, and user experience points of view. We present this research as a first step in many, towards enabling users to meaningfully explore and control journeys in high dimensional latent spaces to construct stories, using and building upon industry standard tools and methods with which they may already be comfortable. Our ultimate goal is to enable users to creatively express themselves, and meaningfully control the way in which they produce time-based media using deep generative models as a medium.

While this article focuses on only image based generative models, there’s nothing inherently constrained to images about this workflow. In fact the sounds in all of these videos were produced in the same way, using GRANNMA (Granular Neural Music and Audio). I’ll be posting a paper and article about that soon.

Acknowledgments

This work has been supported by UK’s EPSRC Centre for Doctoral Training in Intelligent Games and Game Intelligence (IGGI; grant EP/L015846/1).

Stories by Memo Akten on Medium

The Unreasonable Ecological Cost of #CryptoArt (Part 3)

NFTs are a scam

The blockchain does not provide scarcity or authenticity for NFT related content

The blockchain does not guarantee secondary market income

The myths of decentralization

The Dream

Crypto-Artist total income distribution

Footnotes

References

The Unreasonable Ecological Cost of #CryptoArt (Part 2: Extras)

The Unreasonable Ecological Cost of #CryptoArt (Part 2)

Update — Sep 2022

Update — Dec 2021

Additional notes

Introduction

Background: how is the footprint calculated?

Footprint per unit of Gas

Footprint per transaction type

Calculating footprint of NFTs

Footprint per average NFT

Footprint per average transaction relating to a NFT

NFT Statistics

Energy consumption per NFT

Carbon emissions per NFT

NFT Age (time since being minted)

Number of transactions per NFT

Artist Statistics

Number of NFTs minted per artist

Energy consumption per artist

Carbon emissions per artist

Artist Age (time since joining platform)

Income and revenue distributions

NFT primary sales

NFT secondary sales (10% goes to the artist)

Total revenue per NFT (including primary and 10% secondary sales)

Artist primary sales

Artist secondary sales (10% goes to the artist)

Total income per artist (including primary and 10% secondary sales)

Final thoughts

Footnotes

References

The Unreasonable Ecological Cost of #CryptoArt

The Unreasonable Ecological Cost of #CryptoArt (Part 1)

Update — Sep 2022

Update — Dec 2021

Updates

Introduction

What is this? What’s going on?

What? Why?

The problem: the ecological cost

Final thoughts

Additional notes

Acknowledgements

Footnotes

References

Attributions

Building predictive models of people and their thoughts.

Introduction and Background

Making stuff up

Predictive models of people

Private Individual Models

Aren’t they already building models of us?

End-to-end encryption to save the day?

Conclusion

Side note

Science & Religion. Why I’m so obsessed. #1

MEMO M̸e̵h̶m̸e̸t̶ ̴S̴e̷l̴i̷m̸ ̵A̸k̶t̴e̷n̶ on Twitter

Related work

Related texts

“Artificial Intelligence” as the name of a research field vs a characteristic of being.

You haven’t built / you’re not collaborating with “An Artificial Intelligence”.

A semi-mathematical theory of human to human communication

High dimensional thinking

Optimising for beauty - the erasure of heterogeneity and an appeal for a Bayesian world view

Short version

Long version

Acknowledgements

Deep meditations: a meaningful exploration of inner self, a controlled navigation of latent space.

Abstract

**The blockchain does not provide scarcity or authenticity for NFT related content**