A new Stanford preprint was released earlier today (PDF, SI). The authors claim that the true population prevalence of COVID-19 in Santa Clara County is 50–85X higher than the number of confirmed cases. They base their reasoning on a serosurvey of 3330 participants.

If true, this would actually be good news for society. It would mean that the virus had already widely spread, and thus had a lower fatality rate than previously expected, so the disease wasn’t as severe as we thought. Indeed, the authors claim this explicitly, by putting rough caps on the number of deaths and the infection fatality rate (as distinct from the case fatality rate).

However, I am skeptical of this result for several reasons.

Before I begin, I want to say that (1) I’m very glad that the authors are doing serosurveys, (2) I offer this writeup in the spirit of a peer review from a fellow scientist and citizen, (3) I believe I am qualified to render a peer review in the field of molecular diagnostics and statistics, (4) I too would like to get people back to work without causing mass sickness, (5) I hope the feedback from this round of review (even if I myself am mistaken!) allows us all to develop improved serosurveys.

OK, so why I am I skeptical of this paper?

1. The False Positive Rate of the Test is High

The most important raw result by the authors is the claim that 50 out of their 3330 participants tested positive on an antibody test, showing that they were already infected.

However, as we will see, because they reported 2 false positives out of 401 tested samples, there is is a really wide range on what the actual false positive rate could be, and it could be significantly higher than 1.2%. So false positives could account for many if not all of the 50 reported positives in their study.

Above is page 16 of their PDF. A critical question is: how many of these 50 positives might be false positives? To answer this, we need to look at the calibration of the test against gold standard positive and negative results. That is, if you put in a known negative sample into the test, something known not to have COVID-19 antibodies, do you get out a negative result? Or do you get out a positive result? The former is called a “true negative” and the latter is a “false positive”, as shown below.

From their Results section and Supplementary Information, we get some crucial information on the false positive rate. Here are the key bits:

They show that in their internal calibration, 30 out of 30 known negatives tested negative. In the manufacturer’s calibration, however, 369 out of 371 known negatives tested negative. If we follow the authors and just combine these, they had 2 false positives out of 401 total. As Jeffrey Spence observed, this is actually a high enough rate to potentially mess up the whole study.

jeffrey spence on Twitter

In the supplement they say 2 out of 371 + 35 known negative samples tested positive. This means that the 95% confidence interval for the false positive rate is [0.06%, 1.77%]. In their samples from Santa Clara County they had 50 / 3,349 = 1.5% test positive.

To show this, we are going to go through three progressively more sophisticated ways to estimate the false positive rate:

• A simple point estimate: “the false positive rate is 2/401 or ~0.5%”
• A naive confidence interval based on a normality assumption: “the false positive rate is between 0% and 1.2% with 95% confidence”
• A less naive confidence interval based on the reality of non-normality: “the false positive rate is between 0% and a number greater than 1.2% with 95% confidence”

Let’s start with the simple way.

A naive false positive rate is a point estimate: 2/401 or 0.5%

Let’s assume for a second that among the 3330 participants there were actually 0 that were positive. So with a perfect test we’d get 3330 true negative results out and 0 true positive results out.

However, if we have a test with a false positive rate of 2/401, then if we ran it on this population we’d get on average 2/401 * 3330 ~= 17 false positives with the remainder being 3313 true negatives. That number of 17 false positives is roughly one third of the 50 total reported positives, and we’d get that in a sample with zero actual positives. That hints that a solid chunk of the reported positives could be false positives.

Might it be more than that?

Actually, yes, because when you have a low enough numerator like 2 out of 401, you can’t just assume the false positive rate is exactly 2/401= 0.5%. Because if you re-calibrated the assay on a different set of 401 known negative samples, you might get 0/401 false positives (for a false positive rate of 0%), or even 6/401 (for a false positive rate of 1.5%).

So, can we get a confidence interval for what the actual false positive rate could be, given that we saw 2 false positives out of 401 total tests?

A less naive false positive rate is a confidence interval: [0%, 1.2%]

Beyond this point there will be some stats. You don’t need to understand it all. The main point is that the false positive rate could be considerably higher than 0.5%. It might be more like 1.2% or even higher.

How did we come to that conclusion?

Basically, you can think of each of the n=401 calibration tests as a coin flip, with an unknown probability p of coming up heads (a false positive) that we’re trying to put some bounds on. The exact model for the observed number of false positives given (n, p) is the binomial distribution. We divide that by the total number of samples (n) to get something called the sampling distribution of the sample proportion. It lets you put error bars on estimates of proportions that come from real data, like the proportion of false positives.

Under some circumstances (actually invalid in this case, but more on that in a second) you can approximate this distribution with a normal distribution. Here’s a reference with a worked example:

So to recap: this formula is a way to get a confidence interval on a “sample proportion” like the 2/401 number we have, the proportion of false positives. If we assume for the moment that the sample proportion is distributed normally (more on this in a second) you can then add two standard deviations up and down to get a 95% confidence interval for the false positive rate. Plugging in numbers, we get:

Note in the last line we restrict the lower bound of the confidence interval for the false positive rate to be 0 because a proportion can’t be negative (that is, we can’t get less than 0 false positives out of 401 trials).

So under the normality assumption, the 95% confidence interval for the false positive rate is [0, 0.012] or [0%, 1.2%]. The upper end of that range is high.

Let’s return to that idealized population of 3330 participants where zero were actually positive and all 3330 were actually negative. If the false positive rate was actually 1.2% then you’d get 1.2% * 3330 = ~40 false positives.

If we compare to the 50 reported positives in the study, at the upper end that kind of false positive rate would mean 40/50 = 80% of the positives in the study could be false positives.

You could make this calculation exact by starting with a population of 3330 participants, of which there would be a small number of actual positives (call it k) and a much larger number of actual negatives (3330-k). You would then do out the full 2x2 contingency table to figure out how many false positives, false negatives, true positives, and true negatives you get under various assumptions of false positive and false negative rates.

But when you do that out, you would see that so long as k is small (and k is likely less than 50 as that’s the total reported positives in the study), you would find that if the false positive rate is anything like 1.2% then the false positives would dominate the share of reported positives.

An even less naive false positive rate is: [0%, >1.2%]

Now, remember that we assumed normality? That’s not really applicable in this circumstance. When p is around 50%, a normal distribution is a reasonable approximation for the sample proportion.

But when n is large (n = 401) and p is small (p ~= 2/401), a Poisson distribution is a better approximation to the binomial than a normal distribution. There are different rules of thumb for when to use this, but Devore’s says a binomial can be approximated by a Poisson distribution if n>50 and np < 5, which are both satisfied here (n=401, np=2).

To get something similar to the sampling distribution for the sample proportion, you might start by looking at the distribution of the number of false positives if it was modeled by a Poisson distribution with parameter lambda = np = 2, and then dividing by n = 401 to get a false positive rate. Here’s a graph.

Now, just like when we were looking at the sampling distribution of the sample proportion, we don’t actually know what the actual false positive rate (p) is. All we have to go on are two numbers: 2 false positives in 401 samples. But if the actual false positive rate is around 2/401, then that graph of the Poisson distribution with np=2 is close to what the distribution of false positives will be given n=401 tests.

Here’s the issue: this is bad news for the false positive rate. A Poisson distribution with low mean (np = 2) has a long right tail, so when you divide by n=401 to get a false positive rate the actual upper bound on the confidence interval for the false positive rate will be significantly higher than 1.2%.

This is just a sketch of an argument. You can throw a lot of math at this problem if you want to make it complicated. But at this point we’re breaking a butterfly on a wheel. The only raw inputs after all were “2” and “401”. And the main conclusion is that we don’t have high confidence that the false positive rate is low enough.

TLDR: because the authors reported 2 false positives out of 401 tested samples, there is a really wide confidence interval on what our actual false positive rate could be, and it could be significantly higher than 1.2%. This could account for many if not all of the 50 reported positives in their study.

This is one possible failure mode.

2. Were Participants Enriched for COVID-19 Cases?

Recall again that the authors reported 50 positives out of 3330 total participants. They recruited these participants via Facebook ads. They used these positive results to conclude that the population prevalence in Santa Clara County was much higher than expected, indeed 50–85X higher than you’d expect based on confirmed cases.

Put another way, rather than 956 confirmed cases out of 1.928 million in Santa Clara County~= 1/2000=0.05% rate of infection, they think it’s 2.49%-4.16%. That’s how they get 50–85X.

But there’s a problem: what if their group of participants was enriched for positives relative to the general population? What if their participants had a much higher rate of COVID-19 than normal? As a reductio ad absurdum, if you went to a hospital and tested all recovering patients for COVID-19 antibodies, you’d probably get a very high percentage of positives. You wouldn’t be able to generalize from that to say that much of the general population had already gotten the illness. To their credit, the authors acknowledge this might be an issue:

They do try to correct for a possibly unrepresentative sample later in the paper with various renormalizations and so on. The general thrust of their argument is that even if their sample wasn’t representative, it wouldn’t produce a 50–85X increase in positives relative to baseline. However, there are two mechanisms that could do exactly that.

2A. Exposed people may have signed up for the study to get tested

The first mechanism that could significantly enrich the number of COVID-19 cases in the study is if symptomatic or exposed people used the study to get a test they could get no other way.

After all, in the Bay Area in early April, it was really hard to get a test for people with mild symptoms or exposure. So people who thought they were exposed or symptomatic may have signed up for the study to get access to a free COVID-19 test they could get no other way. We only need ~50 out of 3330 to exhibit this behavior. And there are at least some who appear to have done just that.

Jo on Twitter

@balajis @Krzysie75471037 @jonatanpallesen @phl43 This is my friend who signed up to the Stanford study because she travels a lot and thinks she had it. She is posting on fb to get more people who would be interested in getting tested because they think they had it including myself.

2B. Exposed people may have recruited other exposed people for the study

The second mechanism that could significantly enrich the number of COVID-19 cases in the study is if symptomatic or exposed people recruited other symptomatic or exposed people to get tested.

Recall that recruitment of study participants was done through Facebook. People who thought they had symptoms or exposure could be sharing links to the study in private groups, WhatsApp chats, email threads, and the like. If one of those groups was for people who had COVID-19 symptoms or exposure, then it’s game over: you could get a “super-recruiter” event where one person recruits N other enriched people into the study. That could significantly boost the number of positives beyond what you’d see in a random sample of Santa Clara.

2C. How plausible are these enrichment mechanisms?

Could symptomatic/exposed people have signed up to get tested? Or could such people have sent the link around to private groups enriched for other people with COVID-19 symptoms or exposure?

James Cham, a venture capitalist in the Bay Area, told me that “I am pretty sure these ads were forwarded widely on WhatsApp in the Chinese community around Silicon Valley. We saw it more than a couple of times.”

Farhad Manjoo of the New York Times said:

Farhad Manjoo on Twitter

@balajis It was open to any resident of Santa Clara county; you could just sign up, the link was passed on social media. Limit of 2 per household.

Farhad Manjoo on Twitter

@balajis Yeah the recruitment /sampling could be a thing.

And one of the study authors said:

So, this appears a bit contradictory, but the overall impression is that (a) individual links to the study from Facebook ads were unique but (b) anyone could go and sign up for the study once they knew about it. This would mean highly non-random recruitment that could enrich for the number of positives.

3. The Study Would Imply Faster Spread than Past Pandemics

At the risk of paraphrasing incorrectly, the gist of the paper’s argument is that “many people have already got it, they didn’t have severe symptoms, so it’s not as serious as you think, and the total number of deaths and fatalities will be low.”. Here’s a section of the paper that makes this argument explicitly.

Now, we don’t know where the number of deaths will end up. It is possible that social distancing is bending the curve, though as of April 17 the COVIDtracking website doesn’t yet to seem to show a deceleration in daily new deaths for the state of California. I haven’t seen more granular statistics for Santa Clara County. The premise of social distancing is that mass infection will cause mass sickness and death.

However, the paper’s prediction that the infection fatality rate is only 0.12%–0.2% is a statement that even with mass infection we will see far less deaths than expected.

Here’s the issue: we are already seeing the virus boost all-cause mortality in NYC and multiple countries, as the graphs below show:

Balaji S. Srinivasan on Twitter

These are the most important charts to monitor. Because death is sadly the most reliable test. Relative to other tests, it has few false positives, high test/retest reliability, requires no lab. When the spike in all-cause mortality is fading, we may truly be bending the curve. https://t.co/HJMCvGDssb

In order to generate these thousands of excess deaths in just a few weeks with the very low infection fatality rate of 0.12–0.2% claimed in the paper, the virus would have to be wildly contagious. It would mean all the deaths are coming in the last few weeks as the virus goes vertical, churns out millions of infections per week to get thousands of deaths, and then suddenly disappears as it runs out of bodies.

I guess this isn’t strictly impossible, but I was skeptical when this theory was first mooted because it’s different from the way past influenza-like pandemics have played out, including H1N1 in 2009. That took 12+ months to infect 11–21% of the world over multiple waves.

Balaji S. Srinivasan on Twitter

That is, to reconcile: 1) high absolute numbers of severe cases (flooded hospitals, etc) 2) With as *low* a severity rate as they're proposing The scale of virus and its growth rate would have to be incredibly high. All the "rare" severe cases would then turn up at the end.

Balaji S. Srinivasan on Twitter

I definitely do think we should do serosurveys. That's cheaper than a trillion dollars and best way to resolve empirically. But a major counterargument to the theory is that H1N1 pandemic took months to get around the globe. Is this so much faster? https://t.co/47T9c5DkSa

4. Conclusion

To summarize, there are three broad reasons why I am skeptical of this study’s claims.

1. First, the false positive rate may be high enough to generate many of the reported 50 positives out of 3330 samples. Or put another way, we don’t have high confidence in a very low false positive rate, as the 95% confidence interval for the false positive rate is roughly [0%, >1.2%] and the reported positive rate is ~1.5%.
2. Second, the study may have enriched for COVID-19 cases by (a) serving as a test-of-last-resort for symptomatic or exposed people who couldn’t get tests elsewhere in the Bay Area and/or (b) allowing said people to recruit other COVID-19 cases to the study in private groups. These mechanisms could also account for a significant chunk of the 50 positives in 3330 samples.
3. Third, in order to produce the visible excess mortality numbers that COVID-19 is already piling up in Europe and NYC, the study would imply that COVID-19 is spreading significantly faster than past pandemics like H1N1, many of which had multiple waves and took more than a year to run their course.

These points may be mistaken. If so, I welcome corrections. And it would be wonderful news as it would imply we were much closer to herd immunity at a lower cost than people thought.

Alternatively, if these points are correct, we should try to do a second round of serosurveys that (a) aggressively reduces the false positive rate with many controls and possibly multiple independent tests and (b) that uses some form of unbiased recruitment for the serosurvey, potentially similar to jury duty.

While I disagree with the conclusion of the paper, I want to thank the authors for their hard work and hope that these comments prove useful in future serosurveys.

APPENDIX

Morning of April 18, 2020: made some updates to the document, including a link to this comment and edits/typo fixes for clarity. No substantive changes to any of the three points, namely (a) wide confidence interval for false positives, (b) possible significant enrichment for COVID-19 cases, and (c) speed vs past pandemics.

Below is just for the social media preview. You can ignore it!

Foreword

Perhaps the most important document in the world right now is the Chinese National Health Commission’s official guide to how they are diagnosing & treating the novel coronavirus. The Chinese state has been dealing with COVID-19 since January and their guide is now on version 7. It is worthwhile reading for everyone, so we don’t reinvent the wheel.

Yet for some reason it hasn’t been widely circulated in the West. This post is an attempt to remedy that.

What does the document cover? It describes physical characteristics of the virus, strategies for disinfecting surfaces, specific drugs and treatments they’ve found useful, early warning signals, classification of child and adult case types, criteria for patient discharge, and much more. It also does have a subsection on traditional Chinese medicine in their treatment section, which could raise eyebrows, but the majority of it is allopathic medicine. Even if we end up rejecting some of the guide, we should at least know what the Chinese are doing.

How do we know this is a legitimate document? The original Chinese PDF version of the document is on the Chinese National Health Commission (NHC) website here, and an English translation is on the Consulate General of Pusan’s website here. Elsevier also links to this guide on their coronavirus website here.

And who am I? Just a concerned American citizen. My bio is here if you care. I claim no special authority here. I am just taking publicly available information and reformatting it such that it is accessible to Google, Bing, and the English-speaking world.

Again, the translation below is simply a reformatting of the text in the Consulate General’s official translation. Not a single word has been changed. Without further ado:

Diagnosis and Treatment Protocol for Novel Coronavirus Pneumonia (Version 7)

Released by the Chinese National Health Commission & State Administration of Traditional Chinese Medicine on March 3, 2020

Since December 2019, multiple cases of novel coronavirus pneumonia (NCP) have been identified in Wuhan, Hubei. With the spread of the epidemic, such cases have also been found in other parts of China and other countries. As an acute respiratory infectious disease, NCP has been included in Class B infectious diseases prescribed in the Law of the People’s Republic of China on Prevention and Treatment of Infectious Diseases, and managed as an infectious disease of Class A. By taking a series of preventive control and medical treatment measures, the rise of the epidemic situation in China has been contained to a certain extent, and the epidemic situation has eased in most provinces, but the incidence abroad is on the rise. With increased understanding of the clinical manifestations and pathology of the disease, and the accumulation of experience in diagnosis and treatment, in order to further strengthen the early diagnosis and early treatment of the disease, improve the cure rate, reduce the mortality rate, avoid nosocomial infection as much as possible and pay attention to the spread caused by the imported cases from overseas, we revised the Diagnosis and Treatment Protocol for Novel Coronavirus Pneumonia (Trial Version 6) to Diagnosis and Treatment Protocol for Novel Coronavirus Pneumonia (Trial Version 7).

I. Etiological Characteristics

The novel coronaviruses belong to the β genus. They have envelopes, and the particles are round or oval, often polymorphic, with diameter being 60 to 140 nm. Their genetic characteristics are significantly different from SARS-CoV and MERS-CoV. Current research shows that they share more than 85% homology with bat SARS-like coronaviruses (bat-SL-CoVZC45). When isolated and cultured in vitro, the 2019-nCoV can be found in human respiratory epithelial cells in about 96 hours, however it takes about 6 days for the virus to be found if isolated and cultured in Vero E6 and Huh-7 cell lines.

Most of the know-how about the physical and chemical properties of coronavirus comes from the research on SARS-CoV and MERS-CoV. The virus is sensitive to ultraviolet and heat. Exposure to 56°C for 30 minutes and lipid solvents such as ether, 75% ethanol, chlorine-containing disinfectant, peracetic acid, and chloroform can effectively inactivate the virus. Chlorhexidine has not been effective in inactivating the virus.

II. Epidemiological Characteristics

1. Source of infection. Now, the patients infected by the novel coronavirus are the main source of infection; asymptomatic infected people can also be an infectious source.
2. Route of transmission. Transmission of the virus happens mainly through respiratory droplets and close contact. There is the possibility of aerosol transmission in a relatively closed environment for a long-time exposure to high concentrations of aerosol. As the novel coronavirus can be isolated in feces and urine, attention should be paid to feces or urine contaminated environmental that leads to aerosol or contact transmission.
3. Susceptible groups. People are generally susceptible.

III. Pathological changes

Pathological findings from limited autopsies and biopsy studies are summarized below:

1. Lungs

Solid changes of varying degrees are present in the lungs. Alveolar damage involves fibromyxoid exudation and hyaline membrane formation. The exudates are composed of monocytes and macrophages, with plenty of multinucleated syncytial cells. Type II alveolar epithelial cells are markedly hyperplastic, some of which are desquamated. Viral inclusions are observed in type II alveolar epithelial cells and macrophages. Alveolar interstitium is marked with vascular congestion and edema, infiltration of monocytes and lymphocytes, and vascular hyaline thrombi. The lungs are laden with hemorrhagic and necrotic foci, along with evidence of hemorrhagic infarction. Organization of alveolar exudates and interstitial fibrosis are partly present.

The bronchi are filled with desquamated epithelial cells, mucus and mucus plugs. Hyperventilated alveoli, interrupted alveolar interstitium and cystic formation are occasionally seen.

On electron microscopy, cytoplasmic NCP virions are observed in the bronchial epithelium and type II alveolar epithelium. NCP virus antigen positivity in some alveolar epithelia and macrophages are revealed through immunohistochemistry staining, which are positive for NCP virus nucleic acid via RT-PCR.

2. Spleen, hilar lymph nodes and bone marrow

The spleen is evidently shrunk. Lymphocytopenia and focal hemorrhage and necrosis are present. Macrophagocyte proliferation and phagocytosis are noted in the spleen. Lymph nodes are found with sparse lymphocytes and occasional necrosis. CD4+ and CD8+ T cells are present in reduced quantity in the spleen and lymph nodes, revealed by immunohistochemistry staining. Pancytopenia is identified in bone marrow.

3. Heart and blood vessels

Degenerated or necrosed myocardial cells are present, along with mild infiltration of monocytes, lymphocytes and/or neutrophils in the cardiac interstitium. Endothelial desquamation, endovasculitis and thrombi are seen in some blood vessels.

4. Liver and gall bladder

Appearing enlarged and dark-red, the liver is found degenerated with focal necrosis infiltrated with neutrophils. The liver sinusoids are found hyperemic. The portal areas are infiltrated with lymphocytes and monocytes and dotted with microthrombi. The gall bladder is prominently filled.

5. Kidneys

The kidneys are noted with protein exudation in the Bowman’s capsule around glomeruli, degeneration and desquamation of the epithelial cells of renal tubules, and hyaline casts. Microthrombi and fibrotic foci are found in the kidney interstitium.

6. Other organs

Cerebral hyperemia and edema are present, with degeneration of some neurons. Necrosis foci are noted in the adrenal glands. Degeneration, necrosis and desquamation of epithelium mucosae at varying degrees are present in the esophageal, stomach and intestine.

IV. Clinical Characteristics

1. Clinical manifestations

Based on the current epidemiological investigation, the incubation period is one to 14 days, mostly three to seven days.

Main manifestations include fever, fatigue and dry cough. Nasal congestion, runny nose, sore throat, myalgia and diarrhea are found in a few cases. Severe cases mostly developed dyspnea and/or hypoxemia after one week. In severe cases, patients progress rapidly to acute respiratory distress syndrome, septic shock, metabolic acidosis that is difficult to correct, coagulopathy, multiple organ failure and others. It is worth noting that for severe and critically ill patients, their fever could be moderate to low, or even barely noticeable. Some children and neonatal cases may have atypical symptoms, manifested as gastrointestinal symptoms such as vomiting and diarrhea, or only manifested as low spirits and shortness of breath.

The patients with mild symptoms did not develop pneumonia but only low fever and mild fatigue. From current situations, most patients have good prognosis and a small number of patients are critically ill. The prognosis for the elderly and patients with chronic underlying diseases is poor. The clinical course of pregnant women with NCP is similar to that of patients of the same age. Symptoms in children are relatively mild.

2. Laboratory tests

General findings
In the early stages of the disease, peripheral WBC count is normal or decreased and the lymphocyte count decreases. Some patients see an increase in liver enzymes, lactate dehydrogenase (LDH), muscle enzymes and myoglobin. Elevated troponin is seen in some critically ill patients while most patients have elevated C-reactive protein and erythrocyte sedimentation rate and normal procalcitonin. In severe cases, D-dimer increases and peripheral blood lymphocytes progressively decrease. Severe and critically ill patients often have elevated inflammatory factors.

Pathogenic and serological findings

(1) Pathogenic findings: Novel coronavirus nucleic acid can be detected in nasopharyngeal swabs, sputum, lower respiratory tract secretions, blood, feces and other specimens using RT-PCR and/or NGS methods. It is more accurate if specimens from lower respiratory tract (sputum or air tract extraction) are tested. The specimens should be submitted for testing as soon as possible after collection.

(2) Serological findings: NCP virus specific IgM becomes detectable around 3–5 days after onset; IgG reaches a titration of at least 4-fold increase during convalescence compared with the acute phase.

3. Chest imaging

In the early stage, imaging shows multiple small patchy shadows and interstitial changes, apparent in the outer lateral zone of lungs. As the disease progresses, imaging then shows multiple ground glass opacities and infiltration in both lungs. In severe cases, pulmonary consolidation may occur while pleural effusion is rare.

V. Case Definitions

1. Suspect cases

Considering both the following epidemiological history and clinical manifestations:

1.1 Epidemiological history

1.1.1 History of travel to or residence in Wuhan and its surrounding areas, or in other communities where cases have been reported within 14 days prior to the onset of the disease;
1.1.2 In contact with novel coronavirus infected people (with positive results for the nucleic acid test) within 14 days prior to the onset of the disease;
1.1.3 In contact with patients who have fever or respiratory symptoms from Wuhan and its surrounding area, or from communities where confirmed cases have been reported within 14 days before the onset of the disease; or
1.1.4 Clustered cases (2 or more cases with fever and/or respiratory symptoms in a small area such families, offices, schools etc within 2 weeks).

1.2 Clinical manifestations

1.2.1 Fever and/or respiratory symptoms;
1.2.2 The aforementioned imaging characteristics of NCP;
1.2.3 Normal or decreased WBC count, normal or decreased lymphocyte count in the early stage of onset.

A suspect case has any of the epidemiological history plus any two clinical manifestations or all three clinical manifestations if there is no clear epidemiological history.

2. Confirmed cases

Suspect cases with one of the following etiological or serological evidences:

2.1 Real-time fluorescent RT-PCR indicates positive for new coronavirus nucleic acid; 2.2 Viral gene sequence is highly homologous to known new coronaviruses.
2.3 NCP virus specific Ig M and IgG are detectable in serum; NCP virus specific IgG is detectable or reaches a titration of at least 4-fold increase during convalescence compared with the acute phase.

VI. Clinical Classification

1. Mild cases

The clinical symptoms were mild, and there was no sign of pneumonia on imaging.

2. Moderate cases

Showing fever and respiratory symptoms with radiological findings of pneumonia.

3. Severe cases

Adult cases meeting any of the following criteria:

1. Respiratory distress (≧30 breaths/ min);
2. Oxygen saturation≤93% at rest;
3. Arterial partial pressure of oxygen (PaO2)/ fraction of inspired oxygen (FiO2)≦ 300mmHg (l mmHg=0.133kPa).

In high-altitude areas (at an altitude of over 1,000 meters above the sea level), PaO2/ FiO2 shall be corrected by the following formula:

PaO2 / FiO2 x [Atmospheric pressure (mmHg)/760]

Cases with chest imaging that showed obvious lesion progression within 24–48 hours >50% shall be managed as severe cases.

Child cases meeting any of the following criteria:

1. Tachypnea (RR ≥ 60 breaths/min for infants aged below 2 months; RR ≥ 50 BPM for infants aged 2–12 months; RR ≥ 40 BPM for children aged 1–5 years, and RR ≥ 30 BPM for children above 5 years old) independent of fever and crying;
2. Oxygen saturation ≤ 92% on finger pulse oximeter taken at rest;
3. Labored breathing (moaning, nasal fluttering, and infrasternal, supraclavicular and intercostal retraction), cyanosis, and intermittent apnea;
4. Lethargy and convulsion;
5. Difficulty feeding and signs of dehydration.

4. Critical cases

Cases meeting any of the following criteria:

4.1 Respiratory failure and requiring mechanical ventilation;
4.2 Shock;
4.3 With other organ failure that requires ICU care.

VII. Clinical early warning indicators of severe and critical cases

1. Adults.

1.1 The peripheral blood lymphocytes decrease progressively;
1.2 Peripheral blood inflammatory factors, such as IL-6 and C-reactive proteins, increase progressively;
1.3 Lactate increases progressively;
1.4 Lung lesions develop rapidly in a short period of time.

2. Children.

2.1 Respiratory rate increased;
2.2 Poor mental reaction and drowsiness;
2.3 Lactate increases progressively;
2.4 Imaging shows infiltration on both sides or multiple lobes, pleural effusion or rapid progress of lesions in a short period of time;
2.5 Infants under the age of 3 months who have either underlying diseases (congenital heart disease, bronchopulmonary dysplasia, respiratory tract deformity, abnormal hemoglobin, and severe malnutrition, etc.) or immune deficiency or hypofunction (long-term use of immunosuppressants).

VIII. Differential Diagnosis

1. The mild manifestations of NCP need to be distinguished from upper respiratory tract infections caused by other viruses.
2. NCP is mainly distinguished from other known viral pneumonia and mycoplasma pneumoniae infections such as influenza virus, adenovirus and respiratory syncytial virus. Especially for suspect cases, methods such as rapid antigen detection and multiplex PCR nucleic acid detection should be adopted as much as possible for detection of common respiratory pathogens.
3. It should also be distinguished from non-infectious diseases such as vasculitis, dermatomyositis and organizing pneumonia.

IX. Case Finding and Reporting

Health professionals in medical institutions of all types and at all levels, upon discovering suspect cases that meet the definition, should immediately put them in single room for isolation and treatment. If the cases are still considered as suspected after consultation made by hospital experts or attending physicians, it should be reported directly online within 2 hours; samples should be collected for new coronavirus nucleic acid testing and suspect cases should be safely transferred to the designated hospitals immediately. People who have been in close contact with patients who have been confirmed of new coronavirus infection are advised to perform new coronavirus pathogenic testing in a timely manner, even though common respiratory pathogens are tested positive.

If two nucleic acid tests, taken at least 24-hour apart, of a NCP suspect case are negative, and the NCP virus specific IgM and IgG are negative after 7 days from onset, the suspect diagnosis can be ruled out.

X. Treatment

1. Treatment venue determined by the severity of the disease

1.1 Suspected and confirmed cases should be isolated and treated at designated hospitals with effective isolation, protection and prevention conditions in place. A suspect case should be treated in isolation in a single room. Confirmed cases can be treated in the same room.

1.2 Critical cases should be admitted to ICU as soon as possible.

2. General treatment

2.1 Letting patients rest in bed and strengthening support therapy; ensuring sufficient caloric intake for patients; monitoring their water and electrolyte balance to maintain internal environment stability; closely monitoring vital signs and oxygen saturation.

2.2 According to patients’ conditions, monitoring blood routine result, urine routine result, c-reactive protein (CRP), biochemical indicators (liver enzyme, myocardial enzyme, renal function etc.), coagulation function, arterial blood gas analysis, chest imaging and cytokines detection if necessary.

2.3 Timely providing effective oxygen therapy, including nasal catheter and mask oxygenation and nasal high-flow oxygen therapy. If possible, inhalation of mixed hydrogen and oxygen (H2/O2: 66.6%/33.3%) can be applied.

2.4 Antiviral therapy: Hospitals can try Alpha-interferon (5 million U or equivalent dose each time for adults, adding 2ml of sterilized water, atomization inhalation twice daily), lopinavir/ritonavir (200 mg/50mg per pill for adults, two pills each time, twice daily, no longer than 10 days), Ribavirin (suggested to be used jointly with interferon or lopinavir/ritonavir, 500 mg each time for adults, twice or three times of intravenous injection daily, no longer than 10 days), chloroquine phosphate (500 mg bid for 7 days for adults aged 18–65 with body weight over 50 kg; 500 mg bid for Days 1&2 and 500 mg qd for Days 3–7 for adults with body weight below 50 kg), Arbidol (200 mg tid for adults, no longer than 10 days).

Be aware of the adverse reactions, contraindications (for example, chloroquine cannot be used for patients with heart diseases) and interactions of the above- mentioned drugs. Further evaluate the efficacy of those drugs currently being used. Using three or more antiviral drugs at the same time is not recommend; if an intolerable toxic side effect occurs, the respective drug should be discontinued. For the treatment of pregnant women, issues such as the number of gestational weeks, choice of drugs having the least impact on the fetus, as well as whether pregnancy being terminated before treatment should be considered with patients being informed of these considerations.

2.5 Antibiotic drug treatment: Blind or inappropriate use of antibiotic drugs should be avoided, especially in combination with broad-spectrum antibiotics.

3. Treatment of severe and critical cases

3.1 Treatment principle: On the basis of symptomatic treatment, complications should be proactively prevented, underlying diseases should be treated, secondary infections also be prevented, and organ function support should be provided timely.

3.2 Respiratory support:

3.2.1 Oxygen therapy: Patients with severe symptoms should receive nasal cannulas or masks for oxygen inhalation and timely assessment of respiratory distress and/or hypoxemia should be performed.

3.2.2 High-flow nasal-catheter oxygenation or noninvasive mechanical ventilation: When respiratory distress and/or hypoxemia of the patient cannot be alleviated after receiving standard oxygen therapy, high-flow nasal cannula oxygen therapy or non-invasive ventilation can be considered. If conditions do not improve or even get worse within a short time (1–2 hours), tracheal intubation and invasive mechanical ventilation should be used in a timely manner.

3.2.3 Invasive mechanical ventilation: Lung protective ventilation strategy, namely low tidal volume (6–8ml/kg of ideal body weight) and low level of airway platform pressure (< 30cm H2O) should be used to perform mechanical ventilation to reduce ventilator-related lung injury. While the airway platform pressure maintained ≤30cm H2O, high PEEP can be used to keep the airway warm and moist; avoid long sedation and wake the patient early for lung rehabilitation. There are many cases of human-machine asynchronization, therefore sedation and muscle relaxants should be used in a timely manner. Use closed sputum suction according to the airway secretion, if necessary, administer appropriate treatment based on bronchoscopy findings.

3.2.4 Rescue therapy: Pulmonary re-tensioning is recommended for patients with severe ARDS. With sufficient human resources, prone position ventilation should be performed for more than 12 hours per day. If the outcome of prone position ventilation is poor, extracorporeal membrane oxygenation (ECMO) should be considered as soon as possible. Indications include:

1. When Fi02>90%, the oxygenation index is less than 80mmHg for more than 3–4 hours;
2. For patients with only respiratory failure when the airway platform pressure ≥ 35cmH2O, VV-ECMO mode is preferred; if circulatory support is needed, VA- ECMO mode should be used. When underlying diseases are under control and the cardiopulmonary function shows signs of recovery, withdrawal of ECMO can be tried.
3. Circulatory support: On the basis of adequate fluid resuscitation, efforts should be made to improve microcirculation, use vasoactive drugs, closely monitor changes in blood pressure, heart rate and urine volume as well as lactate and base excess in arterial blood gas analysis. If necessary, use non-invasive or invasive hemodynamic monitor such as Doppler ultrasound, echocardiography, invasive blood pressure or continuous cardiac output (PiCCO) monitoring. In the process of treatment, pay attention to the liquid balance strategy to avoid excessive or insufficient fluid intake.

If the heart rate suddenly increases more than 20% of the basic value or the decrease of blood pressure is more than 20% of the basic value with manifestations of poor skin perfusion and decreased urine volume, make sure to closely observe whether the patient has septic shock, gastrointestinal hemorrhage or heart failure.

3.4 Renal failure and renal replacement therapy: Active efforts should be made to look for causes for renal function damage in critical cases such as low perfusion and drugs. For the treatment of patients with renal failure, focus should be on the balance of body fluid, acid and base and electrolyte balance, as well as on nutrition support including nitrogen balance and the supplementation of energies and trace elements. For critical cases, continuous renal replacement therapy (CRRT) can be used. The indications include: 1 hyperkalemia; 2 acidosis; 3 pulmonary edema or water overload; 4 fluid management in multiple organ dysfunction.

3.5 Convalescent plasma treatment: It is suitable for patients with rapid disease progression, severe and critically ill patients. Usage and dosage should refer to Protocol of Clinical Treatment with Convalescent Plasma for NCP Patients (2nd trial version).

3.6 Blood purification treatment: Blood purification system including plasma exchange, absorption, perfusion and blood/plasma filtration can remove inflammatory factors and block “cytokine storm”, so as to reduce the damage of inflammatory reactions to the body. It can be used for the treatment of severe and critical cases in the early and middle stages of cytokine storm.

3.7 Immunotherapy: For patients with extensive lung lesions and severe cases who also show an increased level of IL-6 in laboratory testing, Tocilizumab can be used for treatment. The initial dose is 4–8mg/kg with the recommended dose of 400mg diluted with 0.9% normal saline to 100ml. The infusion time should be more than 1 hour. If the initial medication is not effective, one extra administration can be given after 12 hours (same dose as before). No more than two administrations should be given with the maximum single dose no more than 800mg. Watch out for allergic reactions. Administration is forbidden for people with active infections such as tuberculosis.

3.8 Other therapeutic measures
For patients with progressive deterioration of oxygenation indicators, rapid progress in imaging and excessive activation of the body’s inflammatory response, glucocorticoids can be used in a short period of time (three to five days). It is recommended that dose should not exceed the equivalent of methylprednisolone 1–2 mg/kg/day. Note that a larger dose of glucocorticoid will delay the removal of coronavirus due to immunosuppressive effects. Xuebijing 100ml/time can be administered intravenously twice a day. Intestinal microecological regulators can be used to maintain intestinal microecological balance and prevent secondary bacterial infections.

Child severe and critical cases can be given intravenous infusion of γ-globulin.

For pregnant severe and critical cases, pregnancy should be terminated preferably with c- section.

Patients often suffer from anxiety and fear and they should be supported by psychological counseling.

4. Traditional Chinese Medicine treatment

This disease belongs to the category of plague in traditional Chinese medicine (TCM), caused by the epidemic pathogenic factors. According to the different local climate characteristic and individual state of illness and physical conditions, the following treatment Protocol may vary. The use of over-pharmacopoeia doses should be directed by a physician.

4.1 During medical observation

Clinical manifestation 1: fatigue and gastrointestinal discomfort
Recommended Chinese patent medicine: Huoxiang Zhengqi capsules (pills, liquid, or oral solution)
Clinical manifestation 2: fatigue and fever
Recommended Chinese patent medicine: Jinhua Qinggan granules, Lianhua Qingwen capsules (granules), Shufeng Jiedu capsules (granules), Fangfeng Tongsheng pills (granules)

4.2 During clinical treatment (confirmed cases)
4.2.1 Lung cleansing & detoxifying decoction

Scope of application: It is suitable for light, moderate and severe patients, and can be used reasonably in combination with the actual situation of patients in the treatment of critically ill patients.

Recommended prescription: Ephedra 9g, Zhigancao 6g, Almond 9g, Gypsum 15–30g (fried first), Guizhi 9g, Zixie 9g, Zhuling 9g, Baizhu 9g, Zhiling 15g, Bupleurum 16g, Scutellaria baicalensis 6g, and Pinellia 9g , Ginger 9g, aster 9g, winter flower 9g, shoot dry 9g, asarum 6g, yam 12g, coriander fruit 6g, tangerine peel 6g, aquilegia 9g. Suggested use: Traditional Chinese medicine decoction pieces for decocting in water. One dose per day, twice in the morning and evening (forty minutes after a meal), take with warm water, and three doses a course.

If conditions permit, the patient can take half a bowl of rice soup each time after taking the medicine, and can take up to one bowl if the patient has a dry tongue and is deficient in bodily fluids. (Note: If the patient does not have a fever, the amount of gypsum should be little. If having a fever or strong heat, the amount of gypsum can be increased). If the symptoms improve but do not fully recover, then take the second course of treatment. If the patient has special conditions or other underlying diseases, the prescription of the second course of treatment can be modified based on the actual situation and the medicine should be discontinued when the symptoms disappear.

Source of prescription: Notice on Recommending the Use of ‘Lung cleansing & detoxifying decoction’ in Treatment of NCP by Integrated Traditional Chinese and Western Medicine by the Office of the State Administration of Traditional Chinese Medicine & the General Office of the National Health Commission. (2022 №22)

4.2.2 Mild cases

4.2.2.1 Cold dampness and stagnation lung syndrome

Clinical manifestations: fever, fatigue, sore body, cough, expectoration, chest tightness, suffocation, loss of appetite, nausea, vomiting, sticky stools. Tongue has thin fat tooth mark or is faint red, and the coating is white thick rot or white greasy and the pulse is moisten or slippery.

Recommended prescription: Raw ephedra 6g, raw gypsum 15g, almond 9g, loquat 15g, gardenia 15g, Guanzhong 9g, Dilong 15g, Xu Changqing 15g, Huoxiang 15g, Peilan 9g, Cangzhu 15g, Yunling 45g, Atractylodes 30g, Jiao Sanxian 9g each , Magnolia officinalis 15g, betel coconut 9g, yarrow fruit 9g, ginger 15g.

Suggested use: one dose daily, boiled with 600ml water, take it three times at morning, noon and evening before meal.

4.2.2.2 Dampness and heat-accumulation lung syndrome
Clinical manifestations: low or no fever, slight chills, fatigue, heavy head and body, muscle soreness, dry cough, low phlegm, sore throat, dry mouth, do not want to drink more, or accompanied by chest tightness, no sweat or sweating, Or vomiting and loss of appetite, diarrhea or sticky stool. The tongue is reddish, and the coating is white, thick and greasy or thin yellow, and the pulse is slippery or sloppy.

Recommended prescription: Betel nut 10g, apple 10g, Magnolia 10g, Zhimu 10g, scutellaria baicalensis 10g, Bupleurum 10g, red peony 10g, forsythia 15g, artemisia annua 10g (decocted later), 10g of green leaves, 10g of green leaves, 5g of raw licorice. Suggested use: one dose daily, boiled with 400ml water, take it twice at morning and evening.

4.2.3 Moderate cases
4.2.3.1 Dampness and stagnation lung syndrome

Clinical manifestations: fever, low cough and sputum, or yellow sputum, suffocation, shortness of breath, bloating, and constipation. The tongue is dark red and fat; the coating is greasy or yellow and the pulse is slippery or stringy.

Recommended prescription: raw ephedra 6g, bitter almond 15g, raw gypsum 30g, raw coix seed 30g, grass root 10g, patchouli 15g, artemisia annua 12g, Polygonum cuspidatum 20g, verbena 30g, dried reed root 30g, gardenia 15g 15g of orange red, 10g of raw licorice. Suggested use: one dose daily, boiled with 400ml water, take it twice at morning and evening.

4.2.3.2 Cold dampness lung syndrome

Clinical manifestations: low fever, low body temperature, or no heat, dry cough, low sputum, fatigue, chest tightness, nausea, or nausea. The tongue is pale or red, and the coating is white or greasy, and the veins are pulsating.

Recommended prescription: Atractylodes lancea 15g, Chenpi 10g, Magnolia 10g, Aquilegia 10g, grass fruit 6g, raw ephedra 6g, Zhihuo 10g, ginger 10g, betel nut 10g. Suggested use: one dose daily, boiled with 400ml water, take it twice at morning and evening.

4.2.4 Severe cases
4.2.4.1 Plague poison and lung-closing syndrome

Clinical manifestations: fever, flushing, cough, yellowish phlegm, or blood in sputum, wheezing, shortness of breath, tiredness, fatigue, dryness and stickiness, nausea, food loss, poor stool, and short urination. Red tongue, yellow greasy coating, slippery pulses.

Recommended prescription: Raw ephedra 6g, almond 9g, raw gypsum 15g, licorice 3g, fragrant fragrant 10g (back), Magnolia 10g, atractylodes 15g, grass fruit 10g, pinellia 9g, Poria 15g, raw rhubarb 5g (back) 10g, gardenia 10g, red peony 10g.
Suggested use: one or two doses daily, boiled with 100–200ml water, take it 2–4 times, oral or nasal feeding.

4.2.4.2 Syndrome of flaring heat in qifen and yingfen

Clinical manifestations: Hot fever, thirst, shortness of breath, shortness of breath, blurred vision, or spotted rash, or vomiting blood, bleeding, or convulsions in the limbs. Tongue ridges have few or no moss, and the pulse sinks finely, or floats large and counts. Recommended prescription: 30–60g gypsum (fried first), 30g of Zhimu, 30–60g of raw land, 30g of buffalo horn (fried first), 30g of red sage, 30g of black ginseng, 15g of forsythia, 15g of paeonia, 6g of peony 12g, gardenia 15g, raw licorice 6g.

Suggested use: 1 dose per day, decoction, first decoct gypsum and buffalo horn, then apply other pieces, 100ml-200ml each time, 2–4 times a day, orally or nasally.
Recommended Chinese patent medicines: Xiyanping injection, Xuebijing injection, Reduning injection, Tanreqing injection, Xingnaojing injection. Drugs with similar efficacy can be selected according to individual conditions, or can be used in combination according to clinical symptoms. Traditional Chinese medicine injection can be used in combination with traditional Chinese medicine decoction.

4.2.5 Critical cases (syndrome of inner blocking causing collapse)
Clinical manifestations: dyspnea, dyspnea, asthma or need mechanical ventilation, fainting, irritability, cold sweating, dark purple tongue, thick or dry moss, large floating roots. Recommended prescription: 15g of ginseng, 10g of Heishun tablets (decoct first), 15g of dogwood, delivered with Suhexiang Pill or Angong Niuhuang Pill.

For patients on mechanical ventilation with abdominal distention or constipation: 5–10g of Dahuang. For patients with human-machine asynchronization: 5–10g of Dahuang and 5- 10g of Mangxiao while administering sedatives and muscle relaxants.

Recommended Chinese patent medicines: Xuebijing injection, Reduning injection, Tanreqing injection, Xingnaojing injection, Shenfu injection, Shengmai injection, Shenmai injection. Drugs with similar efficacy can be selected according to individual conditions, or can be used in combination according to clinical symptoms. Traditional Chinese medicine injection can be used in combination with traditional Chinese medicine decoction.

Note: Recommended usage of Chinese medicine injections for severe and critical cases

The use of traditional Chinese medicine injections follows the principle of starting from a small dose and gradually adjusting the dosage according to the instructions of the drug. The recommended usage is as follows:

Viral infection or combined mild bacterial infection: 0.9% sodium chloride injection 250ml plus Xiyanping injection 100mg bid, or 0.9% sodium chloride injection 250ml heated Duning injection 20ml, or 0.9% sodium chloride injection 250ml plus Tanreqing injection 40ml bid.

High fever with disturbance of consciousness: 250ml of 0.9% sodium chloride injection and 20ml bid of Xingnaojing injection.

Systemic inflammatory response syndrome or/and multiple organ failure: 250ml of 0.9% sodium chloride injection and 100ml of Xuebijing injection.

Immunosuppression: 250ml of 0.9% sodium chloride injection and 100ml bid of Shenmai injection.
Shock: 250ml of 0.9% sodium chloride injection plus 100ml bid of Shenfu injection.

4.2.6 Convalescent period
4.2.6.1 Lung and spleen qi deficiency syndrome

Clinical manifestations: shortness of breath, fatigue, fatigue, anorexia, nausea, fullness, weak stool, and uneasiness. The tongue is pale and greasy.
Recommended prescription: French Pinellia 9g, Chenpi 10g, Codonopsis 15g, Sunburn Astragalus 30g, Stir-fried Atractylodes 10g, Poria 15g, Huoxiang 10g, Amomum villosum 6g (later), and Licorice 6g

Suggested use: 1 dose per day, boiled with 400ml of water, twice a day at morning and evening.

4.2.6.2 Qi and Yin deficiency syndrome

Clinical manifestations: Fatigue, shortness of breath, dry mouth, thirst, palpitations, sweating, poor appetite, low or no lever, dry cough and little sputum; dry tongue, fine or weak pulses.

Recommended prescription: North and south radix salviae 10g, 15g ophiopogonis, 6g American ginseng, 6g schisandra, 6g gypsum l5g, 10g light bamboo leaves, 10g mulberry leaves, 15g reed root, 15g salviae miltiorrhiza, 6g raw liquorice.
Suggested use: 1 dose per day, boiled with 400ml of water, twice a day at morning and evening.

XI. Discharge criteria and after-discharge considerations

1.Discharge criteria

1. Body temperature is back to normal for more than three days;
2. Respiratory symptoms improve obviously;
3. Pulmonary imaging shows obvious absorption of inflammation,
4. Nuclei acid tests negative twice consecutively on respiratory tract samples such as sputum and nasopharyngeal swabs (sampling interval being at least 24 hours). Those who meet the above criteria can be discharged.

2. After-discharge considerations

2.1 The designated hospitals should contact the primary healthcare facilities where the patients live and share patients’ medical record, to send the information of the discharged patients to the community committee and primary healthcare facility where the patients reside.

2.2. After discharge, it is recommended for patients to monitor their own health status in isolation for 14 days, wear a mask, live in well-ventilated single room if possible, reduce close contact with family members, separate dinning, practice hand hygiene and avoid going out.

2.3 It is recommended for the patients to return to the hospitals for follow-up and re-visit in two and four weeks after discharge.

XII. Patients Transportation Principles

Patients should be transported in accordance with the Work Protocol for Transfer of the Novel Coronavirus Pneumonia Patients (Trial Version) issued by the National Health Commission.

XIII. Nosocomial Infection Prevention and Control

Measures to prevent and control nosocomial infection should be implemented in accordance with the requirements of the Technical Guidelines for the Prevention and Control of Infection by the Novel Coronavirus in Medical Institutions (First Edition) and the Guidelines on the Usage of Common Medical Protective Equipment against Novel Coronavirus Infection (Trial Version) formulated by the National Health Commission.

The General Office of National Health Commission Office of State TCM Administration

Printed and distributed on March 3, 2020

Once again, here’s the original source: http://busan.china-consulate.org/chn/zt/4/P020200310548447287942.pdf

Context

On February 7, a Recode journalist reached out with a request for comment that immediately raised my eyebrows.

• I had context, and called it out on Twitter in advance here
• Some back and forth ensued.
• Here was the subsequent article (archive link).
• And here was my response as a series of tweets.

Twitter didn’t have space to include citation links for each tweet in the response, so I’m doing that below.

Tweet 1

Balaji S. Srinivasan on Twitter

Incredible. Last week I called out a journalist who I thought was writing a piece that would be a disservice to public health. Unfortunately, my concerns proved well-founded. Recode's virus piece ignores WHO &amp; CDC, gets the science wrong...and focuses on handshakes! Thread: 🧵

• Original tweet
• “This was going to be a story about handshakes”
• “How do you know that without talking to her?”
• “Oh it’s a psychic”

Tweet 2

Balaji S. Srinivasan on Twitter

As context: - WHO has declared a global public health emergency - Director of Mayo Clinic vaccine research: "basically at a pandemic now" - CDC: virus may "take a foothold in the US" - Fmr FDA commissioner: pandemic "likely" That's the grave reality of the current situation.

• CNN: WHO declares coronavirus a public health emergency of national concern
• CNBC: Mayo Clinic doctor: Coronavirus is ‘basically at a pandemic now’ and should be treated as such
• The Hill: The CDC is preparing for coronavirus ‘to take a foothold in the US’
• CNBC: Coronavirus will likely become a pandemic, warns ex-FDA Commissioner Scott Gottlieb

Tweet 3

Balaji S. Srinivasan on Twitter

But this isn't reflected in the article. Instead it has: - clickbait lede - wrong science - author w/o domain expertise - bad faith request for comment - outrage at assuming bad faith - article that proves assumption of bad faith right! See for yourself: https://t.co/dwpVW9jdvX

• Recode: “No handshakes, please”: The tech industry is terrified of the coronavirus
• Archive link

After this point, I won’t keep linking the Recode piece, even though it is cited in many of the tweets below.

Tweet 4

Balaji S. Srinivasan on Twitter

The article contains many statements that are false or omit so much context they mislead readers. Let's go claim by claim. RECODE: "cases...have been contained to those who have recently traveled to Wuhan and their direct family members" FACT CHECK: False, according to CDC/WHO

• STAT - CDC director: More person-to-person coronavirus infections in U.S. likely, but containment still possible
• Director-General of the WHO: “Containment remains our objective, but all countries must use the window of opportunity created by the containment strategy to prepare for the virus’s possible arrival.”
• Director-General of the WHO: The detection of a small number of cases may indicate more widespread transmission in other countries; in short, we may only be seeing the tip of the iceberg.

Tweet 5

Balaji S. Srinivasan on Twitter

RECODE: "chances are...low that you would come into contact at work or in a public setting" CONTEXT: - Confirmed transmission at conferences - One person spread to N people - Hundreds quarantined due to 1 attendee - Major conference canceled - Contagion means risk is increasing

• Reuters: Hunt on for ‘patient zero’ who spread coronavirus globally from Singapore
• BBC News: Coronavirus: UK conference attendees warned over case
• CNBC: Mobile World Congress canceled after coronavirus outbreak makes conference ‘impossible’

Tweet 6

Balaji S. Srinivasan on Twitter

RECODE: "reducing handshakes...not specifically [recommended] for coronavirus" FACT CHECK: False, according to CDC "Human coronaviruses most commonly spread from an infected person to others through...close personal contact, such as touching or shaking hands

• CDC: How Other Coronaviruses Spread
• CNBC: Op-ed: Gottlieb says screening and fist bumps over handshakes will help protect against coronavirus

Tweet 7

Balaji S. Srinivasan on Twitter

RECODE: fecal transmission "still unclear" CONTEXT: - Related SARS virus spread through feces at Amoy Gardens - Infected 300+ people, 42 deaths - #SARSCoV2 has been discovered in feces - Patients have diarrhea - Multiple scientists now implicate fecal transmission

• Japan Times: Coronavirus study implicates fecal transmission
• South China Morning Post: China adds new symptoms to coronavirus diagnostic list. Health authorities also warn that discovery of virus in patients’ faeces suggest that it could be another source of transmission
• ABC News 9 Eugene Oregon / Kezi: How can the coronavirus spread through bathroom pipes? Experts are investigating in Hong Kong.

Tweet 8

Balaji S. Srinivasan on Twitter

RECODE: Quotes non-med/biotech expert on "paranoid fear about disease" CONTEXT: "Zuckerberg SF General is prepared with 50 isolation rooms that will keep patients from transmitting the virus to others." "hospital staff plans to treat patients while wearing protective gear

• Sfist: Local Hospitals Prepare for Coronavirus Patients; Richmond Lab Begins Testing: “While there has yet to be a case of the Wuhan/novel coronavirus in San Francisco, health officials are preparing for the inevitability/possibility that a case will appear here — and Zuckerberg SF General is prepared with 50 isolation rooms that will keep patients from transmitting the virus to others.”

Tweet 9

Balaji S. Srinivasan on Twitter

RECODE: Implies Beijing has no outbreak, and that virus is distant and far away FACT CHECK: False, 370+ cases in Beijing according to JHU dashboard Also, NPR reports Beijing is building another 1000-bed instant hospital to prepare for predicted uptick in coronavirus cases.

• Johns Hopkins: Dashboard of Coronavirus COVID-19 Global Cases by Johns Hopkins CSSE
• NPR: Beijing Braces For Coronavirus Cases As Lunar Holiday Ends

Tweet 10

Balaji S. Srinivasan on Twitter

RECODE: Implies it's paranoid to cancel Lunar New Year to prevent contagion CONTEXT: Wuhan: wanted to cancel events New York: canceled events Miami: canceled events If Wuhan, NYC, Miami all wanted to cancel events...shouldn't SF have done so out of abundance of caution?

• South China Morning Post: Wuhan mayor under pressure to resign over response to coronavirus outbreak…He is slammed for allowing Lunar New Year gathering of more than 40,000 people to go ahead on Sunday
• WSJ: Some Lunar New Year Events in New York Canceled Over Fear of Coronavirus. One organization’s leader says group heard ‘a lot of concerns about having a very big public gathering’
• NBC 6 South Florida: Miami’s Chinese New Year Festival Cancelled Amid Fears of Coronavirus

Tweet 11

Balaji S. Srinivasan on Twitter

RECODE: Quotes SF officials assuring people of "no public health risk" without challenging them CONTEXT: - SF strewn with feces, urine, syringes - NPR: SF like "dirtiest slums in the world" - Berkeley prof: "once fecal matter dries it can become airborne releasing viruses

• NPR: San Francisco Squalor: City Streets Strewn With Trash, Needles And Human Feces
• NBC Bay Area: Survey of Downtown San Francisco Reveals Trash on Every Block, 303 Piles of Feces and 100 Drug Needles

Tweet 12

Balaji S. Srinivasan on Twitter

RECODE: Implies #covid2019 only "scary" to "public" because "new" CONTEXT: Omits - NEJM, Lancet papers on sheer severity - Caijing's courageous reporting on severity - Math of tail risk for a pandemic - Fact that many qualified PhDs/MDs, both in SV &amp; outside, have voiced concern

• The Lancet: Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study
• New England Journal of Medicine (NEJM): First Case of 2019 Novel Coronavirus in the United States
• Caijing: People outside the statistics: They died of “general pneumonia”? (1) Original story archive link, (2) Google Translate, (3) Twitter commentary.
• Wired: Coronavirus Is Bad. Comparing It to the Flu Is Worse

Tweet 13

Balaji S. Srinivasan on Twitter

RECODE: Implies flu is a "greater threat" CONTEXT: Omits - Columbia's Ian Lipkin: "I reject that soundly" - 100X increase in Chinese cases in 3 weeks [JHU] - 100X increase in *non*-Chinese cases in 3 weeks [JHU] - Again, pandemic threat is based on math of tail risk

• CBS News: Interview of Columbia Prof W. Ian Lipkin
• Johns Hopkins: Dashboard of Coronavirus COVID-19 Global Cases by Johns Hopkins CSSE

Tweet 14

Balaji S. Srinivasan on Twitter

RECODE: Calls tech "terrified" and "paranoid" CONTEXT: Omits in-depth discussion of: - Sheer scale of Chinese quarantine - Hospitalization &amp; fatality rates - Contagiousness estimates - 100X growth in non-China cases across 25+ countries - Confirmed transmission at conferences

• NPR: 45 Million Chinese Now Under Quarantine As Officials Try To Halt Coronavirus Spread
• WSJ: How Many People Might One Person With Coronavirus Infect?
• CNBC: China confirms 15,152 new coronavirus cases, 254 additional deaths
• LA Times: How deadly is the new coronavirus? Scientists race to find the answer
• Reuters: WHO investigates Singapore conference linked to foreign virus spread

Tweet 15

Balaji S. Srinivasan on Twitter

RECODE: Also does not discuss: - tech the Chinese are using to fight virus - hardware implications of supply chain disruption - what biotech is doing: antivirals, vaccines, diagnostics This obvious omission was *also* predicted in advance! https://t.co/iS9CHu7OPA

• Original call out tweet

And for color:

Curt Fischer on Twitter

@micsolana This pretty well vindicates @balajis tweet from last week I think https://t.co/WPDkaZTrXZ

Suhail on Twitter

At first this all seems crazy until you realize it's a potential story about VCs and handshakes. https://t.co/CdbNh9eVje

jason@calacanis.com on Twitter

Founders: my advice is to never talk to journalists on the phone-you will likely be misquoted &amp; your answers will not be crisp as email There is so much gotcha journalism &amp; link baiting. Always use email &amp; let the journalist know you plan to publish your answers to your blog.

Tweet 16

Balaji S. Srinivasan on Twitter

It's 2000+ words. I could go on. But there's a point. - I assumed bad faith, as I had context - Not just on this piece, but on tech journalism in 2020 - Some lacked context &amp; didn't understand why I called it out But as you can see, it was indeed a piece about handshakes!

• Original tweet
• “This was going to be a story about handshakes”
• “How do you know that without talking to her?”
• “Oh it’s a psychic”

Tweet 17

Balaji S. Srinivasan on Twitter

Why was I right? Contra one comment, I'm not a psychic. I just have a mental model that correctly predicts the actions of many tech journalists. The sad truth is they are incentivized to produce clickbait. This is partly tech's fault too: social media created these incentives.

Not sure we need any cites here :)

Tweet 18

Balaji S. Srinivasan on Twitter

Again, this article is something of a rubicon for tech journalism, as it crosses the line into a hazard to public health. More later, but for now: - Take virus seriously, WHO/CDC are - Don't take unserious journalists seriously - And avoid handshakes! 🙏 https://t.co/DBMPB3aHWL

That’s all, folks!

Coda

The whole thing can be summarized in a few tweets :)

In May 2015, I became CEO of a company that was in a very difficult situation.

21.co had been started as 21E6, a bitcoin mining company with a sizable data center footprint and a monthly bill to match. The company was set up to return mined bitcoin to its shareholders and generated tens of millions in revenue during its first calendar year of operation. It had thus attracted a substantial amount of investment.

However, it had spent most of this capital on chips, hardware, and data center deployments. And when the Bitcoin hashrate soared and the price of BTC crashed over the course of 2014, the company landed up in a Loudcloud-like scenario with long-term data center leases and falling revenue. Many other mining companies around this time suffered similar fates and died, including Hashfast, Cointerra, Alydian, Aquifer, and others.

While I had helped 21E6 get started back in early 2013, I hadn’t been operationally involved. Instead, I was busy working full time at Andreessen Horowitz as a General Partner. However, once things went wrong, we realized that an advisory role wasn’t going to be sufficient. So I stepped in as full-time CEO with a mandate to fix things. We brought in Lily Liu as a new CFO, and we got to work.

The company we took over in May 2015 was in deep trouble. It had more than $80M in data center and venture debt commitments with less than a year of runway. It had announced a large capital raise, but that was a cumulative figure: the vast majority of the funds had been spent by previous management or locked up in contracts signed before our new team took over. Millions of dollars in bitcoin were being mined every month, but even more millions were being spent in obligate contracts. And to make matters worse, all the BTC returned to shareholders by the mining operation was in danger of being completely clawed back if the company entered a messy bankruptcy.

The new team thus had a very short period of time to do two things: find a way to exit these legacy contracts and figure out a new product — or end up with an $80M hole in the ground.

On the contract side, after an enormous amount of hard work we were able to exit all data center contracts, extinguish all debt, cut the burn by >90%, and extend the company’s runway to the next decade. Some day I’ll write the Hard Thing about Hard Things 2.0 about this process.

On the product side, we’d inherited a risky Hail Mary project called Bitsplit, which tried to salvage some value from the data centers by repurposing the datacenter’s mining chips to embed in phone chargers. Because a phone’s charger was not a power-constrained environment, the idea was that it could be used to power a distributed mining operation, generating a constant stream of BTC that could give every phone owner a free trial of cryptocurrency. We built the requisite software and a hardware development kit to show what this could look like, but by this time Bitcoin micropayments went from potentially feasible to definitely infeasible due to the scaling controversy.

Nevertheless, the open source software we built to handle micropayments ended up being the core of something that did work: namely the paid email and tasks product at the core of what is now Earn.com.

As of today, Earn.com is a fast-growing, cash-flow positive business with a multimillion dollar revenue run rate. Hundreds of thousands of users have signed up to earn free cryptocurrency by answering emails and completing tasks. And thousands of senders have used Earn.com to send out paid emails and surveys to large audiences. I think it’s fair to say that it’s one of the first truly useful blockchain-based applications, where users can earn money in their spare time while senders can pay people to actually reply to their emails and fill out their surveys.

Now we’re about to take the model we’ve proven out at Earn.com and scale it up across Coinbase’s massive user base. As part of this acquisition, I’ll be joining as Coinbase’s new CTO. And with this deal, the total value of cash, cryptocurrency, and equity returned to our shareholders is now in excess of the capital invested in the company.

I’m proud of what our team has accomplished, I’m proud of the fact that we turned what could have been a real problem into a successful product, and I’m excited about the road ahead.

Twitter Moments is a bit of a paradox. It’s Twitter’s in-house news service, and because of recent feature changes, it’s become one of the most visited news outlets in the world. Specifically, mobile Twitter users in many countries now see Moments every time they go to search. Given conservative estimates of monthly uniques that we detail below, this means Twitter Moments probably has a captive audience that’s larger than CNN, the New York Times, or the Washington Post.

Yet surprisingly few people know it by name. And if they have heard of it they often deride it without thinking through the numbers.

That’s really interesting. Whenever we see a technology with empirical traction whose importance is neglected or even derided, it’s a useful signal of an investment or entrepreneurship opportunity. Good examples that I’ve been personally involved with include Soylent and Bitcoin, where skepticism, mockery, and outright hostility was eventually followed by huge returns for those who bet against the crowd.

To be clear, Moments alone likely won’t produce those kinds of outsize financial returns for Twitter. The company has too many other issues that are driving users away, from online mobs to seemingly random moderation policies. Blockchain-based competitors are on the horizon, and Moments is no deus ex machina.

Nevertheless, the new Moments may be the single most interesting thing Twitter is doing, and holds many interesting lessons for other products. Because it’s arguably the first vertically integrated combination of a social network with an international news outlet.

But before diving into that, let’s give a little background.

What is Twitter Moments?

The Moments feature was launched in 2015, and it’s had a bit of a confusing history. It used to be siloed into a separate tab on mobile, it is still a separate lightning bolt tab on desktop, and it has a semi-hidden feature that allows users to create their own moments.

Ignore all that for now. The big new thing that happened this year is that Twitter moved Moments into their so-called “Explore” tab, which the average user would recognize as the search tab with the magnifying glass icon. Here’s a few screens of how that looks in the Twitter iOS app:

See that big photo with the headline in the upper left corner? That’s the top Moment, with the headline and photo written and selected by Twitter. This is not simply showing the most popular hashtag on Twitter; it involves explicit and acknowledged editorial judgment.

If you click that headline within the app, you get a series of screens to swipe through, starting with an intro (again written by Twitter) and followed by Twitter-curated tweets. Try here or here on mobile to see an example. If you instead scroll down, you’ll see other sections of the Explore tab, with other top headlines (“Today’s Moments”), and then individual verticals (like “Tech News”).

If you scroll even further, you see personalized recommendations (“Trends for you” and “Popular articles”), and then finally individual verticals to browse further.

Twitter Moments likely has 92–139M monthly uniques

So that’s what the new Explore feature looks like. Moments are at the top and seen by a large number of global Twitter mobile users on iOS and Android every time they go to search. But just how large is that number?

• As of Q2 2017, Twitter has 328 million monthly active users (MAUs) and an estimated 157 million daily active users (DAUs)
• Assume that 33–50% of these are active in the countries where Moments is live
• Assume further that 85% of Twitter monthly actives use the mobile app
• And assume finally that Twitter mobile app users, like Google users, tend to search on average once per user per day. So each Twitter MAU likely does a search (thus clicking on the Explore tab to view Moments) at least once per month.

Put those together, and you get (328M)(0.85)(0.33–0.5)(1.0), which yields a conservative estimate of 92–139.4M monthly mobile unique views of Moments headlines worldwide, entirely due to users clicking on the Explore tab.

To put this in context, Moments is likely already one of the biggest news outlets in the world by sheer viewership, as we can see by comparing Moments to recent stats from CNN on other organizations:

Twitter Moments is a vertically integrated news outlet + social network

OK, so we’ve established that Twitter Moments has massive international reach, likely more than CNN or the New York Times in terms of sheer monthly mobile uniques. But so does Twitter itself, and Facebook, and all the other big social networks. What’s new here?

The big new point is that Moments are mostly written and curated by Twitter itself and pushed via the Explore tab to 92M+ unique people per month. To my knowledge, that is very different from the behavior of any major social network or search engine to date. Facebook quietly had human editors for a while behind the trending hashtags, but then fired them in search of a more neutral algorithm. And Google doesn’t put much in the way of its corporate opinion on current events in the search box, other than on rare occasions like SOPA. So Twitter Moments is fairly unique with respect to most other social networks in terms of exercising explicit editorial judgment.

Moments is also unique with respect to the other news outlets at comparable scale (CNN, Fox, NYT, etc), because none of those sites are built on a full-blown native social network like Twitter. Among other things, deep integration with the native Twitter social network allows the Moments team to track the progress of each story in minute detail, to personalize the news for each user, to do A/B tests of headlines, and to essentially close the loop with their readers in a way no news outlet has ever been able to do before.

And of course, going in the other direction, the Twitter social network can build features to encourage and prioritize greater engagement with the Twitter Moments news feed (such as putting it in the Explore tab to be seen on every search.)

While the social network piece is probably the hardest to build and most defensible due to network effects, it’s easy to miss the amount of labor that goes into the news side of this equation, because (a) Moments often report on very generic/mainstream content, and (b) Moments are typically very bare-bones — usually a headline, a headline image, an introductory paragraph for context, and then a curated list of tweets.

Nevertheless, they are being created constantly and in realtime. Apparently >27,000 Moments have been written to date, which is about 37 moments written every day since the October 2015 launch of the feature. Again, Moments is a surprisingly large news outlet from a sheer quantity perspective.

But Twitter Moments sucks!

There are tons of common criticisms of Moments. I agree with many of them. Let’s go through a few.

• No one reads Moments! As noted above, we can estimate Moments to have ~92–139M mobile monthly uniques, more than CNN, NYT, or WaPo. The audience might well be a captive audience, but it’s one of the largest news audiences in the world.
• OK, but no one important reads Moments! Assuming most journalists are mobile Twitter users with ~1 search per day, the top Moments headline is likely viewed by a huge number of journalists globally every day. By contrast, it is unlikely that as large a percentage of journalists from organization X refresh (say) the pages of organization Y on a daily basis. So Moments may also be the most widely read news outlet among journalists in the countries where it operates.
• Ok, but no one users ever create Moments! Agreed. Only Twitter-generated moments are high quality as of today. The user interface for user-generated moments is hidden and clunky, and needs integration into the standard tweet compose flow.
• OK, others may use it, but I never click Moments. Maybe so, but people also say they “never click FB/Google ads” — but then many do. Some people also might not know that the headlines under the search tab are “Moments”, or might only use the Twitter desktop app where they are less visible. But it would be surprising if 92–139M monthly uniques viewing a headline wasn’t producing engagement. Only Twitter has the metrics to show this for sure.
• OK, but Moments on desktop sucks. No argument there. It’s far better on mobile, where it’s integrated with search.
• But Moments content is very boring/generic/mainstream! Yes, it is, though (a) it’s not that dissimilar from what you see on CNN or Fox, let alone HuffPo or Buzzfeed and (b) they’ve started to add much more personalization recently.
• Moments is just packaging, it’s not generating any new content/scoops. Yes, the innovation here is on distribution: by putting it on the search tab, Twitter Moments gains a massive captive audience. But the Moments team does write headlines + intro paragraphs + organize Twitter quotes. Millions of people then read these headlines when going to search. Moments doesn’t do original reporting, but they do shape the narrative.
• Moments can’t compare to a real news article! There is of course plenty of tremendous original journalism out there. However, many mainstream articles today are summaries of the Reuters/AP feeds, reprises of someone else’s original reporting, or frankly just long-form Twitter Moments in that they are color commentary on a series of tweets. Axios also has a similar model in some ways to Moments. While Axios spends a lot more energy on high quality reporting, and has many excellent journalists, it also uses an intentionally telegraphic format with a headline, some context, and a few short bullets or quotes.
• Facebook has way more users! Yes, but all Facebook users aren’t seeing the same headline pushed to them, while every Twitter user who does a mobile search in a given time & place sees the same Twitter-created Moments headlines. Facebook did curate hashtags at one point, but doesn’t generate realtime first party content like Moments.
• What about Google News? Google News does do organization and prioritization, but doesn’t do explicit first party editorial. Their algorithms determine what story is “A-1 above the fold”; they don’t manually write headlines or intros.

Conclusion

Frankly, I’m not even a huge advocate for the Moments feature per se, any more than I’m a heavy user of Facebook. I just think it’s got a numerical heft to it that should be more widely appreciated.

Put another way, even if you didn’t use Facebook or care for it, you have to respect the impact of two billion users. In the same way, even if you don’t use Moments or much care for its editorialization or mainstream-ish content, the fact that it likely has more mobile monthly uniques than CNN, the New York Times, or the Washington Post is something to reckon with. Quantity has a quality all its own.

By Balaji S. Srinivasan and Leland Lee

The primary advantage of Bitcoin and Ethereum over their legacy alternatives is widely understood to be decentralization. However, despite the widely acknowledged importance of this property, most discussion on the topic lacks quantification. If we could agree upon a quantitative measure, it would allow us to:

• Measure the extent of a given system’s decentralization
• Determine how much a given system modification improves or reduces decentralization
• Design optimization algorithms and architectures to maximize decentralization

In this post we propose the minimum Nakamoto coefficient as a simple, quantitative measure of a system’s decentralization, motivated by the well-known Gini coefficient and Lorenz curve.

The basic idea is to (a) enumerate the essential subsystems of a decentralized system, (b) determine how many entities one would need to be compromised to control each subsystem, and (c) then use the minimum of these as a measure of the effective decentralization of the system. The higher the value of this minimum Nakamoto coefficient, the more decentralized the system is.

To motivate this definition, we begin by giving some background on the related concepts of the Gini coefficient and Lorenz curve, and then display some graphs and calculations to look at the current state of centralization in the cryptocurrency ecosystem as a whole according to these measures. We then discuss the concept of measuring decentralization as an aggregate measure over the essential subsystems of Bitcoin and Ethereum. We conclude by defining the minimum Nakamoto coefficient as a proposed measure of system-wide decentralization, and discuss ways to improve this coefficient.

The Lorenz Curve and the Gini Coefficient

Even though they are typically concerns of different political factions, there are striking similarities between the concepts of “too much inequality” and “too much centralization”. Specifically, we can think of a non-uniform distribution of wealth as highly unequal and a non-uniform distribution of power as highly centralized.

Economists have long employed two tools for measuring non-uniformity within a population: the Lorenz curve and the Gini coefficient. The basic concept of the Lorenz curve is illustrated in the figure below:

The equation for the Gini coefficient can be calculated from the areas under the Lorenz curve and the so-called “Line of Equality” as shown below:

The coefficient can also be calculated from the individual shares of entities for both continuous and discrete distributions (see here for equations).

Intuitively, the more uniform the distribution of resources, the closer the Gini coefficient is to zero. Conversely, the more skewed to one party the distribution of resources, the closer the Gini coefficient is to one.

This captures our intuitive notions of centralization: in a highly centralized system with G=1, there is one decision maker and/or one entity to capture in order to compromise the system. Conversely, in a highly decentralized system with G=0, there are a multiplicity of decision makers who need to be captured in order to compromise the system. Hence, a low Gini coefficient means a high degree of decentralization.

Cryptocurrency: Gini Coefficients and Lorenz Curves

To build intuition, let’s look at the Lorenz curve and Gini coefficient for a simple example: the distribution of wealth across cryptocurrency market capitalizations. To do this, we took a snapshot of market capitalizations on July 15 2017 for the top 100 digital currencies, calculated the percentage of market share for each, and graphed it as a Lorenz curve with associated Gini coefficient:

If we measure the centralization of market capitalization across the top 100 cryptocurrencies, the Gini coefficient is 0.91. This fits our intuition as about 70% of the market capitalization as of July 2017 is held by the top two cryptocurrencies, namely Bitcoin and Ethereum.

Decentralized Systems are Composed of Subsystems

To apply this concept to the space of public blockchains, we need to make a distinction between a decentralized system and a decentralized subsystem. Specifically, a decentralized system (like Bitcoin) is composed of a set of decentralized subsystems (like mining, exchanges, nodes, developers, clients, and so on). Here are six of the subsystems that compose Bitcoin:

Now, one can argue that some of these decentralized subsystems may be more essential than others; for example, mining is absolutely required for Bitcoin to function, whereas exchanges (as important as they are) are not actually part of the Bitcoin protocol.

Let’s assume however that a given individual can draw a line that identifies the essential decentralized subsystems of a decentralized system. We can then stipulate that one can compromise a decentralized system if one can compromise any of its essential decentralized subsystems.

Quantifying Bitcoin and Ethereum Decentralization

Given these definitions, let’s now calculate Lorenz curves and Gini coefficients for the Bitcoin and Ethereum mining, client, developer, exchange, node, and owner subsystems. We can see how centralized each of them are according to the Gini coefficient and Lorenz curve measures.

Here are the curves for Bitcoin:

And here they are for Ethereum:

Let’s discuss each of these subsystems in turn by reference to the six panels in each of the figures above.

Mining Decentralization

As shown in the top left panel of the figures, Bitcoin mining is surprisingly decentralized as measured by block reward over the past 24 hours. Ethereum mining is somewhat more centralized. There is fairly high variance in these value, so we can track it over time or smooth the result by taking a 7 or 30 day average.

Client Decentralization

As seen in the top middle panel of each figure, Most Bitcoin users use Bitcoin Core, with Bitcoin Unlimited being the second most popular client. This means a fairly high degree of centralization (Gini = 0.92) as measured by the number of different client codebases. For Ethereum clients by codebase, most of the clients (76%) run geth, and another 16% run Parity, which also gives a Gini coefficient of 0.92 as two codebases account for most of the ecosystem.

Dev Decentralization

In the top right panel, we can see that the Bitcoin Core reference client has a number of engineers who have made commits. Though raw commits are certainly an imprecise measure of contribution, directionally it appears that a relatively small number of engineers have done most of the work on Bitcoin Core. For the geth reference client of Ethereum, development is even more concentrated, with two developers doing the lion’s share of commits.

Exchange Decentralization

The volume of Bitcoin and Ethereum traded across exchanges varies a great deal, as do the corresponding Gini coefficients. But we calculated snapshots of the Gini coefficient over the past 24 hours for illustrative purposes in the bottom left panels.

Node Decentralization

Another measure of decentralization (bottom middle panels) is to determine what the node distribution is across countries for Bitcoin and Ethereum.

Ownership Decentralization

In the last panels in the lower right, we look at how decentralized Bitcoin and Ethereum ownership is, as measured by addresses. One important point: if we actually include all 7 billion people on the earth, most of whom have zero BTC or Ethereum, the Gini coefficient is essentially 0.99+. And if we just include all balances, we include many dust balances which would again put the Gini coefficient at 0.99+. Thus, we need some kind of threshold here. The imperfect threshold we picked was the Gini coefficient among accounts with ≥185 BTC per address, and ≥2477 ETH per address. So this is the distribution of ownership among the Bitcoin and Ethereum rich with >$500k as of July 2017.

In what kind of situation would a thresholded metric like this be interesting? Perhaps in a scenario similar to the ongoing IRS Coinbase issue, where the IRS is seeking information on all holders with balances >$20,000. Conceptualized in terms of an attack, a high Gini coefficient would mean that a government would only need to round up a few large holders in order to acquire a large percentage of outstanding cryptocurrency — and with it the ability to tank the price.

With that said, two points. First, while one would not want a Gini coefficient of exactly 1.0 for BTC or ETH (as then only one person would have all of the digital currency, and no one would have an incentive to help boost the network), in practice it appears that a very high level of wealth centralization is still compatible with the operation of a decentralized protocol. Second, as we show below, we think the Nakamoto coefficient is a better metric than the Gini coefficient for measuring holder concentration in particular as it obviates the issue of arbitrarily choosing a threshold.

Maximum Gini Coefficient: A Crude Measure of Blockchain Decentralization

Can we combine these sample measures of subsystem decentralization into a measure of system decentralization? A simple first approach would simply be to take the maximum Gini coefficient over all essential subsystems, as shown below:

So, by this measure, both Bitcoin and Ethereum have a maximum Gini Coefficient of ~0.92, because both of them have nodes with clients that are very highly concentrated in one codebase (Bitcoin Core for Bitcoin, and geth for Ethereum).

Crucially, a different choice of essential subsystems will change these values. For example, one may believe that the presence of a single codebase is not an impediment toward practical decentralization. If so, then Bitcoin’s maximum Gini coefficient would improve to 0.84, and the new decentralization bottleneck would be the node distribution across countries.

We certainly don’t argue that the particular choice of six subsystems here is the perfect one for measuring decentralization; we just wanted to gather some data to show what this kind of calculation would look like. We do argue that the maximum Gini coefficient metric starts to point in the right direction of identifying possible decentralization bottlenecks.

Minimum Nakamoto Coefficient: An Improved Measure of Blockchain Decentralization

However, the maximum Gini coefficient has one obvious issue: while a high value tracks with our intuitive notion of a “more centralized” system, the fact that each Gini coefficient is restricted to a 0–1 scale means that it does not directly measure the number of individuals or entities required to compromise a system.

Specifically, for a given blockchain suppose you have a subsystem of exchanges with 1000 actors with a Gini coefficient of 0.8, and another subsystem of 10 miners with a Gini coefficient of 0.7. It may turn out that compromising only 3 miners rather than 57 exchanges may be sufficient to compromise this system, which would mean the maximum Gini coefficient would have pointed to exchanges rather than miners as the decentralization bottleneck.

There are various ways to surmount this difficulty. For example, we might be able to come up with principled weights for the Gini coefficients of different subsystems prior to combining them.

An alternative approach is to define a spiritually similar metric based on the Lorenz curve from which the Gini coefficient is calculated, which we dub the “Nakamoto coefficient”. A visual is below. In this example, the Nakamoto coefficient of the given subsystem is 8, as it would require 8 entities to get to 51% control.

That is, we define the Nakamoto coefficient as the minimum number of entities in a given subsystem required to get to 51% of the total capacity. Aggregating this measure by taking the minimum of the minimum across subsystems then gives us the “minimum Nakamoto coefficient”, which is the number of entities we need to compromise in order to compromise the system as a whole.

We can also define a “modified Nakamoto coefficient” if 51% is not the operative threshold across each subsystem. For example, perhaps one might require 75% of exchanges to be compromised in order to seriously degrade the system, but only 51% of miners.

We can now use the Lorenz curves from the preceding section to calculate the Nakamoto coefficients for both Ethereum and Bitcoin. Here’s an example of the calculation for Ethereum’s reference client, geth. As we can see, with 2 developers we get to 51% of the commits to geth, so the Nakamoto coefficient is 2.

That illustrates the concept. Here are graphs for all of the subsystems for Bitcoin and Ethereum again, this time with the Nakamoto coefficients calculated:

And here’s a table where we’ve assembled the Nakamoto coefficients of each subsystem:

As we can see, given these essential subsystems, we can say that the Nakamoto coefficient is 1 for both Bitcoin and Ethereum. Specifically, the compromise of the Bitcoin Core or geth codebases would compromise more than 51% of clients, which would result in compromise of their respective networks.

Increasing this for Ethereum would mean achieving a much higher market share for non-geth clients like Parity, after which point developer or mining centralization would become the next bottleneck. Increasing this for Bitcoin would similarly require widespread adoption of clients like btcd, bcoin, and the like.

The Minimum Nakamoto Coefficient depends on Subsystem Definitions

We recognize that some may contend that a high level of concentration in a single reference client for Bitcoin does not impinge upon its decentralization, or that this level of concentration is a necessary evil. We take no position on this issue, because with alternate essential subsystem definitions, one can arrive at different measures of decentralization.

For example, if one considers “founder and spokesman” an essential subsystem, then the minimum Nakamoto coefficient for Ethereum would trivially be 1, as the compromise of Vitalik Buterin would compromise Ethereum.

Conversely, if one considers “number of distinct countries with substantial mining capacity” an essential subsystem, then the minimum Nakamoto coefficient for Bitcoin would again be 1, as the compromise of China (in the sense of a Chinese government crackdown on mining) would result in >51% of mining being compromised.

The selection of which essential subsystems best represent a particular decentralized system will be a topic of some debate that we consider outside the scope of this post. However, it is worth observing that the “founder and spokesman” and “China miner” compromises are two different kinds of attacks for two different chains. As such, if one thinks about comparing the minimum Nakamoto coefficient across coins, some degree of ecosystem diversity may quantitatively improve decentralization.

Conclusion

Many have said that decentralization is the most important property of systems like Bitcoin and Ethereum. If this is true, then it is critical to be able to quantify decentralization. The minimum Nakamoto coefficient is one such measure; as it increases, the minimum number of entities required to compromise the system increases. We believe this corresponds to the intuitive notion of decentralization.

The reason an explicit measure for quantifying decentralization is important is three-fold.

1. Measurement. First, quantitative measures like this can be computed unambiguously, recorded over time, and displayed in dashboards. This gives us the ability to track historical trends toward decentralization within subsystems and at the system level.
2. Improvement. Second, just like we measure performance, a measure like the Nakamoto coefficient allows us to begin measuring improvements and/or reductions of decentralization. This then allows us to begin attributing changes in decentralization to individual deployments of code or other kinds of network activities. For example, given scarce resources, we can measure whether deploying 1000 nodes or hiring two new client developers would provide a greater improvement in decentralization.
3. Optimization. Finally, and most importantly, a quantifiable objective function (in the mathematical sense) determines the outcome of any optimization procedure. Superficially similar objective functions can produce very different solutions. If our goal is to optimize the level of decentralization both across and within decentralized systems, we are going to need quantitative metrics like the Lorenz curve, the Gini coefficient, and the Nakamoto coefficient.

We recognize that there is plenty of room for debate over which subsystems of a decentralized system are essential. However, given a proposed essential subsystem we can now generate a Lorenz curve and a Nakamoto coefficient, and see whether this is plausibly a decentralization bottleneck for the system as a whole.

As such, we think the minimum Nakamoto coefficient is a useful first step towards quantifying decentralization.

Quantifying Decentralization was originally published in news.earn.com on Medium, where people are continuing the conversation by highlighting and responding to this story.

In 2014, we wrote that “Bitcoin is more than money, and more than a protocol. It’s a model and platform for true crowdfunding — open, distributed, and liquid all the way.”

That new model is here, and it’s based on the idea of an appcoin or token: a scarce digital asset based on underlying technology inspired by Bitcoin. While indisputably frothy, as of this writing the token sector sits at a combined market cap in the tens of billions. These new “fat protocols” may eventually create and capture more value than the last generation of Internet companies.

Here we discuss many concepts related to tokens, beginning with the basics for folks new to the space and then moving to advanced ideas.

The most important takehome is that tokens are not equity, but are more similar to paid API keys. Nevertheless, they may represent a >1000X improvement in the time-to-liquidity and a >100X improvement in the size of the buyer base relative to traditional means for US technology financing — like a Kickstarter on steroids. This in turn opens up the space for funding new kinds of projects previously off-limits to venture capital, including open source protocols and projects with fast 2X return potential.

But let’s start with the basics first. Why now?

1. Tokens are possible because of four years of digital currency infrastructure

The last time the public at large heard much about digital currency was in late 2013 to early 2014, when the Bitcoin price last touched its then all-time high of $1242 dollars. Since then, several things happened:

• Bitcoin experienced a massive multi-year crash and recovery all the way down to $173 and back up to the recent all-time highs of $2800+
• Dozens of exchanges arose in many countries to facilitate the conversion of fiat currencies like dollars or yen into digital currencies like Bitcoin and Ethereum
• Major financial institutions began exploring the blockchain technology underpinning Bitcoin to build so-called “private blockchains” or distributed ledgers for internal or consortium use
• The programmable Ethereum blockchain launched, endured its own major crises, brought on major corporate support, and surged in value in early 2017

In 2013, the legality of digital currency was still in question, with many predicting death and others going so far as to call Bitcoin “evil”. Those kneejerk headlines eventually gave way to Satoshi billboards in Davos and the Economist putting the technology behind Bitcoin on its cover.

By 2017, every major country has a digital currency exchange and every major financial institution has a team working on blockchains. The maturation of infrastructure and societal acceptance for digital currencies has set the stage for the next phase: internet-based crowdfunding of novel Bitcoin-like tokens for new applications.

2. Tokens vary in their underlying blockchains and codebases

To first order, a token is a digital asset that can be transferred (not simply copied) between two parties over the internet without requiring the consent of any other party. Bitcoin is the original token, with bitcoin transfers and issuances of new bitcoin recorded in the Bitcoin blockchain. Other tokens also have transfers and changes to their monetary base recorded in their own blockchains.

One key concept is that a token’s codebase is different from its blockchain database. As an offline analogy, imagine if the US banking infrastructure was repurposed to manage Australian dollars: both are “dollars” and have a shared cultural origin, but a completely different monetary base. In the same way, two tokens may use similar codebases (monetary policies) but have different blockchain databases (monetary bases).

The success of Bitcoin inspired several different kinds of tokens:

• Tokens based on new chains and forked Bitcoin code. These were the first tokens. Some of these tokens, like Dogecoin, simply changed parameters in the Bitcoin codebase. Others like ZCash and Dash innovated on privacy-preserving features. Still others like Litecoin also began as simple tweaks to Bitcoin’s code, but eventually became test grounds for new features. All of these tokens initiated their own blockchains, completely separate from the Bitcoin blockchain.
• Tokens based on new chains and new code. The next step was the creation of tokens based on wholly new codebases, of which the most prominent example is Ethereum. Ethereum is Bitcoin-inspired but has its own blockchain and was engineered from the ground up to be more programmable. Though this comes with an increased attack surface, it also comes with new capabilities.
• Tokens based on forked chains and forked code. The most important example here is Ethereum Classic, which was based on a hard fork of the Ethereum blockchain that occurred after a security issue was used to exploit a large smart contract. That sounds technical, but essentially what happened is that a crisis caused the Ethereum community to split 90/10 with two different go-forward monetary policies for each group. A real world example would be if all the citizens of the US who disagreed with the 2008 bailouts changed in their dollars for “classic dollars” and adopted a different Fed.
• Tokens issued on top of the Ethereum blockchain. Examples include Golem and Gnosis, all based on ERC20 tokens issued on top of Ethereum.

In general, it is technically challenging to launch wholly new tokens on new codebases, but much easier to launch new tokens through Bitcoin forks or Ethereum-based ERC20 tokens.

The latter deserves particular mention, as Ethereum makes it so simple to issue these tokens that they are the first example in the Ethereum tutorial! Nevertheless, the ease with which Ethereum-based tokens can be created does not mean they are inherently useless. Often these tokens are a sort of public IOU intended for redemption in a future new chain, or some other digital good.

3. Token buyers are buying private keys

When a new token is created, it is often pre-mined, sold in a crowdsale/token launch, or both. Here, “pre-mining” refers to allocating a portion of the tokens for the token creators and related parties. A “crowdsale” refers to a Kickstarter-style crowdfunding in which internet users at large have the opportunity to purchase tokens.

Given that tokens are digital, what do token buyers actually buy? The essence of what they buy is a private key. For Bitcoin, this looks something like this:

5Kb8kLf9zgWQnogidDA76MzPL6TsZZY36hWXMssSzNydYXYB9KF

For Ethereum, it looks something like this:

3a1076bf45ab87712ad64ccb3b10217737f7faacbf2872e88fdd9a537d8fe266

You can think of a private key as being similar to a password. Just like your private password grants you access to the email stored on a centralized cloud database like Gmail, your private key grants you access to the digital token stored on a decentralized blockchain database like Ethereum or Bitcoin.

There is one major difference, however: unlike a password, neither you nor anyone else can reset your private key if you lose it. If you have the private key, you have possession of your tokens. If you do not, you have lost access.

4. Tokens are analogous to paid API keys

The best existing analogy for tokens may be the concept of a paid API key. For example, when you buy an API key from Amazon Web Services for dollars, you can redeem that API key for time on Amazon’s cloud. The purchase of a token like ether is similar, in that you can redeem ETH for compute time on the decentralized Ethereum compute network.

This redemption value gives tokens inherent utility.

Tokens are similar to API keys in another respect: if someone gains access to your Amazon API keys, they can bill your Amazon account. Similarly, if someone sees the private keys for your tokens, they can take your digital currency. Unlike traditional API keys, though, tokens can be transferred to other parties without the consent of the API key issuer.

So, tokens are inherently useful. And tokens are tradeable. As such, tokens have a price.

5. Tokens are a new model for technology, not just startups

Because tokens have a price, they can be issued and sold en masse at the inception of a new protocol to fund its development, similar to the way startups have used Kickstarter to fund product development.

The money is typically received in digital currency form and goes to the organization issuing the tokens, which can be a traditional company or an open source project funded entirely through a blockchain.

In the same way that boosting sales is an alternative to raising money, token launches can be an alternative to traditional equity-based financings — and can provide a way to fund previously unfundable shared infrastructure, like open source. A word of caution, though: read these three posts and consult a good lawyer before embarking on a token launch!

6. Tokens are a non-dilutive alternative to traditional financing

Tokens aren’t equity, because they have intrinsic use and because they are non-dilutive to the company’s capitalization table. A token sale is more similar to a Kickstarter sale of paid API keys than equity crowdfunding.

However, when considered as an alternative to classic equity financing, token sales yield a >100X increase in the available base of buyers and a >1000X improvement in the time to liquidity over traditional methods for startup finance. The three reasons why: a 30X increase in US buyers, a 20–25X increase in international buyers, and a 1000X improvement in time-to-liquidity.

7. Tokens can be bought by any American (>30X increase in buyers)

A token launch differs from an equity sale — the latter is regulated by the 1934 Act, while the former is more similar to a sale of API keys.

While equities can only be sold in the US to so-called “accredited investors” (the 3% of adults with >$1 million in net worth), the US could not restrict the sale of API keys to accredited investors alone without crippling its IT industry. Thus, if tokens (like API keys) can be sold to 100% of the American population, it would represent an increase of 33x in the available US buyer base relative to a traditional equity financing for a US startup.

Do note, however: some people might want to issue a token and explicitly advertise it as a way to share in the profits of their efforts as a company. For example, the issuer might want to make token holders entitled to corporate dividends and voting rights, or make the company’s total ownership stock denominated in tokens. In these cases, we really are talking about tokenized equity (namely securities issuance), which is very different than the appcoin examples we’ve discussed. Don’t issue tokenized equity unless you want to be limited to accredited investors under US securities laws. The critical distinction is whether the token is simply a useful and tradable digital item like a paid API key. Again: read these three posts and consult a good lawyer before embarking on a token launch!

8. Tokens can be sold internationally over the internet (~20–25X increase in buyers)

Token launches are typically international affairs, with digital currency transfers coming in from all over the world. New bank accounts receiving thousands of wires from all over the world in minutes for millions of dollars would likely be frozen, but a token sale paid in digital currency is always open for business. Given that the US is only ~4–5% of world population, the international availability provides another factor of 20–25X in the available buyer base.

9. Tokens have a liquidity premium (>1000X improvement in time-to-liquidity)

A token has a price immediately upon its sale, and that price floats freely in a global 24/7 market. This is quite different from equity. While it can take 10 years for equity to become liquid in an exit, you can in theory sell a token within 10 minutes — though founders can and should cryptographically lock up tokens to discourage short-term speculation.

Whether or not you choose to sell or use your tokens, the ratio between 10 years and 10 minutes to get the option of liquidity is up to a 500,000X speedup in time, though of course any appreciation in value is likely to be larger and more sustainable over a 10 year window.

This huge liquidity premium alone would cause tokens to predominate whenever they are legally and technically feasible, because the time to liquidity enters inversely in the exponent of the compound annual growth rate. Fast liquidity permits reinvestment in new tokens permits faster growth.

10. Tokens will decentralize the process of funding technology

Because token launches can occur in any country, the importance of coming to the United States in general or Silicon Valley / Wall Street in particular to raise financing will diminish. Silicon Valley will likely remain the world’s leading technology capital, but it will not be necessary to physically travel to the United States as it was for a previous generation of technologists.

11. Tokens enable a new business model: better-than-free

Large technology companies like Google and Facebook offer extremely valuable free products. Despite this, they have sometimes come under fire for making billions of dollars while early adopters only receive the free service.

After the early kinks are worked out, the token launch model will provide a technically feasible way for tech companies (and open source projects in general) to spread the wealth and align their userbase behind their success. This is a better-than-free business model, where users make money for being early adopters. Kik is the first example of this, but expect to see more.

12. Token buyers will be to investors what bloggers/tweeters are to journalists

Tokens will break down the barrier between professional investors and token buyers in the same way that the internet brought down the barrier between professional journalists and tweeters and bloggers.

This will have several implications:

• The internet allowed anyone to become an amateur journalist. Now, millions of people will become amateur investors.
• As with journalism, some of these amateurs will do extremely well, and will use their token-buying track-record to break into professional leagues.
• Just like it eventually became a professional requirement for journalists to use Twitter, investors of every size from seed funds to hedge funds will get into token buying.
• New tools analogous to Blogger and Twitter will be developed that make it easy for people to use, buy, sell, and discuss tokens with others.

We don’t yet have a term for this, but perhaps it will be “commercial media” by analogy to “social media”.

13. Tokens further increase the primacy of the technologist over the traditional executive

Since the rise of Bill Gates in the late 70s, there has been a trend towards ever more tech-savvy senior executives. This trend is going to accelerate with token sales, as folks who are even more predisposed to the pure computer science end of the spectrum end up founding valuable protocols. Many successful token founders will have skillsets more similar to open source developers than traditional executives.

14. Tokens mean instant custody without intermediaries

Because token buyers need only hold private keys to guarantee custody, it changes our notion of property rights. For tokens, the final arbiter of who possesses what property is not a national court system but an international blockchain. While there will be many contentious edge cases to work through, over time blockchains will provide “rule-of-law-as-a-service” as an international, programmable complement to the Delaware Chancery Court.

15. Tokens may be generalizable to every tech company through paid logins

Can the token model can be extended beyond pure protocols like Bitcoin, Ethereum, or ZCash? It’s not hard to imagine selling tokens as tickets — for access to logins, to car-rides, to future products. Or distributing them as rewards to the authors who power social networks and the drivers who power ride-sharing networks. Eventually, tokens can be extended to hardware as well: every time someone buys a slot in line for a Tesla Model 3 or re-sells a ticket, they’re exchanging a primitive token. But the model will need to work for protocols first before being generalized.

Conclusion

The token space is very early, and is likely to experience a dramatic correction over the next few weeks. To deal with the coming profusion of tokens we will need review sites like Coinlist, portfolio management tools like Prism, exchanges like GDAX, and many other pieces of supporting technical and legal infrastructure.

But the world has changed. Tokens represent a 1000X improvement over the status quo, and those don’t come around very often.

PS: If you thought this post was interesting, go join the list at Earn.com/digital-currency/join. You’ll get notified of several upcoming token launches.

Thanks to my friend and colleague Naval Ravikant for helping think through many of the ideas in this post! Go follow him on Twitter at @naval.

Thoughts on Tokens was originally published in news.earn.com on Medium, where people are continuing the conversation by highlighting and responding to this story.