How AI fear crushed the indie creator of ProseCraft.io for no good reason

I’m a writer, but this will be a story criticizing writers.

I plan to debut a novel someday, and I know this may hobble my chances. The lit crowd is a close-knit and cliquey sort.

But I’ll do it anyway, because that’s what you do when giants go curb-stomping underdogs.

The demise of ProseCraft.io

A few days ago, an independent creator had to take down his labor of love—ProseCraft.io—after crushing pressure from the writing community.

ProseCraft was a project that had launched in 2017 and hadn’t actively maintained a blog presence since 2019, so this sudden and acute backlash must’ve been jarring, to say the least.

Even more surprising: ProseCraft.io seemed to just publish summary statistics on novels, with word frequencies and a few exotic metrics like “vividness”.

Not terribly useful, but maybe when plotted on a trend-line over time, or compared across genres, it could uncover some interesting insights. Hard to say without fooling around with it. But it doesn’t seem terribly harmful, either.

So why the outrage? Why the death threats and verbal abuse?

Who are all these angry people that came out of the woodwork to rake this random smalltime hacker over the coals?

There are two main buckets I see, here.

Bucket one: the anti-piracy folks. These writers speculated that ProseCraft likely needed full-text novels to run analytics on them, and that such a large database probably wasn’t paid for out-of-pocket by just one dude.

This is a bucket of people I won’t directly address, because we just fundamentally disagree in our values. For purposes like analytics and research—even if that aggregated data is later sold, e.g. in a scientific journal or as part of consulting services—ripping a book is perfectly fine. I could do much the same by taking my library card, checking a book out of a digital library, analyzing it, and saving those numbers in my notes.

If these people were at all consistent in their views they’d probably also dunk on tools like Sci-Hub or Libby. Maybe they could learn a lesson or two from authors like Wallace Wang, who wrote Steal This Computer Book 4.0. You can probably imagine what his take would be.

Bucket two: the folks who don’t want their work to be used in profit-generating AI. This seemed to be the larger cohort. It wasn’t the data-sourcing that was a problem—it was the fact that it might be used to train some sort of algorithm that a company could use to make money.

The creator of ProseCraft also had a monetized product called Shaxpir, and he made the fatal mistake of mentioning AI somewhere within a 3-mile radius of its product spec. This was all people needed to jump down his throat.

But AI is a broad umbrella term. Some of it can be genuinely harmful—but a lot of it is so unsophisticated that it doesn’t even have the capacity to hurt the writing community.

The latter is what was at play, here.

Technical breakdown

After caving to pressure, Shaxpir removed the features that used algorithms trained on authors’ works. While tragic, it’s useful for our analysis because he identifies the features outright and we don’t have to speculate what he was doing.

These features were:

• A thesaurus function
• A “vividness” and sentiment analysis of prose
• Linguistic percentile analysis

Thesaurus

This feature lets you swap out a word with another one that makes sense given the context. Kind of like a regular thesaurus and its list of synonyms, but far more context-aware.

How does it work?

By using a natural-language processing algorithm that is now positively medieval: word2vec.

How do you train it? In broad strokes:

1. You represent every word as a set of random GPS coordinates that says where the word is in your “word-space”. Numerical representations of words are what let you do fun things like assert that “King - Man + Woman = Queen” and other cute little operations like that.
2. You chop up the text corpus—in this case, novels—into pairs of words that “co-occur”. Whether that means being right next to each other, or just sharing a clause, a sentence, or a whole document is up to you.
3. You jiggle the coordinates of each word such that their distance from each other roughly matches their co-occurrence rate in your document database.
4. Ta-da! You now have a word-space where each word-coordinate is representative of what other words they tend to appear by, and can be used in a narrative-focused thesaurus.

None of this exploits an author’s voice, talent, or style—like maybe some modern Large Language Models might.

It certainly can’t be used to, say, spam beloved sci-fi magazines with factory-made submissions, or mimic an author’s brand to churn out derivative works. You know, actual harmful uses of AI in literature.

ProseCraft and Shaxpir were barely a layer removed from just counting word-pairs. This family of statistical modeling is so unsophisticated that they’re called “bag-of-words” models—because that’s what they do. Put a bunch of words disembodied from their sentences into a bag, and count them up.

Vividness and sentiment analysis

The vividness-analysis feature tells you how many “vivid” sensory words a block of text invokes as well as how intense they are.

This feature might need:

1. Part-of-speech tagging, which is a classic computational linguistics problem people have been hacking at for a while. This can isolate nouns, verbs, and adjectives.
2. Vividness classification, to identify words that count as “vivid”. The Shaxpir blog makes it seem as though they identified which nouns and verbs were vivid by hand. This takes a lot of elbow grease, and means that they personally dedicated time to enrich the dataset.
3. Vividness scoring, since not all vivid words were made equal. The intensity of the vividness appears to be based on word-frequency: so they’d count up how often these words showed up in their library of authors’ works, and the ones that were rarer across the corpus would be more vivid. These rarity metrics would be mapped to scores between 0 and 10.

For sentiment analysis, the analytical technique is much the same, except instead of “vivid” words you’re instead tracking words with positive and negative valence—and the average valence for a block of text is its sentiment.

Again, none of this exploits authors or makes it harder for them to make a living. The words are all ground into a homogenous paste and the software only looks at counts and frequencies—there’s nothing “stolen” here.

Determining that the letter “E” is the most frequently used letter in the English language doesn’t demand consent from every single person who ever picked up a pen—and this kind of statistical analysis certainly doesn’t require buy-in from or remuneration to authors in this text corpus.

Would you really charge someone if they asked you how many times you use the words “tequila” or “underpants” in your sentences? I doubt it.

Linguistic percentiles

This is a meta-feature that works on top of the other features delivered by ProseCraft and Shaxpir—and tells you how a certain metric compares with the rest of the library.

This puts a work in rank-order based on some metric, and then tells you how it placed.

That’s it.

There’s nothing harmful here.

These authors boiled themselves up into a tizzy over next-to-nothing, and then ruined some dude’s day over it because he smelled vaguely tech-adjacent.

Goliaths playing at David

The thing that pisses me off most about this whole affair is how sneering and sanctimonious the offending authors were while engaging in this absolutely unfounded crusade.

They came into the situation with an attitude like they were victims, and it was they who were rising up against some sort of powerful oppressor.

Meanwhile, casual observers would note that:

• Benji Smith, the indie creator of Shaxpir, had less than two thousand followers on Twitter and likely even fewer users of his product, which seemed to have been inactive for several years.
• The authors that dogpiled him were people like Jeff VanderMeer—who has a fucking Hollywood movie made from his books, with around 70,000 Twitter followers—and Robert Brockway, a Cracked-writer turned novelist with 22,000 followers. These people are, compared to Benji, titans.

But the way they speak makes it sound like they just had some kind of world-changing revolution against an empire. Look at this silly nonsense:

Gabino Iglesias on Twitter: "Prosecraft is no more. Why? Because writers got pissed. Because we won't take shit and let our work be fed into machines. The fight is just starting, but we're in this for the long haul. (Not gonna debate AI with anyone, so save your "it's just like using a calculator" comments) pic.twitter.com/TFrXIk0jHA / Twitter"

Prosecraft is no more. Why? Because writers got pissed. Because we won't take shit and let our work be fed into machines. The fight is just starting, but we're in this for the long haul.

Every single one of these “revolutionaries” is hyping themselves up thinking that they just went up against The Man and won—

But guess what, you self-righteous fucks?

You are The Man.

🐣 kabir on Twitter: "Because I'm petty and spiteful, I've created a generative AI trained solely on the text works of every author who jumped onto this dogpile of ProseCraft.Find it now at https://t.co/Ckj9kpDivf - enjoy https://t.co/7pZTZI9Y3b / Twitter"

Because I'm petty and spiteful, I've created a generative AI trained solely on the text works of every author who jumped onto this dogpile of ProseCraft.Find it now at https://t.co/Ckj9kpDivf - enjoy https://t.co/7pZTZI9Y3b

A Pattern-Recognition Critique of Dr. Shiva Ayyadurai

Election fraud in Michigan? Nope, just a huckster

Dr. Shiva Ayyadurai is doubling down on a video which I poked holes in a few days ago. In it, he claimed that Joe Biden stole more than 60,000 votes in Michigan. It involved poor mathematics, and folks like Matt Parker of StandupMaths drew similar conclusions.

Stanford Ph.D and director of MIT’s Election Lab, Charles Stewart III, happens to agree with us.

Interested in the Dr. Shiva nonsense about an algorithm flipping votes in MI? Post by @kabircreates is pretty good on one of the math flaws. My additional issue is that you can't reproduce his graphs using the precinct returns on the Oakland website.https://t.co/zFv21WkjPP

In this new video, Ayyadurai dismisses math-based criticisms by saying that detecting election-fraud is “not a math problem, but a pattern-recognition problem.”

Luckily, pattern recognition is my main discipline and the basis for my professional career — starting at the University of Pennsylvania where I learned how to detect seizures from ECOG data in BE521, developing during SBIR and STTR grant research for the Navy while at Commonwealth Computer Research, Inc, and finally coming into full form at a fin-tech start-up focusing on enhancing employee financial health.

So. We can talk about what he gets wrong, here.

First I want to start by just noting inconsistencies, though, because it doesn’t take an expert in pattern recognition to recognize when someone declares one thing and then says something completely contradictory. Let’s go for it.

Inconsistencies

Composite Curves, Are They Normal Or Bad?

Recall, in the last video, the kinked curve in Ayyadurai’s graphs were a reason for alarm. He pointed at the bend in the curve and said, aha, this must be where the voting machines’ anti-Trump algorithm kicked in! Clear alarm signal!

He goes as far as to say that some kind of transistor function or ramp function is at work (39:05), an algorithm deep in Diebold-esque vote tabulation machines.

Yet in the new video, he says that combined flat and sloped curves are the surefire feature of a “normal” election.

Which one is it? You can’t have it both ways.

Wayne county: was there cheating or not?

This is my favorite graph in all of Ayyadurai’s presentations. It’s the slide of Wayne County, which he flashes again in his new video:

He shows it’s a flattish trend-line and the slide still says there was “no cheating.” But this certainly isn’t a “normal” composite curve, so there must be cheating according to his rule above, no?

That’s not even the best part, though. In this county where there was “no algorithm detected”, I’ve circled in yellow some points that cannot possibly exist. Remember that Dr. Shiva is plotting (% split-ticket Trump votes﹣% straight-ticket Republican votes) on the Y-axis, and % straight-ticket Republican votes on the X-axis.

If you look, there are points at approximately (X: 4%, Y: -20%), or (X: 2%, Y: -10%), etc. Those data-points would imply that Trump got negative split-ticket votes at those precincts. e.g if -20 = SplitTrump﹣4; SplitTrump must be -16.

That would be a surefire piece of evidence that there was cheating, since you can’t have negative votes — but of all the counties he plots for us, this is the only one where there was “no cheating algorithm detected”.

At this point I’m like 30% sure he’s just trolling, and that this is him at peak comedy. But really it just means you can’t trust his plots, because clearly he shows fake data.

Are flat lines good? Or are they signs of fraud?

In the last video, Dr. Shiva Ayyadurai set the expectation that a flat line in his graphs should be a signal of normalcy. He plotted it in the case where sentiment towards Trump was about even with the Republican Party:

In response to a comment from Mitt Romney saying we should expect poor sentiment towards Trump from lifelong Republicans, Ayyadurai then posits this “normal” graph:

This graph should be alarming, because it also has an impossible point on it. On the far left you have one that’s approximately (X: 5, Y: -8) — which would imply it’s “normal” for you to observe -3% split-ticket voting percentages for Trump.

Anywho, besides being inconsistent with itself, it’s inconsistent with what he says in his new video — where he expects a composite curve in the usual case.

Feature Engineering

Let’s talk a little bit about feature engineering. In this video Ayyadurai touches a lot on how important it is to extract information-dense bits of signal from inputs — and this is correct. The majority of a professional machine learning engineer’s job will mostly involve milling raw data into derived features that will correlate well with a final inferred output — in Ayyadurai’s case, that’s “Election Fraud” or “No Election Fraud”.

Ayyadurai’s defense against Matt Parker’s criticism that subtracting two percentages doesn’t have any meaning is: it doesn’t have to mean anything mathematically, it just needs to be a good correlate for distinguishing “Election Fraud” from “No Election Fraud”. He says this is more of an art than it is just mathematics.

It’s true that feature engineering takes creativity, intuition, and artfulness, but it always does come back to down to mathematics — and we’ll see how in a second.

But first let’s get a sense of what Ayyadurai considers important features:

He mentions just the two here, but it’s enough to critique. Some criticisms:

Features shouldn’t be correlated with each other

Generally, features you use for a machine learning model shouldn’t be correlated. It’s hard to tell without knowing what downstream statistical model you’re using — since some won’t care, like decision trees and all the ensemble variations of decision trees—but at worst it’ll screw up your models and at best it just won’t be helpful.

It’s particularly harmful for understanding your multivariate linear regression models, where you make an assumption that there is no multicollinearity.

In this case, one of Ayyadurai’s features (2) is used as a term in another one of his features (1), meaning they are destined to be correlated. This is that big ol’ mean ol’ math criticism Shiva doesn’t like coming back to haunt him, here.

Shiva mentions clustering data in his presentation as well. Correlated features can lead to problems with clustering, since it will warp and skew your feature-space so things aren’t as nicely separable. It’d be like plotting points in a Cartesian space where the axes weren’t at right angles to each other, and then trying to distinguish clusters of points.

Speaking of clustering, though —

What are you even going to cluster?

In the stated research aims of Ayyadurai’s project, clustering is Aim 2.

Clustering is where you take your set of data-points and plot them in feature-space (the space where each feature is one of your axes) — and you split them into distinct groups, where each group has its own label. In Ayyadurai’s problem, each label should presumably be “Election Fraud” or “No Election Fraud”.

Below is an example of some points in feature-space being clustered into three classes by the K-means algorithm.

But here’s what Ayyadurai’s plots look like in his feature-space:

What clusters would you even make here, and what would they mean?

A deeper issue: the points in his feature-space are precincts. What he’s proposing is that his features are able to distinguish between fraud and no-fraud at the precinct level, but in every single one of his talks he mentions fraud being a property of a county, not a precinct.

Which leads me to—

Ayyadurai doesn’t know what a feature is

For a lad who took the first 20 minutes of the whole video fawning over his own accolades in pattern recognition, he sure does seem to be confused.

A feature is a property of the object that can be discriminated into its different classes (or regressed onto a continuous value). In Ayyadurai’s case, that object is a county. He’s always talked about counties as having fraudulent elections or not, and has never made claims that he’s been detecting fraud at individual precincts.

This leads me to believe that what he calls a “signal” is what he actually means as his feature, and what he calls a “feature” is a sub-component used to derive that feature.

So really, his feature is the polynomial degree of the curve he’s plotting, on a county-by-county basis.

A degree of 1 (a line) is bad, and a degree of >1 is good, according to this slide. I don’t know if he just mixed up terms because he was running on low sleep or something— but I suspect he was just desperately trying to save face by trying to play off his choice of (% split-ticket Trump votes﹣% straight-ticket Republican votes) as an “artful feature choice” rather than a mistake/deceit he and his team made.

So there you have it. A small taste of a pattern recognition person’s critique of Dr. Shiva Ayyadurai MIT Ph.D, B.S.E.E, M.S.V.S, M.S.E.E.

A recommendation: you probably shouldn’t trust this guy, since he seems to just be using techno-babble to push an agenda. It is your American right to put political pressure on your Secretary of State to ensure that we have free and fair elections via audits/recounts/hand-counts, etc., but lending your support to scam artists like this who just rile up conflict via disinformation is not the way to do it.

Election Fraud in Michigan? Nope, just being misleading with data.

A few days ago, Dr. Shiva Ayyadurai posted a video that claimed to prove election fraud in Michigan. He is wrong, and I’ll show you how using data from Oakland County, Michigan. My code and data sources are public and replicable — and everything I write is open for comment and discussion.

Previously, I posted a detailed takedown of how his analysis was a mathematical parlor trick — which he uses to generate a “suspicious” result that’s supposed to prove that Biden stole 60,000+ votes from Trump.

But many people aren’t convinced by theory or abstraction, so I figured the best proof would be to play that parlor trick myself: this time “proving” that Trump stole votes from Biden. To be clear, this isn’t a proof of anything. It’s a trick.

To demonstrate this, I will use election data from Oakland County, Michigan.

Recap of Ayyadurai’s Thesis

Before we do any work, let’s recap what Ayyadurai is doing.

The dataset Ayyadurai works with contains:

• Precinct-level % of Republican voters among straight-ticket voters.
• Precinct-level % of Trump votes among split-ticket voters. These are what he calls “Individual Candidate Voters”, people who did not select a party’s “straight-ticket” option on their ballot.

The main quantity Ayyadurai is concerned with is: the % of Trump votes among split-ticket voters MINUS the % of Republicans among straight-ticket votes in a precinct.

He finds it very suspicious that the quantity that he’s plotting has a negative slope, and sees that as evidence that there is foul play. He accuses Michigan of using the Dominion voting machine’s weighted race feature to do this, creating a linear pattern which he notes as suspicious and telling of algorithmic tampering.

What I’ll show you is that you’d get that same negative slope if you flip the Party in question and instead use:

• Precinct-level % of Democrat voters among straight-ticket voters.
• Precinct-level % of Biden votes among split-ticket voters.

Getting Data

To grab the data, I visited the Oakland County site and went to the unofficial election results for November, which routed me to this data source. Here I downloaded their XML data and got to work formatting it into something workable.

Great, now we can obtain those quantities we care about:

• Precinct-level % of Democrat voters among straight-ticket voters.
• Precinct-level % of Biden votes among split-ticket voters.

An important note is: to isolate split-ticket vote counts towards Presidential candidates, you have to subtract the straight-ticket votes towards them. This isn’t perfect since straight-ticket voters can “defect” — but Ayyadurai assumes that straight-ticket voters all vote for their Presidential candidate in his video, so I’ll do the same.

The Parlor Trick

Let’s now plot the quantities Ayyadurai does. On the Y-axis is a difference: the % split-ticket votes for Biden MINUS the % straight-ticket Democrat votes. On the X-axis: just the % straight-ticket Democrat votes.

A negative slope — what do you know. Does that mean Trump was stealing votes from Biden?

Of course not. It just means Ayyadurai’s a hack.

The Parlor Trick, In Reverse

Just so you can see my code is consistent with what Ayyadurai’s doing, here I plot the same quantities, in favor of Trump. So that’d be exactly what he prescribes here:

I do that below:

Also a negative slope.

By design. For details on how this result is pretty much inevitable regardless of your dataset, dig into my previous work.

TL;DR: Ayyadurai says we should expect a flat line in the plots above, and there is only one case that will lead to a flat-line. It would happen only if the slope of the correlation between % of split-ticket votes for Trump and % of straight-ticket Republican votes is exactly 1. We have the data at hand, so I can show you that the true slope is actually more like 0.6.

Since the slope of that trend-line isn’t ≥1, we’d always expect a negative slope in Ayyadurai’s plots. If it were > 1, then we’d expect a positive slope in Ayyadurai’s plots — which is still not evidence of fraud. It’s just… how lines work.

Don’t let this dude fool you — he thinks we’re all rubes to be used for his own personal gain. Buck him off.

Election Fraud in Michigan? Nope: just how lines work

NOTE: On Nov. 16th, Ayyadurai doubled down on his misleading analyses.

On November 10th, Dr. Shiva Ayyadurai posted a video claiming that some simple analytics revealed election fraud in Michigan. It received more than 200,000 views, and claims that Joe Biden stole more than 60,000 votes in Michigan.

The main thrust of his analysis is a mathematical parlor trick. In a separate post, I play that parlor trick myself with Oakland County data to “prove” the opposite conclusion — showing that his analysis is bogus at its core.

I rarely pick on people for making analytical mistakes, because it happens to everyone and every instance is a lesson to learn from. But in this case it seems less like a mistake and more like an intentional deception — and you’ll see that the audacity of it is astounding. The man’s trying to fool us into believing in election fraud by taking advantage of how lines work. Bold move. Let’s see how it goes.

The Data

The dataset Ayyadurai works with contains:

• Precinct-level % of Republican voters among straight-ticket voters.
• Precinct-level % of Trump votes among split-ticket voters. These are what he calls “Individual Candidate Voters”, people who did not select a party’s “straight-ticket” option on their ballot.

The main quantity Ayyadurai is concerned with is: the % of Trump votes among split-ticket voters MINUS the % of Republicans among straight-ticket votes in a precinct.

He shows that the higher the percentage of Republican voters in a precinct, the more negative this difference is. He speciously says that this is an indicator of how much more popular Trump is among split-ticket voters than Republican candidates, and that it’s strange for it to be more negative in more Republican precincts.

According to the good doctor, the pattern you’d expect if there was just normal negative sentiment towards Trump is this flat line:

To Dr. Shiva, this discrepancy is evidence of algorithmic foul play. To him, the negatively sloped line we see is a clear sign that the state of Michigan used the “weighted race” voting feature of vote-tallying machines to steal votes from Trump and grant them to Biden.

The thing is, that you’d always expect the negatively sloping line that he describes as “suspicious”, by definition. Think about what he’s plotting. On the X-axis is the % of straight-ticket Republican voters. On the Y-axis is the % of split-ticket Trump voters MINUS the % of straight-ticket Republican voters.

If we were to draw the equation for that line, it’d be:

If you remember your formulas for lines, you’ll know that this is by construction biasing us to get a line that will slope down 1:1, at 45-degrees. Regardless of the actual split-ticket vote data and how it’s distributed, the quantity Ayyadurai’s plotting is rigged to look like a negatively sloping line from the outset.

A thought exercise: assume that nobody on a split-ticket ever votes for the major two parties, and they all go third-party like true independents. What would you expect from Ayyadurai’s curve? It would just go down and to the right, by design.

And if a dude is artificially constructing a negatively sloped line in front of your eyes and telling you that this “beautiful, too-perfect line!” is evidence of election fraud, you should run the other way. Because he’s trying to take advantage of your good intentions.

Again, here’s the slide where he describes what he’s plotting. It is, from the beginning, set up to create a line of negative slope, since its Y-axis includes a negated term from the X-axis.

If you’re more comfortable with statistical simulations of split-ticket voter data than with this algebra, then let’s dig into some code and some details, as well as answers to some common challenges.

If you don’t like theory/simulations at all and instead want to look at real data, I do that here.

How Ayyadurai Is Misleading You, In Detail

We’re gonna keep this as simple as possible. Let’s say that each precinct has two main populations of people: people who vote with a straight-ticket ballot and people who do a split-ticket vote. Ayyadurai himself frames these as two distinct populations, so I’m rolling with that premise here.

We can focus on percentages of Republican voters among straight-ticket ballots, and percentages of Trump votes among split-ticket ballots (or “Individual Candidate” votes, as he describes). We’re dropping out other voter groups because Ayyadurai doesn’t address them in his analysis at all.

Now, some assumptions we’ll make before we start running some simulations:

• As Ayyadurai assumes: all Republicans who vote straight-ticket will vote for Trump.
• Split-ticket voters, on the other hand, just have some random, non-zero chance of voting for Trump. We will assume this is a flat probability for now (but I show how my conclusions hold for almost any distribution of split-ticket votes, flat or otherwise, later in the article).

We can now simulate the voting outcomes of several precincts. We can simulate the % of Republican straight-ticket voters with a simple uniform sampling.

We can sample what split ticket voters would do by doing a series of weighted coin-flips, using their fixed probability of voting for Trump.

And now we plot the difference between % straight-ticket Republican voters and % votes for Trump among split-ticket voters, just as Dr. Ayyadurai does:

Bam. It’s a line with a negative slope. Just like we actually observe in real life, in counties where we have a nice spread of precincts with different fractions of Republican voters. No evidence of fraud here — just a simple result of the values we decided to plot.

So, what’s going on?

Let’s look closely at what we’re doing.

We’re plotting the line where the Y variable is: (% split-ticket Trump votes MINUS % straight-ticket Republican votes). This is plotted against the X variable, the % of straight-ticket Republican votes.

We can decompose this into two lines:

One line we have is the % of split-ticket Trump votes on the Y axis, vs. the % of straight-ticket Republican votes on the X-axis. If split-ticket voters have a fixed probability of voting Trump that is independent of their precinct’s % of Republicans, this line should be flat.

On the other hand, the other line we have is the % of straight-ticket Republican votes vs. % straight-ticket Republican votes. Otherwise known as the line of unity: a perfectly correlated line, since we’re plotting the same thing on both axes.

To recover Ayyadurai’s original line of (% split-ticket Trump votes MINUS % straight-ticket Republican votes) vs. % straight-ticket Republican votes, we can subtract our second line from our first line. Remember that subtracting a line is the same as the negating a line and adding it. The line of unity, negated, is this:

You can see how it’s obvious that we’ll get a negatively sloping line after doing this. The operation we are performing will produce a downward sloping line by definition.

This dude must think that we were born yesterday. Wow. Again, the sheer audacity of this just kills me.

What if the probability of split-ticket votes for Trump isn’t constant?

Let’s present Ayyadurai’s argument in the strongest light. Here’s a common objection from respondents: why do we assume a flat probability of split-ticket vote percentages? What if the probability of split-ticket votes for Trump is linearly correlated with the % of straight-ticket Republican voters in a precinct?

Let’s indulge this. Let’s say that straight-ticket voters are the majority of a precinct, and so “Republicanism” or “Democraticness” leaks out into the split-ticket voting population and influences their decisions. In that case, we’re no longer subtracting a 1:1, 45 degree line from a constant, flat line.

However, we are subtracting the 1:1 line of unity from some other line.

In the simulation below, I assume that the linear correlation between the % of split-ticket Trump votes and the % of straight-ticket Republican votes has a slope of 0.9.

Here’s the component where I just plot % split-ticket Trump votes. This is no longer flat, because we’re saying there’s a correlation between split-ticket Trump votes and the “Republican-ness” of a precinct.

Plotting the line of unity is the same:

And when you subtract the second from the first, you again get a negatively sloping line!

The only way you’d get the flat line — which Ayyadurai asserts is “expected” — is if the slope of the correlation between split-ticket Trump votes and straight-ticket Republican votes is exactly 1:1. No more, and no less. In every other case, you’d get a line with some slope.

In my work with real data from Oakland County, MI — I show you that this slope is far from 1. It’s far lower than my simulated 0.9, too: it’s just 0.6. We’d always expect Ayyadurai’s “suspicious” plunge, in this case. Biden also experiences this plunge — though it’d have been totally acceptable to see an upward sloping line in either case, too.

Again: not fraud. Just lines behaving like lines.

Invite to Dr. Ayyadurai

If this is truly what Dr. Ayyadurai meant to do, then it’s essentially just a mathematical trick. A lot of build-up and sleight-of hand for an inevitable and mundane result. It’s kinda like if I asked you to think of a number, add 5, subtract 8, and tell me the result. You shouldn’t be impressed if I could guess your number: all I have to do is add 3 to your result and I’ll know what you started with. In the same vein, Dr. Ayyadurai is conjuring a “suspicious” negatively sloped line from any data that you would likely start with.

You could do the same thing with straight-ticket Democrat votes and split-ticket votes for Biden. In fact, I do so here. Is that evidence that Trump is stealing votes from Biden? Of course not. It is crazy to me that Ayyadurai can put on a straight face and pitch this to us as evidence for conspiracy-level election fraud — he must think very little of us.

I know people are very quick to trust something just because it’s got some numbers on it — especially if it agrees with them — but numbers don’t make an analysis bulletproof. Not inspecting even the quant-iest of presentations can leave you vulnerable to a new breed of snake-oil salesman. Case in point: Dr. Shiva Ayyadurai. U.S. Senate Candidate.

That goes double for me, too. Inspect my argument carefully. If you see trouble, send me a rebuttal right here. Dr. Ayyadurai, that goes for you especially. I’ve answered a few responses below:

Some good challenges from respondents so far:

What about Wayne county? It doesn’t display this negative linear trend.

Correct, I should have addressed this in the main body of the article. I think the issue with Wayne is that there isn’t an even spread of precincts with a wide variety of %-Republican voters. He also zooms in on the X-axis, so we’re only looking from 0–30%, where the value he’s plotting is likelier to be positive. I dig into that case in this Git Gist.

One thing I do want to point out is that Ayyadurai’s graph of Wayne county shows impossible data:

If you look, there are points at approximately (X: 4%, Y: -20%), or (X: 2%, Y: -10%), etc. Those data-points would imply that Trump got negative split-ticket votes at those precincts. e.g if -20 = SplitTrump - 4; SplitTrump must be -16.

If these data were real you could argue that there was cheating in Wayne county, because negative votes should be impossible! Yet this is one of the few counties where Ayyadurai says “no algorithm detected”.

My sides. At this point we’re basically just watching a stand-up comedy special.

How do you explain the fact that Ayyadurai’s regression has two line-segments? That doesn’t agree with your thesis that there should be one negatively sloping line.

Dr. Ayyadurai plots this trend-line over his data:

But we could just as easily draw this line:

The choices are arbitrary. The only way to rigorously tell which fit is better is to run the regressions on the data ourselves and see which model minimized the error more.

An important point: in model selection, the aim is always to follow Occam’s Razor and go with the most parsimonious model you can. If you just try and minimize error without controlling the complexity of your model, your “best” model will be a line wildly flitting from exact point to exact point. We call this overfitting.

Oftentimes you’ll compare competing models with something like the Akaike Information Criterion, which punishes models that are more complicated than they need to be (models that have more parameters). Without looking at the raw data itself, I’d say a single regression line would certainly be a simpler model fit than trying to place two linear regressions in a piecewise model like he draws.

If the kink in the curve is real, it could also be due to a non-linear correlation of % straight-ticket Republican voters and % split-ticket Trump voters. A detailed treatment of how the line can be curvy is offered by Dr. Lum in her video here.

EDITED NOTE: Thanks to Carlos Larriba Andaluz and Sam Nim for pointing out a previous analysis’ axes that didn’t match the video’s exactly. Now they do. Generally: a big thank you to everyone who engaged via the responses and helped me sharpen my argument and get closer to the truth.