I feel like we overcomplicate things in data science sometimes…which is saying a lot, coming from me, since empirical evidence suggests that I love to overcomplicate things. Anyway…here are two concepts that a reasonable logically-minded person might talk about:

• False positive: a test comes back positive but the thing the test is testing for isn’t there in reality (two lines on the pregnancy test but you are definitely not with child)
• False negative: a test comes back negative but the thing the test is testing for is there in reality (one line on the pregnancy test but you can feel the baby kicking)

Now, if you think about a binary model prediction (1 or 0, yes or no, true or false, bert or ernie, whatever floats your boat) as a form of a test, which it is, we get a whole pile of new terms- accuracy, precision, recall, type I error, type II error, etc. etc. depending on where you are coming from in the data science ecosystem. (the last two are largely economics terms for false positive rate and false negative rate in case you are not familiar with them) If we are being honest, it’s kind of hard to remember which measure is which, and despite having seen them for like 25 years now I may have just had to google which one was type I and which one was type II error. Add into all of this the fact that people act like Bayes Theorem is an entirely new concept completely unrelated to the terminology soup above and I swear I am going to lose my mind.

Look, in this prediction/testing setup there are only four possibilities:

The upper left box is that the thing exists in reality and the prediction or test gave a positive, and so on and so forth. (this is technically a confusion matrix in prediction model speak, but the whole point here is to not get hung up on terminology) If we put probabilities in each of these boxes, we can then reason out all of the measures mentioned earlier as well as the Bayes theorem brainteasers that are so popular in way too many data science/data engineering interviews. (I personally do not care whether the guy building my data pipelines can calculate conditional probabilities but you do you.) So let’s think about how to do this via an example:

The underlying “base rate,” if you will, is 3%, recall is 98%, and the false positive rate is 1%, but again none of those definitions are necessary in order to move forward…what is necessary, on the other hand, is to understand that the probability of both A and B is the probability of A times the probability of B given A, like so:

• P(A and B) = P(A)*P(B/A)

(technically you could reverse the A and B here if it makes things easier since and is symmetric) Also, the probability of not a thing is 1 minus the probability of the thing, so you can fill in the chart as follows:

or, if you prefer the numbers:

With this information, along with the observation that these probabilities are additive since the boxes are mutually exclusive (for example, the probability of the thing happening in reality is 0.0294 plus 0.0006, or 0.03 as expected), you can get to whatever prediction statistics your little heart desires- for example, accuracy would be 0.0294 + 0.9603 = 0.9897, recall would be 0.0294 / (0.0294 + 0.0006) = 0.98, and so on. (boy I hope I am doing these words right haha)

Even the Bayes thing becomes pretty trivial- in this context, the Bayes question is most likely “what is the probability of the event given that the prediction is yes?”, or

• P(prediction and reality) / P(prediction)

The numerator is just this handy red box:

and the denominator is just the sum of these red boxes:

Easy, right? In case you’re curious, this gives a probability of about 75%, and hopefully you can see how you can reason this out rather than having to rely on terminology and memorization.

You can even play around with the input numbers and see how the results change here.

Originally published at https://jodibeggs.substack.com.

Product Owner: I want to predict how future revenue will develop…

Me: starts spinning up ARIMA model

PO: …if we (insert product or pricing change here)

Me: curses,throws model in virtual trash can, starts coding a regression of some sort

Actually, this is kind of a best-case scenario, since at least the error was caught right away and not once models had been developed and initial results presented. In any case, the point I am trying to make here is that businessey types (I say this with love) throw around the word predict without thinking about what they are really trying to accomplish…sure, in some cases they might truly just want to understand, for example, what the future revenue trajectory is likely to be under status-quo business conditions, or what tomorrow’s stock price is going to be (assuming the stock price can’t be influenced), but in my experience the businessey types are usually asking for analyses in order to figure out what levers they can pull to improve whatever KPI of interest they are currently obsessing over (again, love). And I hate to break it to you, but it’s this desire for lever pulling that takes a business question out of the prediction bucket and puts it squarely onto the causal inference pile.

Why the hate though? It’s mostly because causal inference problems are harder than prediction problems in a lot of ways. Think about it- if all I want to do is predict whether a house will burn down, I can simply note, say, that orange houses burn down more often and do what I can do to avoid orange houses. If I instead want to prevent houses from burning down, I need to confirm that it’s specifically the orange paint that is the problem and not another feature that just happens to ride along with the orange paint (bad taste in general would be my initial hypothesis, personally). This is, of course, just a long-winded way of saying that correlation doesn’t imply causation, but I feel like the concrete example adds a little something. In fact, my favorite illustration of the distinction between prediction and causal inference needs is “are you trying to decide whether the picture is of a cat or are you trying to make the picture into one of a cat?” (predication is the former and causal inference is needed for the latter, in case it wasn’t clear)

Ideally, the businessey types would understand this distinction and choose their words carefully but, honestly, they have a lot of other things to worry about and perfectly-specified requests make me feel less useful as a data scientist. As such, I feel strongly that one of the core competencies of data scientists is to work with the businessey types to get below the surface of a request in order to understand what the real underlying motivation and goal is, like a mathematical therapist if you will. I also feel strongly that a related core competency is knowing what methodologies are valid for causal inference and which are only valid for prediction (not perfectly, but you can kind of think of the first as a subset of the second) and not chuck an inappropriate tool at an analysis.

Personally, my pet peeve is applying prediction techniques to linear regression when causal inference is actually the task at hand. Like, I get it, you have 50 potential explanatory variables and it would be really nice to winnow that down in order to prevent falsely explaining noise and huge confidence intervals and whatnot. The problem is that causal inference cares about the accuracy and significance of the individual parameters being estimated and not so much the accuracy of the prediction itself, especially out of sample predictions. (protip: every time you split a sample into training and testing portions for the purposes of causal inference you are making an economist sad) Put simply, this means that you can’t do causal inference using any procedure that messes around with parameter estimates in order to get better predictions- talking to you, Ridge, LASSO, etc etc.- at least not without being suuuuuper careful and doing other stuff on top of the basic regularization methods.

To see why, let’s think about a situation where there are 10 features that potentially impact a KPI of interest. For reasons that we will just take as given, running a regression on all 10 features results in a mess (probably a collinear mess), so it’s super tempting to employ some sort of familiar regularization technique, get an answer, and move on with your life. Let’s say we care about the impact of feature 5 on the KPI, and we decided to employ an algorithm that selects some subset of the 10 features to use as predictors (eg. LASSO). Let’s say that feature 7 gets dropped in this procedure, perhaps since feature 7 is highly correlated with feature 5. This likely means that feature 5 now gets credit for both its own effect on the KPI but also a lot of the effect of feature 7- while I am happy for feature 5’s increased perceived importance, this is specifically detrimental to the causal task at hand.

The good news is that there are techniques that help deal with the high-dimensionality of explanatory data- double machine learning, yada yada- but this wasn’t meant to be a post about specific data science methods…the more general and relevant (to this article) point is that in order to go down this path a data scientist needs to 1. understand that they are not looking at a prediction problem, and 2. limit themselves to methodologies that are appropriate for causal inference. This latter directive is harder than you would think to implement, mostly because it rules out a number of “sexy” approaches, and it takes a decent amount of wherewithal to commit to not doing the cool thing that people are talking about and demonstrating their mastery of in favor of the simpler seeming analysis. So I propose a sort of data scientist pact: let’s all agree to avoid the “hey look what I know” tendencies and get some good answers to the right questions mmmkay?

Originally published at https://jodibeggs.substack.com.

UL Lafayette on Twitter

@casey_riche We're sorry that the price for the Acct 202 online text has caused so much strife & are working with the publisher to lower it to match the hard copy cost.The $999 price was set w/ good intentions, though we realize now that we needed to explain the rationale behind it better.

$999???????? Here’s the context: students in the accounting course have a required textbook, which is bundled with an access code for the online homework system used in the course. The textbook is customized for the course in some way (likely just adding/deleting chapters and changing order), and the price of the access code/hard copy textbook bundle is a little over $300, whereas the school decided to price the version that doesn’t include a hard-copy textbook (usually an option offered for students who are looking to save money) at $999. This is…not a good look in a number of ways, up to and including the fact that even the $300 price tag is kind of offensive.

I’ve noticed that people have a lot of questions about textbook markets, and, at the risk of making everyone around me angry, I have answers, from both the instructor perspective and as one who’s worked for said textbook publishers.

Q: Why are textbooks so expensive?

A: Historically, instructors didn’t pay a lot of attention to the prices of the textbooks they required for their courses, and students didn’t feel like they had a choice so they just sucked it up and bought the books. This doesn’t give textbook publishers a lot of incentive to price competitively. In addition, putting a textbook together does require more work, and hence cost, than is likely obvious to the outside observer, and this also explains the discrepancy between the prices of textbooks and “trade” books (i.e. non-textbook nonfiction). More recently, it seems like students have started purchasing older editions for cheaper, just going without the textbook (not recommended by the way), or relying on free substitute materials, so textbook publishers have had to get creative. More on this in a sec.

Q: Do instructors get a cut of revenue from the textbooks they assign?

A: Not usually, no, at least not directly, but it’s worth talking about this point. If an instructor has written a textbook that they require for a course, they are entitled to royalties on the textbooks that their students buy, though some instructors donate the royalties or do something similar to avoid conflict of interest. Non-author instructors generally don’t get paid directly, though it’s common for instructors who teach large courses to get treated like royalty by the publishers of the textbooks they assign- dinners, conferences, well-compensated focus group opportunities, and so on. Basically, textbook publishers treat these instructors like pharmaceutical companies treated doctors before regulation was introduced in that industry.

That said, I’ve heard of more than one case where a university has entered into an agreement with a textbook publisher such that the university gets some sort of compensation in return for requiring its instructors to use the publisher’s texts. This is problematic since it can prevent instructors from using the texts that they feel are best for their students, and it can even result in universities losing highly-respected instructors. (In that particular case, the instructor was happy to donate his royalties but felt it was unseemly to require that the royalties be donated to the school itself.)

Q: So why don’t instructors just assign open-access/lower-priced materials or make it clear that an older edition of a textbook is acceptable?

A: Sometimes they do, and there is an increasing trend of instructors being sensitive to the financial burden they are placing on students. BUT…it should not shock you that publishers are getting increasingly creative regarding how to sell their products to instructors. (Make no mistake that they are the real customer here.) Again historically, publishers would entice instructors by providing supplemental materials- test bank questions, lecture slides, etc.- to instructors who assigned their books, giving instructors incentives to choose the text with the best/most helpful instructor materials in order to lower their workload.

Over the last 10–20 years, this concept has been brought into the world of online homework systems. The first company providing this functionality was called Aplia and was founded by Stanford economics professor Paul Romer, reportedly as a result of a realization that he had too many students submitting problem sets and not enough teaching assistants. Not surprisingly, this company was acquired by a textbook publisher in 2007 so that the publisher could bundle the Aplia product with its textbooks. It’s kind of an evil genius move if you think about it- what (likely underpaid adjunct) instructor doesn’t want to assign a text that makes it so they don’t have to grade as much stuff, all at no cost to the instructors themselves?

Nowadays, the pricing you see for a lot of textbooks makes more sense if you think about the access code for the online homework system as the real product and the textbook as the add-on item. For example, I taught a course where the online access code with an online text was $133 and the online access code with both an online and hard-copy text was about $160, so the high price is placed on the item that students can’t get away from purchasing (other than by forfeiting their homework grade, which I have actually seen happen *headdesk*). It’s also convenient from the publisher side that the access code has no resale value once it’s used.

Coming back to the tweet pasted above, it seems like either the school or the publisher was trying to push students toward getting the physical textbook by pricing the online-only version of the product so absurdly high. That said, it could also be the case that it was an awkward attempt to make the $300 price seem less absurd by comparison.

Q: What’s the deal with the textbooks that are customized for a specific course? I’m now suspicious that this is actually helpful.

A: I think the idea is that the customized textbooks make it less likely that students will try to find a used version or an earlier edition and instead will pay the publisher for a new book. To be fair, however, I’ve actually written some of these customized chapters, and in my case the instructors were driving the process because they wanted to teach particular topics that weren’t included in the textbook that wanted to use.

Q: What else should I be looking out for in this space?

A: One textbook publisher, Cengage Learning, recently rolled out a subscription model where students could pay a flat fee for access to all of their Cengage online texts and homework systems. I’m not sure I entirely understand the economics of this from the publisher side (it seems mostly like an attempt to increase market share), but it could help students save money if instructors are actually able to coordinate their textbook choices. Of course then there’s the problem of not choosing the text that is best for each course, so nothing’s perfect I guess.

Q: So what’s your general take on all of this?

A: I do think the textbook companion products- the instructor materials, online homework systems, etc.- are a valuable resource, but I can’t help but notice that the benefits of such products largely accrue to instructors. As such, I’m frustrated that the textbook business model directly passes these costs on to students as a somewhat unanticipated expense rather than forcing instructors and schools to provide a complete educational experience in return for tuition payments. On the flip side, I can’t help but be skeptical that we can provide a full suite of high-quality academic instructional materials without an underlying business model, though I guess that’s kind of what some academics do already via grants and other processes (see, for example, Marginal Revolution University).

Donald J. Trump on Twitter

In speaking with some of the world's top business leaders I asked what it is that would make business (jobs) even better in the U.S. "Stop quarterly reporting & go to a six month system," said one. That would allow greater flexibility & save money. I have asked the SEC to study!

That said, he’s gonna be so mad when he finds out who took up the cause first. Let’s discuss…

Currently, publicly traded companies (“public companies,” though importantly distinct from being a government entity) are required to report their business performance once a quarter. This seems helpful from a transparency perspective, but it also has the unintended consequence of companies being obsessed with “hitting their numbers” every 90 days or so. This doesn’t *have* to be a problem, but it’s entirely likely that focusing on short-term profits comes at the expense of maximizing long-run profitability.

This phenomenon, sometimes referred to as “quarterly capitalism,” isn’t simply the result of companies trying to show off every time they have to release earnings numbers. Instead, they’re mainly responding to market incentives- stock prices tend to respond strongly to quarterly earnings, and the perceived performance of company managers is closely tied to stock performance. This basically creates the corporate equivalent of being judged on your weight-loss progress on a weekly basis, and, just like such reporting can lead to unsustainable binge dieting, quarterly financial reporting can lead companies to engage in analogous, say, inefficient cost-cutting measures.

Do I really think that moving to six-month reporting requirements would solve the quarterly capitalism problem? No, mainly since most useful long-term projects have timelines of far more than six months. (I also do worry about the decrease in transparency nowadays.) But I like that the conversation is being had, since it forces discussion of what is referred to by behavioral economists as “myopic loss aversion.”

“Myopia,” taken literally, means near sighted, and, in a decision-making context, means short sighted, or overly focused on the short run. Loss aversion is the phenomenon where people hate losses more than they like equivalent gains. Myopic loss aversion is thought to lead to irrational choices, or choices that are out of line with maximizing long-term returns (or, somewhat equivalently, happiness.). In a personal investing sense, one of the best pieces of advice I can give is to not look at your stock portfolio every day, since you’ll probably get obsessed with short-term ups and downs and start buying and selling in unproductive ways. In a similar sense, professional investors would often be well served to take the quarterly reports and put them in the desk drawer rather than poring over them…or at least focusing on the information that pertains to long-term plans rather than short-term results. (I’m of course making the assumption that the professional investors are working on behalf of clients who are taking a long-term view of their returns.)

Hopefully this makes sense as a logical matter, but unfortunately it’s harder to act on than it might seem. First, just knowing that they’re doing something unproductive doesn’t always make people able to stop doing the unproductive thing. (The dieting analogy keeps working here I suppose, albeit for different reasons!) In addition, it’s harder to stop being myopic when others involved in a market are still committed to the myopic behavior, and it’s extra hard to convince a whole bunch of people to change their thought processes at the same time. Nevertheless, well…glad we had this talk at least. Baby steps…

Megan McArdle on Twitter

Are those raindrops I feel? Is that a storm coming? Oh yes there is! A TWEETSTORM! Topic du jour: Chris Collins and the terrible, horrible, no good, very bad insider trading allegations. https://t.co/Bf7mRujoZW

To be fair, the article isn’t as bad as the headline, but it’s still mistaken on a crucial mathematical point. Let’s discuss.

The arguments against allowing “insider trading” (i.e. buying and selling stocks based on information you have as a company insider but that the general public is not privy to) run across a pretty wide spectrum, from “it rips people off” on one end to “it erodes trust in markets” on the other. So let’s start with the ripping people off thing- in a broad sense, the stock market is not a zero-sum game since, when stocks increase in value, the pie of wealth to go around gets larger, so in this context there doesn’t have to be a loser for every winner. In a more narrow transactional sense, however, one party’s gain is in fact another party’s loss (or at least forgone gain). Let’s work through a simple example to see why this matters when it comes to insider trading.

Say there’s a stock that is currently trading at $10, but there is non-public information that suggests that the stock is worth $15. Once that information becomes public, the stock will trade at $15, so there is $5 of profit to be had in the course of the stock’s journey from $10 to $15. How this $5 gets split among different investors (savers, to use more precise economic language) depends on how the stock gets bought and sold during the price adjustment process. One simple case is where no insider trading occurs and the stock is only bought and sold once- in this world, some guy buys the stock at $10, sells at $15, and makes $5 profit. Note, however, that whoever sold the stock to this guy at $10 missed out on $5 of profit that he could have had if he had waited to sell- this is what is meant by one person’s gain being another person’s loss. In this scenario, what most likely happened is that one guy (the buyer) got lucky whereas the other guy did not. I could have given you a more complicated scenario that involves the stock changing hands more times, but it still has to be true that both the total profit and the total forgone profit add up to $5.

The insider trading scenario looks pretty similar, except that rather than getting lucky, the insider buyer has a systematic advantage that enables him to capture all $5 of profit, leaving none for, say, the teachers’ union pension fund. (Yes, I’m trying to make a point with this example- the stock market isn’t just douchebags gambling with one another., despite what CNBC might lead you to believe.) Again, the pension fund didn’t literally hand $5 over to someone else, but it missed out on $5 that it would have had if it had not sold the stock.

The argument that insider trading is victimless hinges on the notion that the guy (or organization) who sold the stock to the insider at $10 would have just sold the stock to somebody else for $10 if the insider hadn’t been in the market. This is probably true, but it just means that the first guy transferred $5 of profit to the second guy in a level playing field type scenario. (In other words, the second guy is now the one who got lucky and claimed the $5 profit.) Let’s say there are 100 shares of this stock in the market- this means that there’s a total of $500 in profits to be had from the price movement. Regardless of how I choose to describe it, it mathematically has to be the case that the insider took $5 in profit out of the system and left $495 for everyone else- in other words, I don’t have to specify who lost out in order to conclude that non-insiders are worse off due to insider trading.

Does this mean that ordinary investors can’t earn positive returns if insider trading is prevalent in a market? Not necessarily. Not all price movements are the result of information that could be of an “insider” nature- for example, if bad weather in Florida causes business problems for, say, Tropicana, it’s not like certain fancy people could have been told about the bad weather ahead of time. That said, I’m not convinced that “don’t worry your non-insider returns will just be lower but not totally zero” is terribly comforting to most people.

In addition, a critical mass of insiders in the market could make non-insiders reluctant to buy and sell (or, equivalently, hand their savings over to organizations that buy and sell). Think about it- wouldn’t you hesitate to do business with someone if you had to constantly wonder “what does this guy know that I don’t?” This isn’t all together different from the “lemons” problem that makes buying and selling, say, used cars difficult. (In finance courses, this information asymmetry is described by labeling the insiders as wolves and the non-insiders are sheep- would you want to do business as a sheep if you thought that everyone around you was a wolf?)

Hopefully I’ve made the logic of how insider trading is not a victimless crime a bit more clear…I’m not going to get into the morality of the activity (in general, the Venn diagram of illegal versus immoral is not a circle), but I will acknowledge that the original article lined above was unambiguously right about one thing- insider trading would in fact cause stock prices to adjust to their “correct” values more quickly,simply because inside information is still information. Put another way, it’s not that insider trading is unambiguously inefficient, it’s that we’ve decided as a society that having the right prices isn’t worth it if the system that generates those prices doesn’t operate on a level playing field.

First off, optimistic me was all “ok maybe there’s a huge premium placed on the property because it’s the Brady Bunch house”…but according to Zillow, the median home price in Studio City (Los Angeles, where the house is located) is about $1.3 million, so the price doesn’t seem that out of line with others in the area. Maybe it’s bigger than the others then? The thing is, I remember my classic TV pretty well, and part of the show’s schtick was that all of the kids lived in two bedrooms connected by a bathroom. So if the house is an accurate representation, it has…four bedrooms? (one for Mike and Carol and one for Alice in addition to two for the kids) *checks real-estate listing* The real house has three bedrooms and two bathrooms, so not far off…from either the Brady Bunch storyline or the average house size I guess.

It’s pretty likely that Mike Brady gets a discount on houses that he designs himself (he’s an architect, remember), but I still have a sneaking suspicion that today’s Mike Brady wouldn’t be able to afford the house, which is kind of a bummer. (To be fair, we could be looking at a Friends or Sex and the City situation where the characters couldn’t even afford where they were living at the time.) But it’s not immediately clear whether this is due to changes in housing prices or changes in architect salaries (or both). Maybe architects used to make more money relative to the population than they do today? I looked at some numbers- the median salary for an architect nowadays is about $77,000, which is about the 6oth percentile in terms of household income. (In other words, about 60 percent of households have income of less than $77,000 per year. Also, in case you’re wondering, Carol Brady didn’t have a job.) Mike seemed pretty senior though as far as architects go, so let’s say he’d be at the 80th percentile in terms of income today. If I fiddle with some numbers and project back in time, I get something that looks like this:

(Basically, I fixed both architect and 80th percentile income to an index of 100 in 2013- the latest year for which I had data- and scaled the historical values accordingly.) This suggests that architects like Mike were below the 80th percentile in terms of income at earlier points in time, which certainly doesn’t help solve the puzzle. (The related upside is that architect salaries have been increasing faster than average.) The good news is that the Brady Bunch house isn’t representative of all houses in the U.S.:

(This time, I fixed all of the values to an index of 100 in 1975- the year after the show ended- so that we could compare relative changes over time.) First, holy housing bubble Batman (see that green bump there?)…second, it’s not so much that the median home has gotten more expensive (relative to income) for people like Mike Brady, it’s that houses in places like Southern California have gotten way expensive. In fact, I’m amazed at how well house prices track income here and checked 12 times to make sure that I didn’t make a calculation error somewhere.

But here’s where it gets more interesting…what if we don’t look at the 80th income percentile? What if we look at a more average household versus housing prices? After all, not only the top 20 percent of households own houses.

This…well, is way more interesting than it probably looks on the surface. It suggests that, while housing price increases tracked income gains for this group for a while, we now seem to be in a new world where middle-income families are consistently spending a larger fraction of their incomes on housing than they used to (or buying less nice houses I suppose). Given how income inequality has been increasing over time, this effect is even stronger for the 40th and 20th income percentiles. But wait, there’s more…what if we just look at the prices of new houses rather than all houses?

So new houses are becoming increasingly expensive (again, relative to income) even for households that by any reasonable standards are doing quite well. Now, some of this maybe be unavoidable- if people just happen to like new houses better than…used houses? then new houses should sell at higher prices, but this preference would have to be getting stronger over time in order to give the pattern we’re seeing above. That’s possible, but it’s also possible that developers are targeting the upper end of the market and building new houses that are fancier than existing houses. Let’s do one more graph:

Looking at this, I’m pretty confident in saying that up until 2013 or so, developers were pricing in line with the income gains of the top 5 percent of earners…but what on earth is happening today? Are you all only making houses for the 1 percent nowadays? (Actually this feels kind of right based on my Boston-based anecdotal evidence.)

Before we get too upset at this observation, I should point out that building more fancy housing should still make houses more affordable for lots of people- some high earners will move into the new fancy houses, leaving fewer people to buy the less fancy houses and keeping prices of existing houses down as a result. But geez…I’m basically saying that building more fancy housing is better than building nothing, but making the housing stock fancier will obviously lead to higher prices than building new less fancy houses would. I guess Mike Brady was doing his part to keep housing prices down by cramming all of those kids into 2 bedrooms, but there’s also a “you get what you pay for” lesson here.

To see why, consider the “monopsony”… I love monopsonies- for starters, it’s a fun word to say, and it makes people do a double take when they initially think you said “monopoly.” Definitionally, a monopsony is a market with only one buyer- basically the economic counterpart to a monopoly, which is a market with only one seller. Not surprisingly, the buyer in a monopsony has all of the market power in much the same way that the seller in a monopoly does, and largely for a mathematically analogous reason.

In labor markets, companies are the buyers (or, perhaps more accurately given the 13th amendment, the renters) and people are the sellers of labor. When the number of employers in an area competing for workers is small, firms are generally thought to have “monopsony power.” This is a little different from literally being a monopsony, but you can think of monopsony power as the ability to act like a monopsony but to a lesser degree. Now, I could try to explain to you how minimum wages work in monopsonies using some fun graphs and whatnot, but I’ve come to learn that not everyone’s brain wants to operate at an abstract conceptual level, so let’s work through a simple example instead.

Ready? I’m gonna try to do this without using any econ-specific terms (they’re not as fun to say as monopsony), and we’re going to talk about a company that makes widgets- yes, I know this is a tired econ thing, but Widget is the name of my mom’s cat so I contend it’s fair game.

First, here are some numbers on how many widgets workers can produce:

You’ll note that the “incremental widgets” numbers are decreasing as more workers are added- this is generally reasonable, since most production faces a diminishing returns or “too many cooks in the kitchen” problem. Now let’s say that the company can sell as many widgets as it wants at a price of $10 each- now we can see how much each worker would add to the company’s revenue (does revenue count as an econ term?)

If we want to figure out how many workers the company wants to hire, we need to know how much the workers cost. The thing about workers is that some of them will work more cheaply than others (even if they’re of equivalent quality), and this is relevant for a monopsony, since they’re not so small that they can increase their hiring without pushing up the going market wage. (In fact, this feature is the crux of why monopsonies look different from competitive labor markets!) Let’s assume the wage required to get each worker to work looks like this: (you can think of everything in per hour terms, per day, whatever, it doesn’t really matter)

We can now calculate how much it would cost to hire 1 worker, 2 workers, etc., and we can also calculate the incremental cost of each additional worker:

I feel like this requires some explanation since it’s not really intuitive…if you want to hire the first worker, things are pretty simple, but if you want to hire the second worker, you not only have to pay the second worker the $20 to get him to work, but you also have to pay the first guy $20 rather than $10- there are a number of reasons for this, but you can just imagine that the first guy will get really pissed if he makes less money than second guy for doing the same job. This means that the incremental cost of hiring an additional worker includes not only his wage but also the additional compensation that has to be given to everyone who came before. (This is seriously the most important point so keep staring at it until it sinks in. I’ll wait.)

So take a look at the relevant numbers so far- how many workers does the company want to hire?

Does the company want to hire the first worker? Sure, since that worker brings in $120 and only costs $10. (just assume that labor is the only cost of production) Does the company want to hire the second worker? Sure, since the second worker brings in an additional $100 but only adds $30 to the company’s costs. If we keep analyzing in this way, we can conclude that the company will want to stop hiring after it hires 3 workers.

Now how do things change if a minimum wage is imposed? Let’s assume that a minimum wage of $40 is set. (Now I look real generous to those people who interpreted the numbers as being per hour.)

Workers 1, 2, 3, and 4 are happy to work for the minimum wage, so the company can hire up to 4 workers at $40 each. After 4 workers are hired, however, things change dramatically- in order to hire the 5th guy, the company not only has to pay him $50 but also has to pay each of the 4 other workers $50 rather than $40. How many workers does the company want to hire now? (This is the last set of numbers, I swear.)

Now the company would find it profitable to hire the first 4 workers but not the 5th worker. BOOM- I just showed you a case where a minimum wage would increase rather than decrease the number of workers hired.

So why did this happen? Put simply, the minimum wage took away the “well if I hire you I have to pay everyone else more too” feature of the company’s analysis, at least for the first 4 workers. That said, the minimum wage still creates somewhat of a double-edged sword in terms of employment, since it also made it incrementally more expensive to hire the 5th guy than it was before. As a result, minimum wages placed on monopsonistic markets can both encourage companies to hire more workers and discourage companies from setting wages just above the minimum wage. (This is relevant for companies that pay just above what would be the minimum wage, since it could actually cause them to decrease wages!)

I’m not trying to convince you that minimum wages are good, I’m just walking you through some numbers so that you don’t do the “I took one econ class once and models of competitive markets are always applicable” thing that causes economists to call you an ass clown behind your back. One last thing- in case you (understandably) have the follow-up question of “how prevalent is monopsony power in labor markets?”, here’s some reading material for you to peruse. Spoiler alert: the answer is “too prevalent to ignore.”

SCOTUSblog on Twitter

SCOTUS holds requiring nonmembers of public-sector unions to pay fees to cover collective-bargaining activities violates the First Amendment, overruling longstanding precedent

In case you’re curious, you name of the case is Janus v…well, the name of his union, and you can see the full decision here. I’m not here to discuss whether this decision is correct from a legal perspective. Instead, my goal is to walk through the relationship between the decision and the feasibility of unions- in other words, to explain why some people are describing the ruling as a death blow to unions and to assess whether they are correct. (In the discussion that follows, I’m not distinguishing between public-sector unions and unions more generally- this is because there is speculation that the court’s decision could technically apply to all unions. Also, the distinction isn’t crucial to the logic of the situation.)

Since the 4th of July is coming up, I think it makes sense to talk about fireworks displays. (I swear I have a point here and am not just bringing them up because one freaked out my dog last night.) Logistically, it’s hard to restrict a fireworks display to paying customers- for example, I live in Boston and can see the big 4th of July fireworks from the roof of my building…unless somebody’s going to put up a reallllllly tall screen, I get to watch whether I help pay for them or not. So let’s say I like both fireworks and keeping my money- would I pay money to watch the fireworks display? Probably not.

This is what people are referring to when they talk about the free-rider problem- technically, me watching the fireworks display without contributing a payment is me free-riding off of others, and the free-rider problem itself is the suboptimal outcome where no fireworks happen (even though creatures who are not my dog like fireworks) because everybody is waiting for others to pay so they can free ride. (protip: if you’re thinking about how to get something for free, so is everyone else) In the context of the unions, the idea is that unions would shut down due to lack of funding if those who benefit from the union’s negotiation activities aren’t required to contribute to its funding, since everyone would just try to free ride off of others. (In this particular case at least, the union is required to represent the interests of both members and nonmembers.)

In Econ 101, we definitely teach that economically rational individuals will free ride whenever possible, and, as a result, there will be fewer unions and fireworks displays than is optimal for society. This is particularly awkward to teach in summer courses, since it tends to be followed closely by “now here are the best places to watch the giant fireworks display tomorrow.” So what gives? In the case of the fireworks, they’re usually paid for either by governments or by companies that want to sponsor the event. In the case of unions, there’s talk of trying to have the government pay for them, which I guess works in principle (though is potentially still questionable in a legal sense).

Note, however, that “economically rational” was doing some heavy lifting in the previous paragraph, and it’s well-documented that people often behave in “irrational” ways. (This is why the entire field of behavioral economics exists.) So what does behavioral economics (and psychology, to be fair) have to say on the matter? One set of data points shows that “pay what you want” pricing schemes generate significantly more than zero revenue, suggesting that people aren’t pure free riders when given the opportunity. Another data point concerns what economists call “public goods,” of which the fireworks display is a classic example. (note that, while public goods are often provided by governments in practice, this is not why they are called as such) Since public goods are not restricted to paying customers, a temptation to free ride exists…but this temptation is apparently not overwhelming, since there’s lots of documented evidence that people contribute to public goods.

Given these findings, it’s helpful to explore the factors that encourage people to contribute rather than free ride…generally, they’re pretty much what you’d expect. People contribute when the benefits of the good to them is higher, and when people can communicate and coordinate more effectively among themselves. Group size also plays a role, but whether bigger groups are good or bad for cooperation is situation dependent. In addition, people sometimes contribute “by accident”- i.e. they don’t realize that free riding is a possible or desirable option. One warning though- the evidence also suggests that cooperation declines over time, essentially meaning that people learn to free ride off of others, so observing that unions stay solvent in the short run doesn’t mean that that trend is likely to continue.

Overall, however, it appears that unions don’t have to be facing a death sentence, but the inability to coerce payment makes it crucial to think through how to design them to make people excited about participating…which, come to think of it, is probably something that unions should think about regardless of legal restrictions.

The “trade balance” is just exports minus imports- in other words, the amount of stuff that we produce in the US and sell to foreigners minus the amount that is produced in foreign countries that US consumers buy. (If you’re not reading this from the US, just pretend for a few minutes that you are.) When exports are greater than imports there is a “trade surplus” (positive trade balance) and when imports are greater than exports there is a “trade deficit” (negative trade balance). We can talk about the trade balance in a global sense, as I did above, or we could talk about the trade balance with a particular country in an analogous fashion.

This all sounds pretty simple, except…well, as globalization becomes a bigger thing, it gets a little tricky. For example, let’s say Apple get iPhone parts from China, finishes producing it and whatnot in the US, and sells it to someone in Germany. Technically speaking, the value of the parts counts as an import and the value of the finished iPhone counts as an export, and the difference contributes to the trade balance. But it can get even weirder- one of the things I like to tell my students is that my work is contributing positively to the trade balance. How is this, given that I don’t get shipped overseas to give my lectures?

Here, I think the Brookings Institution can explain it better than I can:

If you are not an economist, the idea that America’s education of foreign students is an export may seem strange. What makes a service like education an export is that foreigners pay U.S. institutions, so money flows into the U.S. from abroad. Education of a French student in the United States is an export in exactly the same way that the visit of a Chinese tourist to Disneyland is an export, or the provision of stock brokering services on the New York Stock Exchange to a German financial company is an export. When we provide a service that leads to foreigners sending money into the U.S., that’s an export with exactly the same economic effects as when we sell soybeans or coal abroad.

Weird, but it makes perfect sense if you think about it.

Now that we understand the trade balance, here’s a quiz: consider the following scenario. What impact does this transaction have on the US trade balance?

A Canadian citizen comes to Michigan to buy a pair of shoes. He buys the shoes and then brings them back to his home in Canada. (For the sake of simplicity, assume that the shoes were made in Michigan.)

Spoiler alert: it’s a bit of a trick question. Conceptually, this obviously counts as an export from the perspective of the US. In terms of data collection, however, it’s a little less clear. Specifically, I guess it depends on whether the shoes are declared at customs and such, since the store isn’t going to directly report that they sold shoes to a visiting Canadian. In terms of the actual economic contribution, it counts whether it gets reported or not.

By now you may have guessed that the scenario that I gave you is not random, and you’re right:

Daily Mail US on Twitter

Trump says Canadians are smuggling scuffed shoes out of the U.S. to avoid tariffs and America can no longer be 'the stupid country' on trade https://t.co/WMu6cMtOyc

There’s…well, a lot to unpack here. First, I did a little digging, and I learned that despite the government’s lamentations, NAFTA is still in effect. This means that there’s no reason to smuggle the US shoes into Canada, since there’s no tariff on shoes made in the US anyway. But let’s make things more interesting and assume instead that the shoes weren’t made in the US and instead made in a country that Canada doesn’t have a trade agreement with. In this case, Canadians can still bring a limited amount of these goods home with them from the US without any tariff consequences, provided that they spend at least 48 hours in the US. (You can see more detail here if you are curious.)

I almost didn’t want to tell you any of that though, since it’s a bit like missing the forest for the trees. EVEN IF Canadians were smuggling shoes back home with them, it has a far more negative effect on Canada than it does on the US- in fact, it probably doesn’t have a negative impact on the US at all. Let’s see how it stacks up:

Canada: doesn’t sell a pair of shoes (since the Canadian bought the US shoes instead), doesn’t get tariff revenue that it might otherwise be entitled to

US: sells a pair of shoes, might not get national income accounting credit for it as an export

This makes me wonder…did Trump forget that we’re not Canada? As a pure point of logic, it doesn’t make sense to complain both about trade deficits and about things that decrease the trade deficit. (Even if the shoes were made outside the US, there’s likely some added value provided by the retailer that counts as an export.) I mean, the only even remotely reasonable thing to complain about in this process from the US perspective is that the US may not get official credit for the export. Or, put a slightly different way:

Greg Ip on Twitter

This would of course suggest the U.S. trade surplus with Canada is much bigger than reported. https://t.co/DPZo4y1DRJ

For the nerds in the audience that remember your Y=C+I+G+NX, this implies that some items that should be in NX get counted as C instead. For most macroeconomic purposes this doesn’t mater a *whole* lot I guess, unless of course you’re specifically obsessed with the trade deficit.

Wonkblog on Twitter

A minimum-wage worker can't afford a 2-bedroom apartment anywhere in the U.S. https://t.co/05sfZfk60P

Anywhere? Are you sure? Because, not gonna lie, my backup plan for when I get sick of the world is to go take a minimum wage job and live in the middle of nowhere. (I’ll have to take nowhere literally now I guess.)

Before I totally panicked, though, I read the article (and the study, which I generally recommend at least skimming as articles aren’t infallible)…ok, here’s the actual deal: the test being used here to determine “afford” is whether a minimum wage worker working full time (40 hours a week, 52 weeks a year) can rent a “fair market rent two-bedroom home” by spending no more than 30% of their income. In looking at the full report, I learned that “fair market rent” is defined as the price of the 40th percentile two-bedroom rental home in the area. *insert screeching noise here*

By definition, 40 percent of two-bedroom rental homes in an area cost less than this “fair market rent.” (See here for a review on how percentiles work.) On the up side, I feel better than my minimum-wage self might be able to afford a number of those 40 percent of rentals below the fair market rent. (Similarly, my grad student self certainly couldn’t afford such a fair market rent home, but there were a number of options that were financially accessible.) On the down side, I’m incredibly annoyed at the study’s misleading use of both math and words here, particularly since it took me more than trivial effort to read through and find the 40th percentile thing. I’m also annoyed at the article for not being at least a little critical of the garbage affordability metric. (Not to mention that the average two-bedroom home is described in the article as “modest” and “decent” interchangeably and I’m not sure which one bothers me more.)

Is it a cause for concern that a minimum wage worker can’t afford this 40th percentile rental home? I honestly can’t tell without further information. Some other things I would like to know: How many minimum-wage workers are the only worker in a larger household? (In other words, how badly do these workers need two bedrooms in order to live without going insane? How many minimum-wage incomes represent total household income?) Are one-bedroom homes available at lower prices in the area? (This may not be the case in some places.) How spread out is the lower part of the price distribution for these homes? (In other words, how much cheaper are the 40 percent of homes that are below fair market rent? How many of those are affordable?) Are the lower-end homes total dens of iniquity or are they actually livable? Could the minimum-wage workers buy something rather than rent? (Yeah, I heard it too, but I’m trying to be analytically exhaustive.)

After stewing over this for the better part of a day, I realized something particularly interesting- unless I’m doing something very wrong, it takes pretty specific conditions to result in a scenario where a minimum-wage worker can afford this fair market rent home, which implies that the aforementioned lack of affordability has to exist almost by construction. Why is this? The amount of housing in an area is fixed to some degree (this is a topic for another day) and higher incomes tend to bid up prices when supply doesn’t adjust to compensate. If there’s a general correspondence between quality and price, then we’re going to see, roughly speaking, the highest-income person in the most expensive home, the average-income person in the average home, and so on. As long as minimum-wage workers are a small fraction of workers (this report is at a federal level but suggests a small percentage), it’s unlikely that aminimum-wage worker won’t get outbid for that 40th percentile home. (One caveat: this conclusion could change if higher-income households overwhelmingly buy rather than rent.)

At the risk of sounding like a jerk, I’m going to say I don’t see the lack of affordability as described in the study as a problem in itself as long as the lower-end housing isn’t a total slum situation. I’m not a total monster, though, so there are statistics out there that could get me on the outrage train- for example, housing affordability metrics typically compare median (50th percentile) house price to median income, and I’d definitely be worried if such a ratio got too high. This doesn’t really speak to the welfare of low-income workers, however, so I would also likely be concerned if, say, minimum-wage workers represent 10 percent of the households in the rental market in an area but can only afford 5 percent of the homes for rent.

To slightly paraphrase economist Ronald Coase, “ If you torture the data long enough, it will confess.” In other words, there’s often enough wiggle room that you can make numbers say what you want to say, especially if you choose definitions and thresholds strategically, drop inconvenient outliers, and all sorts of other fun stuff. So what can you, as a consumer of quantitative information, do? Be suspicious. Look behind the headline to see what numbers are being calculated. Ask yourself “self, does this information actually answer the questions that it claims it does?”

As I’m writing this (I swear this is not made up), my TV is showing one of those auto insurance commercials that claim “people who switch save an average of $412” or whatever, and I literally yelled “of course they did, what moron would switch if he wasn’t going to save a significant amount of money?” so at least you know I practice what I preach.

More where this came from

This story is published in Noteworthy, where thousands come every day to learn about the people & ideas shaping the products we love.

Follow our publication to see more stories featured by the Journal team.