Think Bayesian Dice!

This comes from the second edition of this book: THINK BAYES 2e! by author and educator Allen Downey (with an accompanying website and Python code, all intended to be beginner-friendly!).

Gemini Pro images (see the prompt I used in Sources and References, below)

I approach the subject with Bayesian networks instead of Python

Here is the question (originally from another book, Bernoulli’s Fallacy by Aubrey Clayton):

Suppose the sequence of those 10 rolls gives these 10 clues: G, G, L, E, L, L, L, E, G, L < T

Suppose in one round of the game we had this sequence of outcomes, with G representing a greater roll, L a lesser roll, and E an equal roll:

G , G, L, E, L, L, L, E, G, L

Based on this data, what is the posterior distribution of T? What is your best guess for the value of T?

Allen Downey fully works this example with Python and makes his solution available on Collab. My aim here is to rework this example, but with a Bayesian Network (I use Netica).

One way to start is with the target node T and just a single observation node, and then build from there. Let’s call the mystery roll T1, and the clue is that T1 > T (hence G). A Boolean node, R1, encodes the first clue that T1 >T.

We only know the relationship between T and T1, namely T1 > T

Compiling the equation gives this table:

Sequence of True/False for when T1 > T

We can now activate this very small network to see how the distributions are changed:

That one clue that T1 > T tells us that the target number T is not 6, and T1 is not 1

But we have 9 more clues to worry about. Surely we’re not going to add all 10 nodes …?

Notice that the equations are as simple as they can be but the network

Yes, this is, of course, brute force, but we had to start somewhere. We can refine the model and make this much more compact, but let’s first activate this network and see what happens to the distribution for target node T:

We get the distribution for T, and the most probable value for T (at almost 50%) , given the 10 clues, is 4

Note that with this full model we also get the distributions for each of the 10 dice roll T1 through T10 (even though this was not asked for)

We can significantly reduce and simplify the network structure, but at the cost of making the equations slightly more complex. We still have the discrete central node T representing the first dice roll. In a sense, T represents 6 hypotheses, which are initially equally probable. The ten evidence nodes (T1, …, T10) will tell us how each roll is related to T (greater than, less than, or equal to T):

The target node T is the hypothesis in question.

Since we don’t need to know the distribution of each of the ten dice rolls, our evidence node needs only to track if each roll was greater than (G), less than (L), or equal (E) to the target dice T. This is the CPT for each evidence node:

The CPT table

A useful exercise is to fill this table manually with unnormalized count data (Netica automatically normalizes afterwards) :

Unnomalized count data for T1

This means, for example, that if T = 1, there are 5 values of T1 greater than T: only 1 is equal to T, and of course none can be less than T.

Get Pascal Bercker’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

More simply (but equivalently), we can also compile this equation:

The equatioon is the same for each of T1 … T10

After we compile the equations, we get :

The conditional distribution of the data given the parameter

We do the same with all 10 nodes, and once compiled, we have this updated network:

We are now ready to enter the data in each observation node T1 through T10

Let’s watch how that updating works. In particular, watch how the distribution for the target node T changes with each added observation (but notice what happens with node R4 and R8):

Did yyou see what happened with T4 and T8?

Along the way, we noticed that T4 and T8 (both E, for equal) made no difference, meaning that they gave no useful evidence either way and could have been omitted.

Distribution for target node T does not change even when omitting T4 and T8

But can we still do better? Yes, we can, with the Multinomial Distribution, a generalization of the Binomial Distribution. We use this since there are 3 possible outcomes (G, E, L).

In our case m = 3 outcomes, n = 10 trials,

We have m = 3 possible outcomes (G, E, L), and we track with count nodes count _G, Count_E, and Count_L.

Notice that the equation is very nearly identical to what we used earlier, except that this one node will do the business of all ten nodes we needed earlier!

Note how the probabilities depend on the target node T

Let’s do a sanity check to be sure that this works as expected. We will activate the network and then step through the target node T, 1 through 6. We are watching to see that the count nodes work as expected. For example when T = 1, then count_L should be zero, since none can be less than 1.

This passes our sanity check and works as expected.

So now lets enter our count where Count_G = 3, Count_E = 2 and Count_L = 5 (for a total of 10).

Notice that Count_L automatically went to 5 since the constraint was the total count = 10

So our final answer for the distribution of T and our best guess for its value:

Just as before, our best guess is that the target dice landed on 4

SOURCES & REFERENCES

This was the prompt to Gemini Pro, which I asked ChatGPT to generate:

Think Bayes 2

Bayesian Billiards

The M&M PUZZLE —DON’T JUST THINK BAYES — THINK BAYESIAN NETWORKS!

Bernoulli's Fallacy: Statistical Illogic and the Crisis of Modern Science - Aubrey Clayton

Norsys - Netica Application

Multinomial distribution - Wikipedia