adding autocorrelation to chapter 3

Author: CamDavidsonPilon

Chapter1_Introduction/Chapter1_Introduction.ipynb

@@ -17,7 +17,7 @@
      "language": "python",
      "metadata": {},
      "outputs": [],
-     "prompt_number": 1
+     "prompt_number": 2
     },
     {
      "cell_type": "markdown",
@@ -35,6 +35,9 @@
       "The Philosophy of Bayesian Inference\n",
       "------\n",
       "\n",
+      "Before we explore Bayesian solutions, it would help to ask a question first: what is the inference problem? This is a very broad question, so I'll provide a broad answer. Inference is the task of discovering non-observed variables from information that is *somehow* related. One can think of it as the reverse-engineering problem: given output, infer input. This broad, definition provides enough room for many candidate solutions to the inference problem. One such solution is Bayesian methods. Consider the following vignette:\n",
+      "\n",
+      "\n",
       "  \n",
       "> You are a skilled programmer, but bugs still slip into your code. After a particularly difficult implementation of an algorithm, you decide to test your code on a trivial example. It passes. You test the code on a harder problem. It passes once again. And it passes the next, *even more difficult*, test too! You are starting to believe that there are may be no bugs present...\n",
       "\n",
@@ -57,7 +60,7 @@
       "\n",
       "Notice in the paragraph above, I assigned the belief (probability) measure to an *individual*, not to Nature. This is very interesting, as this definition leaves room for conflicting beliefs between individuals. Again, this is appropriate for what naturally occurs: different individuals have different beliefs of events occuring, because they possess different *information* about the world.\n",
       "\n",
-      "Think about how we can extend this definition of probability to events that are not *really* random. That is, we can extend this to anything that is fixed, but we are unsure about: \n",
+      "- I flip a coin, and we both guess the result. We would both agree, assuming the coin is fair, that the probability of heads if 1/2. Assume, then, that I peek at the coin. Now I know for certain what the result is: I assign probability 1.0 to either heads or tails. Now what is *your* belief that the coin is heads? My knowledge of the outcome has not changed the coin's results. Thus we assign different probabilities to the result. \n",
       "\n",
       "-  Your code either has a bug in it or not, but we do not know for certain which is true. Though we have a belief about the presence or absence of a bug.  \n",
       "\n",
@@ -71,11 +74,13 @@
       "\n",
       "John Maynard Keynes, a great economist and thinker, said \"When the facts change, I change my mind. What do you do, sir?\" This quote reflects the way a Bayesian updates his or her beliefs after seeing evidence. Even --especially-- if the evidence is counter to what was initially believed, the evidence cannot be ignored. We denote our updated belief as $P(A |X )$, interpreted as the probability of $A$ given the evidence $X$. We call the updated belief the *posterior probability* so as to contrast it with the *prior probability*. For example, consider the posterior probabilities (read: posterior belief) of the above examples, after observing some evidence $X$.:\n",
       "\n",
-      "1\\.   $P(A): \\;\\;$  This big, complex code likely has a bug in it. $P(A | X): \\;\\;$ The code passed all $X$ tests; there still might be a bug, but its presence is less likely now.\n",
+      "1\\. $P(A): \\;\\;$ the coin has a 50 percent chance of being heads. $P(A | X):\\;\\;$ You look at the coin, observe a heads, denote this information $X$, and trivially assign probability 1.0 to heads and 0.0 to tails.\n",
+      "\n",
+      "2\\.   $P(A): \\;\\;$  This big, complex code likely has a bug in it. $P(A | X): \\;\\;$ The code passed all $X$ tests; there still might be a bug, but its presence is less likely now.\n",
       "\n",
-      "2\\.  $P(A):\\;\\;$ The patient could have any number of diseases. $P(A | X):\\;\\;$ Performing a blood test generated evidence $X$, ruling out some of the possible diseases from consideration.\n",
+      "3\\.  $P(A):\\;\\;$ The patient could have any number of diseases. $P(A | X):\\;\\;$ Performing a blood test generated evidence $X$, ruling out some of the possible diseases from consideration.\n",
       "\n",
-      "3\\.  $P(A):\\;\\;$ That beautiful girl in your class probably doesn't have a crush on you.  $P(A | X): \\;\\;$ She sent you an SMS message about this Friday night. Interesting... \n",
+      "4\\.  $P(A):\\;\\;$ You believe that the probability that the lovely girl in your class likes you is low.  $P(A | X): \\;\\;$ She sent you an SMS message about this Friday night. Interesting... \n",
       "\n",
       "It's clear that in each example we did not completely discard the prior belief after seeing new evidence, but we *re-weighted the prior* to incorporate the new evidence (i.e. we put more weight, or confidence, on some beliefs versus others). \n",
       "\n",

Chapter3_MCMC/IntroMCMC.ipynb

@@ -31,13 +31,13 @@
      "cell_type": "markdown",
      "metadata": {},
      "source": [
-      "Statisticians can be a sour bunch. Instead of considering their winnings, they only measure how much they are losing. In fact, they consider their wins to be negative loses. But what's interesting is *how they measure their losses.*\n",
+      "Statisticians can be a sour bunch. Instead of considering their winnings, they only measure how much they are losing. In fact, they consider their wins as *negative loses*. But what's interesting is *how they measure their losses.*\n",
       "\n",
       "The author Nassim Taleb of *The Black Swan* and *Antifragility* stresses the importance of the *payoffs* of decisions, *not the accuracy*. For example, consider the following vignette:\n",
       "\n",
       ">   A meteorologist is predicting the probability of a possible hurricane striking his city. He estimates, with 95% confidence, that the probability of it *not* striking is between 99% - 100%. He is very happy with his precision and advises the city that a major evacuation is unecessary. Unfortunately, the hurricane does strike and the city is flooded. \n",
       "\n",
-      "The stylized example shows the flaw in using a pure accuracy metric to measure how good your estimate is, while an appealing and *objective* thing to do, misses the point: results of decision making. Taleb goes on to say \"I would rather be vaugely right than very wrong.\"  "
+      "This stylized example shows the flaw in using a pure accuracy metric to measure outcomes. Using a measure that emphasizes etimation accuracy, while an appealing and *objective* thing to do, misses the point of why you are even performing the statistical inference in the first place: results of inference. Taleb distills this quite succiently: \"I would rather be vaugely right than very wrong.\"  "
      ]
     },
     {

Chapter4_TheGreatestTheoremNeverTold/LawOfLargeNumbers.ipynb

@@ -53,7 +53,10 @@ Interactive notebooks + examples can be downloaded by cloning! )
     We explore the gritty details of PyMC through code and examples. Examples include:
     -  Analysis on real-time GitHub repo stars and forks.
 
-
+**More questions about PyMC?**
+Please post your modeling, convergence, or any other PyMC question on [cross-validated](http://stats.stackexchange.com/), the statistcs stack-exchange.
+    
+    
 Using the book
 -------