Yesterday, I was thinking about the first time I realized the difference between two random variables being disjoint rather than independent. The difference was very confusing at the time for me, and I think the reason is related to Venn diagrams.
Venn diagrams are the method usually used to teach the probability concepts in schools (and to some extent universities) these days. Although to me, they have a fundamental problem in that sense. For the reason, assume you have two random variables P and Q. When the following figure is given:
The first thing which came to my mind was that P and Q are independent. Obviously that is not true, because if we know that P has happened, we are sure that Q will not happen. That is a dependency right there, as giving information about one variable, gives you some information about the other one. Two variables are independent, if having some information about one of them, does not change our knowledge about the other one. So for two variables to be independent, the Venn diagram should have the following form:
The problem with this diagram is that the diagram itself does not tell us if P and Q are dependent or independent. For that we should have all the information about area of P, Q and .
So I looked around to see if I can find a better diagram, and I came across the Eikosograms, proposed by Wayne Oldford. In this method, shaded areas represent probabilities of different events inside a unite square. For example for a binary random variable P who can take values 0 and 1, the eikosogram will look like this:
where we have , and so we have . Sounds reasonable, now assume we have another binary random variable Q, which can also take two values (0 and 1), and our goal is to measure the conditional probability . We can represent this concept using the following diagram:
In this diagram we have: . The interesting metaphor is that you can imagine the square as a container, and the shaded area as the water in the container. The vertical line, or the conditional probability is a barrier separating the water into two parts. If we remove this barrier the water will settle down and the result is the marginal probability of the variable P (go ahead and do the calculation).
Finally for the probabilistic independence, if we draw the eikosogram, we can clearly observe the independence as the “water level” in both sides of the “barrier” are equal. Or knowledge about status of Q does not change our knowledge over status of P.
I strongly recommend everyone to read his paper and get a better intuition of the Bayes Theorem (I was going to write a whole post about that, but I think Oldford did a complete job in section 4.1.3).
Finally looking at his work, I came across a video on YouTube, where he presents the following puzzle:
We have three gas stations, they always have different prices for their gas. On any given day, any one of the three might have the cheapest gas, with equal probability. All three are on route, but once you pass a gas station there is no turning back. You must buy gas at one and only one of the gas stations. As you reach each station, you can see its price advertised but not the price of the next. We want the cheapest gas with high probability. Does it matter what you do?
For the answer look at his video clip.