Yesterday, I was thinking about the first time I realized the difference between two random variables being disjoint rather than independent. The difference was very confusing at the time for me, and I think the reason is related to Venn diagrams.

Venn diagrams are the method usually used to teach the probability concepts in schools (and to some extent universities) these days. Although to me, they have a fundamental problem in that sense. For the reason, assume you have two random variables P and Q. When the following figure is given:

Disjoint random variables.

Disjoint random variables.

The first thing which came to my mind was that P and Q are independent. Obviously that is not true, because if we know that P has happened, we are sure that Q will not happen. That is a dependency right there, as giving information about one variable, gives you some information about the other one. Two variables are independent, if having some information about one of them, does not change our knowledge about the other one. So for two variables to be independent, the Venn diagram should have the following form:

Dependent or independent random variables.

Dependent or independent random variables.

The problem with this diagram is that the diagram itself does not tell us if P and Q are dependent or independent. For that we should have all the information about area of P, Q and P \cap Q.

So I looked around to see if I can find a better diagram, and I came across the Eikosograms, proposed by Wayne Oldford. In this method,  shaded areas represent probabilities of different events inside a unite square. For example for a binary random variable P who can take values 0 and 1, the eikosogram will look like this:


Eikosogram for binary random variable P.


where we have Pr\{P=0\}=\frac{1}{3}, and so we have Pr\{P=1\}=\frac{2}{3}. Sounds reasonable, now assume we have another binary random variable Q, which can also take two values (0 and 1), and our goal is to measure the conditional probability Pr\{P|Q\}. We can represent this concept using the following diagram:


Eikosogram for conditional probability of P given Q.


In this diagram we have: Pr\{P=0|Q=0\}=\frac{2}{3} . The interesting metaphor is that you can imagine the square as a container, and the shaded area as the water in the container. The vertical line, or the conditional probability is a barrier separating the water into two parts. If we remove this barrier the water will settle down and the result is the marginal probability of the variable P (go ahead and do the calculation).

Finally for the probabilistic independence, if we draw the eikosogram, we can clearly observe the independence as the “water level” in both sides of the “barrier” are equal. Or knowledge about status of Q does not change our knowledge over status of P.


Eikosogram for independent random variables P and Q.


I strongly recommend everyone to read his paper and get a better intuition of the Bayes Theorem (I was going to write a whole post about that, but I think Oldford did a complete job in section 4.1.3).

Finally looking at his work, I came across a video on YouTube, where he presents the following puzzle:

We have three gas stations, they always have different prices for their gas. On any given day, any one of the three might have the cheapest gas, with equal probability. All three are on route, but once you pass a gas station there is no turning back. You must buy gas at one and only one of the gas stations. As you reach each station, you can see its price advertised but not the price of the next. We want the cheapest gas with high probability. Does it matter what you do?

For the answer look at his video clip.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s