You are at the end of a game show after answering the most questions, and the host shows you three doors: A, B and C. Behind one is a brand new luxury car, filled with money. Behind the other two are watermelons. The game is:
1. You choose a door (let's say A);
2. The host shows you what's behind one of the doors you didn't choose (let's say C);
3. You decide whether to stick to your choice of door (A) or switch to the remaining door (B);
4. The host then opens your final choice and you get what's behind
Important: the host knows which door holds the money, and won't be showing you that door!
The question is: do you switch or stick?
The answer is not particularly intuitive, and a lot of people can follow the logic, but just can't accept the answer. Will you?
Let's start with the priors. Before any door was opened you would be equally confident that the car was behind any of the three doors:
p(q=A) = 1/3
p(q=B) = 1/3
p(q=C) = 1/3
Now let's look at the likelihood function, which answers the question "What is the probability that the host opened door C, after you selected door A?":
l(X|q=A) = ½
Because if the door you chose is correct, the host could have selected either door B or door C, each with 50% probability
l(X|q=B) = 1
Because if you chose door A and the car was behind door B, the host had to choose door C
l(X|q=C) = 0
Because we would have seen the car when the host opened this door
This gives the posterior distribution:
f(A|X) = 1/3
f(B|X) = 2/3
f(C|X) = 0
In other words, we are twice as confident in the door we could switch to as the one we chose, so we should switch!
How can that be? Surely the probability of door B being the right one doesn't get bigger than door A's probability by opening C? No, it doesn't. They have probabilities of zero or one - they aren't random. What is happening is that we are changing our confidence because our observation was a function of the parameter q (the location of the car) so it contained information.