Solving Bayes Theorem | Good Organization
Like many people, math was far from my favorite subject in school. “When will I ever need this?” I thought. Cut to my adult life where I would end up needing it more than most due to its unmatched ability to provide evidence in all kinds of analysis. Still, I can’t help but still feel disdain for the subject. Unless, of course, the usefulness of the topic at hand is immediately evident. Bayes theorem is one such topic. Bayes theorem is more than just a mathematical equation, it’s a method for checking your instinctual assumptions. In a world where many of our beliefs are constantly challenged and continually proven incorrect or inadequate, few things can help us consider context more adequately. For this reason, knowing how to solve Bayes's problems can help you not only pass that upcoming statistics test but also be a subtle reminder that history, background, and biases all play a role in what we think we know about the world.
Before we start trying to solve anything, what is Bayes' theorem? If you have found your way here, I am going to assume that this is not your first ever encounter with it just give you a quick refresher. Bayes’ theorem is a method by which we can calculate the probability of specific circumstances.
The simple formula for Bayes is:
It is one approach to statistical inference, known as Bayesian inference, but has many practical applications in the real world in many fields including the medical field, machine learning, and credit/finance to name a few. For a longer explanation, I recommend checking out this video and this video. Now, we’re ready to work on a problem.
Example: Jiji (hey, that’s me!) is a meme queen. The internet is rich with endless sources for the dankest of memes. Memes often try to be funny or otherwise ironic, but they can also spread awareness of major events that have occurred. Good memes do a better job of spreading such awareness because memes that are not good are unlikely to spark people’s curiosity about what event the meme is referring to. Note that not all memes are referring to major events, but there are ways of learning about major events other than through memes.
The probability that a meme will be good is 30%. At the same time, major events often result in the entire meme generating community hyper-focusing on one aspect and churning out new memes faster than EA releases DLC after a new game launch. Sometimes Jiji learns about major events because she has encountered a good meme, this happens 80% of the time. Unfortunately, other times she learns an event occurred because of a meme that is not good, this happens 25% of the time? Let us take for granted that on an important day in history, Jiji reads one random meme, which may be good or not good, and a major event occurs. What is the probability that the meme she reads is good given that she learns about the major event?
Step 1: This problem is more than a thinly veiled excuse to include as many memes as possible in my blog, it is also a totally valid use of the Bayes theorem. But first, we must better define the problem we are trying to solve. What exactly is the question? What is our P(A|B)? For this, we usually look to the last sentence.
What is the probability that the meme she reads is good given that she learns about the major event on that day?
This tells us that what we want to know is the probability that a meme is good (A) given that she has learned a major event (B) has occurred.
Step 2: Now that we know what we’re trying to solve. We need to look at what information we are given. Generally, you will be given three pieces of information, though, which three can vary from problem to problem. In this problem we are given the following:
- The probability that a meme is good is 30%.
- Jiji learns a major event occurred due to a good meme 80% of the time.
- She learns a major event occurred due to a meme that is not good 25% of the time.
We know that our P(A) is the relating to the memes and our P(B) is relating to the events and with that information, we can now assign values to our variables. Our first bullet point is our P(A), the probability of a meme being good in general. The second bullet point is our P(B|A), the probability that she learns a new event occurred given that she sees a good meme. The third bullet point is our P(B|A’), the probability she learns a new event has occurred given she sees a meme that is not good.
Step 3: We have noted all the information given to us explicitly but is that really all the information? Nope! Now it is time to note the information we have from the inverses of the information given. For example, if we know that P(A) is 30% then the probability of not ‘A’ must be 70%. With other combinations, it can be a bit trickier to keep straight. Note for example that the inverse of P(B|A) is not P(B|A’) rather it is P(B’|A). Here are all of the inverses to help you stay on track as we continue to organize our data:
With the above chart in mind, we can now map out more data. Since now we know we were given twice as much information as we originally thought. Though we may not end up using all of this information in the final equation, it may be helpful to write it down anyway.
STEP 4: Now we have truly gathered all the information available to us from the original question and we can switch focus to the piece(s) of information we don't have, the ever so crucial P(B). And, good news everyone! The formula needed is one you have probably already seen:
We can now use the information we have collected and the formula in the denominator above to calculate the final piece of the puzzle. Huzzah!
Step 5: The final step is just a matter of plug and play. Inserting all of our values into the standard Bayes theorem formula.
So it appears that Jiji has a 58% of seeing a good meme given a major event has occurred. Which are pretty good odds if you ask me but also somewhat counterintuitive to what you might believe if you are a regular purveyor of the meme-istic arts. This is the point, Bayes theorem proves that sometimes, maybe even often, what we might think is true lacks context. In this case, even though I only see good memes 30% of the time, it does not counteract how much more often it is following a major event.
In the end, there are many ways to approach a Baysean problem whether you choose to draw a picture, solve part of it in your head, or solve it thru organization as I do. What we should agree on is the importance of using this and many other methods to constantly challenge our own predispositions. Nevertheless, I hope this method can be one way you go from seeing this as Bayes theorem to bae’s theorem.
It will likely take more than one example to help you get there but once you do, you are likely to find that this is one concept with uses well beyond the classroom or lab. With practice, soon you too can go from this:
To this:
