Bayes’ Rule - Machine Learning Mindset

Bayes’ Rule

Lesson 7 Chapter 2

In order to explain the Bayes' rule, let's start with something simple as a special case. Assume we have two events $A$ and $B$ . Do you agree with the following statement?

$A = A \cap \Omega = A \cap (B \cup B^c) = (A \cap B) \cup (A \cap B^c)$

$A = A \cap \Omega$ : The intersection of any event with the sample space is that particular event.
$\Omega = B \cup B^c$ : The union of any event with its complement event is the whole sample space.

NOTE: $E_1 = A \cap B$ and $E_2 = A \cap B^c$ events are mutually exclusive. Think why?

Now, let's calculate the probability of event $A$ :

(1) $\begin{equation*} \begin{split}P(E) & = P(A \cap (B \cup B^c)) = P(A \cap B) + P(A \cap B^c) \\& = P(A | B) P(B) + P(A | B^c) P(B^c)\end{split}\end{equation*}$

Above, we used the conditional probability rules to expand the event intersections. The above equation asserts that the probability of event $A$ is a weighted average of the conditional probability of $A$ given that $B$ has occurred and the conditional probability of $A$ given that $B$ has not happened. This is a very useful formula as a lot of times, directly calculating the probability of an event such as $A$ may not be easy or even possible. This rule conditions the likelihood of an event on different events.

The fact is, in Bayes' rule, the probability of an event is going to be conditioned on different mutually exclusive events ( $B$ and $B^c$ ) which together form the whole sample space as $B \cup B^c = \Omega$ .

The special case of above, can be extended to the below more general rule called the Bayes' rule.

Bayes' rule

Assume we have some mutually exclusive events as $\Omega = \bigcup_{i=1}^{n}E_i$ , i.e., at least one of the $E_i$ event MUST and will occur. Then, the probability of the event $A$ is calculated as below:

(2) $\begin{equation*} \begin{split}P(A) = \sum_{i=1}^{n}P(A|E_i)P(E_i)\end{split}\end{equation*}$

Share0

Tweet0

Share0

Bayes' rule

60

SHARES