Chapter 3 - Probability: Conditioning
Probability: Conditioning
- Conditional probability: The conditional probability is defined as
$\mathbb{P}(A|B)$ $=$ probability of $A$, given that event $B$ occurred or certain.
$\Rightarrow$ use new information to revise a model
- $B$ becomes our new universe (we are certain that $B$ occurs)
Example: Consider the weather of the 6th of February of the last 10 years
| Year | 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Raining | X | X | X | X | X | X | X | X | 8 Days | ||
| Windy | X | X | X | X | X | X | X | 7 Days | |||
| Humid | X | X | X | X | X | 5 Days |
$\mathbb{P}(A|B) = \frac{P(A \cap B)}{\mathbb{P}(B)}$
- Assumption: $\mathbb{P}(B) \neq 0$
$\lvert\Omega\rvert = 12, \; \lvert A\rvert = 5, \; \lvert B\rvert = 6, \; \lvert A \cap B\rvert = 2$
$\mathbb{P}(A) = \frac{5}{12}, \; \mathbb{P}(B) = \frac{6}{12}, \; \mathbb{P}(A \cap B) = \frac{2}{12}$
- Consider that $B$ is the new universe $\Rightarrow \mathbb{P}(A \vert B) = \frac{2}{6}$ and $\mathbb{P}(B \vert B) = 1$
- Consequence (symmetry):
$\mathbb{P}(B \vert A) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)}$
- Consider the previous example:
Example: Consider the two rolls of a tetrahedral die
$\Omega = {(1, 1), (1, 2), \ldots, (4, 4)} = 16$ elements
Event: $B = \{\text{min}(X, \; Y) = 2\} \Rightarrow B = \{(2, \; 2),(2, \; 3),(2, \; 4),(3,\; 2),(4, \; 2)\} \rightarrow \lvert B \rvert = 5 \Rightarrow \mathbb{P}(B) = \frac{5}{16}$
Event: $M = \{\text{max}(X, \; Y) = 2\} \Rightarrow M = \{(1, \; 2),(2, \; 1),(2, \; 2)\} \rightarrow \lvert C \rvert = 3 \Rightarrow \mathbb{P}(M) = \frac{3}{16}$
\[\mathbb{P}(M \vert B) = \frac{\mathbb{P}(M \cap B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(2, \; 2)}{\mathbb{P}(B)} = \frac{\frac{1}{16}}{\frac{5}{16}} = \frac{1}{5}\]
Example:
Event: $B = \{\text{min}(X, \; Y) = 2\} = \{(2, 2),(2, 3),(2, 4),(3, 2),(4, 2)\} \rightarrow \lvert B \rvert = 5 \Rightarrow \mathbb{P}(B) = \frac{5}{16}$
Event: $M = \{\text{max}(X, \; Y) = 1\} = \{(1,1)\} \rightarrow \lvert M \rvert = 1 \Rightarrow \mathbb{P}(M) = \frac{1}{16} \rightarrow M \cap B = \emptyset$
\[\mathbb{P}(M \vert B) = \frac{\mathbb{P}(M \cap B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(\emptyset)}{\mathbb{P}(B)} = 0\]Probability: Model based on conditional probability
Example:
For headaches three out of five patients take aspirin (or equivalent), two out of five take a drug M.
- With aspirin, 75% of the patients have been relieved.
- With drug M, 90% of the patients have been relieved.
- What is the overall rate of people relieved?
- What is the likelihood that a patient has taken aspirin knowing that he has been relieved?
Modeling the problem:
- Universe: $\Omega = \{\text{5 person have headaches}\}$
- Event A: $\{\text{3 patients took aspirin}\} \rightarrow \mathbb{P}(A) = \frac{3}{5}$
- Event B: $\{\text{2 patients took drug M}\} \rightarrow \mathbb{P}(B) = \frac{2}{5}$
- Event C: $\{\text{Patient is relieved}\}$
- We have $\mathbb{P}(C \vert A) = 0.75$
- We have $\mathbb{P}(C \vert B) = 0.9$
Solution:
1. The overall rate of people who relieved $= \mathbb{P}(C)$
- In this example, we have $A \cup B = \Omega$ and $A \cap B = \emptyset$
- The event $C$ can be written as: $C = \Omega \cap C = (A \cup B) \cap C = (A \cap C) \cup (B \cap C)$
$\mathbb{P}(C) = \mathbb{P}[(A \cap C) \cup (B \cap C)]$
$= \mathbb{P}(A \cap C) + \mathbb{P}(B \cap C) - \mathbb{P}(A \cap B \cap C)$
$= \mathbb{P}(A \cap C) + \mathbb{P}(B \cap C) - \mathbb{P}(\emptyset \cap C)$
$= \mathbb{P}(A \cap C) + \mathbb{P}(B \cap C) - \mathbb{P}(\emptyset)$
$= \mathbb{P}(A \cap C) + \mathbb{P}(B \cap C)$
$= \mathbb{P}(A)\mathbb{P}(C \vert A) + \mathbb{P}(B)\mathbb{P}(C \vert B)$
$= \frac{3}{5} \times (0.75) + \frac{2}{5} \times (0.9) =$ $0.81$
2. The likelihood that a patient has taken aspirin knowing that he has been relieved? $\rightarrow \mathbb{P}(A \vert C)$
$\mathbb{P}(A \vert C) = \frac{\mathbb{P}(A \cap C)}{\mathbb{P}(C)} = \frac{\mathbb{P}(A)\mathbb{P}(C \vert A)}{\mathbb{P}(C)} = \frac{\frac{3}{5}(0.75)}{0.81} =$ 0.5556
3. The likelihood that a patient has taken drug M knowing that he has been relieved $\rightarrow \mathbb{P}(B \vert C)$
$\mathbb{P}(B \vert C) = \frac{\mathbb{P}(B \cap C)}{\mathbb{P}(C)} = \frac{\mathbb{P}(B) \mathbb{P}C \vert B}{\mathbb{P}(C)} = \frac{\frac{2}{5}(0.9)}{0.81} =$ $0.4444$
Probability: Conditioning
Properties of conditional probability
- $\mathbb{P}(A \vert \Omega) = \frac{\mathbb{P}(A \cap \Omega)}{\mathbb{P}(\Omega)} = \frac{\mathbb{P}(A)}{\mathbb{P}(\Omega)} = \frac{\mathbb{P}(A)}{1} = \mathbb{P}(A)$ and $\mathbb{P}(B \vert \Omega) = \mathbb{P}(B)$
- $\mathbb{P}(\Omega \vert B) = \frac{\mathbb{P}(\Omega \cap B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(B)}{\mathbb{P}(B)} = 1$
- $\mathbb{P}(B \vert B) = \frac{\mathbb{P}(B \cap B)}{\mathbb{P}(B)} = \frac{\mathbb{P}(B)}{\mathbb{P}(B)} = 1$
- $\mathbb{P}(A \cap B) = \mathbb{P}(B)\mathbb{P}(A \vert B)$ and $\mathbb{P}(A \cap B) = \mathbb{P}(A)\mathbb{P}(B \vert A)$
If $A \cap C = \emptyset$, we have $\mathbb{P}((A \cup C) \vert B) = \mathbb{P}(A \vert B) + \mathbb{P}(C \vert B)$
$\mathbb{P}(A \cup C \vert B) = \frac{P\mathbb{P}((A \cup C) \cap B)}{\mathbb{P}(B)}$
$= \frac{\mathbb{P}[(A \cap B) \cup (C \cap B)]}{\mathbb{P}(B)}$
$= \frac{\mathbb{P}(A \cap B) + \mathbb{P}(C \cap B) - \mathbb{P}[(A \cap B) \cap (C \cap B)]}{\mathbb{P}(B)}$
$= \frac{\mathbb{P}(A \cap B) + \mathbb{P}(C \cap B) - \mathbb{P}(\emptyset)}{\mathbb{P}(B)} = \frac{\mathbb{P}(A \cap B)}{P(B)} + \frac{\mathbb{P}(C \cap B)}{\mathbb{P}(B)}$
$= \mathbb{P}(A \vert B) + \mathbb{P}(C \vert B)$
Consequence: $A \cap A^{c} = \emptyset \Rightarrow \mathbb{P}(\underbrace{(A \cup A^{c})}_{\Omega} \vert B) = \mathbb{P}(A \vert B) + \mathbb{P}(A^{c} \vert B) = 1$ $\Rightarrow$ $\mathbb{P}(A \vert B) = 1 - \mathbb{P}(A^{c} \vert B)$
Probability: Radar model based on conditional probability
- Event $A$: An airplane is flying above $\rightarrow$ $A^{c}$: Nothing is flying above.
- Event $B$: Something registers on the radar’s screens $\rightarrow$ $B^{c}$: The radar is not detecting anything.
- Let $\mathbb{P}(A) = 0.05$
- Let $\mathbb{P}(B \vert A) = 0.99$ $\Rightarrow \mathbb{P}(B^{c} \vert A) = 1 - 0.99 = 0.01$
- Let $\mathbb{P}(B \vert A^{c}) \rightarrow$ False alarm $\Rightarrow \mathbb{P}(B^{c} \vert A^{c}) = 1 - 0.1 = 0.90$
Question:
What is the probability that an airplane is flying above when something registers on the radar’s screen (we want to check the reliability of the radar).
$\mathbb{P}(A \vert B) = \; ?$
- Event $A$: An airplane is flying above
- Event $B$: Something registers on the radar’s screens
- By giving a conditional probability, can we compute $\mathbb{P}(A \cap B)$ and $\mathbb{P}(B)$?
$\mathbb{P}(B \vert A) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)} \Rightarrow$ $\mathbb{P}(A \cap B)$ $= \mathbb{P}(A) \times \mathbb{P}(B \vert A) = (0.50)(0.99) =$ $0.0495$
$\mathbb{P}(A^{c} \cap B)$ $= \mathbb{P}(A^{c}) \times \mathbb{P}(B \vert A^{c}) = (0.95)(0.1) =$ $0.095$
$\mathbb{P}(B)$ $= \mathbb{P}((A \cup A^{c}) \cap B) = \mathbb{P}((A \cap B) \cup (A^{c} \cap B)) =$ $\mathbb{P}(A \cap B) + \mathbb{P}(A^{c} \cap B)$
$= 0.0495 + 0.095 =$ $0.1445$
$\mathbb{P}(A \vert B) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(B)} = \frac{0.0495}{0.1445} = \underline{0.342}$
The radar is not reliable: “Most of the time there is nothing but the radar detect a flying plane with a rate of 10%” $\rightarrow$ “false alarms are pretty common”
Probability: Bayes’s rule
(Thomas Bayes, British Mathematician, 1701-1761)
Baye’s rule:
$\mathbb{P}(A \vert B) = \frac{\mathbb{P}(B \vert A) \times \mathbb{P}(A)}{\mathbb{P}(B)}$ and $\mathbb{P}(B \vert A) = \frac{\mathbb{P}(A \vert B) \times \mathbb{P}(B)}{\mathbb{P}(A)}$