Chapter 1 - Probability: Set theory
Fundamentals of Probability
- Set theory, Probability spaces.
- Probability Axioms.
- Discrete uniform law.
- Continuous uniform law.
Probability: Set theory
Set theory:
- The mathematical basis of probability is the theory of sets. For that we will review material alreeady learned and introduce the notation and terminology as set, element, union, intersection, and complement
- A set is a collection of things:
- The symbol $\in$ denotes set inclusion. Thus $x$ $\in$ $A$ means “x is an element of the set A.”
- The symbol $\notin$ is the opposite of $\in$. Thus $c$ $\notin$ $A$ means “$c$ is not an element of $A$ or $c$ is not in $A$.”
Subset
- The notion of a “subset” describes a relationship between two sets: $A$ is subset of $B$ if every member of $A$ is also a member of $B$. Thus $A \subset B$ is a mathematical nontation for the statment
$A \subset B$ if and only if "for all $x \in A \Rightarrow x \in B$".
\[A = \{0, 17, 46\}, B = \{0, 46\} \subset A, B = \{0\} \subset A\]- The definition of set’s “equality” is:
$A = B$ if and only if $B \subset A$ and$A \subset B$
$\{0, 17, 46\} = \{17, 0, 46\} = \{46, 0, 17\}$ are all the same set.
- The “null” set, or the “empty” set, is a set that has no elements. The notation for the empty set is $\emptyset$ $\Rightarrow \emptyset$ is a subset of every set.
$A = \{$ $\emptyset$ $, 0, 17, 46\}$
Union
- The “union” of sets $A$ and $B$ is the set of all elements that are either in $A$ or in $B$, or in both, and it is denoted by $A \cup B$. Formally, the definition states
$x \in A \cup B$ if and only if $x \in A$ OR $x \in B$.
\[A = \{0, 17, 46\}, B = \{5, 24, 17, 3\} \Rightarrow A \cup B = \{0, 17, 46, 5, 24, 3\}\]
Intersection
- The intersection of two sets $A$ and $B$ is the set of all elements which are contained in both $A$ and $B$, and it is denoted by $A \cap B$. Formally, the definition is
$x \in A \cap B$ if and only if $x \in A$ AND $x \in B$.
\[A = \{0, 17, 46\}, B = \{5, 24, 17, 3\} \Rightarrow A \cap B = \{ 17 \}\]
- Remark: $(A \cap B)$ $\subset$ $(A \cup B)$
Complement
- The complement of a set $A$, denoted as $A^{c}$, is the set of all elements in the universal set $(S)$ that are not in $A$. Formally,
$x \in A^{c}$ if and only if $x \notin A$.
$\Rightarrow$ $A^{c}$ $= \{3, 12, 21, 35, 40, 51\}$
- We have the following properties: $A \cup A^{c} = S, \ A \cap A^{c} = \emptyset$
Difference
- The difference set is a combination of intersection and comeplement, such that the different between $A$ and $B$ is a set $A - B$ that contains all elements of $A$ that are not elements of $B$. Formally,
$x \in A - B$ if and only if $x \in A$ and $x \notin B$
\[A = \{0, 17, 46\}, B = \{5, 24, 17, 3\} \Rightarrow A - B = \{0, 46\}\]
Example 1
Let the universal set $S = \{10, 13, 22, 49, 55, 132, 102, 67, 91\}$. Let the subsets $A = \{10, 55, 132, 49, 102\}$ and $B = \{13, 49, 55\}$ of $S$.
- $A^{c} = \{13, 22, 67, 91\}$
- $B^{c} = \{10, 22, 132, 102, 67, 91\}$
- $A \cap B = \{55, 49\}$
- $A \cup B = \{10, 55, 132, 49, 102, 13\}$
- $A - B = \{10, 132, 102\}$
Probability: Set theory-Properties
We have the following properties:
1. $A - B = A \cap B^{c}$
- $A - B \subset A \cap B^{c}$ ?
- $x \in A - B \Rightarrow x \in A$ and $x \notin B$
- $\Rightarrow x \in A$ and $x \in B^{c}$
- $\Rightarrow x \in A \cap B^{c}$
- $A \cap B^{c} \subset A - B$ ?
- $x \in A \cap B^{c} \Rightarrow x \in A$ and $x \in B^{c}$
- $\Rightarrow x \in A$ and $x \notin B$
- $\Rightarrow x \in A - B$
2. $A^{c} = S - A$ (with $A \subset S$)
- $A^{c} \subset S - A$ ?
- $x \in A^{c} \Rightarrow x \in S$ and $x \notin A$
- $\Rightarrow X \in S - A$
- $S - A \subset A^{s}$ ?
- $s \in S - A \Rightarrow x \in S$ and $x \notin A$
- $\Rightarrow x \in A^{c}$
3. - The DeMorgan’s Laws:
$(A \cap B)^{c} = A^{c} \cup B^{c}$ and $(A \cup B)^{c} = A^{c} \cap B^{c}$
- $(A \cap B)^{c} \subset A^{c} \cup B^{c}$ ?
- $x \in (A \cap B)^{c} \Rightarrow x \notin (A \cap B)$
- $\Rightarrow x \notin A$ or $x \notin B \Rightarrow x \in A^{c}$ or $x \in B^{c}$
- $\Rightarrow x \in A^{c} \cup B^{c}$
- $A^{c} \cup B^{c} \subset (A \cap B)^{c}$ ?
- $x \in A^{c} \cup B^{c} \Rightarrow x \in A^{c}$ or $x \in B^{c}$
- $\Rightarrow x \notin A$ or $x \notin B \Rightarrow x \notin (A \cap B)$
- $\Rightarrow x \in (A \cap B)^{c}$
- In the same way we can proof the second law.
Let the sets of Example 1:
$S = \{10, 13, 22, 49, 55, 132, 102, 67, 91\}$.
$A = \{10, 55, 132, 49, 102\}$.
$B = \{13, 49, 55\}$.
- $A^{c} = \{13, 22, 67, 91\}$
- $B^{c} = \{10, 22, 132, 102, 67, 91\}$
- $A \cap B = \{55, 49\}$
- $A \cup B = \{10, 55, 132, 49, 102, 13\}$
- $A - B = \{10, 132, 102\}$.
$\Rightarrow$ $A^{c} \cup B^{c} = \{13, 22, 67, 91, 10, 132, 102\}$
$\Rightarrow$ $(A \cap B)^{c} = \{10, 132, 22, 132, 102, 67, 91\}$
- We conclude that $(A \cap B)^{c} = A^{c} \cup B^{c}$
$\Rightarrow$ $A^{c} \cap B^{c} = \{22, 67, 91\}$
$\Rightarrow$ $(A \cup B)^{c} = \{22, 67, 91\}$
- We conclude that $(A \cup B)^{c} = A^{c} \cap B^{c}$
4. $(A - B) \subset A$ for all $x \in A - B \Rightarrow x \in A$ and $x \notin B \Rightarrow (A - B) \subset A$
5. $(A^{c})^{c} = A$.
6. $A$ $\cup$ $(B$ $\cap$ $C) = (A$ $\cup$ $B)$ $\cap$ $(A$ $\cup$ $C)$.
7. $A$ $\cap$ $(B$ $\cup$ $C) = (A$ $\cap$ $B)$ $\cup$ $(A$ $\cap$ $C)$.
Set theory
- In working with probablity we will frequently refer to two important properties of the collections of sets: mutually exclusive and collectively exhaustive.
- A collection of sets $A_{1}, …, A_{n}$ is mutually exclusive if and only if
$A_{i} \cap A_{j} = \emptyset,$ for all $i \neq j$.
- If we have only two sets in the collection, we say that these sets are disjoints.
$A$ and $B$ are disjoints if and only if $A \cap B = \emptyset$.
-
We will use a shorthand for intersection of $n$ sets: $\bigcap\limits_{i=1}^n A_{i} = A_{1} \cap A_{2} \cap … \cap A_{n}$
-
A collection of sets $A_{1}, …, A_{n}$ is collectively exhaustive if and only if
- We will use a shorthand for intersection of $n$ sets: $\bigcup\limits_{i=1}^n A_{i} = A_{1} \cup A_{2} \cup … \cup A_{n}$
Example 2: Let the sets $S = \{10, 13, 22, 49, 55, 132, 102, 67, 91\}$.
$A = \{10, 132, 102\}$.
$B = \{13, 22, 67\}$.
- $A^{c} = \{13, 22, 67, 55, 49, 91\}$
- $B^{c} = \{10, 132, 102, 55, 49, 91\}$
- $A \cap B = \emptyset \Rightarrow$ $A$ and $B$ are disjoint
- $A \cup B = \{10, 132, 67, 102, 13, 22\} \neq S \rightarrow$ $A$ and $B$ are not collectively exhaustive
- $A - B = A \cap B^{c} = A$.
$\Rightarrow A^{c} \cup B^{c} = \{13, 22, 67, 55, 49, 91, 10, 132, 102\} = S$
$\Rightarrow (A \cap B)^{c} = \emptyset^{C} = S = \{13, 22, 67, 55, 49, 91, 10, 132, 102\}$
- Which satisfy the Morgan’s law: $(A \cap B)^{c} = A^{c}\cup B^{c}$
$\Rightarrow A^{c} \cap B^{c} = \{55, 49, 91\}$
$\Rightarrow (A \cup B)^{c} = \{55, 49, 91\}$
- Which satisfy the Morgan’s law: $(A \cup B)^{c} = A^{c} \cap B^{c}$
Example 3: Let the sets $S = \{10, 13, 22, 49, 55, 132, 102, 67, 91\}$.
$A = \{10, 132, 102, 55, 49\}$
$B = \{13, 22, 67, 91\}$.
- $A^{c} = \{13, 22, 67, 91\} = B$
- $B^{c} = \{10, 132, 102, 55, 49\} = A$
- $A \cap B = \emptyset \Rightarrow$ $A$ and $B$ are disjoint
- $A \cup B = S = \{10, 13, 22, 49, 55, 132, 102, 67, 91\} \Rightarrow$ $A$ and $B$ are exhaustive.
Remark: Consider the following sets
We have $A \cap C \neq \emptyset, B \cap C \neq \emptyset$, but $A \cap B \cap C = \emptyset$
$\Rightarrow A, B$, and $C$ are no mutually exclusive.
- Consider the following sets
We have $A \cap C = \emptyset, B \cap C = \emptyset$, and $A \cap B \cap C = \emptyset$
$\Rightarrow A, B$, and $C$ are mutually exclusive.
Remark1:
We can write the union $A \cup B \cup C$ as an union of disjoint sets as follow:
$A \cup B \cup C =$ $A$ $\cup$ $(B \cap A^{c})$ $\cup$ $C \cap (A^{c} \cap B)^{c}$ $\leftarrow$ union of disjoint sets.
- Note that the green part is computed as:
Remark2:
We can write the union $A \cup B \cup C$ as an union of disjoint sets as all following:
$A \cup B \cup C = A \cup (C - A) \cup (B - (A \cup C))$
$= A \cup (C \cap A^{c}) \cup (B \cap (A \cup C)^{c})$
$= A \cup (C \cap A^{c}) \cup (B \cap (A^c \cap C^{c}))$
$= A \cup (C \cap A^{c}) \cup (B \cap (A^c \cap C^{c}))$
$A \cup B \cup C = B \cup (A - B) \cup (C - (A \cup B))$
$= B \cup (A \cap B^{c}) \cup (C \cap (A \cup B)^{c})$
$= B \cup (A \cap B^{c}) \cup (C \cap (A^{c} \cap B^{c}))$
$= B \cup (A \cap B^{c}) \cup (C \cap A^{c} \cap B^{c})$
$A \cup B \cup C = C \cup (B - C) \cup (A - (B \cup C))$
$= C \cup (B \cap C^{c}) \cup (A \cap (B \cup C)^{c})$
$= C \cup (B \cap C^{c}) \cup (A \cap (B^{c} \cap C^{c}))$
$= C \cup (B \cap C^{c}) \cup (A \cap B^{c} \cap C^{c})$
- We can make more combinations.
Problem 1: Let the Figure 1:
- Express $A \cup B$ and $B \cup A$ in different ways.
- Express $A \cup B$ as a union of disjoint sets.
Problem 1-Solution:
1. Let the Figure 1:
$A \cup B = \Omega \cap (A \cup B)$
$A \cup B = (\Omega \cap A) \cup (\Omega \cap B)$
$A \cup B = \Omega - (A^{c} \cap B^{c})$
$A \cup B = A \cup (\Omega - B^{c})$
$A \cup B = (\Omega - A^{c}) \cup B$
$A \cup B = (\Omega - A^{c}) \cup (\Omega - B^c)$
$A \cup B = (A \cap B^{c}) \cup (B \cap A^c) \cup (A \cap B)$
——————————————————————————————————————
$A \cap B = \Omega \cap (A \cap B)$
$A \cap B = \Omega \cap A \cap B$
$A \cap B = \Omega - (A^{c} \cup B^{c})$
$A \cap B = A \cap (\Omega - B^{c})$
$A \cap B = (\Omega - A^{c}) \cap B$
$A \cap B = (\Omega - A^{c}) \cap (\Omega - B^{c})$
2. Express $A \cup B$ as an union of disjoints sets.
\[A \cup B = A \cup (B - A) = A \cup (B \cap A^{c})\] \[A \cup B = B \cup (A - B) = B \cup (A \cap B^{c})\]Problem 2: Write in different ways the following sets:
- $A \cup (B \cup C)^{c}$
- $[A \cup (B^{c} \cup C^{c})]^{c}$
- $[(A \cap B)^{c} \cup (B \cap C)]^{c}$
Problem 2-Solution:
- $A \cup (B \cup C)^{c} = A \cup (B^{c} \cap C^{c}) = (A \cup B^{c}) \cap (A \cup C^{c})$
- $[A \cup (B^{c} \cup C^{c})]^{c} = A^{c} \cap (B^{c} \cup C^{c})^{c} = A^{c} \cap (B^{c})^{c} \cap (C^{c})^{c} = A^{c} \cap (B \cap C)$
- $[(A \cap B)^c \cup (B \cap C)]^c = (A \cap B) \cap (B \cap C)^c = (A \cap B) \cap (B^c \cup C^c)$
$= (A \cap B \cap B^c) \cup (A \cap B \cap C^c) = (A \cap \emptyset) \cup (A \cap B \cap C^c)$
$= \emptyset \cup (A \cap B \cap C^c) =$ $(A \cap B \cap C^c)$