Lecture #3: Independence & Conditional Probability

Formulas for \(P(A \cup B)\) and \(P(A \cap B)\)

Recall that: \[ P(A \cup B) = P(A) + P(B) - P(A \cap B), \]

and if \(A\) and \(B\) are mutually exclusive, then \(P(A \cap B) = 0\) and:

\[ P(A \cup B) = P(A) + P(B). \]

Independence

Two events are independent if the occurrence of one event occuring does not affect the probability of the other event occuring.

In our formal notation, that is:

\[ P(A \cap B) = P(A) \times P(B) \]

Independence occurs when knowing information about one event or random variable does not give us any information about the other event or random variable.

Independence Examples

1. What is the probability of rolling two \(1\)s on a six-sided die?

2. What is the probability of seeing ten heads when flipping ten fair coins?

3. Can two events be both mutually exclusive and independent?

Non-Independence

Most events are not independent, however. We might try to assume independence for many statistical operations, but that’s not always a sound assumption to make.

How then do we calculate \(P(A \cap B)\) if we can’t assume independence?

Conditional Probability

The conditional probability of \(A\) given \(B\), denoted by \(P(A \,\vert\, B)\), is defined by

\[ P(A \,\vert\, B) = \frac{P(A \cap B)}{P(B)}. \]

Conditional Probability Example

Example: E-mail sent to a person can either be “spam” or not “spam”. It can also contain the word “offer” or not contain the word. Suppose 160 emails are sampled and the following table is computed:

1. What is the probability an e-mail contains the word “offer” given that it is spam?

2. What is the probability an e-mail doesn’t contain the word “offer” given that it is spam?

Conditional Probability Example

Example: Gregor Mendel was a monk who, in 1865, suggested a theory of inheritance based on the science of genetics. He identified heterozygous individuals for flower color that had two alleles (one \(r\) = recessive white color allele and one \(R\) = dominant red color allele). When these individuals were mated, \(\frac{3}{4}\) of the offspring were observed to have red flowers, and \(\frac{1}{4}\) had white flowers.

The following table summarizes this mating; each parent gives one of its alleles to form the gene of the offspring.

Conditional Probability Example (Continued)

We assume that each parent is equally likely to give either of the two alleles and that, if either one or two of the alleles in a pair is dominant (R), the offspring will have red flowers. What is the probability that an offspring has:

1. at least one dominant allele?
2. at least one recessive allele?
3. one recessive allele, given that the offspring has red flowers?

The Law of Total Probability

The Law of Total Probability:

\[ P(A) = P(A \,\vert\, B) \times P(B) + P(A \,\vert\, \overline{B}) \times P(\overline{B}) \]

Partition

For some positive integer \(k\), let the sets \(B_1, B_2, \dots, B_k\) be such that:

1. \(S = B_1 \cup B_2 \cup \dots \cup B_k\).
2. \(B_i \cap B_j = \varnothing\), for \(i \ne j\).

Then the collection of sets \(\{B_1, B_2, \dots, B_k \}\) is said to be a partition of \(S\).

The Law of Total Probability also holds if the partition consists of a countably infinite collection of sets.

The Law of Total Probability Example

A population of voters contains \(40\%\) Republicans and \(60\%\) Democrats. It is reported that \(30\%\) of the Republicans and \(70\%\) of the Democrats favor an election issue. A person chosen at random from this population is found to favor the issue in question. What is the probability that this person is a Democrat?

Bayes’ Theorem

Bayes’ Theorem:

If we want to “flip” the events in a conditional probability, we can thank Thomas Bayes (1701–1761). \[ P(A \,\vert\, B) = \frac{P(B \,\vert\, A) \times P(A)}{P(B)} \]

Bayes’ Theorem

We can combine this with the Law of Total Probability, to get the formulation: \[ P(A \,\vert\, B) = \frac{P(B \,\vert\, A) \times P(A)}{P(B \,\vert\, A) \times P(A) + P(B \,\vert\, \overline{A}) \times P(\overline{A})} \]

or:

\[ P(A_j \,\vert\, B) = \frac{P(B \,\vert\, A_j) \times P(A_j)}{\displaystyle\sum_{i \in I} \big[ P(B \,\vert\, A_i) \times P(A_i) \big]} \]

Bayes’ Theorem Example

Example: An airport explosives detector lights up with probability \(0.75\) when it senses an explosive, but also lights up with probability \(0.01\) when it doesn’t. Suppose that a piece of luggage has only a \(0.0001\) chance of containing an explosive. If the detector lights up, what is the probability the luggage contains an explosive?

Bayes’ Theorem Example

Example: Suppose there is a genetic disorder that occurs in \(1\) in every \(100000\) births. A team of researchers develop a test to see if you have it.

If you have the disorder, there is a \(99.9\%\) chance that the test will return a positive result.

If you do not have the disorder, there is a \(99.95\%\) chance that the test will return a negative result.

You have the test run on yourself, and you get a positive result. What is the probability that you have the disease?