Lecture #10: The Basics of Probability

STAT 200 – Introductory Statistics

Torin Quinlivan
tquinlivan@knox.edu

Knox College

January 27, 2026

Probability

Imagine you’re playing a hand of high-stakes Texas Hold ’Em, and, to stay alive, you need an spade on the river to save you. How likely is that?

Or suppose you’re playing Monopoly, and you want to buy Boardwalk before your opponents to pair with your Park Place, but you’re currently twelve squares away. What are the chances you roll that twelve on your next turn?

This Wednesday night, the New York Knicks are on the road playing the Toronto Raptors. If I want to bet on that game, who should I expect to win that game?

To help determine any of these situations, we need to discuss the concepts of probability.

Probability

The probability of an event occurring is the long-term relative frequency of the event occurring.

In other words, if we repeat an event many times, over and over again, like turning over the river card or rolling two dice, what proportion of times would our desired outcome occur?

That proportion is called the probability of that event.

Randomness vs. Chaos

What is the difference between “randomness” and “chaos”?

A random phenomenon has the unique characteristic of being predictable in the long run.

For example, if we flip a fair coin a large number of times, we know that in the long run about 50% of the flips will land “heads”, but I can’t know what the next flip will result in.

Chaos is not predictable in any way; think of the white noise static on a TV.

Definitions

The sample space, \(\mathcal{S}\) is the set or list of all possible outcomes of an experiment.

An event is a subset of the sample space; a single outcome or set of outcomes.

Sample Space Example

Consider rolling a six-sided die.

  1. What is the sample space, \(\mathcal{S}\)?
  2. Let event A be the event that you roll a 6. What does that event look like?
  3. Let event B be the event you roll a 3 or lower. What does that event look like?

Probability

For any random phenomenon, each attempt of a basic experiment is called a trial.

Each trial produces an outcome.

For example, a given toss of a fair coin is a trial, the sample space is \(\mathcal{S} = \{Heads, Tails\}\) and the outcome might be “Tails”.

We combine possible outcomes to create events.

In the previous example, the event \(B\) was the event we roll a three or less on a six-sided die. The event \(B\) is the combination of the outcomes 1, 2 and 3, i.e. \(B = \{ 1, 2, 3 \}\).

The Law of Large Numbers

The Law of Large Numbers (or LLN) states if we repeat a random, basic process over and over again under identical conditions, the proportion of times the event occurs will settle down, in the long run, to its true probability.

This is different than the fallacious “Law of Averages” that some people imagine.

The Law of Large Numbers only governs long-run relative frequencies. Even if you haven’t rolled a 6 in your last twenty rolls, you aren’t due a 6 on your next roll.

Don’t fall for statements like “It’s been a mild autumn, so we’re in for a harsh winter!” or “I’ve been getting crap hands all night, I’m due for a big hand!”

Probability Notation

We denote the probability of event A occurring \(\mathbb{P}(A)\), and say:

\[ \mathbb{P}(A) = \frac{\text{number of outcomes in event A}}{\text{total number of outcomes in sample space } \mathcal{S}} \]

\[ \mathbb{P}(A) = \frac{\text{number of ways event A can occur}}{\text{total number of possible outcomes}} \]

NOTE: For this equation to work, all outcomes must be equally likely!

Example #1: Sample Space

For each of the following, list the sample space and tell whether you think the events are equally likely:

  1. Roll two dice; record the sum of the numbers.
  2. A family has 3 children; record each child’s sex in order of birth.
  3. What is the probability of having three children of the same sex?

The Axioms of Probability

  1. All probabilities must be between zero and one, \(0 \le \mathbb{P}(A) \le 1\).
    • The closer a probability is to zero, the less likely that event is.
    • The closer a probability is to one, the more likely that event is.
  2. \(\mathbb{P}(\varnothing) = 0\), where \(\varnothing\) is the empty set.
  3. \(\mathbb{P}(\mathcal{S)} = 1\), where \(\mathcal{S}\) is the sample space.

Disjoint Events

Two events \(A\) and \(B\) are disjoint, or mutually exclusive, if there are no outcomes in common between \(A\) and \(B\).

In other words, there is no way for both \(A\) and \(B\) to occur simultaneously.

Rolling a 2 and rolling a 5 on the same die are disjoint events.

Rolling a 2 on one die and rolling a 5 on another die are not disjoint events.

Addition Rule

If \(A\) and \(B\) are disjoint events, then the probability of \(A\) or \(B\) occurring is:

\[ \mathbb{P}(A \textrm{ or } B) = \mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) \]

If \(A\) and \(B\) are not disjoint events, then the probability of \(A\) or \(B\) occurring is:

\[ \mathbb{P}(A \textrm{ or } B) = \mathbb{P}(A \cup B) = \mathbb{P}(A) + \mathbb{P}(B) - \mathbb{P}(A \cap B) \]

\(A \cup B\) is the union of \(A\) and \(B\), you can think of this as \(A\) or \(B\).

\(A \cap B\) is the intersection of \(A\) and \(B\), you can think of this as \(A\) and \(B\).}

Example #2: Valid Distributions

The plastic arrow on a spinner for a child’s game stops rotating to point at a color that will determine what happens next. Which of the following probability assignments are legitimate?

Complement Rule

The complement of an event, often denoted as \(A^C\) or \(A'\), is the set of all outcomes in the sample space not in the event \(A\).

Let \(A\) be an event, and \(A^C\) be the complement of that event. Then:

\[ \mathbb{P}(A^C) = 1 - \mathbb{P}(A) \]

Example #3: Blood Type

The American Red Cross says that about \(45\%\) of the U.S. population has Type O blood, \(40\%\) Type A, \(11\%\) Type B, and the rest Type AB.

  1. What is the probability that a person has Type AB blood?
  2. What is the probability that a person doesn’t have Type O blood?
  3. What is the probability that a person has either Type A or Type B blood?
  4. What is the probability that a person doesn’t have the antigen A, i.e. Type B blood or Type O?

Independent Events

Two events \(A\) and \(B\) are independent if the outcome of one event does not affect the outcome of the other event.

Getting heads on a coin flip and rolling a 4 on a die are independent events.

You having red hair and your mother having red hair are not independent events.

If we say two people are “randomly chosen” in a word problem, that usually implies independence.

Can two events be both mutually exclusive and independent?

Multiplication Rule

If \(A\) and \(B\) are independent events, then the probability of \(A\) and \(B\) occurring is:

\[ \mathbb{P}(A \textrm{ and } B) = \mathbb{P}(A \cap B) = \mathbb{P}(A) \times \mathbb{P}(B) \]

Example #3: Blood Type

The American Red Cross says that about \(45\%\) of the U.S. population has Type O blood, \(40\%\) Type A, \(11\%\) Type B, and the rest Type AB.

  1. If two people are randomly selected, what is the probability that they both have Type A blood?
  2. If two people are randomly chosen, what is the probability that the first one has Type B blood and the second one doesn’t?
  3. If seven people are randomly chosen, what is the probability that at least one of the seven has type O blood?

Conditional Probability

If we know that an event \(A\) has occurred already, the probability that an event \(B\) then happens, or “the probability of \(B\) given \(B\)”, is:

\[ \mathbb{P}(B \mid A) = \frac{\mathbb{P}(A \cap B)}{\mathbb{P}(A)} \]

This is the difference between “What is the likelihood that the Chiefs win the Super Bowl next year?” and “If the Chiefs win the division next year, what is the likelihood that the Chiefs then win the Super Bowl next year?”

Multiplication Rule

If \(A\) and \(B\) are not independent events, then the probability of \(A\) and \(B\) occurring is:

\[ \mathbb{P}(A \textrm{ and ) B} = \mathbb{P}(A \cap B) = \mathbb{P}(A \mid B) \times \mathbb{P}(B) = \mathbb{P}(B \mid A) \times \mathbb{P}(A) \]

Example

Suppose \(C\) and \(D\) are events such that \(\mathbb{P}(C) =0.50\), \(\mathbb{P}(D) = 0.30\) and \(\mathbb{P}(C \mid D) = 0.40\).

  1. Find \(\mathbb{P}(C \text{ and } D)\).
  2. Find \(\mathbb{P}(D \mid C)\).

Bayes’ Theorem

Combining these definitions give us a very important result, Bayes’ Theorem.

Let \(A\) and \(B\) be events. The conditional probability of \(A\) given \(B\) can be expressed as:

\[ \mathbb{P}(B \mid A) = \frac{\mathbb{P}(A \mid B) \times \mathbb{P}(B)}{\mathbb{P}(A)} \]

Bayes’ Theorem

We can also write this as (by using the Law of Total Probability):

\[ \mathbb{P}(B \mid A) = \frac{\mathbb{P}(A \mid B) \times \mathbb{P}(B)}{\mathbb{P}(A \mid B) \times \mathbb{P}(B) + \mathbb{P}(A \mid B^C) \times \mathbb{P}(B^C)} \]

This can be extended to any partition of the sample space, no matter how many subsets are in the partition.

Example #4: Bayes’ Theorem

Suppose there is a genetic disorder that occurs in \(1\) in every \(1000\) births. A team of researchers develop a test to see if you have it.

If you have the disorder, there is a \(99\%\) chance that the test will return a positive result.

If you do not have the disorder, there is a \(95\%\) chance that the test will return a negative result.

You have the test run on yourself, and you get a positive result. What is the probability that you have the disease?

Bayesian Statistics

Bayes’ Theorem is the foundation of the field of statistics known as “Bayesian statistics”. Bayesian statistics is arguably the most common school of thought in modern statistics.

My research work and my dissertation was actually in Bayesian statistics and the applications on complex data sets.

Bayesian statistics follows the way we consider things in real life, updating our assumptions and perceptions of the world when we get new information.

The math in Bayesian statistics can get very complex very quickly, so it was limited for a long time by the difficulties in computation, but as computers have gotten more and more powerful it has opened new doors in the field.