Lecture #12: The Bernoulli, Binomial and Poisson Distributions

Definition of Bernoulli Experiment

A random variable with only two possible outcomes follows a Bernoulli distribution with success probability parameter, \(p\).

Examples of Bernoulli Experiments

Examples:

• head on a single flip of a coin.
• getting a 6 on one roll of a die.
• correct choice on a True/False question.
• using a credit card for a transaction.
• voting in an election.
• stopping at Starbucks in the morning.
• war breaking out between two countries in a year.

Probability Mass Function of a Bernoulli Variable

Recall that the probability mass function (pmf) provides the probability of each element of the sample space.

For a Bernoulli random variable, there are only two outcomes: failure and success – 0 and 1.

From this, the probability mass function for a Bernoulli random variable is always

\[ \mathbb{P}[X = x] = \begin{cases} p & \textrm{if } x = 1 \\ 1 - p & \textrm{if } x = 0 \\ 0 & \textrm{otherwise} \end{cases} \]

Probability Mass Function of a Bernoulli Variable

This can also be written as

\[ \mathbb{P}[X = x] = p^x\, (1 - p)^{1 - x}, \qquad x \in \{0, 1\} \]

Expected Value of a Bernoulli Variable

The expected value of a Bernoulli random variable is:

\[ \begin{aligned} \mathbb{E}[X] &= \sum_{x \in S} x\,\mathbb{P}[X = x] \\ &= \left( 0 \times \mathbb{P}[X = 0] \right) + \left( 1 \times \mathbb{P}[X = 1] \right) \\ &= 0\,(1 - p) + 1\,(p) \\ &= p \end{aligned} \]

Variance of a Bernoulli Variable

The variance of a Bernoulli random variable, where \(\mu = p\), is:

\[ \begin{aligned} \mathbb{V}[X] &= \sum_{x \in S} (x - \mu)^2 \, \mathbb{P}[X = x] \\ &= (0 - p)^2 \left( \mathbb{P}[X = 0] \right) + (1 - p)^2 \left( \mathbb{P}[X = 1] \right) \\ &= p^2\,(1 - p) + (1 - p)^2\,p \\ &= p\,(1 - p)\,\left( p + (1 - p) \right) \\ &= p\,(1 - p) \end{aligned} \]

Cumulative Distribution Function of a Bernoulli R.V.

Recall the probability mass function (pmf):

\[ \mathbb{P}[X = x] = \begin{cases} p & \textrm{if } x = 1 \\ 1 - p & \textrm{if } x = 0 \\ 0 & \textrm{otherwise} \end{cases} \]

Median of a Bernoulli Variable

The median depends on the Bernoulli parameter, \(p\). When \(p \ne 0.5000\), the rule to calculate the median is to start low and calculate the cumulative probability until that sum is at least 0.5000. This is most easily done with a cumulative distribution function.

Median of a Bernoulli Variable

Why is that the formula for the median?

Think: As I increase \(x\), when does the cumulative probability attain 0.500, when \(p < 0.500\) (as here)? That will be the median, \(\widetilde{X} = 0\).

Median of a Bernoulli Variable

Why is that the formula for the median?

Think: As I increase \(x\), when does the cumulative probability attain 0.500, when \(p > 0.500\) (as here)? That will be the median, \(\widetilde{X} = 1\).

Median of a Bernoulli Variable

Why is that the formula for the median?

Think: As I increase \(x\), when does the cumulative probability attain 0.500, when \(p = 0.500\)? That will be the median, \(0 \le \widetilde{X} \le 1\) (or \(\widetilde{X} = \frac{1}{2}\) by convention).

Definition of a Binomial Random Variable

A binomial random variable is the sum of \(n\) independent and identically distributed Bernoulli random variables.

Definition of a Binomial Random Variable

This definition leads to the five requirements of a Binomial random variable:

1. The number of trials, \(n\), is known.
2. Each trial has two possible outcomes (i.e. is Bernoulli).
3. The success probability, \(p\), is constant from trial to trial.
4. The trials are independent.
5. The random variable is the number of successes.

Examples of Binomial Random Variables

Examples of Binomial random variables are:

• times getting heads on \(n\) flips of a coin.
• times getting a 6 on \(n\) rolls of a die.
• times getting the correct choice on \(n\) True/False questions.
• times using a credit card for \(n\) transactions.
• times voting in \(n\) elections.
• times stopping at Starbucks in \(n\) mornings.
• times war breaks out between two countries in \(n\) years.

Examples of Binomial Random Variables

The difference between Binomial random variables and Bernoulli random variables:

• The Bernoulli random variable is the number of successes in one trial.
• The Binomial random variable is the number of successes in \(n\) trials.

Probability Mass Function of a Binomial R.V.

Recall that a Binomial random variable is just the sum of \(n\) Bernoulli random variables. From this, we will be able to determine the pmf of the Binomial distribution:

\[ \mathbb{P}[X = x] = \binom{n}{x}\, p^x\, (1 - p)^{n - x}, \qquad x = 0, 1, \dots, n \]

Binomial Parameters

The expected value and variance of a Binomial are easily calculated from its statistical definition:

Let \(X_i \stackrel{\textrm{iid}}{\sim} \text{Bernoulli}(p)\). Define \(Y = \sum_{i=1}^n X_i\). Then \(Y \sim \text{Binomial}(n, p)\).

Expected Value of a Sum of Random Variables

The expected value of a linear combination of random variables is the sum of the expected values of the random variables:

\[ \mathbb{E}[aX + bY + c] = a\,\mathbb{E}[X] + b\,\mathbb{E}[Y] + c \]

Expected Value of a Binomial R.V.

Using this property and the definition of the Bernoulli distribution, we can show:

\[ \begin{aligned} \mathbb{E}(Y) &= \mathbb{E}\left(\sum_{i=1}^n X_i\right) \\ &= \sum_{i=1}^n \mathbb{E}(X_i) \\ &= \sum_{i=1}^n p \\ &= n\,p \end{aligned} \]

Variance of a Sum of Random Variables

The variance of a linear combination of independent random variables is the sum of the variances of the random variables:

\[ \mathbb{V}[aX + bY] = a^2\,\mathbb{V}[X] + b^2\,\mathbb{V}[Y] \]

Variance of a Binomial Random Variable

Using this property and the definition of the Bernoulli distribution, we can show:

\[ \begin{aligned} \mathbb{V}(Y) &= \mathbb{V}\left(\sum_{i=1}^n X_i\right) \\ &= \sum_{i=1}^n \mathbb{V}(X_i) \\ &= \sum_{i=1}^n p\,(1 - p) \\ &= n\,p\,(1 - p) \end{aligned} \]

Binomial Distribution in R

In R, there are a set of four functions to calculate various quantities from the Binomial distributions

• dbinom(x, size, prob) is the PMF, giving \(\mathbb{P}(X = x)\).
• pbinom(x, size, prob) is the CDF, giving \(\mathbb{P}(X \le x)\).
• qbinom(p, size, prob) is the quantile function, i.e. the smallest \(x\) with \(\mathbb{P}(X \le x) \ge p\).
• rbinom(n, size, prob) generates \(n\) random values from a Binomial(size, prob) distribution.

Example #1: Fair Coins

Let \(Y\) be the number of heads flipped in \(n = 10\) flips of a coin. If the coin is fair, then it is clear that \(Y \sim \text{Binomial}(n = 10, p = 0.500)\).

1. What is the probability of getting three heads?
2. What is the probability of getting three tails?
3. What is the probability of getting at most three heads or at least seven heads?

Example #2: Cancer in Dogs

A Knox student studied a type of canine cancer (hemangiosarcoma) and a medicine that is supposed to increase longevity. Without the medicine, the median survival time for the dog is 90 days from detection.

She wanted to determine if the medicine helped dogs live longer. To do this, she gave the treatment to 10 dogs with the cancer and measured how long they lived. Of the 10 dogs given the new treatment, eight lived longer than 90 days.

What is the probability of observing eight or more dogs living longer than 90 days if the medicine doesn’t work?

Example #3: Hurricanes in the Atlantic

A researcher has a theory that climate change is causing the intensity of hurricanes in the Atlantic Ocean to increase. Historical data shows that 43% of all Atlantic hurricanes are determined to be a “major hurricane” (Category 3, 4 or 5).

The researcher looks at 40 Atlantic hurricanes from the last three seasons, and finds that 20 of them were named “major hurricanes”.

Is this evidence that the amount of “major hurricanes” in the Atlantic has increased in recent years?

Definition of Poisson Experiment

The Poisson distribution is used to model a random variable that is the count of successes over an area or a time period.

Examples of Poisson Experiments

Examples:

• heads flipped in an hour.
• number of dents on a car.
• errors on a page.
• number of terrorist attacks in a year.
• bacteria in a swimming pool.
• influenza cases in a week.
• wars in a year.

Poisson Probability Mass Function

Recall that the probability mass function (PMF) provides the probability of each element of the sample space.

For a Poisson random variable, there are an infinite number of possible outcomes:

\[ S = \{ 0, 1, 2, \dots \} \]

Poisson Probability Mass Function

Using some calculus, it can be proven that the probability mass function is:

\[ \mathbb{P}(X = x) = \begin{cases} \dfrac{e^{-\lambda}\, \lambda^x}{x!} & x \in S \\[10pt] 0 & \textrm{otherwise} \end{cases} \]

The parameter \(\lambda\) (“lambda”) represents the average rate (successes per time period, per area, per volume, etc.).

Poisson Probability Mass Function

The graph of the probability mass function of a \(\text{Poisson}(\lambda = 2)\) distribution looks like this:

Expected Value of Poisson Random Variable

The expected value of a \(\text{Poisson}(\lambda)\) random variable is:

\[ \mathbb{E}(X) = \lambda \]

I state this without proof. If you would like the proof, come see me. The proofs only require infinite sums, but I am not sure it is helpful in understanding statistics.

Variance of Poisson Random Variable

The variance of a \(\text{Poisson}(\lambda)\) random variable is:

\[ \mathbb{V}(X) = \lambda \]

I state this without proof. If you would like the proof, come see me. The proofs only require infinite sums, but I am not sure it is helpful in understanding statistics.

Poisson Distribution in R

In R, there are a set of four functions to calculate various quantities from the Poisson distributions.

• dpois(x, lambda) is the pmf, giving \(\mathbb{P}(X = x)\).
• ppois(x, lambda) is the CDF, giving \(\mathbb{P}(X \le x)\).
• qpois(p, lambda) is the quantile function, i.e. the smallest \(x\) with \(\mathbb{P}(X \le x) \ge p\).
• rpois(n, lambda) generates \(n\) random values from a Poisson(lambda) distribution.

Example #4: Fair Coins

Let \(X\) be the number of heads flipped in a minute, given that I average 24 flips per minute. If the coin is fair, then it is clear that \(X \sim \text{Poisson}(\lambda = 12)\).

1. What is the probability of getting no heads in that minute?
2. What is the probability of getting at most 20 heads in that minute?
3. What is the probability of getting at least one head in the first 6 seconds?

Example #5: Crime in Galesburg

We would like to develop a system to determine if the crime rate in Galesburg has significantly increased. To do this, we will compare the crime rate from last year with the number of crimes reported over the past week.

According to the US Department of Justice, the number of violent crimes reported in Galesburg in 2020 was just 96.

For the sake of using this information, let us assume that Galesburg experienced 4 violent crimes last week.

Does this suggest that there is a significant increase in the violent crime rate?

Example #5: Crime in Galesburg

Let’s extend the previous example and illustrate the importance of sample size on our conclusions.

Suppose that there were 16 violent crimes in Galesburg over the past four weeks (so the same rate, but over a longer period of time).

With this new information, is there significant evidence of an uptick in the crime rate?

Note that the previous example had the same rate (4 per week) but with a shorter period of time (1 week). Here, we are measuring over a longer period of time (4 weeks), meaning we have more data now.

Example #5: Crime in Galesburg

Now let’s suppose there were 208 violent crimes in Galesburg this year (so the same rate, but over a year rather than a month).

With this new information, is there significant evidence of an uptick in the crime rate?

Now we have much more data than we did before.