Lecture #13: Continuous Distributions and the Normal Distribution

Continuous Random Variables

A continuous random variable is a random variable with a sample space consisting of an interval of values.

Examples:

• Student height
• The age of your car
• Time spent at a stoplight
• Distance a baseball goes

Near-Continuous Random Variables

Frequently, a discrete random variable is “essentially” continuous because the spacing between adjacent values is insignificant when compared to the useful data range.

Examples:

• Grade point average
• Annual salary
• Gross domestic product (GDP)
• GDP per capita
• Crime rate
• The number of ears of corn grown in Illinois
• The number of COVID-19 cases in a week on campus.

Near-Continuous Random Variables

Why does this matter?

• In terms of analysis, there is no difference in how continuous and near-continuous variables are handled. Thus, we will ignore the distinction between near-continuous and continuous.
• Statistical procedures dealing with continuous variables are (usually) identical to those dealing with near-continuous variables.
• As such, the near-continuous are treated as continuous for all intents and purposes.

The Uniform Distribution

The uniform distribution is a continuous distribution that describes random variables whose likelihood of occurring is constant across a specified interval.

The Uniform Distribution

The Uniform distribution is characterized by these features:

• It has two parameters: minimum and maximum values (\(a\) and \(b\)).
• The sample space is \(S = [a, b]\).
• The expected value is \(\mathbb{E}(X) = \frac{a + b}{2}\).
• The variance is \(\mathbb{V}(X) = \frac{(b-a)^2}{12}\).

Probability Density Function of a Uniform R.V.

The probability density function for the Uniform distribution is given by:

\[ f(x; a, b) = \begin{cases} \dfrac{1}{b-a} & x \in [a, b] \\[10pt] 0 & \textrm{otherwise} \end{cases} \]

Probability Density Function of a Uniform R.V.

Notes on the Probability Density Function

Like the probability mass function for a discrete random variable, the probability density function uniquely defines a continuous distribution.

Note that the probability density function does not represent \(\mathbb{P}(X = x)\), unlike the probability mass function of a discrete random variable. This can be hard to understand at first!

Technically, \(\mathbb{P}(X = x) = 0\) for any specific value of \(x\).

Probabilities are defined as areas under the density curve.

Example #1: Main and Academy

There is a particularly pesky stoplight between my home and campus in the morning. It regularly cycles among green (for \(175\) seconds), yellow (\(5\) seconds), and red (\(180\) seconds).

Given that I stop at the light, what is the probability that I wait at most \(60\) seconds?

Example #1: Main and Academy

Because I am equally likely to arrive at any moment during the red light, and because there is a definite lower- and upper-bound to my wait time (\(0\) and \(180\) seconds, respectively), the wait time distribution follows a Uniform distribution.

Example #1: Main and Academy

Here is that depicted graphically. The large rectangle is the pdf of \(\text{Uniform}(0, 180)\), and the dark rectangle is the area of \(\mathbb{P}(T \le 60)\).

Example #1: Main and Academy

Because the pdf of the Uniform distribution is just a rectangle, and because areas in pdfs are probabilities, we just need to calculate the area of the region \(T \le 60\).

Cumulative Distribution Function

The cumulative distribution function (CDF) of a probability distribution is defined as

\[ F(x) = \mathbb{P}(X \le x) \]

Cumulative Distribution Function

The CDF is frequently difficult to calculate. Usually, it requires integral calculus. Specifically:

\[ F(x) = \mathbb{P}(X \le x) = \int_{-\infty}^x f(t) dt \]

However, for the Uniform distribution, we can rely on middle-school geometry.

Cumulative Distribution Function of a Uniform R.V.

The cumulative distribution function (CDF) is the function that describes \(F(x) = \mathbb{P}(X \le x)\).

Cumulative Distribution Function of a Uniform R.V.

Let \(X \sim \text{Uniform}(a, b)\).

The probability \(\mathbb{P}(X \le x)\) is the area of the rectangle with height \(f(x) = \frac{1}{b - a}\) and width \(x - a\).

Quantile Function

The CDF starts with a value of the random variable, \(x\), and calculates a cumulative probability, \(p\), such that \(\mathbb{P}(X \le x) = p\).

Uniform Distribution in R

In R, there are a set of four functions to calculate various quantities from the Uniform distributions:

• dunif(x, min, max) is the pdf, giving \(f(x)\).
• punif(x, min, max) is the CDF, giving \(F(x) = \mathbb{P}(X \le x)\).
• qunif(p, min, max) is the quantile function, i.e. solving \(F(x) = \mathbb{P}(X \le x) = p\) for \(x\).
• runif(n, min, max) generates \(n\) random values from a Uniform(min, max) distribution.

The Exponential Distribution

The Exponential distribution is a continuous distribution that describes waiting times where shorter waits are more likely than longer ones. It is frequently used to model “time until” something happens when there is no upper bound.

The Exponential Distribution

The Exponential distribution is characterized by these features:

• It has one parameter: \(\lambda\), the rate of the event occurring.
• The sample space is \(S = (0, \infty)\).
• The expected value is \(\mathbb{E}(X) = \frac{1}{\lambda}\).
• The variance is \(\mathbb{V}(X) = \frac{1}{\lambda^2}\).

Probability Density Function of an Exponential R.V.

The probability density function for the Exponential distribution is given by:

\[ f(x;\, \lambda) = \begin{cases} \lambda e^{-\lambda x} & x > 0 \\ 0 & \textrm{otherwise} \end{cases} \]

Probability Density Function of an Exponential R.V.

The CDF of the Exponential Distribution

The cumulative distribution function of the Exponential distribution is: \[ \mathbb{P}(X \le x) = \begin{cases} 0 & x < 0 \\ 1 - e^{-\lambda x} & x \ge 0 \end{cases} \]

Exponential Distribution in R

In R, there are a set of four functions to calculate various quantities from the Exponential distributions

• dexp(x, rate) is the pdf, giving \(f(x)\).
• pexp(x, rate) is the CDF, giving \(F(x) = \mathbb{P}(X \le x)\).
• qexp(p, rate) is the quantile function, i.e. solving \(F(x) = \mathbb{P}(X \le x) = p\) for \(x\).
• rexp(n, rate) generates \(n\) random values from an Exponential(rate) distribution.

Characteristics of the Normal Distribution

Here are some important features of the Normal distribution (or Gaussian distribution):

• It is named after Carl Friedrich Gauss (1777–1855) who advocated for its use.
• The Normal distribution arises from modeling observed randomness (errors) in astronomical and geodetic data.
• It arises in variables that have a specific expected value, but demonstrate minor random variation.
• It has two parameters: \(\mu\) (expected value) and \(\sigma\) (standard deviation).
• It is the most important distribution in statistics because of the Central Limit Theorem.

Carl Friedrich Gauss (1777–1855)

Carl Friedrich Gauss is one of the most famous and important mathematicians ever:

Probability Density Function of the Normal Distribution

If a random variable follows a Normal distribution with mean \(\mu\) and standard deviation \(\sigma\), i.e. \(X \sim \text{Normal}(\mu, \sigma)\), then the probability density function (pdf) is:

\[ f(x;\, \mu, \sigma) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left\{ -\frac{(x - \mu)^2}{2 \sigma^2} \right\} \]

Standard Normal Distribution

This function depends on the values of \(\mu\) and \(\sigma\). If we standardize \(X\), meaning take the \(z\)-score for all the values, we get the Standard Normal distribution

\[ Z = \frac{X - \mu}{\sigma} \sim \text{Normal}(0, 1) \]

This has mean 0 and standard deviation 1.

If you’ve ever used a \(Z\)-table, this is the Normal distribution that it describes.

Shape of the Normal Distribution

Shape of the Normal Distribution

Cumulative Distribution Function of the Normal

Recall that the Normal distribution is a continuous distribution, so it’s defined by its cumulative distribution function, and \(\mathbb{P}(X = x) = 0\) for any \(x\).

Normal Distribution in R

In R, there are a set of four functions to calculate various quantities from the Normal distributions

• dnorm(x, mean, sd) is the pdf, giving \(f(x)\).
• pnorm(x, mean, sd) is the CDF, giving \(F(x) = \mathbb{P}(X \le x)\).
• qnorm(p, mean, sd) is the quantile function, i.e. solving \(F(x) = \mathbb{P}(X \le x) = p\) for \(x\).
• rnorm(n, mean, sd) generates \(n\) random values from a Normal(mean, sd) distribution.

Example #2: Intelligence Quotient (IQ)

By design, intelligence quotient (IQ) measures in the United States follow a Normal distribution with \(\mu = 100\) and \(\sigma = 15\). What proportion of Americans have an IQ score less than 90?

Example #2: Intelligence Quotient (IQ)

Solution: We are asked to calculate \(p\) such that

\[ \mathbb{P}(X \le 90) = p \]

given that \(X \sim \text{Normal}(100, 15)\).

Example #3: Intelligence Quotient (IQ)

By design, intelligence quotient (IQ) measures in the United States follow a Normal distribution with \(\mu = 100\) and \(\sigma = 15\). What proportion of Americans have an IQ score above 90?

Example #3: Intelligence Quotient (IQ)

Solution: We are asked to calculate \(p\) such that

\[ \mathbb{P}(X > 90) = p \]

given that \(X \sim \text{Normal}(100, 15)\).

Example #4: Intelligence Quotient (IQ)

By design, intelligence quotient (IQ) measures in the United States follow a Normal distribution with \(\mu = 100\) and \(\sigma = 15\). What proportion of Americans have an IQ score between 87 and 122?

Example #4: Intelligence Quotient (IQ)

Solution: We are asked to calculate \(p\) such that

\[ \mathbb{P}(87 < X < 122) = p \]

given that \(X \sim \text{Normal}(100, 15)\).

Example #5: Intelligence Quotient (IQ)

By design, intelligence quotient (IQ) measures in the United States follow a Normal distribution with \(\mu = 100\) and \(\sigma = 15\). What IQ value separates the lower 30% from the upper 70%?

Example #5: Intelligence Quotient (IQ)

Solution: We are asked to calculate \(x_p\) such that

\[ \mathbb{P}(X < x_p) = 0.30 \]

given that \(X \sim \text{Normal}(100, 15)\).