Lecture #7: The Normal Distribution

MATH 321 – Mathematical Statistics I

Torin Quinlivan
tquinlivan@knox.edu

Knox College

October 14, 2025

The Normal Distribution

A random variable \(Y\) is said to have a normal (or Gaussian) probability distribution if and only if, for \(\sigma > 0\) and \(−\infty < \mu < \infty\), the density function of \(Y\) is \[ f_Y(y) = \frac{1}{\sqrt{2 \pi \sigma^2}}\exp\left\{ -\frac{(x - \mu)^2}{2\sigma^2} \right\}, \qquad -\infty < y < \infty. \]

We write \[ Y \sim \text{Normal}(\mu,\, \sigma^2) \] to say that \(Y\) is drawn from a normal distribution with parameters \(\mu\) and \(\sigma^2\).

The Support of a Random Variable

The support of a random variable is the range of values the random variable can take on with non-zero probability.

In the case of the binomial distribution, the support of \(Y\) is \(\{ 0, 1, 2, \dots, n \}\).

In the case of the normal distribution, the support of \(Y\) is \(\mathbb{R}\).

The Graph of the Normal Distrbution

Facts about Definite Integrals

A function \(f\) is even if \(f(x) = f(-x)\) for all \(x \in \mathbb{R}\).

If \(f(x)\) is even, then for any \(a > 0\): \[ \int_{-a}^a f(x) \, dx = 2 \int_0^a f(x) \, dx \]

A function \(f\) is odd if \(-f(x) = f(-x)\) for all \(x \in \mathbb{R}\).

If \(f(x)\) is odd, then for any \(a > 0\): \[ \int_{-a}^a f(x) \, dx = 0 \]

Expected Value and Variance of the Normal Distribution

Theorem: If \(Y \sim \text{Normal}(\mu, \sigma^2)\), then \[\begin{align*} E[Y] &= \mu & &\text{and} & V[Y] = \sigma^2 \end{align*}\]

Proof of Expected Value of the Normal Distribution

\[\begin{aligned} E[Y] &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty y \exp\left\{ -\frac{(y - \mu)^2}{2 \sigma^2} \right\} \, dy \\ &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty (u + \mu) \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty u \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &\qquad\quad + \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \mu \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &= 0 + \mu \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \right) \\ &= \mu. \end{aligned}\]

Proof of Variance of the Normal Distribution

\[\begin{aligned} V[Y] &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty (y - \mu)^2 \exp\left\{ -\frac{(y - \mu)^2}{2 \sigma^2} \right\} \, dy \\ &= \frac{\sigma^2}{\sqrt{2 \pi}} \int_{-\infty}^\infty z^2 e^{ -\frac{z^2}{2} } \, dz \\ &= \ \frac{\sigma^2}{\sqrt{2 \pi}} \left( -z e^{-\frac{z^2}{2}} \bigg\rvert_{-\infty}^\infty \right)+ \frac{\sigma^2}{\sqrt{2 \pi}} \int_{-\infty}^\infty e^{-\frac{z^2}{2}} \, dz \\ &=0 + \sigma^2 \left( \frac{1}{2\pi} \int_{-\infty}^\infty e^{-\frac{z^2}{2}} \, dz \right) \\ &\sigma^2 \end{aligned}\]

The Standard Normal Distribution

The standard Normal (or Gaussian) random variable \(Z\) has a PDF \[ f_Z(z) = \frac{1}{\sqrt{2 \pi}} \exp\left\{ -\frac{z^2}{2} \right\}. \] That is, \(Z \sim \text{Normal}(0,1)\) is a Normal random variable with mean \(\mu = 0\) and \(\sigma^2 = 1\).

The cumulative distribution function (CDF) of the standard Normal is defined as the function \(\Phi(\cdot)\) \[ \Phi(z) \overset{def}{=} F_Z(z) = P(Z \le z) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^z e^{\frac{t^2}{2}} \, dt. \]

CDF of an Arbitrary Normal Distribution

Theorem (CDF of an arbitrary Normal): Let \(X \sim \text{Normal}(\mu, \sigma^2)\). Then \[ F_X(x) = P(X \le x) = \Phi\left( \frac{x - \mu}{\sigma} \right) \]

Examples:

  1. Find \(P(Z \le 2.00)\).
  2. Let \(X \sim \text{Normal}(3, 4)\). Find \(P(X \le 11)\).

Properties of Normal Distributions

Theorem: Let \(X \sim \text{Normal}(\mu, \sigma^2)\). Then the following results hold:

  1. \(\Phi(y) = 1 - \Phi(-y)\).
  2. \(P(X \ge b) = 1 - \Phi\left( \frac{b - \mu}{\sigma} \right)\).
  3. \(P(\lvert X \rvert \ge b) = 1 - \Phi\left( \frac{b - \mu}{\sigma} \right) + \Phi\left( \frac{-b-\mu}{\sigma} \right)\).

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

  1. Find the probability that a randomly chosen person has an IQ less than 92.
  2. Find the probability that a randomly chosen person has an IQ of 120 or greater.
  3. Find the probability that a randomly chosen person has an IQ between 83 and 114.

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

  1. What IQ score is greater than 40% of people?
  2. What IQ score is less that the top 15% of people
  3. What IQ scores are the bounds of the middle 95% of people?

Skewness and Kurtosis

For a random variable \(X\) with PDF \(F_X(x)\), define the following central moments as: \[\begin{gather*} \text{mean:} \quad E[X] \overset{def}{=} \mu, \\[0.3em] \text{variance:} \quad E[(X - \mu)^2] \overset{def}{=} \sigma^2, \\[0.3em] \text{skewness:} \quad E\left[ \left( \frac{X - \mu}{\sigma} \right)^3 \right] \overset{def}{=} \gamma, \\[0.3em] \text{kurtosis:} \quad E\left[ \left( \frac{X - \mu}{\sigma} \right)^4 \right] \overset{def}{=} \kappa. \end{gather*}\]

Skewness and Kurtosis

The kurtosis of a distribution deals with the “peakedness” or “tailedness” of the distribution.

We define excess kurtosis to be \(\kappa - 3\). Excess kurtosis is easier to understand/interpret than “raw” kurtosis.

The normal distribution has skewness \(0\) and excess kurtosis \(0\).

Percentiles of Prebability Distributions

Let \(X\) be a continuous random variable with proability denisty function \(f_X\). The median of \(X\) is a point \(c \in \mathbb{R}\) such that \[ F_X(c) = \int_{-\infty}^c f_X(x) \, dx = \int_c^\infty f_X(x) \, dx. \]

Theorem: The median of a random variable \(X\) is the point \(c\) such that \[ F_X(c) = \frac{1}{2}. \]

Percentiles of Prebability Distributions

What about other percentiles, like the \(35^{th}\) percentile?

The \(k^{th}\) percentile of a random variable \(X\) is the point \(c\) such that \[ F_X(c) = \frac{k}{100}. \]

The quantile function is the inverse of the CDF. These are hard to compute for most distributions, so we tend to use R.

Percentile Example

A train arrives to the station between 0 and 15 minutes late, distributed uniformly:

  1. What is the median arrival time after it was scheduled?
  1. What is the 35th percentile of the amount of time the train will be late?

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

  1. Mensa International famously only allows people who score at the 98th percentile or higher on a standardised, supervised IQ to join. What IQ score must you have to join Mensa?