Lecture #7: The Normal Distribution

The Normal Distribution

A random variable \(Y\) is said to have a normal (or Gaussian) probability distribution if and only if, for \(\sigma > 0\) and \(-\infty < \mu < \infty\), the density function of \(Y\) is \[ f_Y(y) = \frac{1}{\sqrt{2 \pi \sigma^2}}\exp\left\{ -\frac{(y - \mu)^2}{2\sigma^2} \right\}, \qquad -\infty < y < \infty. \]

The Support of a Random Variable

The support of a random variable is the set of values where its PMF or PDF is positive: \(\{ y : p_Y(y) > 0 \}\) for a discrete random variable, or \(\{ y : f_Y(y) > 0 \}\) for a continuous random variable.

The Graph of the Normal Distribution

Facts about Definite Integrals

A function \(f\) is even if \(f(x) = f(-x)\) for all \(x \in \mathbb{R}\).

Facts about Definite Integrals

A function \(f\) is odd if \(-f(x) = f(-x)\) for all \(x \in \mathbb{R}\).

Expected Value and Variance of the Normal Distribution

Theorem: If \(Y \sim \text{Normal}(\mu, \sigma^2)\), then \[ E[Y] = \mu \quad \text{and} \quad V[Y] = \sigma^2. \]

Proof of Expected Value of the Normal Distribution

Substitute \(u = y - \mu\), so \(du = dy\): \[ \begin{aligned} E[Y] &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty y \exp\left\{ -\frac{(y - \mu)^2}{2 \sigma^2} \right\} \, dy \\ &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty (u + \mu) \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty u \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &\qquad\quad + \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \mu \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &= 0 + \mu \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \right) \\ &= \mu. \end{aligned} \]

Proof of Expected Value of the Normal Distribution

\[ \begin{aligned} E[Y] &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty u \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &\qquad\quad + \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \mu \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \\ &= 0 + \mu \left( \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty \exp\left\{ -\frac{u^2}{2 \sigma^2} \right\} \, du \right) \\ &= \mu. \end{aligned} \] The first integral is \(0\) because the integrand is odd and \(\displaystyle\int_{-\infty}^\infty \lvert u \rvert e^{-u^2/(2\sigma^2)} \, du < \infty\).

Proof of Variance of the Normal Distribution

Substitute \(z = \dfrac{y - \mu}{\sigma}\), so \(dy = \sigma \, dz\), then integrate by parts: \[ \begin{aligned} V[Y] &= \frac{1}{\sqrt{2 \pi \sigma^2}} \int_{-\infty}^\infty (y - \mu)^2 \exp\left\{ -\frac{(y - \mu)^2}{2 \sigma^2} \right\} \, dy \\ &= \frac{\sigma^2}{\sqrt{2 \pi}} \int_{-\infty}^\infty z^2 e^{ -\frac{z^2}{2} } \, dz \\ &= \frac{\sigma^2}{\sqrt{2 \pi}} \left( -z e^{-\frac{z^2}{2}} \bigg\rvert_{-\infty}^\infty \right) + \frac{\sigma^2}{\sqrt{2 \pi}} \int_{-\infty}^\infty e^{-\frac{z^2}{2}} \, dz \\ &= 0 + \sigma^2 \left( \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty e^{-\frac{z^2}{2}} \, dz \right) \\ &= \sigma^2. \end{aligned} \]

The Standard Normal Distribution

The standard Normal (or Gaussian) random variable \(Z\) has a PDF \[ f_Z(z) = \frac{1}{\sqrt{2 \pi}} \exp\left\{ -\frac{z^2}{2} \right\}. \] That is, \(Z \sim \text{Normal}(0,1)\) is a Normal random variable with mean \(\mu = 0\) and \(\sigma^2 = 1\).

The Standard Normal Distribution

The cumulative distribution function (CDF) of the standard Normal is defined as the function \(\Phi(\cdot)\) \[ \Phi(z) \overset{def}{=} F_Z(z) = P(Z \le z) = \frac{1}{\sqrt{2 \pi}} \int_{-\infty}^z e^{-\frac{t^2}{2}} \, dt. \]

CDF of an Arbitrary Normal Distribution

Theorem (CDF of an arbitrary Normal):

Let \(Y \sim \text{Normal}(\mu, \sigma^2)\). Then \[ F_Y(y) = P(Y \le y) = \Phi\left( \frac{y - \mu}{\sigma} \right) \]

Properties of Normal Distributions

Theorem: Let \(Y \sim \text{Normal}(\mu, \sigma^2)\). Then the following results hold:

1. \(\Phi(z) = 1 - \Phi(-z)\).
2. \(P(Y \ge b) = 1 - \Phi\left( \frac{b - \mu}{\sigma} \right)\).
3. \(P(\lvert Y \rvert \ge b) = 1 - \Phi\left( \frac{b - \mu}{\sigma} \right) + \Phi\left( \frac{-b-\mu}{\sigma} \right)\), for \(b \ge 0\).

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

1. Find the probability that a randomly chosen person has an IQ less than 92.
2. Find the probability that a randomly chosen person has an IQ of 120 or greater.
3. Find the probability that a randomly chosen person has an IQ between 83 and 114.

Skewness and Kurtosis

For a random variable \(X\) with PDF \(f_X(x)\), define the following moments as: \[ \begin{gather*} \text{mean:} \quad E[X] \overset{def}{=} \mu, \\[0.3em] \text{variance:} \quad E[(X - \mu)^2] \overset{def}{=} \sigma^2, \\[0.3em] \text{skewness:} \quad E\left[ \left( \frac{X - \mu}{\sigma} \right)^3 \right] \overset{def}{=} \gamma, \\[0.3em] \text{kurtosis:} \quad E\left[ \left( \frac{X - \mu}{\sigma} \right)^4 \right] \overset{def}{=} \kappa. \end{gather*} \]

Skewness and Kurtosis

The kurtosis of a distribution measures the “tailedness” of the distribution: how much probability lies in its extreme tails.

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

4. What IQ score is greater than 40% of people?
5. What IQ score is less than the top 15% of people?
6. What IQ scores are the bounds of the middle 95% of people?

Percentiles of Probability Distributions

Let \(X\) be a continuous random variable with probability density function \(f_X\). The median of \(X\) is a point \(c \in \mathbb{R}\) such that \[ F_X(c) = \int_{-\infty}^c f_X(x) \, dx = \int_c^\infty f_X(x) \, dx. \]

Percentiles of Probability Distributions

What about other percentiles, like the \(35^{th}\) percentile?

Percentile Example

A train arrives at the station between 0 and 15 minutes late, distributed uniformly:

1. What is the median number of minutes the train is late?

2. What is the 35^th percentile of the amount of time the train will be late?

Normal Distribution Example

Intelligence Quotient (IQ) scores are defined to follow a Normal distribution with a mean of 100 and a standard deviation of 15. Use this information to answer the following questions:

7. Mensa International famously only allows people who score at the 98th percentile or higher on a standardized, supervised IQ test to join. What IQ score must you have to join Mensa?