Lecture #13: The Bivariate and Multivariate Normal Distributions

MATH 321 – Mathematical Statistics I

Torin Quinlivan

Knox College

November 10, 2025

Normal Distribution

We’ve now dealt with the single variable Normal distribution a lot.

\[\begin{gather*} X \sim \text{Normal}(\mu, \sigma^2) \\ f_X(x) = \frac{1}{\sqrt{2 \pi \sigma^2}} \exp\left\{ -\frac{(x - \mu)^2}{2\sigma^2} \right\} \end{gather*}\]

How might we generalize this to several variables?

The Bivariate Normal Distribution

We could use independence.

Suppose that: \[ \begin{aligned} X &\sim \text{Normal}(\mu_1, \sigma_1^2) \\ Y &\sim \text{Normal}(\mu_2, \sigma_2^2) \\ \end{aligned} \]

If \(X\) and \(Y\) are independent, then: \[ f_{XY}(x, y) = \frac{1}{2 \pi \sigma_1 \sigma_2} \exp\left\{ -\frac{1}{2} \left[ \left( \frac{x - \mu_1}{\sigma_1} \right)^2 + \left( \frac{x - \mu_2}{\sigma_2} \right)^2 \right] \right\} \]

The Bivariate Normal Distribution

But what if \(X\) and \(Y\) are not independent?

The Bivariate Normal Distribution

But what if \(X\) and \(Y\) are not independent?

The Bivariate Normal Distribution

Two random variables \(X\) and \(Y\) are said to be bivariate normal, or jointly normal, if \(aX + bY\) has a Normal distribution for all \(a, b \in \mathbb{R}\).

The Standard Bivariate Normal Distribution

Two random variables \(X\) and \(Y\) are said to have the standard bivariate normal distribution with correlation coefficient \(\rho\) if their joint PDF is given by \[ f_{XY}(x, y) = \frac{1}{2\pi \sqrt{1 - \rho^2}} \exp\left\{ -\frac{1}{2(1 - \rho^2)} \left( x^2 - 2\rho x y + y^2 \right) \right\}, \] where \(\rho \in (-1, 1)\). If \(\rho = 0\), then we just say \(X\) and \(Y\) have the standard bivariate normal distribution.

The Standard Bivariate Normal Distribution

Theorem: Let \(X\) and \(Y\) be two bivariate normal random variables. Then there exist independent standard normal random variables \(Z_1\) and \(Z_2\) such that \[ \begin{aligned} X &= \sigma_X Z_1 + \mu_X \\ Y &= \sigma_Y \Big( \rho Z_1 + \sqrt{1 - \rho^2} Z_2 \Big) + \mu_Y. \end{aligned} \]

The Standard Bivariate Normal Distribution

Two random variables \(X\) and \(Y\) are said to have the bivariate normal distribution with parameters \(\mu_X\), \(\sigma_X^2\), \(\mu_Y\), \(\sigma)Y^2\) and \(\rho\) if their joint PDF is given by \[ \begin{aligned} f_{XY}(x, y) &= \frac{1}{2\pi \sigma_X \sigma_Y \sqrt{1 - \rho^2}} \\ &\qquad \times \exp\Bigg\{ -\frac{1}{2} \Bigg[ \left( \frac{x - \mu_X}{\sigma_X} \right)^2 + \left( \frac{y - \mu_Y}{\sigma_Y} \right)^2 \\ &\qquad \qquad \qquad - 2\rho \left( \frac{(x - \mu_X)(y - \mu_Y)}{\sigma_X \sigma_Y} \right) \Bigg] \Bigg\} \end{aligned} \] where \(\mu_X, \mu_Y \in \mathbb{R}\), \(\sigma_X, \sigma_Y > 0\) and \(\rho \in (-1, 1)\) are all constants.

Conditional Expectation and Variance

Theorem: Suppose \(X\) and \(Y\) are jointly normal random variables with parameters \(\mu_X\), \(\sigma_X^2\), \(\mu_Y\), \(\sigma_Y^2\) and \(\rho\). Then, given \(X = x\), \(Y\) is normally distributed with \[\begin{gather*} E[Y \mid X = x] = \mu_Y + \rho \sigma_Y \left( \frac{x - \mu_X}{\sigma_X} \right). \\ V[Y \mid X = x] = (1 - \rho^2) \sigma_Y^2. \end{gather*}\]

Theorem: If \(X\) and \(Y\) are bivariate normal and uncorrelated, then they are independent.

The Multivariate Normal Distribution

But we can take this further!

The random variables \(X_1, X_2, \dots, X_n\) have a multivariate normal distribution if their joint PDF is given by: \[ p(x; \mu, \Sigma) = \frac{1}{(2 \pi)^{\frac{n}{2}} \lvert \Sigma \rvert^{\frac{1}{2}}} \exp\left\{ -\frac{1}{2} (\tilde{x} - \tilde{\mu})^T \Sigma^{-1} (\tilde{x} - \tilde{mu}) \right\}, \] where \(\tilde{\mu} = [\mu_1, \mu_2, \dots, \mu_n]^T\) is a vector of the means, and \(\Sigma\) is the covariance matrix.

The Multivariate Normal Distribution

The marginals of a joint normal distribution is normal.

The conditional distributions of a joint normal distribution is normal.

You can think of linear regression as a joint normal distribution from the independent errors.