Lecture #15: The Central Limit Theorem
STAT 200 – Introductory Statistics
February 7, 2026
The Central Limit Theorem
Let \(X\) be a random variable with mean \(\mu\) and finite variance \(\sigma^2\). Let us draw a random sample of size \(n\) from this distribution.
Then, the distribution of the sample sums converges to a Normal distribution as \(n\) gets larger. Specifically,
\[ \sum_{i=1}^n X_i \overset{d}{\longrightarrow} \text{Normal}(n\mu, n\sigma^2). \]
The proof of this theorem is beyond the scope of this course. It is first proven in MATH 321: Mathematical Statistics I.
Consequences of the Central Limit Theorem
The Central Limit Theorem (CLT) tells us the following:
- The sum of independent random variables is more Normal than the distribution of the variable itself, unless the variable is Normal distributed or if it has an infinite variance.
- The Binomial is a sum of independent Bernoulli random variables.
- The Poisson is a sum of independent Poisson random variables.
- Because the sample mean is just the sample sum, divided by a constant (\(n\)), the Central Limit Theorem tells us that the distribution of sample means will tend towards the Normal distribution.
- The speed of convergence depends on how closely the data distribution is to a Normal distribution. The closer, the faster.
Example #1: The Uniform Distribution
Let \(X \sim \text{Uniform}(a, b)\). Use the Central Limit Theorem to estimate the distribution of the sum of a sample of size \(n\).
By the CLT,
\[ T \sim \text{Normal}(n\mu, n\sigma^2). \]
From our knowledge of the Uniform distribution, this means
\[ T \sim \text{Normal} \left( n\left(\frac{a + b}{2}\right), n\left(\frac{(b - a)^2}{12} \right) \right) \]
Example #2: The Exponential Distribution
Let \(X \sim \text{Exponential}(\lambda)\). Use the Central Limit Theorem to estimate the distribution of the sum of a sample of size \(n\).
By the CLT,
\[ T \sim \text{Normal}(n\mu, n\sigma^2). \]
From our knowledge of the Exponential distribution, this means
\[ T \sim \text{Normal} \left( n\left(\frac{1}{\lambda}\right), n\left(\frac{1}{\lambda^2} \right) \right) \]
Example #3: The Binomial Distribution
Let \(X \sim \text{Binomial}(n, p)\). Use the Central Limit Theorem to approximate the distribution of \(X\).
Note that the Binomial is just the sum of \(n\) independent Bernoulli distributions. That is, if \(Y_i \sim \text{Bernoulli}(p)\), then \(Y_i \sim \text{Binomial}(1, p)\) and,
\[ X = \sum_{i = 1}^n Y_i \sim \text{Binomial}(n, p). \]
Thus, by the Central Limit Theorem, we know
\[ X \sim \text{Normal}(np, np(1-p)). \]
The Distribution of the Sample Mean
Let \(X\) be a random variable with mean \(\mu\) and finite variance \(\sigma^2\). Let us draw a random sample of size \(n\) from this distribution.
Then, the distribution of the sample mean is
\[ \overline{X}_n \overset{d}{\longrightarrow} \text{Normal} \left( \mu, \frac{1}{n} \sigma^2 \right) \]
Proof of the Distribution of the Sample Mean
Proof: From the Central Limit Theorem, we know
\[ \sum_{i=1}^n X_i \sim \text{Normal}(n\mu, n\sigma^2). \]
Thus,
\[ \overline{X}_n = \frac{1}{n} \sum_{i=1}^n X_i \sim \text{Normal} \left(\mu, \frac{1}{n} \sigma^2 \right) \]
Proof of the Distribution of the Sample Mean
Proof of Expectation:
\[ \begin{aligned} \mathbb{E}(\overline{X}_n) &= \mathbb{E}\left(\dfrac{1}{n} T\right) \\ &= \dfrac{1}{n} \mathbb{E}(T) \\ &= \dfrac{1}{n} \left( n\mu \right) \\ &= \mu \end{aligned} \]
Proof of the Distribution of the Sample Mean
Proof of Variance:
\[ \begin{aligned} \mathbb{V}(\overline{X}_n) &= \mathbb{V}\left(\dfrac{1}{n} T\right) \\ &= \dfrac{1}{n^2} \mathbb{V}(T) \\ &= \dfrac{1}{n^2} \left( n\sigma^2 \right) \\ &= \dfrac{1}{n} \sigma^2 \end{aligned} \]
NOTE: Sample Means vs. Sample Values
Before we move onto examples, I want to make an important distinction.
The Central Limit Theorem describes the behavior of the sample mean, not the sample values!
The sample values will stay in whatever distribution the population takes. The sample means will approach Normality.
It is a subtle difference, but a very important one.
Example #1: A First Look
I draw a sample of size \(n = 14\) from a population with mean \(\mathbb{E}(X) = 126\) and variance \(\mathbb{V}(X) = 42\). What is the approximate distribution of the sample means?
Example #2: Heights
I have been told that the average adult height for males in the United States has mean \(\mu = 69\) inches and standard deviation \(\sigma = 3\) inches. What is the probability of having the mean of a sample of size \(2\) being less than \(65\) inches?
Example #3: More Heights
I have been told that the average adult height for males in the United States has mean \(\mu = 69\) inches and standard deviation \(\sigma = 3\) inches. What is the probability of having the mean of a sample of size \(10\) being less than \(65\) inches?
Example #4: Crime
The 2000 violent crime rate for the \(50\) states (and the District of Columbia) are given in the data file crime. What is a \(95\%\) central confidence interval for the mean violent crime rate?
Example #4: Crime (Bootstrapping)
We could also estimate the confidence interval from the data. This process is called “bootstrapping”, and here is the code:
This gives a \(95\%\) confidence interval of 380 to 510.
Question: Why is there a difference between the two confidence intervals?
The Distribution of the Sample Proportion
Let \(X \sim \text{Binomial}(n, p)\) be a random sample of size \(n\) from Bernoulli random variables.
Then, the distribution of the sample proportion is
\[ P = \dfrac{1}{n} X \overset{d}{\longrightarrow} \text{Normal} \left( p, \dfrac{p(1-p)}{n} \right) \]
Proof of the Distribution of the Sample Proportion
Proof: From the Central Limit Theorem, we know
\[ X \sim \text{Normal}(np, np(1-p)). \]
Thus,
\[ \begin{gathered} \mathbb{E}(P) = \mathbb{E}\left(\dfrac{X}{n}\right) = \dfrac{\mathbb{E}(X)}{n} = \dfrac{np}{n} = p, \\ \mathbb{V}(P) = \mathbb{V}\left(\dfrac{X}{n}\right) = \dfrac{\mathbb{V}(X)}{n^2} = \dfrac{np(1-p)}{n^2} = \dfrac{p(1-p)}{n}. \end{gathered} \]
Proof of the Distribution of the Sample Proportion
So we have: \[ P \sim \text{Normal} \left( p, \frac{p(1-p)}{n} \right). \]
Example #1: Poverty
According to the US Census, \(18\%\) of Americans are below the poverty line. If I randomly sample \(n = 10\) people from the United States, what is the probability that more than \(20\%\) of them are below the poverty line?
Example #2: More Poverty
According to the US Census, \(18\%\) of Americans are below the poverty line. If I randomly sample \(n = 100\) people from the United States, what is the probability that more than \(20\%\) of them are below the poverty line? ## Example #3: Much More Poverty
According to the US Census, \(18\%\) of Americans are below the poverty line. If I randomly sample \(n = 1000\) people from the United States, what is the probability that more than \(20\%\) of them are below the poverty line?
Example #4: Even More Poverty
According to the US Census, \(18\%\) of Americans are below the poverty line. If I randomly sample \(n = 10000\) people from the United States, what is the probability that more than \(20\%\) of them are below the poverty line?
