Lecture #16: The Theory of Confidence Intervals
STAT 200 – Introductory Statistics
February 7, 2026
Observation Intervals
An observation interval is a set of values that contain the observed vales a given proportion of the time.
An observation interval describes your data.
Prediction Intervals
A prediction interval is a set of values that contain a future observation a given proportion of the time.
A prediction interval predicts a future observation.
Confidence Intervals
A confidence interval is a set of values that theoretically contain the population parameter a given proportion of the time when the experiment is performed many times.
A confidence interval estimates a parameter of the population (usually the mean, the proportion/success probability, or the variance: \(\mu\), \(p\) or \(\sigma^2\)).
Example #1: Enrollment Percentage
One of the variables in the crime data file is the percent of children enrolled in school in 2000 (enroll00). Calculate the three intervals for this variable. Use a \(95\%\) level for each.
Observation Interval:
This tells us that the \(95\%\) observation interval is from \(85.575\) to \(97.700\). In other words, about \(95\%\) of these observations are between \(85.575\) and \(97.700\), inclusive.
Example #1: Enrollment Percentage
One of the variables in the crime data file is the percent of children enrolled in school in 2000 (enroll00). Calculate the three intervals for this variable. Use a \(95\%\) level for each.
Prediction Interval:
This tells us that the \(95\%\) prediction interval is from \(82.559\) to \(100.936\). In other words, about \(95\%\) of the time the next state (future observation) will have an enrollment between \(82.559\) and \(100.936\), inclusive.
Example #1: Enrollment Percentage
One of the variables in the crime data file is the percent of children enrolled in school in 2000 (enroll00). Calculate the three intervals for this variable. Use a \(95\%\) level for each.
Confidence Interval:
This tells us that the 95% confidence interval is from 90.473 to 93.021. In other words, we are 95% confident that the population mean is between 90.473 and 93.021, inclusive.
Definitions
A confidence interval is a set of values that theoretically contain the population parameter a given proportion of the time when the experiment is performed many times.
If \((L, U)\) is a \((1 - \alpha) \times 100\%\) confidence interval for the parameter, then \((1 - \alpha) \times 100\%\) of the intervals – under repeated experiments – contain the parameter.
Definitions
Note that the confidence interval:
- gives some information about the population parameter.
- is a set of reasonable values for that parameter.
- is a function of the data (and thus a random variable).
- is a result of a probability distribution calculation.
Note that the confidence interval does not provide a probability statement on the parameter.
That is, one cannot state something like “There is a 95% chance that the parameter is between 4 and 9.”
Definitions
The confidence level is the theoretical frequency (or proportion) of possible confidence intervals that contain the true value of their corresponding parameter.
- The value of the confidence level is \(c = 1 - \alpha\).
- By default, we will use \(\alpha = 0.05\) in this course.
- The value \(\alpha\) will be re-introduced when we cover hypothesis testing. It will represent the theoretical Type I error rate. It is selected by the research to reflect tradition in the discipline and the peculiarities of the current research.
Bootstrapping
Previously, we introduced bootstrapping as a way of estimating – and obtaining – a confidence interval for a population parameter. Let’s look at the code:
dt = read.csv("http://rfs.kvasaheim.com/data/geography.csv")
attach(dt)
st = numeric()
for (i in 1:1e4) {
x = sample(Score, replace = TRUE)
st[i] = mean(x)
}
quantile(st, c(0.025, 0.975)) 2.5% 97.5%
1.166667 2.722222 Thus, we are 95% confident that the population mean, \(\mu\), of the average understanding of geography, is between 1.167 and 2.722 (out of 6).
Bootstrapping
One thing I want to point out here is the difference between the distribution of the sample values and the distribution of the sample means:
Bootstrapping
One thing I want to point out here is the difference between the distribution of the sample values and the distribution of the sample means:
The \(Z\) Theory
This is fine if all we have is the data. However, we know that sample means tend toward Normality as the sample size increases (CLT). Thus, we may be able to create confidence intervals without bootstrapping and with using additional information.
Recall that for large \(n\),
\[ \overline{X} \sim Normal \left(\mu, \dfrac{\sigma^2}{n} \right) \]
The \(Z\) Theory
Applying the \(z\)-transform (i.e., standardizing formula, \(z\)-score formula) to \(\overline{X}\) gives
\[ \frac{\overline{X} - \mu}{\sqrt{\frac{\sigma^2}{n}}} \sim Normal(0, 1) \]
This means the expression on the left is between \(-1.96\) and \(1.96\) approximately 95% of the time.
The \(Z\) Theory
This is because the \(2.5^{th}\) percentile of the standard Normal distribution is \(-1.96\) and the \(97.5^{th}\) percentile of the standard Normal distribution is \(1.96\).
Thus, by definition, a theoretical 95% interval for \(\dfrac{\overline{X} - \mu}{\sqrt{\frac{\sigma^2}{n}}}\) is between \(-1.96\) and \(1.96\).
The \(Z\) Theory
Since a theoretical 95% interval for \(\dfrac{\overline{X} - \mu}{\sqrt{\frac{\sigma^2}{n}}}\) is between \(-1.96\) and \(1.96\), we can solve for \(\mu\) to get a 95% confidence interval for \(\mu\), the population parameter.
The \(Z\) Theory
One bound of a 95% confidence interval for \(\mu\) is:
\[ \begin{aligned} -1.96 = \dfrac{\overline{X} - \mu}{\sqrt{\frac{\sigma^2}{n}}} \\ -1.96 \sqrt{\frac{\sigma^2}{n}} = \overline{X} - \mu \\ -\overline{X} - 1.96 \sqrt{\frac{\sigma^2}{n}} = -\mu \\ \overline{X} + 1.96 \sqrt{\frac{\sigma^2}{n}} = \mu \end{aligned} \]
The \(Z\) Theory
Similarly, the other bound of a 95% confidence interval for \(\mu\) is:
\[ \begin{aligned} 1.96 = \dfrac{\overline{X} - \mu}{\sqrt{\frac{\sigma^2}{n}}} \\ 1.96 \sqrt{\frac{\sigma^2}{n}} = \overline{X} - \mu \\ -\overline{X} + 1.96 \sqrt{\frac{\sigma^2}{n}} = -\mu \\ \overline{X} - 1.96 \sqrt{\frac{\sigma^2}{n}} = \mu \end{aligned} \]
The \(Z\) Theory
Thus, the endpoints of a confidence interval for \(\mu\) are
\[ \overline{X} \pm z_{\frac{\alpha}{2}} \sqrt{\frac{\sigma^2}{n}}. \]
Here, \(z_\frac{\alpha}{2}\) is the absolute value of the \(\frac{\alpha}{2}\) quantile for the \(Z \sim Normal(0, 1)\) distribution. If \(\alpha = 0.05\) as we usually use, then \(z_{\frac{\alpha}{2}} = z_{0.025} = 1.96\).
The \(Z\) Theory
This confidence interval for \(\mu\) works if:
- You are estimating \(\mu\).
- You are able to calculate \(\overline{x}\) and \(n\).
- You know \(\sigma^2\).
- The data are generated from a Normal process (or \(n\) is sufficiently large).
The \(Z\) Theory
Recall the definition of the population variance. To calculate it, you need to know \(\mu\). However, since we are estimating \(\mu\) in this procedure, we rarely know \(\sigma^2\).
Since we cannot use the \(z\)-procedure, we need to use a different procedure.
Going forward, all our procedures will have requirements to consider. Before you use any procedures, you need to consider if the requirements are reasonable to apply to your problem.
The \(t\) Theory
We rarely know the population variance, \(\sigma^2\), but we can always calculate the sample variance, \(s^2\). Can we use \(s^2\) in place of \(\sigma^2\)?
Thanks to William Sealy Gossett and Karl Pearson in 1908, it can be show that the following is true:
\[ \frac{\overline{X} - \mu}{\sqrt{\frac{s^2}{n}}} \sim t_\nu, \]
where \(\nu = n - 1\) (in this one-sample case).
The \(t\) Theory
From this, we can derive the following confidence interval:
\[ \overline{X} \pm T_{\frac{\alpha}{2}, \nu}\, \sqrt{\frac{s^2}{n}} \]
Here, \(T_{\frac{\alpha}{2}, \nu}\) is the \(\frac{\alpha}{2}\) quantile for the \(t\) distribution with \(\nu = n - 1\) degrees of freedom.
For instance, if \(\alpha = 0.05\) and \(n = 4\), then \(\nu = n - 1 = 3\) and \(T_{\frac{\alpha}{2}, \nu} = t_{0.025, 3} = 3.182\).
William Sealy Gossett
Student’s \(t\) Distribution
The \(t\) Theory
This confidence interval for \(\mu\) works if:
- You are estimating \(\mu\).
- You are able to calculate \(\overline{x}\), \(n\) and \(s\).
- The data are generated from a Normal process (or \(n\) is “large enough”).
Other Procedures
Note the process we followed to obtain the endpoints of the confidence intervals:
- Create a test statistic.
- Determine the distribution of that statistic.
- Determine the quantiles of that distribution.
- Solve the test statistic for the parameter of interest.
This process can/will be repeated for other parameters to create appropriate analysis procedures. You will use it many times in MATH 322: Mathematical Statistics II.
