STAT 200 – Introductory Statistics
February 25, 2026
Understanding the persistence of violent crime in cities can help determine which cities are being successful in reducing it and which cities are not. What is the relationship between the violent crime rate in 1990 and in 2000?
Note that we want to know more than just if there is a significant relationship. We want to:
What is the relationship between these two variables?
To summarize the relationship, we use a “line of best fit”:
\[ \hat{y} = \beta_0 + \beta_1 x \]
However, this raises the following question: What do we mean by the “best” fit?
This is the most important question we can answer. Different answers lead to different fitting methods.
MATH/STAT 225 covers a few of the different methods. (There’s always slots open in the winter term! Dr. Forsberg would love to have you!) It also covers the easiest method in great detail.
In this course, we will only look at the easiest method: ordinary least squares (OLS).
It is based on minimizing the sum of the squared residual values.
Thus, our estimators of the \(y\)-intercept and slope (effect) are:
\[ \begin{aligned} b_0 &= \overline{y} - b_1 \overline{x} \\ b_1 &= \frac{\sum x_i y_i - n \overline{x} \overline{y}}{\sum x_i^2 - n \overline{x}^2} \end{aligned} \]
By the way, the second formula can also be written as:
\[ \begin{aligned} b_1 &= \frac{\sum (x_i - \overline{x}) (y_i - \overline{y})}{\sum (x_i - \overline{x})^2} \\ &= r \left( \frac{s_y}{s_x} \right) \end{aligned} \]
This gives some insight into the slope, the correlation, and the relationship between them.
It can be shown that the slope estimate has the following distribution:
\[ b_1 \sim Normal \left( \beta_1, \frac{\sigma_e^2}{\sum (x_i - \overline{x})^2} \right) \]
With this distribution, we are able to formulate confidence intervals and \(p\)-values for hypotheses about the slope (the effect of \(x\) on \(y\)).
The proof of this is rather elementary and follows from the assumptions we make on the residuals. The three-line proof is provided in STAT 225. Also in that course is the proof of the confidence intervals and the \(p\)-values based on this distribution. (These are also three-line proofs.)
Note: Here, \(\sigma_e^2\) is the population variance of the residuals.
Sometimes, we want to draw conclusions about the model, as a whole.
To do this, we rely on ANOVA (does the model give information about the dependent variable?). As such, we need to calculate the following variations:
| \(SSM = \displaystyle\sum_i (\hat{y}_i - \overline{y})^2\) | Explained by Model |
| \(SSE = \displaystyle\sum_i (y_i - \hat{y}_i)^2\) | Remaining Error |
| \(SST = \displaystyle\sum_i (y_i - \overline{y})^2\) | Total |
The ANOVA table for the linear regression model with 1 independent variable:
| Source | \(\boldsymbol{SS}\) | \(\boldsymbol{df}\) | \(\boldsymbol{MS}\) | \(\boldsymbol{F}\) | \(\boldsymbol{p}\) |
|---|---|---|---|---|---|
| Model | \(\displaystyle\sum_i (\hat{y}_i - \overline{y})^2\) | \(1\) | \(\dfrac{SSM}{1}\) | \(\dfrac{MSM}{MSE}\) | \(\mathbb{P}(F \ge f)\) |
| Error | \(\displaystyle\sum_i (y_i - \hat{y}_i)^2\) | \(n - 2\) | \(\dfrac{SSE}{n-2}\) | ||
| Total | \(\displaystyle\sum_i (y_i - \overline{y})^2\) | \(n - 1\) |
The ANOVA table is useful for drawing conclusions about the model as a whole.
Finally, we have a measure of how well the model fits the data. This is the so-called “\(R\)-squared” value. There are two ways of calculating it:
\[ R^2 = \frac{\textrm{Variation Explained}}{\textrm{Variation Remaining}} = \frac{SSR}{SST} = \frac{SST - SSE}{SST} = 1 - \frac{SSE}{SST}, \]
and:
\[ R^2 = r^2 \]
Here, \(r\) is the correlation we learned about in the past.
RHelp} Of course, for any real data set, it is unreasonable to do calculations by hand. The calculations can be done in R, of course:
This is the code to start to answer the questions raised at the start of this section:
# Load and attach the data
dt = read.csv("crime.csv")
attach(dt)
### Create the model
modOLS = lm(vcrime00 ~ vcrime90)
### Print the results
summary.aov(modOLS) Df Sum Sq Mean Sq F value Pr(>F)
vcrime90 1 2556238 2556238 349.3 <2e-16 ***
Residuals 49 358600 7318
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The Interpretation:
Because the \(p\)-value is less than our usual \(\alpha = 0.05\), we reject the null hypothesis that the model does not offer significant understanding of the relationship between the violent crime rates in 1990 and 2000.
The ANOVA output tells us if the individual independent variables are statistically significant in describing the dependent variable. That is all the ANOVA output tells us.
If we want to determine things like the effect estimates, we use the “linear model” summary: summary.lm(modOLS)
Side Note: If you fit using the lm function, the summary function is equivalent to the summary.lm function. That is, summary(modOLS) gives the same results.
From this output, we at least know the following:
The previous code provides the point estimates for the intercept and slope. However, as we already know, we also need a confidence interval to communicate the precision of our estimates.
Partial Conclusion:
We are 95% confident that the true effect of the 1990 violent crime rate on the 200 violent crime rate is between 0.518 and 0.643.
Since we can estimate a value of the dependent variable, we also need to calculate a confidence interval to indicate the precision of our estimate. Recall that the original problem had us estimate the 2000 violent crime rate for a state with a 1990 violent crime rate of 1500.
fit lwr upr
1 537.0600 510.8882 563.2318
2 301.1418 272.7264 329.5571
3 313.1032 285.3497 340.8566
4 192.6763 156.6817 228.6709
5 360.3100 334.7013 385.9187
6 431.0332 406.9339 455.1326
7 485.5563 461.0229 510.0897
8 458.4399 434.2988 482.5811
9 359.7874 334.1595 385.4154
10 403.4524 379.0335 427.8712
11 384.6973 359.8605 409.5342
12 671.2483 636.7591 705.7374
13 568.4732 540.8010 596.1453
14 207.8313 173.0311 242.6315
15 287.2642 258.0287 316.4998
16 283.5481 254.0839 313.0122
17 524.8663 499.1807 550.5520
18 795.2171 750.2101 840.2241
19 204.0570 168.9626 239.1514
20 471.5626 447.2745 495.8507
21 369.4843 344.1951 394.7735
22 489.9692 465.3399 514.5985
23 643.1448 610.7509 675.5387
24 1536.8818 1416.6710 1657.0925
25 183.3859 146.6425 220.1292
26 263.2253 232.4493 294.0013
27 562.5505 535.1868 589.9142
28 676.5903 641.6875 711.4931
29 548.6730 521.9865 575.3595
30 832.0303 783.6325 880.4282
31 336.2130 309.6088 362.8173
32 498.7951 473.9479 523.6424
33 520.9760 495.4332 546.5187
34 307.1805 279.1048 335.2563
35 418.5493 394.3498 442.7487
36 631.1834 599.6356 662.7312
37 427.4332 403.3126 451.5538
38 551.6343 524.8088 578.4598
39 202.0248 166.7710 237.2785
40 269.6125 239.2596 299.9653
41 284.5352 255.1321 313.9382
42 414.9492 390.7071 439.1913
43 185.8827 149.3417 222.4236
44 488.3434 463.7505 512.9364
45 274.3738 244.3300 304.4175
46 152.4372 113.1193 191.7552
47 400.7814 376.3127 425.2500
48 403.8007 379.3882 428.2133
49 716.4229 678.3042 754.5416
50 414.0783 389.8249 438.3316
51 272.6318 242.4757 302.7880Partial Conclusion:
We are 95% confident that the expected violent crime rate in 2000 for a state with a violent crime rate of 1500 in 1990 is between 918 and 1043, with a point estimate of 981.
Finally, we can predict a value of the dependent variable for the next state with that value of the independent variable.
fit lwr upr
1 537.0600 363.165289 710.9547
2 301.1418 126.895283 475.3882
3 313.1032 138.963404 487.2429
4 192.6763 17.034563 368.3180
5 360.3100 186.499189 534.1209
6 431.0332 257.438367 604.6281
7 485.5563 311.900640 659.2120
8 458.4399 284.839247 632.0406
9 359.7874 185.973771 533.6011
10 403.4524 229.812831 577.0919
11 384.6973 210.998548 558.3961
12 671.2483 495.908875 846.5877
13 568.4732 394.346345 742.6000
14 207.8313 32.430415 383.2321
15 287.2642 112.882114 461.6463
16 283.5481 109.127485 457.9686
17 524.8663 351.044135 698.6885
18 795.2171 617.509399 972.9248
19 204.0570 28.597583 379.5165
20 471.5626 297.941451 645.1838
21 369.4843 195.720249 543.2484
22 489.9692 316.300006 663.6385
23 643.1448 468.205472 818.0841
24 1536.8818 1327.108026 1746.6555
25 183.3859 7.589187 359.1826
26 263.2253 88.578333 437.8723
27 562.5505 388.472467 736.6286
28 676.5903 501.169027 852.0115
29 548.6730 374.700086 722.6459
30 832.0303 653.433711 1010.6270
31 336.2130 162.252732 510.1734
32 498.7951 325.094852 672.4954
33 520.9760 347.174835 694.7771
34 307.1805 132.989105 481.3720
35 418.5493 244.940451 592.1581
36 631.1834 456.398781 805.9680
37 427.4332 253.835382 601.0310
38 551.6343 377.640031 725.6286
39 202.0248 26.533370 377.5161
40 269.6125 95.039554 444.1854
41 284.5352 110.124897 458.9454
42 414.9492 241.334473 588.5640
43 185.8827 10.128176 361.6372
44 488.3434 314.679338 662.0075
45 274.3738 99.854362 448.8932
46 152.4372 -23.915563 328.7900
47 400.7814 227.134826 574.4279
48 403.8007 230.162101 577.4394
49 716.4229 540.333578 892.5122
50 414.0783 240.461930 587.6946
51 272.6318 98.093019 447.1707Partial Conclusion:
We are 95% confident that the actual violent crime rate in 2000 for a state with a violent crime rate of 1500 in 1990 is between 797 and 1164, with a point estimate of 981.
What is the relationship between the violent crime rate in 2000 and the school enrollment in 1990?
The code for this analysis:
Call:
lm(formula = vcrime00 ~ enroll90)
Residuals:
Min 1Q Median 3Q Max
-406.58 -160.05 -55.30 98.19 1005.73
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -211.613 804.596 -0.263 0.794
enroll90 7.095 8.732 0.813 0.420
Residual standard error: 242.3 on 49 degrees of freedom
Multiple R-squared: 0.01329, Adjusted R-squared: -0.006843
F-statistic: 0.6602 on 1 and 49 DF, p-value: 0.4204Brief Conclusion:
Because the \(p\)-value of 0.4204 is greater than our usual \(\alpha = 0.05\), we cannot reject the null hypothesis. We did not detect a statistically significant relationship between the school enrollment in 1990 with the violent crime rate in 2000.
What is the relationship between the violent crime rate in 2000 and average wealth of the state in 1990? Also, what is the predicted 2000 violent crime rate for a state with average weatlh $50,000?
The code for this analysis:
Call:
lm(formula = vcrime00 ~ gspcap90)
Residuals:
Min 1Q Median 3Q Max
-413.6 -123.9 -44.5 126.1 427.8
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 74.32013 88.97655 0.835 0.408
gspcap90 0.01598 0.00366 4.365 6.55e-05 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 207 on 49 degrees of freedom
Multiple R-squared: 0.28, Adjusted R-squared: 0.2653
F-statistic: 19.06 on 1 and 49 DF, p-value: 6.545e-05The code for this analysis:
Brief Conclusion:
Because the \(p\)-value is much less than our usual \(\alpha = 0.05\), we can reject the null hypothesis. We did detect a statistically significant relationship between the state’s average wealth in 1990 with the violent crime rate in 2000.
For every $1,000 increase in average wealth in the state, the 2000 violent crime rate increase by an average of between 8.6 and 23.3.
We predict that a state with a GSP per capita of $50,000 in 1990 will have a violent crime rate in 2000 between 409 and 1338, with a prediction of 873.
What is the relationship between the violent crime rate in 2000 and the unemployment rate in 1990? If there is a relationship, estimate the 2000 violent crime rate given the 1990 unemployment rate is 10%.
The code for this analysis:
Call:
lm(formula = vcrime00 ~ unemp1990)
Residuals:
Min 1Q Median 3Q Max
-346.28 -146.21 -53.13 91.02 1035.79
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 293.51 168.04 1.747 0.087 .
unemp1990 27.06 30.09 0.899 0.373
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 241.9 on 49 degrees of freedom
Multiple R-squared: 0.01624, Adjusted R-squared: -0.003835
F-statistic: 0.809 on 1 and 49 DF, p-value: 0.3728Interpretation:
Because the \(p\)-value of 0.373 is greater than out usual \(\alpha = 0.05\), we cannot reject the null hypothesis. We did not detect a statistically significant relationship between the unemployment rate in 1990 with the violent crime rate in 2000.
The previous example had one numeric dependent variable and one numeric independent variable, hence Simple Linear Regression. However, research usually posits that multiple variables affect the variable of interest (response variable). For example:
Nothing we have discussed so far can handle this many independent variables.
As a part of her research, a former Knox student hypothesized that there was a relationship between the violent crime rate and the percent of the population that is urban. She also hypothesized that this relationship differed across the four census regions.
Her actual research focused on policing policies across the United States and their effects on crime rate. This statistical work was just a small part of her effort. In fact, it was a very small part once she had the data.
From her literature review, she further hypothesized that there was a positive relationship between the violent crime rate and the urban percent, and that the relationship would be strongest in the Southern census region of the United States.
The code for the first part of this analysis is:
Df Sum Sq Mean Sq F value Pr(>F)
urbanp1990 1 178031 178031 5.402 0.0249 *
census4 3 950724 316908 9.615 5.61e-05 ***
urbanp1990:census4 3 368879 122960 3.731 0.0181 *
Residuals 43 1417204 32958
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Why summary.aov? There is a categorical independent variable and we want to determine if the variable and/or its interaction is important.
Note that the interaction between the two variables is significant (\(p= 0.0181 < 0.05 = \alpha\)). As such, both variables are important in the understanding of the violent crime rate in 2000 (in this model).
Furthermore, this result supports her second hypothesis, that the effect of urbanness on the violent crime rate varies from region to region.
Note: Only three of the four regions are listed here. This is because all of the effects (estimates) are measured with respect to this “base” category (Midwest). Thus, the region effect of the South is 281.398 less than for the Midwest. Also, the interaction effect for the South is 7.263 greater than for the Midwest.
Because the table on the previous slide is difficult to understand (for everyone), here is a graphic of the results.
Because it is important to illustrate your results with specific examples, let us estimate the 2000 violent crime rate for a specific southern state with an urban population of 90%:
fit lwr upr
1 922.8865 758.8959 1086.877From these results, a 95% confidence interval for the 2000 violent crime rate for a southern state with a 90% urban population is from 759 to 1087, with a point estimate of 923 violent crimes per 100,000 people.
Again, because it is important to illustrate your results with specific examples, let us predict the 2000 violent crime rate for a southern state with an urban population of 90%:
fit lwr upr
1 922.8865 521.7188 1324.054From these results, a 95% prediction interval for the 2000 violent crime rate for a southern state with a 90% urban population is from 522 to 1324, with a point estimate of 923 violent crimes per 100,000 people.
What is the relationship between the 2000 violent crime rate and the 1990 property crime rate, when controlling for the four census regions?
The code for the first part of this analysis is:
Df Sum Sq Mean Sq F value Pr(>F)
pcrime90 1 1322873 1322873 52.352 5.88e-09 ***
census4 3 423136 141045 5.582 0.00252 **
pcrime90:census4 3 82263 27421 1.085 0.36556
Residuals 43 1086565 25269
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Note that the interaction between the two variables is not statistically significant (\(p= 0.36556 > 0.05 = \alpha\)). As such, we need to modify our model by removing this interaction.
This decision arises from Occam’s Razor: All things being equal, the simpler model is the more useful. This translates as one should generally remove non-significant variables.
This leads to the following model:
Df Sum Sq Mean Sq F value Pr(>F)
pcrime90 1 1322873 1322873 52.063 4.33e-09 ***
census4 3 423136 141045 5.551 0.00245 **
Residuals 46 1168828 25409
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Both variables have a detectable effect. As such, this is our model.
Call:
lm(formula = vcrime00 ~ pcrime90 + census4)
Residuals:
Min 1Q Median 3Q Max
-364.84 -84.29 -15.69 62.73 500.83
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -143.37481 96.65541 -1.483 0.1448
pcrime90 0.12889 0.01999 6.448 6.15e-08 ***
census4Northeast -67.79721 70.31720 -0.964 0.3400
census4South 78.62681 62.83648 1.251 0.2172
census4West -155.24230 65.02444 -2.387 0.0211 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 159.4 on 46 degrees of freedom
Multiple R-squared: 0.599, Adjusted R-squared: 0.5641
F-statistic: 17.18 on 4 and 46 DF, p-value: 1.101e-08Again: Note that only three of the four regions are listed here. This is because all of the effects (estimates) are measured with respect to this “base” category (Midwest). Thus, for instance, the region effect of the South is 78.63 greater than for the Midwest.
The following is a graphic of the relationships between the property and the violent crime rate across the four census regions.
What is the relationship between the 2000 violent crime rate and the 1990 property crime rate, when controlling for the level of education level of the state?
Note that this is a very different analysis than the previous ones. The two independent variables are both numeric. The analysis will be very similar, but the graphic will be quite different.
The ANOVA output for this model is:
Df Sum Sq Mean Sq F value Pr(>F)
pcrime90 1 1322873 1322873 44.273 2.74e-08 ***
waea90 1 181846 181846 6.086 0.0173 *
pcrime90:waea90 1 5777 5777 0.193 0.6622
Residuals 47 1404342 29880
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Note that the interaction between the two variables is not statistically significant (\(p= 0.6622 > 0.05 = \alpha\)). As such, we need to modify our model by removing this interaction.
This leads to the following model:
Df Sum Sq Mean Sq F value Pr(>F)
pcrime90 1 1322873 1322873 45.03 2.04e-08 ***
waea90 1 181846 181846 6.19 0.0164 *
Residuals 48 1410119 29377
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Both variables have a detectable effect. As such, this is our model.
The important parts of the regression table are:
Call:
lm(formula = vcrime00 ~ pcrime90 + waea90)
Residuals:
Min 1Q Median 3Q Max
-305.65 -98.89 -17.21 75.19 575.08
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 555.94745 321.06193 1.732 0.0898 .
pcrime90 0.14060 0.02039 6.897 1.06e-08 ***
waea90 -12.54722 5.04317 -2.488 0.0164 *
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 171.4 on 48 degrees of freedom
Multiple R-squared: 0.5162, Adjusted R-squared: 0.4961
F-statistic: 25.61 on 2 and 48 DF, p-value: 2.7e-08Interpretation: For every increase in the property crime rate of 1 in 100,000, the violent crime rate tends to increase by 0.14 points, all things being equal. Similarly, for ever increas in the weighted average educational achievement of one point, the violent crime rate drops by 12.55 points.
So, how do we illustrate the relationship between/among the two independent numeric variables and the dependent variable? It is quite difficult.
The key is to determine which variable you want to emphasize in your graphic. We shall designate this variable as the “\(x\)-variable”. Your \(x\)-variable will be assigned a range of values. All other variables need to be set to a specific value (or values).
Depending on how complicated your graphic, those specific values may be means, medians, or quartiles. This is where a conversation with others (and experience) is very useful. Remember that education is also a community activity. The best work is done with others.
Also, this is where acknowledging “good” is good enough is helpful. No graphic is perfect. They are (usually) good enough. They always have weaknesses, especially because different people look for different features in a graphic.
The following is a graphic of the relationships between the weighted average educational achievement and the violent crime rate for three exemplar property crime rates.
At this point in the course, we are technically able to model a Normally-distributed response variableand that’s all. In the future, you may wish to model dependent variables that are not Normal, that are:
These are covered in STAT 225. Here are some results you will be able to perform by the end of that course.
The following is a graphic of the relationship between a homeowner’s age and the likelihood that the homeowner has purchased flood insurance.
The following is a graphic of the relationship between a vote and the likelihood it was invalidated in the 2011 presidential election in Uganda.
The following is a graphic estimating the number of citizen’s initiatives in states based on their population and geographic region (for states that allow the initiative).
