Lecture #2: Sampling Theory

What’s Your Intuition Say?

Suppose there is a genetic disorder that occurs in \(1\) in every \(100000\) births. A team of researchers develop a test to see if you have it.

If you have the disorder, there is a \(99.9\%\) chance that the test will return a positive result.

If you do not have the disorder, there is a \(99.95\%\) chance that the test will return a negative result.

You have the test run on yourself, and you get a positive result. What is the probability that you have the disease?

Today’s Objective

After today’s lecture, you should:

• understand some basic statistical terms
• distinguish between population and sample
• distinguish between the two types of populations
• know the five basic sampling schemes
• clearly distinguish between cluster and stratified sampling
• understand the interplay between bias and variance
• understand how these terms relate to how a sample represents a population

Basic Statistics Terminology

Let’s start with some definitions:

Basic Statistics Terminology

Let’s start with some definitions:

A sampling frame is a physical list of all members of the sampled population.

Imagining Data as a Spreadsheet

Basic Statistics Terminology

A population parameter is a numerical description of a population characteristic.

Sample vs. Population

It is very, very, very important to be mindful of the relationship between your population and your sample. Don’t get them mixed up.

Sample vs. Population

The target population is our population of interest. It is the group we want to learn about.

Sample vs. Population

The Literary Digest vs. George Gallup

In 1936, Democratic President Franklin Delano Roosevelt was running for his second term against the Republican governor of Kansas, Alf Landon. The Literary Digest, one of the most respected and popular publications of the day, decided on an ambitious plan: one of the largest and most expensive polls ever conducted to predict the winner. They surveyed around 2.4 million people, which accounted for over 5% of the total turnout, chosen from subscriber lists, telephone directories and other available lists, and predicted that Landon would win in a landslide, 57% to Roosevelt’s 43%. Meanwhile, George Gallup ran a focused poll of only 50,000 voters.

The Literary Digest vs. George Gallup

Roosevelt went on to win 63% of the vote in what is one of the largest landslide elections in American history, a result George Gallup predicted. Roosevelt won the second highest percentage of votes in a contested election, behind only LBJ in 1968, and the second highest electoral vote total, behind only Reagan in 1984.

The Gallup Organization is now one of the most respected names in polling.

The Literary Digest vs. George Gallup

What issues with The Literary Digest’s poll can you identify?

Example

Determine what is the population of interest, the sampled population, the sample, the population parameter and the sample statistic in the following example.

A candidate for Galesburg Mayor wants to determine what percentage of Knox College students will support them in the upcoming election. They ask 145 students who walked through the SMC Atrium who they are going to vote for in the election. Of those surveyed, 68% said they will support the candidate.

Example

Determine what is the population of interest, the sampled population, the sample, the population parameter and the sample statistic in the following example.

Researchers want to know what the average high school GPA of all Northwestern University students was. They go to the closest dorm, Sargent Hall, and knock on all the doors in the building. Of the 37 students that answered, the average high school GPA was 3.46.

Bias and Variance

Two related, but different, concepts in statistics are bias and variance.

Bias-Variance Tradeoff

In a perfect world, we’d want a statistic with no bias (or unbiased) and no variance.

Typical Sampling Methods

The five main methods of sampling we will cover in this course are:

• Simple Random Sampling (SRS)
• Stratified Sampling
• Cluster Sampling
• Systematic Sampling
• Convenience Sampling

Simple Random Sampling

A sampling method in which each element from the sampled population has an equal chance of being selected.

Advantages

• unbiased (if we perform SRS over and over and over again, the average of the sample statistics should be perfectly representative of the population)

Simple Random Sampling

A sampling method in which each element from the sampled population has an equal chance of being selected.

Disavantages

• high resource cost
• high variability of estimates
• may not be possible to conduct (we might not have a list of all the everyone from the population)

Stratified Sampling

A sampling method in which the population is divided into subgroups (called strata) according to a variable that is related to the variable being estimated. Elements are chosen from each strata.

Advantages

• unbiased (if we know the proper proportions of each strata in the population)

Stratified Sampling

A sampling method in which the population is divided into subgroups (called strata) according to a variable that is related to the variable being estimated. Elements are chosen from each strata.

Disavantages

• moderate resource cost
• moderate variability of estimates
• easy to bias, if we use the wrong proportions

Cluster Sampling

A sampling method in which the sampled population is divided into subgroups (called clusters) according to a variable that is not related to the variable being estimated. Elements are chosen from one (or more) clusters.

Advantages

• unbiased (if the grouping variable is uncorrelated with what is being estimated)

Cluster Sampling

A sampling method in which the sampled population is divided into subgroups (called clusters) according to a variable that is not related to the variable being estimated. Elements are chosen from one (or more) clusters.

Disavantages

• can have high resource cost, depending on the choice of grouping variable
• high variability of estimates

Systematic Sampling

A sampling method in which we utilize an algorithm (such as every \(n^{th}\) member of the population is selected), after a random starting point.

Advantages

• unbiased (as long as the gap (\(n\)) is large enough to avoid overlap between groups or the rule is chosen well)

Systematic Sampling

A sampling method in which we utilize an algorithm (such as every \(n^{th}\) member of the population is selected), after a random starting point.

Disavantages

• moderate resource cost
• low variability of estimates
• may not be possible to perform

Convenience Sampling

Convenience sampling is a method of collecting samples by taking samples that are conveniently located without attempting to ensure that the sample is representative of the population.

Examples

Which of the sampling schemes is described by each of the following research activities? Is the method appropriate in each case?

• A researcher wants to determine the popularity of House of the Dragon among students. To do this, he stands outside the Gizmo between 11am and 3pm on Monday asking every \(10^{th}\) person their opinion on the show.
• A researcher wants to determine if Knox seniors tend to have GPAs greater than \(3.50\). To determine this, the researcher asks the GPA of the first \(150\) Knox seniors who walk through the SMC Atrium, starting at Monday at 3:00pm (Week 3 in the Fall term).

Examples

Which of the sampling schemes is described by each of the following research activities? Is the method appropriate in each case?

• In order to determine the student population’s satisfaction with the MATH classes on offer at Knox, the department asks \(30\) freshmen, \(30\) sophomores, \(30\) juniors and \(30\) seniors their opinions.
• A researcher wants to determine the proportion of students on campus supporting the equality of non-binary and gender non-conforming students. To do this, they randomly select \(500\) students from the campus directory.

Examples

Which of the sampling schemes is described by each of the following research activities? Is the method appropriate in each case?

• A researcher wants to determine which of the five “burger joints” on Henderson (Culver’s, McDonald’s, Steak and Shake, Wendy’s, and Dairy Queen) is most popular. To determine this, she mails a random sample of \(1000\) people in Galesburg using addresses in the white pages. A total of \(14\) people reply.
• Consumer Reports wants to see how much airline passengers pay for their tickets. They decide to go to O’Hare and chose 5 outgoing planes, and ask each passenger as they get on board.

What’s Your Intuition Say?

Suppose there is a genetic disorder that occurs in \(1\) in every \(100000\) births. A team of researchers develop a test to see if you have it.

If you have the disorder, there is a \(99.9\%\) chance that the test will return a positive result.

If you do not have the disorder, there is a \(99.95\%\) chance that the test will return a negative result.

You have the test run on yourself, and you get a positive result. What is the probability that you have the disease?

Solution

This is a problem involving Bayes’ Theorem, which helps us calculate the probability of having a disease given a positive test result. Let:

• Let \(D\) be the event that you have the disorder.
• Let \(D^c\) be the event that you do not have the disorder.
• Let \(T^+\) be the event that the test result is positive.
• Let \(T^-\) be the event that the test result is negative.

We want to compute the probability:

\[ P(D \mid T^+) \]

Solution

What we know:

• Prevalence of disorder: \(P(D) = \frac{1}{100000} = 0.00001\)
• Probability of not having the disorder: \(P(D^c) = 1 - P(D) = 0.99999\)
• Sensitivity (true positive rate): \(P(T^+ \mid D) = 0.999\)
• Specificity (true negative rate): \(P(T^- \mid D^c) = 0.9995\)
• False positive rate: \(P(T^+ \mid D^c) = 1 - 0.9995 = 0.0005\)

Solution

We want to compute the probability:

\[ P(D \mid T^+) \]

We will use Bayes’ Theorem:

\[ P(D \mid T^+) = \frac{P(T^+ \mid D) P(D)}{P(T^+)} \]

Solution

Applying Bayes’ Theorem:

\[ \begin{aligned} P(D \mid T^+) &= \frac{P(T^+ \mid D)P(D)}{P(T^+)} \\ &= \frac{(0.999)(0.00001)}{0.000509985} \\ &\approx \frac{0.00000999}{0.000509985} \\ &\approx 0.0196 \end{aligned} \]

Solution

Applying Bayes’ Theorem:

\[ \begin{aligned} P(D \mid T^+) &= \frac{P(T^+ \mid D)P(D)}{P(T^+)} \\ &= \frac{(0.999)(0.00001)}{0.000509985} \\ &\approx \frac{0.00000999}{0.000509985} \\ &\approx 0.0196 \end{aligned} \]

The probability that you have the disease given a positive test result is approximately \(1.96\%\).