Statistical Claims and Studies

Evaluate statistical claims and study designs

Statistical Claims and Studies

Types of Studies

1. Observational Study

Definition: Researchers observe and collect data without manipulating variables.

Example: Survey students about study habits and compare to grades.

Limitation: Can show correlation but NOT causation
❌ Cannot prove studying CAUSES better grades (other factors may be involved)

2. Experiment

Definition: Researchers assign treatments to groups and control variables.

Example: Randomly assign students to study methods and measure results.

Advantage: Can establish causation if properly designed
✓ Can prove method A CAUSES better results than method B

Key Study Design Concepts

Randomization

Why it matters: Eliminates bias by ensuring groups are similar

Example: Flip a coin to assign students to study groups (not let them choose)

Control Group

Purpose: Provides baseline for comparison

Example: Group that studies with no intervention vs. group with new method

Placebo Effect

What it is: People improve simply because they think they're receiving treatment

Solution: Use a blind or double-blind design where participants (and sometimes researchers) don't know who gets the real treatment

Sample Selection

Random Sample

✓ Every member has equal chance of selection
✓ Allows generalization to the population

Biased Samples

Common types:

  • Convenience sample: Survey only people nearby (not random)
  • Voluntary response: Only people who choose to respond (biased toward strong opinions)
  • Undercoverage: Some groups aren't included in sampling frame

SAT Question Types

Type 1: Can This Study Show Causation?

Ask yourself:

  1. Was it an experiment or observational study?
  2. Was there random assignment?
  3. Was there a control group?

If all yes → Can show causation
If any no → Can only show correlation

Type 2: Can Results Be Generalized?

Ask yourself:

  1. Was the sample random?
  2. Was it large enough?
  3. Was the population well-defined?

Example:
Study of 1000 randomly selected US adults → Can generalize to US adults
Study of 50 college students at one university → Cannot generalize to all students

Type 3: Identify the Bias

Look for:

  • How was sample selected?
  • Who was excluded?
  • What incentive did people have to respond?

Quick Decision Tree

Question: Does X cause Y?

  • Is it an experiment?
    • YES → Was there random assignment?
      • YES → Can show causation ✓
      • NO → Association only
    • NO (observational) → Association only, NOT causation

Common SAT Mistakes

❌ Saying observational studies prove causation
❌ Generalizing from non-random samples
❌ Ignoring confounding variables
❌ Not recognizing bias in sample selection

Red Flag Words

Causation claims to watch for:

  • "proves that X causes Y" (need experiment)
  • "X is the reason for Y" (need experiment)
  • "shows a relationship" (✓ okay for observational)
  • "associated with" (✓ okay for observational)

📚 Practice Problems

1Problem 1easy

❓ Question:

A researcher wants to determine if a new study method improves test scores for all high school students. Which of the following study designs would best support a generalization to all high school students?

A) Survey 50 students from one honors class B) Randomly select 200 students from various schools, grades, and academic levels C) Ask for volunteers from one school and test those who sign up D) Select the top 100 students from a single school district

💡 Show Solution

To generalize results to ALL high school students, the sample must be representative of the entire population.

A) One honors class: • NOT representative - only high-achieving students • One class - very small, limited diversity • Cannot generalize ✗

B) Randomly select 200 from various schools/grades/levels: • Random selection reduces bias • Diverse sources (various schools, grades, levels) • Large enough sample (200) • Representative of population ✓ • Can generalize ✓

C) Volunteers from one school: • Volunteer bias (self-selection) • Only motivated students sign up • One school - limited diversity • Cannot generalize ✗

D) Top 100 students: • Only high achievers - NOT representative • One district - limited geographic diversity • Cannot generalize ✗

Answer: B) Randomly select 200 students from various schools, grades, and academic levels

Key Principles: • Random selection reduces bias • Sample should represent population diversity • Larger, diverse samples allow better generalization • Avoid convenience sampling and volunteer bias

2Problem 2medium

❓ Question:

A study found that students who eat breakfast score 10% higher on tests than students who skip breakfast. Which conclusion is most appropriate?

A) Eating breakfast causes higher test scores B) There is an association between eating breakfast and test scores, but causation cannot be determined C) Skipping breakfast causes lower test scores D) All students should be required to eat breakfast

💡 Show Solution

This is about correlation vs. causation.

What we know: • Students who eat breakfast score higher (correlation/association) • This is an observational study (not an experiment)

Problems with claiming causation:

  1. Confounding variables: • Students who eat breakfast might have more structured home lives • Might get more sleep • Might have better overall health habits • Family income/resources

  2. Reverse causation possible: • Maybe good students have better habits in general • Correlation doesn't tell us direction

  3. Not a controlled experiment: • No random assignment to breakfast/no breakfast groups • Can't isolate breakfast as the cause

A) Claims causation - NOT supported ✗ B) States association, acknowledges can't determine causation - CORRECT ✓ C) Claims causation - NOT supported ✗ D) Makes policy recommendation beyond data - NOT supported ✗

Answer: B) There is an association between eating breakfast and test scores, but causation cannot be determined

SAT Key Point: Observational studies show correlation/association, but only randomized controlled experiments can demonstrate causation.

3Problem 3hard

❓ Question:

A pharmaceutical company conducts a study where 100 participants are randomly assigned to either receive a new drug or a placebo. Neither the participants nor the researchers know who receives which treatment. After 8 weeks, the drug group shows 20% improvement while the placebo group shows 5% improvement. What features of this study design strengthen the conclusion that the drug causes improvement?

I. Random assignment II. Control group (placebo) III. Double-blind procedure

A) I only B) I and II only C) II and III only D) I, II, and III

💡 Show Solution

Let's analyze each feature:

I. Random assignment: • Participants randomly assigned to drug or placebo • Eliminates selection bias • Creates comparable groups • Confounding variables distributed equally • Allows causal conclusions ✓ • STRENGTHENS causation

II. Control group (placebo): • Comparison group that doesn't receive treatment • Shows what happens without the drug • 20% vs 5% - can see drug effect beyond placebo effect • Essential for determining if drug makes a difference ✓ • STRENGTHENS causation

III. Double-blind procedure: • Neither participants nor researchers know who gets drug • Eliminates expectation bias (placebo effect) • Eliminates researcher bias in evaluation • Ensures objective results ✓ • STRENGTHENS causation

All three features strengthen the conclusion!

Why each matters: • Random assignment → causal inference possible • Control group → provides baseline comparison • Double-blind → eliminates bias

Answer: D) I, II, and III

This is a GOLD STANDARD study design: • Randomized • Controlled • Double-blind • Can make strong causal claims

🎴

Practice with Flashcards

Review key concepts with our flashcard system

📖

Browse All Topics

Explore other calculus topics