Statistical Claims and Studies

Types of Studies

1. Observational Study

Definition: Researchers observe and collect data without manipulating variables.

Example: Survey students about study habits and compare to grades.

Limitation: Can show correlation but NOT causation
❌ Cannot prove studying CAUSES better grades (other factors may be involved)

2. Experiment

Definition: Researchers assign treatments to groups and control variables.

Example: Randomly assign students to study methods and measure results.

Advantage: Can establish causation if properly designed
✓ Can prove method A CAUSES better results than method B

Key Study Design Concepts

Randomization

Why it matters: Eliminates bias by ensuring groups are similar

Example: Flip a coin to assign students to study groups (not let them choose)

Control Group

Purpose: Provides baseline for comparison

Example: Group that studies with no intervention vs. group with new method

Placebo Effect

What it is: People improve simply because they think they're receiving treatment

Solution: Use a blind or double-blind design where participants (and sometimes researchers) don't know who gets the real treatment

Sample Selection

Random Sample

✓ Every member has equal chance of selection
✓ Allows generalization to the population

Biased Samples

Common types:

• Convenience sample: Survey only people nearby (not random)
• Voluntary response: Only people who choose to respond (biased toward strong opinions)
• Undercoverage: Some groups aren't included in sampling frame

SAT Question Types

Type 1: Can This Study Show Causation?

Ask yourself:

1. Was it an experiment or observational study?
2. Was there random assignment?
3. Was there a control group?

If all yes → Can show causation
If any no → Can only show correlation

Type 2: Can Results Be Generalized?

Ask yourself:

1. Was the sample random?
2. Was it large enough?
3. Was the population well-defined?

Example:
Study of 1000 randomly selected US adults → Can generalize to US adults
Study of 50 college students at one university → Cannot generalize to all students

Type 3: Identify the Bias

Look for:

• How was sample selected?
• Who was excluded?
• What incentive did people have to respond?

Quick Decision Tree

Question: Does X cause Y?

• Is it an experiment?
  • YES → Was there random assignment?
    • YES → Can show causation ✓
    • NO → Association only
  • NO (observational) → Association only, NOT causation

Common SAT Mistakes

❌ Saying observational studies prove causation
❌ Generalizing from non-random samples
❌ Ignoring confounding variables
❌ Not recognizing bias in sample selection

Red Flag Words

Causation claims to watch for:

• "proves that X causes Y" (need experiment)
• "X is the reason for Y" (need experiment)
• "shows a relationship" (✓ okay for observational)
• "associated with" (✓ okay for observational)