Observational Studies vs Experiments
Distinguish between observational studies and experiments, and understand causation vs. association.
Try the Interactive Version!
Learn step-by-step with practice exercises built right in.
🔬 Observational Studies vs Experiments
Key Distinction
Observational Study
- Researchers observe/record without intervention
- Subjects in treatment groups already exist or self-select
- Can only establish association
- Examples: survey, cohort study, case-control study
Experiment
- Researchers randomly assign subjects to treatment groups
- Researcher controls treatment (independent variable)
- Can establish causation
- Examples: randomized controlled trial, laboratory experiment
Why Causation Requires Random Assignment
Confounding Variables (Lurking Variables)
- Variable that affects both treatment and outcome
- Creates spurious association
- Example: Ice cream sales → drowning deaths (both ↑ in summer)
Without random assignment:
- Treatment groups may differ in pre-existing ways
- Cannot isolate effect of treatment from confounders
- Example: patients choosing statins may already have healthier behaviors
With random assignment:
- Confounders randomly distributed across groups (on average)
- Expected value of confounding effect = 0
- Treatment difference attributable to treatment, not confounders
Study Design Types
Cohort Study (Observational)
- Follow subjects forward in time
- Identify baseline exposure status
- Track outcomes over time
- Can calculate risk/relative risk
- Long, expensive; expensive
- Can study multiple outcomes
Case-Control Study (Observational)
- Identify subjects with and without outcome
- Look back at past exposure
- Calculate odds ratio
- Efficient for rare outcomes
- Prone to recall bias
- Example: interviews of heart attack patients vs controls about smoking
Randomized Controlled Trial (RCT) (Experiment)
- Randomly assign to treatment or control
- Gold standard for causation
- Expensive, time-consuming
- Ethical concerns if withholding known benefit
- Example: vaccine trials with placebo control
Common Mistakes
- Claiming causation from observational data
- Ignoring confounders in observational analysis
- Confusing cohort (forward-looking) with case-control (backward-looking)
Decision Rule
Need causal inference? → Must use random assignment Only have observational data? → Report association, NOT causation; identify potential confounders
AP Exam Tip
Always distinguish between "associated with" (observational) and "causes" (experimental). If a study lacks random assignment, you cannot conclude causation no matter how strong the association.
📚 Practice Problems
1Problem 1easy
❓ Question:
A researcher records the average daily coffee consumption and sleep hours for 100 people. She finds that people who drink more coffee sleep fewer hours. Can she conclude that coffee causes reduced sleep? Explain.
💡 Show Solution
No, she cannot conclude causation. This is an observational study — data is collected without imposing a treatment. The researcher simply observes existing behaviors.
Confounding variables may explain the relationship:
- People with demanding jobs drink more coffee AND naturally sleep less (the job, not coffee, causes reduced sleep)
- Stress increases coffee consumption and reduces sleep
- Night-shift workers naturally drink more coffee and sleep differently
The correlation observed doesn't prove causation. To conclude that coffee causes sleep reduction, you'd need a controlled experiment where people are randomly assigned to drink coffee or not, with other factors held equal.
2Problem 2medium
❓ Question:
In a pharmaceutical trial, some patients receive a new drug while others receive a placebo, randomly assigned. Both groups are followed for 6 months. Is this observational or experimental? Identify the treatment, response variable, and control group.
💡 Show Solution
Type: This is an EXPERIMENT (also called a Randomized Controlled Trial or RCT).
Why? The researcher actively assigns a treatment (new drug vs. placebo) through random assignment, rather than just observing.
Components:
Treatment (Explanatory Variable): Type of pill — either the new drug or placebo
Response Variable: Outcome measured — likely symptom improvement, side effects, or recovery time over 6 months
Experimental Group: Patients receiving the new drug
Control Group: Patients receiving the placebo
Random Assignment: Crucial for validity — ensures the treatment groups are comparable. Without it, we can't isolate the drug's effect from other differences.
Key advantage: Random assignment minimizes confounding variables, making causation conclusions possible.
3Problem 3hard
❓ Question:
Design either an observational study or experiment to investigate whether a new study app improves student exam scores. Clearly state which you'd choose and justify your choice.
💡 Show Solution
Best choice: EXPERIMENT (Randomized Controlled Trial)
Justification:
- The question asks about causation (does the app improve scores — cause-and-effect)
- Observational studies can't eliminate confounding (students who choose to use the app might be more motivated, study harder for other reasons, etc.)
- An experiment with random assignment isolates the app's true effect
Study Design:
Participants: 200 high school students across multiple schools
Random Assignment: Randomly divide into:
- Treatment group (100 students): Use the new study app for 8 weeks
- Control group (100 students): Use no app (business as usual)
Blinding (optional but ideal): If possible, control group could use a placebo app (same interface, no learning features) to match the experience
Response Variable: Exam scores on a standardized exam administered at week 9 (after 8 weeks of treatment)
Analysis: Compare average exam scores between groups; if treatment mean >> control mean, the app shows causal benefit
Why not observational? Students self-selecting to use the app differ systematically from non-users in motivation, study habits, etc., making it impossible to isolate the app's effect.
⚠️ Common Mistakes: Observational Studies vs Experiments
Avoid these 3 frequent errors
Practice with Flashcards
Rate this topic's cards with spaced repetition. Cards join your deck when you finish a topic's lesson and take its exit quiz.
Browse All Topics
Explore more AP Statistics topics