Experimental Design
Design experiments using control, randomization, replication, and blocking.
Try the Interactive Version!
Learn step-by-step with practice exercises built right in.
🧪 Experimental Design Principles
Key Components
Random Assignment
- Each subject has equal probability of assignment to each treatment
- Balances known and unknown confounders
- Implementation: random number generator, random digit table
Control Group
- Receives placebo, standard treatment, or no treatment
- Provides baseline for comparison
- Without control, cannot assess treatment effect
Blinding
- Single-blind: subjects don't know treatment assignment
- Double-blind: neither subjects nor researchers know (most rigorous)
- Prevents bias in treatment delivery and assessment
- Impossible for some treatments (surgery, counseling)
Placebo Effect
- Improvement from expectation alone
- Blinding controls for placebo effect
- Ethical requirement: inform subjects placebo possible (in informed consent)
Blocking
- Divide subjects into homogeneous blocks
- Randomly assign treatments within each block
- Reduces within-group variability
- Example: block by gender, then randomly assign to drug vs placebo within each gender
Matched Pairs Design
- Special case of blocking: per block
- Pair subjects by relevant characteristics
- Randomly assign one to each treatment per pair
- More efficient than separate random assignment
- Example: before-after measurements on same subject
Replication
- Repeat experiment multiple times
- Increases sample size, reduces sampling variability
- Two meanings: (1) number of subjects per group, (2) independent repetition of entire experiment
- Large → smaller standard error
Steps for Well-Designed Experiment
- Identify variables: independent (treatment), dependent (outcome), potential confounders
- Select subjects: random sampling from population (if inferring to population)
- Random assignment: to treatment groups
- Control: use control group, blinding, blocking if relevant
- Replicate: sufficient sample size
- Standardize: same procedure, conditions for all subjects
- Measure response: reliable, valid outcome measurement
- Analyze: compare groups, account for variability, assess statistical significance
Common Mistakes
- Confusing random sampling (who gets in study) with random assignment (who gets which treatment)
- Omitting control group
- Failing to double-blind when possible
- Insufficient replication (too few subjects)
- Blocking on variable unrelated to outcome (wastes degrees of freedom)
Decision Rule
Can you randomly assign? → Experiment (can infer causation) Cannot randomly assign? → Observational (report association only)
AP Exam Tip
"Design an experiment" prompts require: statement of hypotheses, identification of variables, treatment groups, control group, blinding method, randomization procedure, outcome measurement, and conclusion.
📚 Practice Problems
1Problem 1easy
❓ Question:
An experiment tests whether adding fertilizer increases tomato plant yield. All 60 plants are kept in the same greenhouse at the same temperature, watered equally, but half receive fertilizer and half don't. Why is this good experimental design?
💡 Show Solution
Why this design is strong:
-
Random Assignment: The 60 plants are randomly divided into two groups (30 fertilized, 30 not), ensuring the groups are comparable before treatment.
-
Control of Variables: All other factors are held constant — same greenhouse, temperature, water, light, soil type. This eliminates confounding variables so differences in yield come from fertilizer, not other causes.
-
Control Group: The unfertilized group serves as a baseline to compare against; without it, we wouldn't know if the treated group's yield is high.
-
Replication: Using 60 plants instead of 1 or 2 gives multiple observations, reducing the effect of individual plant variation and increasing reliability.
Result: If fertilized plants significantly outyield unfertilized ones, we can confidently conclude fertilizer causes higher yield because confounding factors were controlled.
2Problem 2medium
❓ Question:
A researcher wants to compare three new cancer drugs on 150 patients. Suggest a design that uses blocking and explain why blocking improves the experiment.
💡 Show Solution
Design with Blocking:
Block by disease stage (three blocks):
- Block 1: 50 early-stage patients
- Block 2: 50 mid-stage patients
- Block 3: 50 late-stage patients
Within each block, randomly assign 1/3 to Drug A, 1/3 to Drug B, 1/3 to Drug C (so ~17 per group per block).
Response: Measure survival time or remission rate after 12 months.
Why blocking improves the experiment:
Disease stage is a confounding variable — late-stage patients naturally have different outcomes than early-stage regardless of drug. Without blocking, if by chance more late-stage patients got Drug A, its apparent effectiveness would be artificially lowered (confounded with stage).
Blocking isolates the drug effect from stage effect by ensuring each block (stage level) has equal numbers on each drug. We can then:
- Within each block: Compare drugs fairly (stage-specific effects)
- Across blocks: See if effects are consistent regardless of stage
This increases precision and makes differences between drugs clearer.
3Problem 3hard
❓ Question:
Design a double-blind experiment testing whether a memory supplement improves test performance. Explain what 'double-blind' means and why both levels of blinding are essential.
💡 Show Solution
Study Design:
Participants: 100 college students
Groups:
- Treatment: 50 students receive memory supplement pills
- Control: 50 students receive placebo pills (identical in appearance, taste, packaging)
Random Assignment: Students randomly assigned to supplement or placebo
Blinding:
- Single-blind: Students don't know which group they're in (don't know if taking supplement or placebo)
- Double-blind: BOTH students AND researchers don't know who got which pill until after data collection
Implementation: A neutral third party prepares identical-looking pills, labels them only with code numbers. After the study, codes are revealed.
Response: Test performance measured 8 weeks after starting pills
Why double-blinding is essential:
-
Eliminates placebo effect (student level): If students knew they got the real supplement, improved performance might come from psychological expectation, not the pill itself. Blinding ensures any improvement is physiological.
-
Eliminates experimenter bias (researcher level): If researchers knew who had supplements, they might unconsciously encourage supplement students differently, treat them with more enthusiasm, or score their tests more leniently. Blinding prevents this.
Result: Any difference in performance between groups is reliably caused by the supplement, not expectations or bias.
Without double-blinding, results are unreliable — we can't tell if the supplement or expectations caused improvement.
⚠️ Common Mistakes: Experimental Design
Avoid these 3 frequent errors
Practice with Flashcards
Rate this topic's cards with spaced repetition. Cards join your deck when you finish a topic's lesson and take its exit quiz.
Browse All Topics
Explore more AP Statistics topics