Experimental Design

Principles of Good Experimental Design

1. Control

• Use a control group that receives no treatment (or a placebo)
• Keep all other variables constant (controlled variables)

2. Randomization

• Randomly assign subjects to treatment groups
• This balances out confounding variables (both known and unknown)

3. Replication

• Use enough subjects to reduce the effect of chance variation
• More subjects = more precise results

4. (Optional) Blinding

• Single-blind: Subjects don't know which treatment they receive
• Double-blind: Neither subjects nor evaluators know
• Prevents placebo effect and experimenter bias

Vocabulary of Experiments

| Term | Definition | |------|-----------| | Experimental units | The individuals being studied | | Subjects | Experimental units that are people | | Factor | An explanatory variable that is manipulated | | Level | A specific value of a factor | | Treatment | A specific combination of factor levels | | Response variable | What is measured as the outcome |

Completely Randomized Design (CRD)

All experimental units are randomly assigned to treatments with no grouping.

Randomized Block Design

1. Group subjects into blocks of similar individuals
2. Randomly assign treatments within each block
3. This reduces variability due to the blocking variable

Example: Block by gender, then randomly assign treatments within each gender group.

Note: Blocking is like stratifying in sampling — but blocks are for experiments, strata are for surveys.

Matched Pairs Design

A special case of blocking where:

• Each "block" has only 2 units (matched on key characteristics)
• One gets Treatment A, the other gets Treatment B

OR:

• Each subject serves as their own control (before/after)

Inference and Scope

| Design | Can conclude causation? | Can generalize? | |--------|------------------------|-----------------| | Experiment + Random assignment | ✅ Yes | Only if random selection was used | | Random selection + Observational | ❌ No | ✅ Yes, to population | | Neither | ❌ No | ❌ No |

AP Tip: A well-designed experiment has random assignment (to establish causation) and ideally random selection (to generalize to a population).