Observational Studies vs Experiments

The Fundamental Difference

Observational Study: Observe/measure without intervention
Experiment: Actively impose treatment to observe effect

Key distinction: In experiments, researchers control the explanatory variable; in observational studies, they don't.

Observational Studies

Definition: Researchers observe individuals and measure variables without assigning treatments.

Characteristics:

• No manipulation by researcher
• Observe "natural" conditions
• Can show association but NOT causation

Examples:

• Survey asking smokers about health
• Study comparing test scores by study method (students choose method)
• Medical records analysis of diet and disease

Types:

Sample Survey: Collect data from sample at one point in time
Retrospective Study: Look back at historical data
Prospective Study: Follow subjects forward in time

Advantages:

• Can study variables that can't be manipulated ethically
• Often cheaper and faster
• Observes real-world behavior

Disadvantages:

• Cannot establish causation (correlation ≠ causation!)
• Confounding variables
• Less control over conditions

Experiments

Definition: Researchers deliberately impose treatments and observe responses.

Characteristics:

• Researcher assigns treatments
• Control over conditions
• Can establish cause-and-effect

Examples:

• Randomly assign students to study methods, compare test scores
• Give some patients medicine, others placebo
• Test fertilizer by applying different amounts to randomly selected plots

Key Components:

Explanatory Variable (Factor): What researcher manipulates
Treatments: Different values/levels of explanatory variable
Response Variable: Outcome measured
Experimental Units: Individuals/items receiving treatments

Advantages:

• Can establish causation (if well-designed)
• Control over conditions
• Can isolate effect of treatment

Disadvantages:

• May be unethical (can't randomly assign smoking!)
• Artificial setting may not reflect real life
• Often expensive and time-consuming

Why Experiments Can Show Causation

Three criteria for causation:

1. Association: Variables are related
2. Time order: Cause precedes effect
3. No plausible alternative explanation

Experiments satisfy all three:

• Random assignment controls for confounding
• Researcher imposes treatment before measuring response
• Controlled conditions eliminate alternatives

Observational studies can show association but struggle with #3 (confounding variables provide alternative explanations)

Confounding Variables

Confounding Variable: Variable related to both explanatory and response variables that distorts the relationship.

Example:

• Observational study: Coffee drinkers have higher heart disease rates
• Confounding variable: Smoking (coffee drinkers more likely to smoke; smoking causes heart disease)
• Conclusion: Can't tell if coffee causes heart disease or if it's the smoking!

Experiments address confounding through randomization (discussed in Experimental Design topic)

Association vs Causation

Association (Correlation): Variables are related
Causation: One variable causes changes in another

All causation involves association, but NOT all association implies causation!

Famous example: Ice cream sales and drowning deaths are associated (both higher in summer), but ice cream doesn't cause drowning. Confounding variable: weather/temperature.

Choosing Between Observational and Experimental

Use Observational Study when:

• Can't manipulate variable ethically (smoking, genetics)
• Want to study relationships in natural setting
• Experiment not feasible

Use Experiment when:

• Want to establish causation
• Can ethically manipulate variable
• Need controlled conditions

Ethical Considerations

Cannot experiment on:

• Harmful treatments (cigarette smoking)
• Variables you can't control (age, gender, genetics)
• Situations where withholding beneficial treatment would be harmful

Must use observational studies for many important questions!

Scope of Inference

Generalization (to population): Requires random sampling
Causation (cause-effect): Requires random assignment (experiment)

Four scenarios:

| Random Sample? | Random Assignment? | Can Infer Causation? | Can Generalize? | |----------------|-------------------|---------------------|-----------------| | Yes | Yes | Yes | Yes | | Yes | No | No | Yes | | No | Yes | Yes | No | | No | No | No | No |

Best case: Random sample + random assignment (rare!)
Worst case: Convenience sample + observational (common but weakest)

Quick Reference

Observational Study:

• Observe without intervention
• Shows association
• Cannot establish causation
• Confounding variables problematic

Experiment:

• Impose treatments
• Can show causation
• Random assignment controls confounding
• May not reflect real-world conditions

Remember: The gold standard for establishing causation is a well-designed, randomized experiment. Observational studies can suggest relationships but cannot prove cause-and-effect!