⚠️ Type I and Type II Errors

Type I Error (α)

A Type I error occurs when we reject a true null hypothesis.

In words: We conclude there's an effect/difference when actually there isn't one.

Probability of Type I error = α (the significance level)

Example:

• $H_0$: The defendant is innocent (truth)
• $H_a$: The defendant is guilty
• Type I error: Convicting an innocent person (reject true $H_0$)
• Consequence: Innocent person punished

In medical testing:

• $H_0$: Patient does not have disease
• Type I error: Diagnosing disease when patient is healthy
• Consequence: Unnecessary treatment, patient anxiety

Type II Error (β)

A Type II error occurs when we fail to reject a false null hypothesis.

In words: We conclude there's no effect/difference when actually there is one.

Probability of Type II error = β (often unknown)

Example:

• $H_0$: The defendant is innocent (false; they actually committed crime)
• Type II error: Acquitting a guilty person (fail to reject false $H_0$)
• Consequence: Criminal goes free

In medical testing:

• $H_0$: Patient does not have disease
• Type II error: Saying patient is healthy when they actually have disease
• Consequence: Missed diagnosis, delayed treatment

Power of a Test

The power of a test is the probability of correctly rejecting $H_0$ when $H_a$ is true.

$\text{Power} = 1 - \beta$

• High power (close to 1): Good chance of detecting true effect if it exists
• Low power (close to 0): High risk of missing true effect

Interpretation: If the alternative is true, power is the probability we'll find it.

Error Summary Table

Worked Example

Scenario: Testing if a new drug is effective.

• $H_0$: Drug has no effect
• $H_a$: Drug has effect
• α = 0.05

Type I error (α = 0.05): Conclude drug works when it doesn't. Risk: 5% (set by significance level)

Type II error (β): Conclude drug doesn't work when it actually does. Risk: Unknown, but reduced by:

• Larger sample size
• Larger effect size
• Less variable data

Power = 1 − β: Probability we detect the drug's effect if it exists. Should be high (0.80 or 0.90 typical targets)

Factors Affecting Type I and Type II Errors

Trade-off Between Errors

Lowering α (e.g., from 0.05 to 0.01) automatically increases β. You can't simultaneously minimize both errors with fixed sample size.

Strategy depends on consequence:

• High Type I cost (e.g., convicting innocent): Use low α (e.g., 0.01)
• High Type II cost (e.g., missing disease): Use higher α (e.g., 0.10) or larger n

Sample Size and Power

To increase power for fixed α:

• Increase sample size n: More data provides stronger evidence
• Formula: Larger n → smaller SE → larger test statistic → higher power

Example: If power is too low (say 0.60), increasing n to 200 might increase power to 0.85.

AP Exam Tip

Know the definitions of Type I and II errors cold; these appear frequently. Context matters: identify which error is worse, then design accordingly. Power problems ask: "What sample size gives power = 0.90?" Use technology or power tables. Always interpret in context (what does rejecting/failing to reject mean for the actual situation?).