Hypothesis Testing Framework

The Logic of Hypothesis Testing

1. Assume the null hypothesis ($H_0$) is true
2. Collect data and compute a test statistic
3. Determine how likely (or unlikely) the data is under $H_0$
4. Make a decision: reject or fail to reject $H_0$

Hypotheses

Null Hypothesis ($H_0$): The "no effect" or "no difference" claim. Always includes $=$.

Alternative Hypothesis ($H_a$): What we're trying to find evidence for.

Types of alternative hypotheses:

• Two-sided: $H_a: p \neq p_0$ or $H_a: \mu \neq \mu_0$
• One-sided (right): $H_a: p > p_0$ or $H_a: \mu > \mu_0$
• One-sided (left): $H_a: p < p_0$ or $H_a: \mu < \mu_0$

Significance Level ($\alpha$)

The significance level $\alpha$ is the threshold for deciding when to reject $H_0$.

Common values: $\alpha = 0.05$ (most common), $\alpha = 0.01$, $\alpha = 0.10$

Test Statistic

The test statistic measures how far the sample result is from what $H_0$ predicts:

$\text{test statistic} = \frac{\text{statistic} - \text{parameter under } H_0}{\text{standard error}}$

P-Value

The p-value is the probability of obtaining a test statistic as extreme as (or more extreme than) the observed value, assuming $H_0$ is true.

• Small p-value → evidence against $H_0$
• Large p-value → no convincing evidence against $H_0$

Decision Rule

• If $p\text{-value} \leq \alpha$: Reject $H_0$. There is convincing evidence for $H_a$.
• If $p\text{-value} > \alpha$: Fail to reject $H_0$. There is not convincing evidence for $H_a$.

Never say "accept $H_0$" — we only fail to reject it.

Statistical Significance

A result is statistically significant at level $\alpha$ if the p-value $\leq \alpha$.

Statistical significance ≠ practical significance. A very large sample can detect tiny, meaningless differences.

Four-Step Process (AP)

1. State: Define parameter, state hypotheses, choose $\alpha$
2. Plan: Name the test, check conditions
3. Do: Calculate test statistic and p-value
4. Conclude: Compare p-value to $\alpha$, state conclusion in context

AP Tip: Always state your conclusion in context: "Since the p-value of 0.03 is less than $\alpha = 0.05$, we reject $H_0$. There is convincing evidence that [context about $H_a$]."