Inference for Regression

The Regression Model

The population regression line: $\mu_y = \alpha + \beta x$

We estimate this with the sample regression line: $\hat{y} = a + bx$

The key question: Is there a significant linear relationship? (Is $\beta \neq 0$?)

Conditions for Regression Inference

LINE conditions:

• Linear: The true relationship is linear (check residual plot)
• Independent: Observations are independent (10% condition)
• Normal: For each value of $x$, the responses are Normally distributed (check Normal probability plot of residuals)
• Equal variance: The standard deviation of $y$ is the same for all $x$ (check residual plot for constant spread)

Hypothesis Test for the Slope

Hypotheses:

• $H_0: \beta = 0$ (no linear relationship)
• $H_a: \beta \neq 0$ (there is a linear relationship)

Test Statistic: $t = \frac{b - 0}{SE_b}$

where $SE_b = \frac{s}{\sqrt{\sum(x_i - \bar{x})^2}}$ and $s = \sqrt{\frac{\sum(y_i - \hat{y}_i)^2}{n-2}}$

Degrees of freedom: $df = n - 2$

Confidence Interval for $\beta$

$b \pm t^* \cdot SE_b$

with $df = n - 2$

Interpretation: "We are [C]% confident that the true slope of the relationship between [x] and [y] is between [lower] and [upper]."

Reading Computer Output

A typical regression output includes:

| | Coef | SE Coef | T | P | |---|---|---|---|---| | Constant | $a$ | $SE_a$ | $t_a$ | $p_a$ | | [x variable] | $b$ | $SE_b$ | $t_b$ | $p_b$ |

$S =$ [standard deviation of residuals] $R\text{-}sq =$ [$r^2$ value]

The row for the x-variable gives you everything you need:

• $b$ = slope estimate
• $SE_b$ = standard error of slope
• $t$ = test statistic for $H_0: \beta = 0$
• $P$ = p-value for the test

Interpreting Regression Output

1. Slope ($b$): For each 1-unit increase in $x$, the predicted $y$ changes by $b$ units
2. Standard error ($SE_b$): Measures the precision of the slope estimate
3. t-statistic: How many SEs the slope is from 0
4. P-value: Probability of observing this slope (or more extreme) if $\beta = 0$

Example Conclusion

"Since the p-value of 0.002 is less than $\alpha = 0.05$, we reject $H_0$. There is convincing evidence of a linear relationship between [x] and [y]."

AP Tip: You MUST be able to read computer regression output. Practice identifying $b$, $SE_b$, $t$, $p$, $s$, and $r^2$ from output tables. Also, always check LINE conditions before doing inference.