Least-Squares Regression

The Regression Line

The least-squares regression line (LSRL) is the line that minimizes the sum of squared residuals:

$\hat{y} = a + bx$

where:

• $\hat{y}$ = predicted value of $y$
• $b$ = slope
• $a$ = y-intercept

Computing the LSRL

Slope: $b = r \cdot \frac{s_y}{s_x}$

Y-intercept: $a = \bar{y} - b\bar{x}$

The LSRL always passes through the point $(\bar{x}, \bar{y})$.

Interpreting the Slope

"For each additional [unit of x], the predicted [y variable] changes by [b] [units of y]."

Example: If $\hat{y} = 2.5 + 0.8x$ where $x$ = study hours and $y$ = exam score: "For each additional hour of studying, the predicted exam score increases by 0.8 points."

Interpreting the Y-Intercept

"When [x variable] = 0, the predicted [y variable] is [a] [units]."

Caution: The y-intercept often doesn't have practical meaning (e.g., 0 hours of studying may not make sense in context).

Making Predictions

Substitute the $x$-value into the equation to get $\hat{y}$.

Extrapolation

Extrapolation = predicting $y$ for $x$-values outside the range of the data. This is dangerous because the linear pattern may not continue.

Residuals

$\text{residual} = y - \hat{y} = \text{observed} - \text{predicted}$

• Positive residual: actual > predicted (point above line)
• Negative residual: actual < predicted (point below line)
• $\sum \text{residuals} = 0$ for the LSRL

Properties of LSRL

1. Minimizes $\sum(y_i - \hat{y}_i)^2$
2. Passes through $(\bar{x}, \bar{y})$
3. Sum of residuals = 0
4. $r$ and $b$ have the same sign
5. Regression toward the mean: predicted values are closer to $\bar{y}$ than observed values

AP Tip: When interpreting slope, always use "predicted" — not "will increase by." The relationship is an estimate, not a guarantee.