Least-Squares Regression

Regression Line (Line of Best Fit)

Purpose: predict response variable y from explanatory variable x

Goal: minimize vertical distances (residuals) from points to line

Form: $\hat{y} = a + bx$

where:

• \(\hat{y}\) = predicted y value (not actual y)
• a = y-intercept
• b = slope

Finding the Regression Line

Slope: $b = r \cdot \frac{s_y}{s_x}$

where r = correlation, \(s_x\) = std dev of x, \(s_y\) = std dev of y

Y-intercept: $a = \bar{y} - b\bar{x}$

Key fact: regression line always passes through \((\bar{x}, \bar{y})\)

Interpreting Slope and Intercept

Slope (b): predicted change in y for each 1-unit increase in x

• Positive slope: as x increases, predicted y increases
• Negative slope: as x increases, predicted y decreases
• Units: same ratio as y-units per x-unit

Example: \(\hat{\text{Score}} = 60 + 5 \cdot \text{Hours}\)

• Slope b = 5: for each additional hour studied, predicted score increases by 5 points
• Intercept a = 60: predicted score if 0 hours studied (often not meaningful in context)

Y-intercept (a): predicted y value when x = 0

• Meaningful only if x = 0 is reasonable in context
• Example: predicted exam score with 0 hours studied may not make sense

Making Predictions

Prediction for given x:

1. Substitute x into regression equation
2. Calculate \(\hat{y}\)
3. Caveat: only valid for x-values in range of data

Extrapolation: predicting beyond range of data

• Risky: relationship may not hold outside observed range
• Avoid extrapolation unless strong theoretical reason

Example: If data ranges 1–6 hours, predicting score for 50 hours is extrapolation (unreliable)

Worked Example

Data: 6 students, hours studied (x) vs. exam score (y)

Given: \(\bar{x} = 4.67, \bar{y} = 83, s_x ≈ 1.97, s_y ≈ 10.05, r ≈ 0.97\)

Step 1: Calculate slope $b = 0.97 \cdot \frac{10.05}{1.97} ≈ 0.97 \cdot 5.10 ≈ 4.95$

Step 2: Calculate intercept $a = 83 - 4.95 \cdot 4.67 ≈ 83 - 23.1 ≈ 59.9$

Regression line: \(\hat{\text{Score}} ≈ 59.9 + 4.95 \cdot \text{Hours}\)

Prediction: If student studies 7 hours: $\hat{\text{Score}} = 59.9 + 4.95(7) = 59.9 + 34.65 = 94.55 ≈ 94.6\text{ points}$

(This is interpolation; 7 hours is within data range 2–8)

Residuals and Residual Plots

Residual: actual y − predicted y = \(y - \hat{y}\)

Residual plot: scatterplot of residuals vs. x values

• If residuals randomly scattered around 0 → linear model is appropriate
• If residuals show pattern (curved, increasing) → linear model inadequate

Sum of residuals: always ≈ 0 for least-squares regression

Common Mistakes

1. Swapping x and y: regression of y on x ≠ regression of x on y
2. Over-interpreting intercept: a = 60 (x = 0) may have no real meaning
3. Extrapolating recklessly: don't predict far outside data range
4. Confusing \(\hat{y}\) with y: \(\hat{y}\) is prediction, not actual value
5. Ignoring residuals: always check residual plot to validate linear assumption

AP Exam Tip

FRQ response format:

1. Show work: state formula for slope \(b = r \cdot \frac{s_y}{s_x}\) and intercept \(a = \bar{y} - b\bar{x}\)
2. Write regression equation: \(\hat{\text{variable}} = a + b \cdot \text{variable}\)
3. Interpret slope in context: "For each additional [x-unit], predicted [y-variable] increases by [slope value] [y-units]."
4. Caveat on predictions: "This prediction assumes the linear relationship continues in this range" or note if extrapolating

Example: "\(\hat{\text{Score}} = 59.9 + 4.95 \cdot \text{Hours}\). For each additional hour studied, we predict the exam score increases by 4.95 points."