Least-Squares Regression

📈 Scatterplots and Correlation

Part 1 of 7 — Exploring Bivariate Relationships

Describing Scatterplots

When examining a scatterplot, describe:

Correlation Coefficient $r$

The correlation $r$ measures the strength and direction of a linear relationship. On the AP exam, $r$ is calculated using technology (calculator or computer output); you interpret it rather than compute it by hand.

Important Properties of $r$

• $-1 \leq r \leq 1$ always
• $r$ has no units
• $r$ is not affected by changes in units (e.g., inches to cm)
• $r$ measures only linear association — a strong curved relationship can have $r \approx 0$
• A single point that doesn't fit the pattern can change $r$ a lot
• An $r$ close to $-1$ or $1$ does not by itself mean a linear model is appropriate: always look at the scatterplot (and later, the residual plot)

🔑 Correlation does NOT imply causation. A strong correlation between two variables does not mean one causes the other.

Correlation Check 🎯

Correlation Practice 🧮

1) If $r = 0.72$, what is $r^2$? (Round to 2 decimal places)

2) What percentage of variation in $y$ is explained by the linear relationship with $x$? (Use $r^2$ from #1, express as a whole number)

3) If every data point falls exactly on the line $y = 3x + 2$, then $r =$?

📊 Least-Squares Regression Line

Part 2 of 7 — The LSRL

Topics in This Part

🔑 Key Concept: The least-squares regression line (LSRL) is the line that minimizes the sum of the squared residuals — the best-fit line through a scatterplot.

The LSRL Equation

$\boxed{\hat{y} = a + bx}$

where:

• $b$ is the slope and $a$ is the $y$-intercept; both are calculated using technology (calculator or computer output)
• $\hat{y}$ ("y-hat") is the predicted value of the response for a given $x$
• The line always passes through the point $(\bar{x}, \bar{y})$, so if you know $b$ and the two means you can find $a = \bar{y} - b\bar{x}$

What LSRL Minimizes

$\text{LSRL minimizes } \sum (y_i - \hat{y}_i)^2 = \sum (\text{residual})^2$

This is the sum of squared residuals — hence "least squares."

Interpreting Slope

Template: "For each additional [1 unit of $x$], the predicted [y variable] changes by [$b$ units]."

Example: $\hat{y} = 12 + 3.5x$ where $x$ = hours studied, $y$ = exam score.

✅ "For each additional hour studied, the predicted exam score increases by 3.5 points."

⚠️ AP Tip: Say "predicted" (or "on average"): the slope describes the model's prediction, not a guaranteed change for every individual.

Interpreting Intercept

Template: "When $x = 0$, the predicted [y variable] is [$a$]."

Example: $a = 12$ in $\hat{y} = 12 + 3.5x$.

✅ "When a student studies 0 hours, the predicted exam score is 12 points."

⚠️ Caution: The intercept sometimes has no reasonable interpretation: $x = 0$ may be far outside the data (extrapolation), or the predicted value may be impossible, such as a negative height.

Predictions & Extrapolation

⚠️ Extrapolation is less reliable the farther you go. The linear relationship may not hold outside the data range.

LSRL Concepts 🎯

LSRL Calculations 🧮

Technology gives the slope of the LSRL as $b = 1.7$. The means are $\bar{x} = 10$ and $\bar{y} = 25$.

1) What is the intercept $a$?

2) What is $\hat{y}$ when $x = 12$?

3) An observed point is $(12, 30)$. What is its residual?

Interpretation Practice 🔍

$\hat{y} = 50 - 0.8x$ where $x$ = temperature (°F), $y$ = hot chocolate sales (cups). The data cover temperatures from 20°F to 70°F.

Exit Quiz — LSRL ✅

📊 Residuals and Residual Plots

Part 3 of 7 — Assessing the Fit of a Linear Model

Topics in This Part

🔑 Key Concept: A residual is the vertical distance from a data point to the regression line. Residual plots help us assess whether a linear model is appropriate.

Residual Formula

$\boxed{\text{residual} = y - \hat{y} = \text{observed} - \text{predicted}}$

The LSRL is the line that makes the sum of the squared residuals as small as possible.

Residual Plots

A residual plot is a scatterplot of the residuals (vertical axis) versus the predicted values $\hat{y}$ or the explanatory variable $x$ (horizontal axis).

Reading Residual Plots

🔑 AP Tip: A correlation close to $\pm 1$ is not enough. Look at the residual plot before trusting a linear model.

Worked Example

$\hat{y} = 10 + 2x$. Data point: $(5, 23)$.

$\hat{y} = 10 + 2(5) = 20$ $\text{residual} = 23 - 20 = 3$

The residual is $+3$: the observed value is 3 units above the predicted value, so the model underpredicts this point by 3.

Residual Concepts 🎯

Residual Calculations 🧮

LSRL: $\hat{y} = 15 + 4x$

1) Point $(3, 30)$. Residual $=$

2) Point $(5, 33)$. Residual $=$

3) Point $(2, 20)$. The model ___ $y$ for this point (enter "overestimates" or "underestimates").

Residual Plot Patterns 🔍

Exit Quiz — Residuals ✅

📊 Coefficient of Determination

Part 4 of 7 — Understanding $r^2$

Topics in This Part

🔑 Key Concept: $r^2$, the coefficient of determination, is the proportion of the variation in the response variable $y$ that is explained by the linear relationship with the explanatory variable $x$.

The Definition

In simple linear regression, $r^2$ is the square of the correlation coefficient:

$\boxed{r^2 = r \times r}$

Technology reports it directly (often as "R-sq" or "$R^2$", sometimes as a percent). Because it is a square, $0 \leq r^2 \leq 1$, and it carries no sign.

Interpretation Template

$\text{"[}r^2 \times 100\text{]\% of the variation in [y context] is explained by the linear relationship with [x context]."}$

Example: $r^2 = 0.72$, $x$ = hours studied, $y$ = exam score.

✅ "72% of the variation in exam scores is explained by the linear relationship with hours studied."

⚠️ AP Tip: Always say "variation in [y]" and "linear relationship with [x]." Do not say "caused by," and $r^2$ is NOT the percent of points on the line.

The rest, $1 - r^2$, is the proportion of variation in $y$ that the linear model leaves unexplained.

$r$ vs. $r^2$

🔑 Key Insight: Going from $r^2$ back to $r$, take the square root and give it the sign of the slope: $r^2 = 0.49$ with a negative slope means $r = -0.7$. Even $r = 0.7$ explains only 49% of the variation.

$r^2$ Concepts 🎯

$r^2$ Calculations 🧮

1) $r = 0.9$. What is $r^2$?

2) $r^2 = 0.49$. What percentage of the variation in $y$ is explained?

3) $r^2 = 0.49$ and the slope is negative. What is $r$?

Interpretation Practice 🔍

Exit Quiz — $r^2$ ✅

📊 Unusual Points, Extrapolation and Interpreting the Line

Part 5 of 7 — Reading a Regression Carefully

Topics in This Part

🔑 Key Concept: A regression line is only as good as the pattern behind it. Before using a line, look for unusual features, check whether a linear model fits, and use it only for $x$ values like the ones in the data.

Unusual Features of a Scatterplot

A full description of a scatterplot gives form, direction, strength and unusual features. Unusual features are:

Example: For 12 used cars, price falls steadily with age, except for one 6-year-old classic car priced far above the trend. That car is a point that doesn't fit the pattern: the line badly underpredicts its price, so its residual is large and positive.

When $r$ Misleads

• An $r$ close to $-1$ or $1$ does not necessarily mean a linear model is appropriate. Gently curved data can give $r = 0.97$ while the residual plot shows clear curvature.
• An $r$ near 0 means no linear association; a strong curved relationship is still possible.
• Correlation does not imply causation: a confounding variable may drive both variables.

Unusual Features Check 🎯

📏 Interpolation vs. Extrapolation

Example: For children aged 3 to 12 years, $\hat{y} = 30.2 + 2.4x$, where $x$ = age (years) and $y$ = height (inches).

• Age 8: $\hat{y} = 30.2 + 2.4(8) = 49.4$ inches (interpolation).
• Age 40: $\hat{y} = 30.2 + 2.4(40) = 126.2$ inches, over 10 feet (extrapolation; growth stops, so the line no longer applies).

📝 Interpreting Slope and Intercept in Context

Slope: "For each additional year of age, the predicted height increases by 2.4 inches."

Intercept: "At age 0, the predicted height is 30.2 inches." But age 0 is outside the 3-to-12-year data, so this is an extrapolation and should not be trusted.

The intercept also has no logical interpretation when it is impossible for the response, for example a negative predicted height, weight or count.

⚠️ AP Tip: Always name both variables, use "predicted," and give units. Say whether $x = 0$ is inside the data before interpreting the intercept.

Using the Height Model 🧮

$\hat{y} = 30.2 + 2.4x$, with $x$ = age (years, data from 3 to 12) and $y$ = height (inches).

1) Predicted height at age 8?

2) Predicted height at age 40 (an extrapolation)?

3) A 10-year-old is 52 inches tall. What is the residual?

Reading the Line Carefully 🔍

Exit Quiz — Unusual Points, Extrapolation & Interpretation ✅

📊 Problem-Solving Workshop

Part 6 of 7 — Full Regression Analysis Problems

Workshop Goals

🔑 AP Tip: Free-response regression questions typically ask you to interpret slope/$r^2$ in context, describe the residual plot, and discuss whether the model is appropriate.

Worked Example 1 — Temperature and Ice Cream Sales

A manager records daily high temperature ($x$, °F) and ice cream sales ($y$, $100s) for 15 summer days.

Technology output:

$r^2 = 91.1\%$

Step 1 — Write the LSRL: $\hat{y} = -3.50 + 0.15x$

Step 2 — Interpret the slope: "For each additional degree Fahrenheit in daily high temperature, the predicted ice cream sales increase by $15 (0.15 hundreds of dollars)."

Step 3 — Interpret $r^2$: "91.1% of the variation in ice cream sales is explained by the linear relationship with daily high temperature."

Step 4 — Predict: At $x = 85°$F: $\hat{y} = -3.50 + 0.15(85) = -3.50 + 12.75 = 9.25 \text{ (\$925 in sales)}$

Step 5 — Check appropriateness:

• Residual plot shows no obvious pattern → linear model is appropriate
• $r^2 = 0.911$ → most of the variation in sales is explained
• No points stand far from the pattern in the residual plot

Worked Example 2 — Study Hours and GPA

A sample of 30 college students. $x$ = weekly study hours, $y$ = GPA.

LSRL: $\hat{y} = 1.85 + 0.052x$, $r = 0.68$, $r^2 = 0.462$

Most students study 5–25 hours; one student studies 42 hours/week and has a GPA of 3.9.

Analysis:

1. Slope interpretation: "For each additional hour of weekly studying, the predicted GPA increases by 0.052 points."

2. $r^2$ interpretation: "46.2% of the variation in GPA is explained by the linear relationship with weekly study hours."

3. The 42-hour student:

   • $\hat{y} = 1.85 + 0.052(42) = 4.034$, so the predicted GPA is 4.034
   • Residual $= 3.9 - 4.034 = -0.134$, a small residual
   • Although this $x$ is far from the other students, the point fits the general pattern

4. Prediction for 50 hours: $\hat{y} = 1.85 + 0.052(50) = 4.45$

   • This is extrapolation (beyond the data), and the prediction exceeds 4.0, the maximum GPA, so it is unreliable!

Common Mistakes on the AP Exam

Regression Analysis Practice 🎯

Computations 🧮

LSRL: $\hat{y} = 5.2 + 1.3x$, $r = 0.85$

1) Predict $y$ when $x = 10$.

2) What is $r^2$? (two decimal places)

3) Observed $y = 22$ when $x = 10$. What is the residual?

Interpretation Decisions 🔍

Exit Quiz — Regression Workshop ✅

📊 Review & Applications

Part 7 of 7 — Comprehensive Linear Regression Review

Complete Reference

Interpretation Templates (AP Exam Ready)

Slope: "For each additional [1 unit of x], the predicted [y in context] [increases/decreases] by [|b|] [units of y]."

Intercept: "When [x in context] is 0, the predicted [y in context] is [a] [units of y]." (Only if $x = 0$ is within the data and the value makes sense.)

$r$: "There is a [strong/moderate/weak], [positive/negative], linear association between [x] and [y]."

$r^2$: "[$r^2 \times 100$]% of the variation in [y in context] is explained by the linear relationship with [x in context]."

Residual: "The actual [y in context] was [e] [units] [above/below] the value predicted by the model."

Key Concepts Summary

Decision Guide

$\text{Is the relationship linear?}$ $\downarrow$ $\text{Check scatterplot → Fit LSRL → Check residual plot}$ $\downarrow$ $\text{Random scatter → Linear model OK}$ $\text{Curved pattern → Linear model not the most appropriate}$ $\downarrow$ $\text{Interpret: slope, } r, r^2 \text{ in context}$ $\downarrow$ $\text{Note unusual features}$ $\downarrow$ $\text{Make predictions (within data range only)}$

🔑 AP Exam Strategy: Regression appears on the exam every year. Master the interpretation templates — they earn you full credit on free-response questions.

Comprehensive Review 🎯

Mixed Calculations 🧮

Technology gives $b = 1.5$ and $r = 0.75$. The means are $\bar{x} = 10$ and $\bar{y} = 25$.

1) Find the intercept $a$.

2) Predict $y$ when $x = 14$.

3) What is $r^2$? (two decimal places)

Concept Connections 🔍

Final Exam — Linear Regression ✅