Linear Regression and Correlation

📈 Linear Regression and Correlation

Part 1 of 5 — Scatterplots & Describing a Relationship

Topics in This Part

🔑 Key Concept: When two numerical variables are measured on the same individuals — like hours studied and test score — we plot the pairs as points. The picture that emerges (a scatterplot) tells us whether the variables move together, and how tightly.

Bivariate Data & Scatterplots

Bivariate means two variables. Each individual contributes an ordered pair $(x, y)$.

Plotting these five points gives a scatterplot. Here the points climb from lower-left to upper-right — more hours tends to mean a higher score.

We always put the explanatory variable (the one we think does the explaining) on the horizontal $x$-axis and the response variable (the one we think responds) on the vertical $y$-axis.

💡 In "hours studied vs. test score," hours is explanatory ($x$) and score is response ($y$). Ask yourself: which one would I change to affect the other? That one is $x$.

Concept Check 🎯

Direction, Form, and Strength

Describe every scatterplot with three words:

• Direction — Positive (points rise left-to-right) or negative (points fall).
• Form — Linear (points follow a straight-line pattern) or nonlinear (a curve).
• Strength — How tightly the points hug the pattern: strong, moderate, or weak.

⚠️ Always look at the scatterplot first. Everything else in this lesson — the correlation coefficient, the line of best fit — only makes sense when the form is linear. A strong curved pattern can fool a number into looking weak.

Describe the Scatterplot 🔽

A scatterplot of outdoor temperature vs. ice cream sales shows points climbing steadily from lower-left to upper-right, clustered tightly around a straight line.

Negative Relationships

A negative relationship is just as common: as one variable goes up, the other goes down, so the points fall from upper-left to lower-right.

💡 A negative relationship can still be strong — strength is about how tightly the points hug the line, not which way the line tilts. A tightly falling cloud of points is strong and negative.

Concept Check 🎯

Why This Matters

Once we know a relationship is linear, two powerful tools open up:

1. The correlation coefficient $r$ — a single number measuring direction and strength (Part 2).
2. The line of best fit $\hat{y} = a + bx$ — an equation that lets us predict (Parts 3–4).

You can now read a scatterplot and describe it in three words. Next we turn "strong/moderate/weak" into an exact number.

📈 Linear Regression and Correlation

Part 2 of 5 — The Correlation Coefficient $r$

🔑 The Idea: The correlation coefficient $r$ squeezes the direction and strength of a linear relationship into one number between $-1$ and $+1$.

What $r$ Tells You

$-1 \le r \le 1$

• The sign of $r$ = the direction. Positive $r$ → positive (rising) trend; negative $r$ → negative (falling) trend.
• The size of $r$ (its distance from $0$) = the strength.

💡 $r = -0.9$ is stronger than $r = +0.6$. Strength depends on how far $r$ is from $0$, not on its sign. The sign only tells direction.

Concept Check 🎯

Computing $r$ by Hand

For a small data set you can compute $r$ from sums of squares:

$r = \frac{S_{xy}}{\sqrt{S_{xx}\, S_{yy}}}, \quad\text{where}\quad S_{xy} = \sum (x-\bar{x})(y-\bar{y})$

and $S_{xx} = \sum (x-\bar{x})^2$, $\;S_{yy} = \sum (y-\bar{y})^2$.

Worked Example

Data: $x = 0,1,2,3,4$ and $y = 1,3,2,5,4$. Means: $\bar{x} = 2$, $\bar{y} = 3$.

$r = \frac{8}{\sqrt{10 \cdot 10}} = \frac{8}{10} = 0.8$

✅ A moderate-to-strong positive correlation. In practice you'll usually let a calculator do this — but doing it once shows you exactly what $r$ measures.

Compute $r$ 🧮

A different data set gives these sums: $S_{xx} = 16,\quad S_{yy} = 25,\quad S_{xy} = 18$

1) Find $\sqrt{S_{xx}\, S_{yy}} = \sqrt{16 \cdot 25} = \,?$ 2) Find $r = \dfrac{S_{xy}}{\sqrt{S_{xx} S_{yy}}} = \,?$ (decimal, round to 2 places)

The Coefficient of Determination $r^2$

Square the correlation to get $r^2$ (always between $0$ and $1$). It has a concrete meaning:

🔑 $r^2$ is the proportion of the variation in $y$ that is explained by the linear relationship with $x$.

If the Hours vs. Score data gives $r = 0.99$, then $r^2 = (0.99)^2 \approx 0.98 = 98\%.$ So about 98% of the variation in test scores is explained by hours studied; the remaining 2% is due to everything else.

⚠️ Because $r^2$ comes from squaring, it loses the sign — it cannot tell you the direction. A correlation of $r=-0.6$ and one of $r=+0.6$ both give $r^2 = 0.36$.

Concept Check 🎯

📈 Linear Regression and Correlation

Part 3 of 5 — The Line of Best Fit

🔑 The Goal: Find the single straight line $\hat{y} = a + bx$ that comes closest to all the points. We call it the least-squares regression line, and it lets us predict $y$ from $x$.

The Least-Squares Regression Line

Out of all possible lines, the least-squares line is the one that makes the total of the squared vertical distances from the points to the line as small as possible.

$\hat{y} = a + bx$

The hat on $\hat{y}$ ("$y$-hat") signals a predicted value, not an observed one.

• $b$ = slope — how much $\hat{y}$ changes for each +1 increase in $x$.
• $a$ = $y$-intercept — the predicted $\hat{y}$ when $x = 0$.

The slope and correlation are linked through the standard deviations $s_x$ and $s_y$:

$b = r \cdot \frac{s_y}{s_x}, \qquad a = \bar{y} - b\,\bar{x}$

💡 The slope $b$ always has the same sign as $r$: a positive correlation gives an upward-sloping line, a negative correlation a downward-sloping one.

Concept Check 🎯

Building the Line: Worked Example

Use the Hours vs. Score data: $x = 1,2,3,4,5$ and $y = 50,65,70,80,90$.

Step 1 — Means. $\bar{x} = 3$, $\bar{y} = 71$.

Step 2 — Sums of squares. Working from the deviations gives $S_{xx} = 10, \qquad S_{xy} = 95.$

Step 3 — Slope. The slope can be found directly as $b = \dfrac{S_{xy}}{S_{xx}}$: $b = \frac{95}{10} = 9.5.$

Step 4 — Intercept. Use $a = \bar{y} - b\,\bar{x}$: $a = 71 - 9.5(3) = 71 - 28.5 = 42.5.$

The line: $\hat{y} = 42.5 + 9.5x$

✅ Check: the regression line always passes through the point of averages $(\bar{x}, \bar{y}) = (3, 71)$. Test it: $42.5 + 9.5(3) = 42.5 + 28.5 = 71$ ✓

Order the Steps 🔽

You're finding the regression line for a data set with $\bar{x} = 4$, $\bar{y} = 20$, $S_{xx} = 8$, and $S_{xy} = 24$.

The Two-Step Recipe

Every regression problem in Algebra 1 boils down to the same two moves, in order:

$\boxed{\;b = \frac{S_{xy}}{S_{xx}}\;}\qquad\text{then}\qquad\boxed{\;a = \bar{y} - b\,\bar{x}\;}$

You must find the slope $b$ first, because the intercept formula needs it. Then assemble $\hat{y} = a + bx$.

🔑 Sanity check every line you build: plug in $\bar{x}$ and you should get exactly $\bar{y}$, because the line always passes through $(\bar{x}, \bar{y})$.

Build the Line 🧮

A data set has $\bar{x} = 5$, $\bar{y} = 10$, $S_{xx} = 20$, $S_{xy} = 30$.

1) Slope $b = \dfrac{S_{xy}}{S_{xx}} = \,?$ 2) Intercept $a = \bar{y} - b\,\bar{x} = \,?$

📈 Linear Regression and Correlation

Part 4 of 5 — Predictions, Residuals & Cautions

🔑 The Payoff: With the line $\hat{y} = a + bx$ in hand, plug in any $x$ to predict $y$. But predictions come with rules — and a famous warning about causation.

Making Predictions

To predict, substitute the $x$-value into the line.

Using $\hat{y} = 42.5 + 9.5x$ (Hours vs. Score):

Predict the score for 6 hours of study: $\hat{y} = 42.5 + 9.5(6) = 42.5 + 57 = 99.5$

Predict the score for 2.5 hours: $\hat{y} = 42.5 + 9.5(2.5) = 42.5 + 23.75 = 66.25$

Interpolation vs. Extrapolation

• Interpolation — predicting inside the range of the data ($x$ between 1 and 5 here). Generally safe.
• Extrapolation — predicting outside the data range (e.g. $x = 20$ hours). Risky — the linear pattern may not hold out there.

⚠️ Extrapolation is the #1 prediction trap. Our line would "predict" a score of $42.5 + 9.5(20) = 232.5$ for 20 hours — impossible on a 100-point test. Never trust a prediction far outside the data.

Predict with the Line 🧮

A delivery company models cost ($y$, dollars) against distance ($x$, miles) with $\hat{y} = 3 + 2x.$

1) Predicted cost for a 10-mile trip: $\hat{y} = \,?$ (dollars) 2) Predicted cost for a 25-mile trip: $\hat{y} = \,?$ (dollars)

Residuals: How Far Off Was the Prediction?

A residual is the vertical gap between an actual data point and the line:

$\text{residual} = y - \hat{y} = (\text{observed}) - (\text{predicted})$

Worked Example

In the Hours vs. Score data, the student who studied 2 hours actually scored 65. The line predicts $\hat{y} = 42.5 + 9.5(2) = 61.5.$ So the residual is $65 - 61.5 = +3.5.$

A positive residual means the point sits above the line (the model under-predicted). A negative residual means it sits below the line.

💡 The least-squares line is built so the residuals always sum to zero — positives and negatives perfectly cancel. That's what makes it the "best fit."

Compute Residuals 🧮

Use $\hat{y} = 42.5 + 9.5x$. A student studied 4 hours and scored 80.

1) Predicted score $\hat{y} = 42.5 + 9.5(4) = \,?$ 2) Residual $= y - \hat{y} = 80 - \,?$ — enter the residual (it is negative)

Correlation Is Not Causation

A strong correlation means two variables move together — it does not prove one causes the other.

⚠️ The classic example: Across a town, ice-cream sales and drowning deaths are strongly positively correlated. Ice cream doesn't cause drownings! A hidden lurking variable — hot summer weather — drives both up at once.

To establish causation you need a controlled experiment, not just a high $r$. Always ask: Could a third variable explain both?

Concept Check 🎯

📈 Linear Regression and Correlation

Part 5 of 5 — Mixed Practice & Mastery Check

You can now (1) describe a scatterplot, (2) interpret $r$ and $r^2$, (3) build the line of best fit, and (4) predict, find residuals, and avoid the causation trap. Let's put it all together.

Quick Reference

⚠️ Remember: the sign of $r$ matches the sign of $b$; $r^2$ drops the sign; extrapolation is risky; and correlation never proves causation.

Mixed Practice 🎯

One Full Problem, Start to Finish

A typical exam question hands you the summary numbers and asks for the line and a prediction. The flow is always the same:

1. Slope: $b = \dfrac{S_{xy}}{S_{xx}}$
2. Intercept: $a = \bar{y} - b\,\bar{x}$
3. Assemble: $\hat{y} = a + bx$
4. Predict: substitute the requested $x$

Try the complete cycle below — find the slope, the intercept, and then make a prediction.

Build & Predict 🧮

A data set has $\bar{x} = 4$, $\bar{y} = 12$, $S_{xx} = 10$, $S_{xy} = 25$.

1) Slope $b = S_{xy}/S_{xx} = \,?$ 2) Intercept $a = \bar{y} - b\bar{x} = \,?$ 3) Using $\hat{y} = a + bx$, predict $\hat{y}$ when $x = 6$. $\hat{y} = \,?$

You're Ready

You can describe a scatterplot, read $r$ and $r^2$, build $\hat{y} = a + bx$, predict and interpret residuals, and steer clear of the causation trap. One last set of three questions seals the lesson.

💡 On each Exit-Quiz item, decide which tool it's testing first — strength of $r$, a residual, or causation — then apply the matching move.

Exit Quiz ✅

Answer all three to finish the lesson.