Inference for Regression

📐 Inference for Linear Regression

Part 1 of 7 — Regression Model Assumptions

The Population Regression Model

$y = \beta_0 + \beta_1 x + \epsilon$

where $\epsilon \sim N(0, \sigma)$ — errors are normally distributed with constant spread.

Conditions for Inference (LINE)

🔑 The residual plot is the most important diagnostic tool. Look for random scatter around zero.

Regression Assumptions 🎯

📊 T-Test for Slope

Part 2 of 7 — Is There a Linear Relationship?

Topics in This Part

🔑 Key Concept: The $t$-test for slope tests whether the true population slope $\beta$ is zero (no linear relationship) or nonzero.

The Linear Regression Model

The population model is:

$y = \alpha + \beta x + \varepsilon$

where $\varepsilon \sim N(0, \sigma)$ (errors are independent and Normally distributed with constant variance).

• $\beta$ = true population slope
• $b$ = sample slope (our estimate of $\beta$)
• $\text{SE}_b$ = standard error of the slope

Hypotheses

$H_0: \beta = 0 \quad \text{(no linear relationship)}$ $H_a: \beta \neq 0 \quad \text{(or } \beta > 0 \text{ or } \beta < 0\text{)}$

⚠️ AP Tip: Most AP problems use the two-sided alternative $\beta \neq 0$. One-sided tests are less common but do appear.

The Test Statistic

$\boxed{t = \frac{b - 0}{\text{SE}_b} = \frac{b}{\text{SE}_b}}$

with $\text{df} = n - 2$ (two parameters estimated: $a$ and $b$).

Conditions (LINE)

Worked Example

A researcher studies 20 pine trees. $x$ = diameter (inches), $y$ = height (feet).

Computer output:

Step 1 — Hypotheses: $H_0: \beta = 0$ (no linear relationship between diameter and height) $H_a: \beta \neq 0$ (there is a linear relationship)

Step 2 — Conditions:

• L: Residual plot shows random scatter ✓
• I: Trees randomly selected; $20 < 10\%$ of all pine trees ✓
• N: Histogram of residuals approximately Normal ✓
• E: No fan shape in residual plot ✓

Step 3 — Test statistic: $t = b / \text{SE}_b = 2.35 / 0.42 = 5.60$, df $= 20 - 2 = 18$

Step 4 — P-value: $P < 0.001$ (from the table or computer output)

Step 5 — Conclusion: "Since $P < 0.001 < \alpha = 0.05$, we reject $H_0$. There is convincing evidence of a linear relationship between tree diameter and tree height."

🔑 AP Tip: Always state "convincing evidence" (not "proof") and reference the context.

T-Test for Slope Concepts 🎯

Calculating the $t$-Statistic 🧮

1) $b = 3.6$, $\text{SE}_b = 1.2$. What is $t$?

2) $n = 30$ data points. What are the degrees of freedom?

3) $b = -0.45$, $\text{SE}_b = 0.15$. What is $|t|$?

Conditions and Conclusions 🔍

Exit Quiz — $t$-Test for Slope ✅

📊 Confidence Interval for Slope

Part 3 of 7 — Estimating the True Slope

Topics in This Part

🔑 Key Concept: A confidence interval for $\beta$ gives a range of plausible values for the true population slope.

The Formula

$\boxed{b \pm t^* \cdot \text{SE}_b}$

where:

• $b$ = sample slope
• $t^*$ = critical value from a $t$-distribution with df $= n - 2$
• $\text{SE}_b$ = standard error of the slope (from computer output)

Interpretation Template

$\text{"We are [C]\% confident that the true slope of the linear relationship}$ $\text{between [x context] and [y context] is between [lower] and [upper]."}$

Example: 95% CI for $\beta$: $(1.52, 3.18)$, $x$ = diameter (in), $y$ = height (ft).

✅ "We are 95% confident that the true increase in height per additional inch of diameter is between 1.52 and 3.18 feet."

Connection to Hypothesis Testing

If the confidence interval does not contain 0, there is evidence of a linear relationship.

Conditions

Same LINE conditions as the $t$-test:

• Linear relationship
• Independent observations
• Normal residuals
• Equal variance of residuals

Worked Example

Study: $n = 15$ students. $x$ = hours using phone/day, $y$ = GPA.

Computer output: $b = -0.12$, $\text{SE}_b = 0.04$

Build a 95% CI: df $= 15 - 2 = 13$, so $t^* = 2.160$.

$-0.12 \pm 2.160(0.04) = -0.12 \pm 0.0864$

$(-0.2064, -0.0336)$

Interpretation: "We are 95% confident that the true slope of the relationship between daily phone use and GPA is between $-0.206$ and $-0.034$. For each additional hour of daily phone use, GPA is predicted to decrease by between 0.034 and 0.206 points."

Connection to test: Since 0 is NOT in the interval, we would reject $H_0: \beta = 0$ at $\alpha = 0.05$.

⚠️ AP Tip: You can read $b$ and $\text{SE}_b$ directly from computer output. The AP formula sheet provides the CI formula.

CI for Slope Concepts 🎯

Building a CI 🧮

$b = 4.5$, $\text{SE}_b = 1.5$, $t^* = 2.101$ (95%, df $= 18$)

1) Margin of error $= t^* \times \text{SE}_b =$

2) Lower bound of the CI $=$

3) Upper bound of the CI $=$

Interpretation Practice 🔍

Exit Quiz — CI for Slope ✅

📊 Computer Output Interpretation

Part 4 of 7 — Reading Regression Output Like a Pro

Topics in This Part

🔑 Key Concept: The AP exam always provides computer output. You must know where to find each number and what it means.

Standard Computer Output Table

Below the table: $S = s_e \quad R\text{-}sq = r^2 \quad R\text{-}sq(\text{adj}) = r^2_{\text{adj}}$

What Each Value Means

⚠️ Important: The $P$-value in the table tests $H_0: \beta = 0$ vs. $H_a: \beta \neq 0$ (two-sided). For a one-sided test, divide by 2.

Worked Example — Reading Output

$S = 4.12 \quad R\text{-}sq = 76.3\% \quad R\text{-}sq(\text{adj}) = 74.8\%$

From this output:

• LSRL: $\hat{y} = 15.8 + 2.45x$
• Slope: For each additional study hour, predicted score increases by 2.45 points
• $t = 2.45 / 0.38 = 6.45$ ✓ (matches output)
• $P < 0.001$ → reject $H_0: \beta = 0$ (strong evidence of a linear relationship)
• $R^2 = 76.3\%$ → 76.3% of variability in scores is explained by study hours
• $S = 4.12$ → typical prediction error is about 4.12 points

Building a CI from Output

Using the same output with $n = 22$:

• df $= 22 - 2 = 20$, $t^* = 2.086$ (95%)
• CI: $2.45 \pm 2.086(0.38) = 2.45 \pm 0.793 = (1.657, 3.243)$

🔑 AP Tip: Verify: the $P$-value in the table is for the two-sided test. The CI and test should agree — if 0 is not in the CI, the $P$-value should be $< \alpha$.

Common Mistakes

Reading Output 🎯

Extracting Values from Output 🧮

$S = 2.80 \quad R\text{-}sq = 83.0\%$, $n = 20$

1) What is the $t$-statistic for the slope?

2) What is the LSRL equation? (Write the slope value only)

3) Degrees of freedom $=$

Output Interpretation 🔍

Exit Quiz — Computer Output ✅

📊 Prediction Intervals

Part 5 of 7 — Predicting Individual Values vs. Mean Responses

Topics in This Part

🔑 Key Concept: A prediction interval for a single future observation is always wider than a confidence interval for the mean response at the same $x$-value.

Two Types of Intervals at a Given $x^*$

1. Confidence Interval for Mean Response $\mu_{y|x^*}$

• Estimates the average $y$-value for all individuals with $x = x^*$
• Variability comes only from estimating the line (uncertainty in $a$ and $b$)

2. Prediction Interval for Individual Response $y_{\text{new}}$

• Predicts a single new $y$-value when $x = x^*$
• Variability comes from estimating the line AND the natural scatter of individuals around the line

Formulas (Conceptual)

Both center on $\hat{y} = a + bx^*$, but the standard errors differ:

$\text{CI for mean: } \hat{y} \pm t^* \cdot \text{SE}_{\hat{\mu}}$

$\text{PI for individual: } \hat{y} \pm t^* \cdot \text{SE}_{\text{pred}}$

where $\text{SE}_{\text{pred}} > \text{SE}_{\hat{\mu}}$ because:

$\text{SE}_{\text{pred}}^2 = \text{SE}_{\hat{\mu}}^2 + S^2$

The extra $S^2$ accounts for the individual-to-individual scatter.

⚠️ AP Note: The formulas for these SEs are not on the AP formula sheet. You should understand the concept — why prediction intervals are wider — but you will not be asked to compute them by hand on the AP exam.

Visual Intuition

At a given $x^*$:

$\underbrace{\text{Narrow band}}_{\text{CI for mean}} \quad \subset \quad \underbrace{\text{Wide band}}_{\text{Prediction interval}}$

Both intervals are narrowest near $\bar{x}$ and widen as $x^*$ moves away from $\bar{x}$.

Why Wider at Extreme $x$?

The further $x^*$ is from $\bar{x}$:

• More uncertainty in where the true line is → wider CI for mean
• Same extra scatter for individuals → prediction interval grows similarly
• At the extremes of the data, both intervals are widest

This is related to the concept of extrapolation — predicting outside the data range is unreliable because both intervals become very wide.

Summary Comparison

🔑 Key Insight: Even with infinite data, a prediction interval never shrinks to zero width because individual variability ($S$) always remains.

Prediction Interval Concepts 🎯

Conceptual Calculations 🧮

$\hat{y} = 50$ at $x^* = 10$. The CI for the mean is $(47, 53)$.

1) What is the margin of error of the CI for the mean?

2) Would the prediction interval at $x^* = 10$ be narrower or wider? (narrower/wider)

3) If $x^* = \bar{x}$, the intervals are at their ___ width. (narrowest/widest)

Applying the Concepts 🔍

Exit Quiz — Prediction Intervals ✅

📊 Problem-Solving Workshop

Part 6 of 7 — Full Inference for Regression Problems

Workshop Goals

🔑 AP Tip: Inference for regression is one of the most commonly tested topics on the AP exam. Master the 4-step process: hypotheses → conditions → mechanics → conclusion.

Worked Example 1 — Chirps and Temperature

A biology student records cricket chirps per minute ($x$) and outdoor temperature ($y$, °F) for 15 observations.

Computer output:

$S = 3.83 \quad R\text{-}sq = 71.9\%$

Step 1 — Hypotheses: $H_0: \beta = 0$ (no linear relationship between chirp rate and temperature) $H_a: \beta \neq 0$ (there is a linear relationship)

Step 2 — Conditions (LINE):

• L: Scatterplot shows a linear pattern; residual plot shows random scatter ✓
• I: Observations taken on different days; 15 < 10% of all possible days ✓
• N: Histogram of residuals is approximately Normal ✓
• E: Residual plot shows constant spread ✓

Step 3 — Mechanics: $t = b/\text{SE}_b = 3.29/0.57 = 5.77$, df $= 15 - 2 = 13$ $P < 0.001$

Step 4 — Conclusion: "Since $P < 0.001 < \alpha = 0.05$, we reject $H_0$. There is convincing evidence of a linear relationship between cricket chirps per minute and outdoor temperature."

95% CI for slope: df $= 13$, $t^* = 2.160$ $3.29 \pm 2.160(0.57) = 3.29 \pm 1.231 = (2.059, 4.521)$ "We are 95% confident that for each additional chirp per minute, the true increase in temperature is between 2.06°F and 4.52°F."

Worked Example 2 — Fertilizer and Yield

An agronomist tests 25 plots. $x$ = fertilizer (kg/hectare), $y$ = crop yield (tons/hectare).

$S = 0.45 \quad R\text{-}sq = 12.8\%$

Analysis at $\alpha = 0.05$:

• $t = 0.035/0.019 = 1.84$, df $= 23$
• $P = 0.078 > 0.05$
• Conclusion: "We fail to reject $H_0$. There is not convincing evidence of a linear relationship between fertilizer amount and crop yield."
• $R^2 = 12.8\%$ — fertilizer explains very little of the variability in yield

95% CI: $t^* = 2.069$ (df $= 23$) $0.035 \pm 2.069(0.019) = 0.035 \pm 0.039 = (-0.004, 0.074)$ The interval contains 0, consistent with failing to reject $H_0$.

Common AP Mistakes

Inference Workshop Practice 🎯

Practice from Output 🧮

$n = 32$

1) What is $t$ for the slope?

2) What is df?

3) 95% CI margin of error if $t^* = 2.042$: $t^* \times \text{SE}_b =$

Decision Making 🔍

Exit Quiz — Inference Workshop ✅

📊 Review & Applications

Part 7 of 7 — Comprehensive Inference for Regression Review

Complete Formula Reference

LINE Conditions Summary

Interpretation Templates (AP Exam Ready)

$t$-Test Conclusion (Reject): "Since $P = [\text{value}] < \alpha = [\text{level}]$, we reject $H_0$. There is convincing evidence of a linear relationship between [x in context] and [y in context]."

$t$-Test Conclusion (Fail to Reject): "Since $P = [\text{value}] > \alpha = [\text{level}]$, we fail to reject $H_0$. There is not convincing evidence of a linear relationship between [x in context] and [y in context]."

CI for Slope: "We are [C]% confident that the true slope is between [lower] and [upper]. For each additional [unit of x], [y in context] changes by between [lower] and [upper] [units of y]."

Key Concept Connections

Decision Flowchart

$\text{Read computer output} \to \text{Identify } b, \text{SE}_b, t, P, S, R^2$ $\downarrow$ $\text{State } H_0: \beta = 0 \text{ and } H_a$ $\downarrow$ $\text{Check LINE conditions}$ $\downarrow$ $\text{Report } t = b/\text{SE}_b, \text{df} = n-2, P\text{-value}$ $\downarrow$ $\text{Compare } P \text{ to } \alpha \to \text{Reject or Fail to Reject}$ $\downarrow$ $\text{State conclusion in context}$

🔑 AP Exam Strategy: Inference for regression appears on the AP exam nearly every year, often as a full free-response question. The 4-step process is your blueprint for full credit.

Comprehensive Review 🎯

Mixed Review Calculations 🧮

$n = 27$, $R\text{-}sq = 48.0\%$

1) $t$-statistic for the slope $=$

2) df $=$

3) 95% CI lower bound if $t^* = 2.060$: $b - t^* \cdot \text{SE}_b =$

Concept Connections 🔍

Final Exam — Inference for Regression ✅