Linear Regression (Line of Best Fit)

In four steps. First determine the means x̄ and ȳ. Second compute for each data point the deviation products (xᵢ − x̄)(yᵢ − ȳ) and the squares (xᵢ − x̄)². Third b = sum of products / sum of squares. Fourth a = ȳ − b·x̄. Example with (0|1), (1|3), (2|5): x̄ = 1, ȳ = 3; products: (−1)(−2) + 0 + (1)(2) = 4; squares: 1 + 0 + 1 = 2; so b = 2, a = 3 − 2 = 1 and ŷ = 2x + 1. Check: the line must pass through (x̄|ȳ), here 2·1 + 1 = 3 ✓. A table with columns for xᵢ, yᵢ, deviations and products keeps the calculation tidy.

r measures strength and direction of the linear relationship between two quantities and always lies between −1 and +1. r = +1 means all points lie exactly on a rising line; r = −1 exactly on a falling one. Values near 0 mean no linear relationship. Rough school reading: |r| from about 0.8 strong, around 0.5 moderate, below 0.3 weak. Two warnings: first, r measures only LINEAR coupling; a perfect parabola can yield r ≈ 0 although a clear relationship exists (look at the scatter plot!). Second, even r = 0.99 says nothing about cause and effect. Example: for (1|2), (2|3), (3|5), (4|6), r = 7/√50 ≈ 0.99, an almost perfect linear trend.

For three reasons. First the sign problem: positive and negative deviations would cancel each other when simply summed; a line could look perfect "on average" and still be far from all points. Squares are always positive. Second the weighting: squaring penalizes large outliers disproportionately, pulling the line towards points that would otherwise be missed badly. Third the mathematics: the sum of squares is differentiable, and setting the derivatives to zero yields unique closed formulas for a and b; with absolute values instead of squares there would be no such smooth solution. The price: single extreme outliers can tilt the line noticeably, so inspect the scatter plot first.

Within the data range (interpolation) yes, with judgement: for x-values between the observed data, ŷ = a + bx gives usable estimates when |r| is high. Example: ŷ = 1.4x + 0.5 from data with x from 1 to 4 may safely be evaluated at x = 3.5. Extrapolation far beyond is critical: the linear trend is only supported within the observed range; outside it the relationship may flatten, tip over or break entirely. A classic: children's growth data extended linearly to age 30 yields absurd body heights. Exams expect exactly this assessment: compute the prediction AND judge its reliability based on data range and r.

No, correlation is not causation. r only measures that two quantities vary together, not why. Often a third quantity is behind it (confounder): ice cream sales and sunburn cases correlate strongly, but the cause of both is sunny weather. The direction can also be unclear (does x influence y or vice versa?), and in small data sets high correlations even arise by chance. The sound approach: the regression describes the relationship and allows predictions; causal claims additionally require an experiment with controlled conditions or at least a plausible mechanism. In exam answers the sentence "a high r-value does not prove a causal relationship" is almost always a required assessment element.