Lab 23 · Lecture 23 · Least Squares and Linear Regression
Least Squares Lab
Fit straight lines to your own data, see the residuals, and solve multiple regression problems through the normal equations.
Type the x and y values (spaces or commas). The table lists fitted values and residuals; the note gives S, R² and the prediction.
Solve AᵀA c = Aᵀy for Problem 2 (house prices). AᵀA is symmetric positive definite, so Cholesky works; try Gaussian elimination too.
Straight line
Try this
Multiple regression
Try this
Q1 What is the principle of least squares?
Choose the parameters of the model so that the sum of the squares of the residuals is as small as possible.
Q2 Write the normal equations for fitting .
and .
Q3 How does least squares differ from interpolation?
Interpolation passes through every data point (as many coefficients as points); least squares uses fewer coefficients than points and passes close to them, smoothing the noise.
Q4 What does mean?
95% of the variation of about its mean is explained by the model; it does not prove that the model is correct, so look at the residuals too.
Q5 Why can solving the normal equations be inaccurate?
cond() = cond()², so nearly dependent columns (e.g. x = years) lose about twice as many digits; centre the data or use QR.