Lab 24: Curve Fitting Lab · Numerical Methods

Choose the model; for a polynomial set the degree, for the trigonometric model the period. The table lists fitted values and residuals.

Ten noisy samples of sin 2πx. Move the degree; the right plot shows training and test error for every degree. Turn on ridge regularisation (λ) to tame high degrees.

Polynomial fit

Try this

Run it. Compare with Problem 1.
Set m = 1 and m = 3: how much does S change?
Use the data of Problem 4 and check a = 1.42, b = −1.07, c = 0.55.

Exponential fit

Try this

Run it. Compare with Problem 2.
Change the data to homework 2.
Change the program to fit a power law (use ln t; drop t = 0).

Q1 Write the normal equations for fitting

y = a + bx + cx^2

.

na + b\sum x + c\sum x^2 = \sum y

;

\;a\sum x + b\sum x^2 + c\sum x^3 = \sum xy

;

\;a\sum x^2 + b\sum x^3 + c\sum x^4 = \sum x^2y

.

Q2 How do you fit

y = ae^{bx}

by least squares?

Take logarithms:

\ln y = \ln a + bx

. Fit a straight line to the points

(x_i, \ln y_i)

; the slope is

b

and

a = e^{\text{intercept}}

.

Q3 How do you fit

y = ax^{b}

?

\ln y = \ln a + b\ln x

: fit a straight line to

(\ln x_i, \ln y_i)

.

Q4 Why is fitting

\ln y

not the same as fitting

y

?

It minimises the squared errors of

\ln y

, i.e. roughly relative errors, so small values of

y

get more weight; the result differs slightly from minimising

\sum (y_i - ae^{bx_i})^2

.

Q5 What is overfitting and how is it detected?

A model with too many parameters fits the noise of the training data: training error is tiny but error on new data is large. It is detected by measuring the error on data not used for fitting (validation or test data).