Example 1: The least squares method
Find the least squares solution for the system
x1 - x2 = 2
x1 + x2 = 4
2 x1 + x2 = 8

Solution:

The coefficient matrix is

A = 1 -1
1 1
2 1
The least squares solution satisfies ATA = AT.

We have

ATA = 1 1 2
-1 1 1
1 -1
1 1
2 1
= 6 2
2 3
and
AT = 1 1 2
-1 1 1
2
4
8
= 22
10
so that
6 2
2 3
= 22
10
We use the Gaussian elimination to solve :
6 2 22
2 3 10
~ 2 3 10
0 -7 -8
~ 1 0 23/7
0 1 8/7
-Add the 2nd row multiplied by -3 to the 1st row
-Interchange the first two rows
-Add the 2nd row multiplied by 3/7 to the 1st row
-Divide the 1st row by 2
-Divide the 2nd row by -7
-We have:
x1 = 23/7
x2 = 8/7
We now know the least squares solution .

The residual vector (the error) is

= - A = 2
4
8
- 1 -1
1 1
2 1
23/7
8/7
= -1/7
-3/7
2/7
and ||||22 = 2/7
Go back to theory