Operation Counts and MATLAB Code for Gaussian Elimination

Here we explain what the MATLAB command A\b actually does when A is an n by n nonsingular matrix and b is a given n-vector. We count the number of operations (additions, subtractions, multiplications, and divisions) required to solve the linear system Ax = b.

Step 1. If a11 = 0, exchange rows so that the (1,1) element is nonzero. For each row i = 2, ¼, n, replace row i by itself minus ai1/a11 times row 1:

æ
ç
ç
ç
ç
ç
ç
ç
è
a11
a12
a13
¼
a1n
|
b1
a21
a22
a23
¼
a2n
|
b2
a31
a32
a33
¼
a3n
|
b3
:
:
:
:
|
:
an1
an2
an3
¼
ann
|
bn
ö
÷
÷
÷
÷
÷
÷
÷
ø
® æ
ç
ç
ç
ç
ç
ç
ç
ç
ç
ç
è
a11
a12
a13
¼
a1n
|
b1
0
~
a
 

22 
~
a
 

23 
¼
~
a
 

2n 
|
~
b
 

2 
0
~
a
 

32 
~
a
 

33 
¼
~
a
 

3n 
|
~
b
 

3 
:
:
:
:
|
:
0
~
a
 

n2 
~
a
 

n3 
¼
~
a
 

nn 
|
~
b
 

n 
ö
÷
÷
÷
÷
÷
÷
÷
÷
÷
÷
ø

Operation Count:

n-1 divisions to compute the numbers ai1/a11; n(n-1) multiplications and n(n-1) subtractions to do the elimination. Total: approximately 2 n2 operations. [Note: We are interested in the operation count for large n. Hence it will not matter much whether it is 2 n2 or 2n(n-1). The coefficient of the largest term n2 is still 2.]

MATLAB code for this step (after rows have been exchanged so that the (1,1) element is nonzero):

% Eliminate first column using first row.
for i=2:n,
  mult = A(i,1)/A(1,1);
  for k=1:n,
    A(i,k) = A(i,k) - mult*A(1,k);
  end;
  b(i) = b(i) - mult*b(1);
end;

Note that the inner for loop can be replaced by the statement:

 
A(i,:) = A(i,:) - mult*A(1,:);   % This does the same thing as the inner 
                                 % loop above.

Step 2. If [a\tilde]22 = 0, exchange rows so that the (2,2) element is nonzero. For each row i = 3, ¼, n, replace row i by itself minus [a\tilde]i2/ [a\tilde]22 times row 2.

æ
ç
ç
ç
ç
ç
ç
ç
ç
ç
ç
è
a11
a12
a13
¼
a1n
|
b1
0
~
a
 

22 
~
a
 

23 
¼
~
a
 

2n 
|
~
b
 

2 
0
~
a
 

32 
~
a
 

33 
¼
~
a
 

3n 
|
~
b
 

3 
:
:
:
:
|
:
0
~
a
 

n2 
~
a
 

n3 
¼
~
a
 

nn 
|
~
b
 

n 
ö
÷
÷
÷
÷
÷
÷
÷
÷
÷
÷
ø
® æ
ç
ç
ç
ç
ç
ç
ç
ç
ç
ç
ç
ç
è
a11
a12
a13
¼
a1n
|
b1
0
~
a
 

22 
~
a
 

23 
¼
~
a
 

2n 
|
~
b
 

2 
0
0
~
~
a
 
 


33 
¼
~
~
a
 
 


3n 
|
~
~
b
 
 


3 
:
:
:
:
|
:
0
0
~
~
a
 
 


n3 
¼
~
~
a
 
 


nn 
|
~
~
b
 
 


n 
ö
÷
÷
÷
÷
÷
÷
÷
÷
÷
÷
÷
÷
ø

Operation Count: n-2 divisions to compute the numbers [a\tilde]i2/ [a\tilde]22; (n-1)(n-2) multiplications and (n-1)(n-2) subtractions to do the elimination. Total: approximately 2 (n-1)2 operations.

MATLAB code for this step (after rows have been exchanged so that the (2,2) element is nonzero):

for i=3:n,
  mult = A(i,2)/A(2,2);
  A(i,2:n) = A(i,2:n) - mult*A(2,2:n);  % Note that we work only with
                                        % columns 2 through n.
end;

Step 3. If [[a\tilde]\tilde]33 = 0, exchange rows so that the (3,3) element is nonzero. For each row i = 4, ¼, n, replace row i by itself minus [[a\tilde]\tilde]i3/ [[a\tilde]\tilde]33 times row 3.

Operation Count: 2( n-2 )2.

Etc.

Total Operation Count for Gaussian Elimination:

2[ n2 + (n-1 )2 + (n-2 )2 + ¼+ 12 ] = 2 n(n+1)(2n+1)
6
» 2 n3
3

MATLAB Code:

%  Given an n by n nonsingular matrix A and an n-vector b, does Gaussian
%  elimination (with pivoting) to bring A to upper triangular form.

for j=1:n-1,
  [cmax,jmax] = max(abs(A(j:n,j))); % Find index of maximum element in column j.
  temp = A(j,:);                    % Exchange rows j and j+jmax-1.
  A(j,:) = A(j+jmax-1,:); 
  A(j+jmax-1,:) = temp;
  tempb = b(j);                   % Also exchange elements j and j+jmax-1 of b.
  b(j) = b(j+jmax-1);
  b(j+jmax-1) = tempb;
  for i=j+1:n,                      % Eliminate in column j using row j.
    mult = A(i,j)/A(j,j);
    A(i,j:n) = A(i,j:n) - mult*A(j,j:n);
    b(i) = b(i) - mult*b(j);
  end;
end;

The matrix is now upper triangular. We still must backsolve to find the solution of the linear system:

æ
ç
ç
ç
ç
ç
è
u11
u12
¼
u1n
0
u22
¼
u2n
···
unn
ö
÷
÷
÷
÷
÷
ø
æ
ç
ç
ç
ç
ç
è
x1
x2
:
xn
ö
÷
÷
÷
÷
÷
ø
= æ
ç
ç
ç
ç
ç
è
f1
f2
:
fn
ö
÷
÷
÷
÷
÷
ø
.

For i = n,n-1, ¼, 1,

xi = ( fi - n
å
j = i+1 
uij xj ) / uii .
This requires approximately
2 n
å
i = 1 
( n-i+1 ) = 2 n(n+1)
2
» n2
This is lower order (n2) than the work required to get A into upper triangular form (n3).

The MATLAB code for backsolving is:

% Solve the linear system Ax=b once A has been reduced to upper triangular
% form.
x = zeros(n,1);      % Force x to be a column vector.
x(n) = b(n)/A(n,n);
for i=n-1:-1:1,
  x(i) = (b(i) - A(i,i+1:n)*x(i+1:n))/A(i,i);
end;


File translated from TEX by TTH, version 1.59.