View
234
Download
2
Embed Size (px)
Citation preview
Gauss EliminationGauss Elimination
9.1 Solving small numbers of equations 9.2 Naive Gauss Elimination 9.3 Pivoting 9.4 Tridiagonal Systems
MATLAB M-files GaussNaive, GaussPivot, Tridiag
Small MatricesSmall Matrices
For small numbers of equations, can be solved by hand
Graphical Cramer's rule Elimination
nnnn22n11n
2nn2222121
1nn1212111
bxaxaxa
bxaxaxa
b xaxaxa
Graphical MethodGraphical Method
12
12
21
21
x3x3x2x
rearrange 3xx
3xx2
2x1 – x2 = 3x1 + x2 = 3
One solution
Cramer’s RuleCramer’s RuleCompute the determinant D2 x 2 matrix
3 x 3 matrix
211222112221
1211 aaaaaa
aaD
3231
222113
3331
232112
3332
232211
333231
232221
131211
aa
aaa
aa
aaa
aa
aaa
aaa
aaa
aaa
D
Cramer’s RuleCramer’s RuleTo find xk for the following system
Replace kth column of as with bs (i.e., aik bi ) nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxabxa...xaxa
)
)(
ijk D(a
matrix newDx
ExampleExample3 x 3 matrix
333231
232221
131211
aaa
aaa
aaa
D
33231
22221
11211
33
33331
23221
13111
22
33323
23222
13121
11
baa
baa
baa
D
1
D
Dx
aba
aba
aba
D
1
D
Dx
aab
aab
aab
D
1
D
Dx
Ill-Conditioned SystemIll-Conditioned System
What happen if the determinant D is very small or zero?
Divided by zero (linearly dependent system)Divided by a small number: Round-off errorLoss of significant digits
0AdetD
Eliminate x2
Subtract to get
2222121
1212111
bxaxa
bxaxa
2122221212112
1222122211122
baxaaxaa
baxaaxaa
aaaa
babax
aaaa
babax
babaxaaxaa
21122211
1212112
21121122
2121221
2121221211211122
Elimination MethodElimination Method
Not very practical for large number (> 4) of equations
MATLAB’s MethodsMATLAB’s Methods
Forward slash ( / )Back-slash ( \ )Multiplication by the inverse of the
quantity under the slash
b*Ainvx
bAxbAx
bAx1
)(
\
Gauss EliminationGauss Elimination
Manipulate equations to eliminate one of the unknownsDevelop algorithm to do this repeatedlyThe goal is to set up upper triangular matrix
Back substitution to find solution (root)
nn
n333
n22322
n1131211
a
aa
aaa
aaaa
U
Basic Gauss EliminationBasic Gauss Elimination
Direct Method (no iteration required)Forward eliminationColumn-by-column elimination of the
below-diagonal elementsReduce to upper triangular matrixBack-substitution
Naive Gauss EliminationNaive Gauss Elimination
Begin with
Multiply the first equation by a21 / a11 and subtract from
second equation
nnnn22n11n
2nn2222121
1nn1212111
bxa...xaxa
bxa...xaxa
bxa...xaxa
nnnn22n11n
111
212nn1
11
21n2212
11
2122111
11
2121
1nn212111
bxa...xaxa
ba
abxa
a
aa...xa
a
aaxa
a
aa
bxa...xaxa
Gauss EliminationGauss Elimination
Reduce to
Repeat the forward elimination to get
nnnn22n11n
2nn2222
1nn1212111
bxa...xaxa
bxa...xa
bxa...xaxa
nnnn22n
2nn2222
1nn1212111
bxa...xa
bxa...xa
bxa...xaxa
Gauss EliminationGauss Elimination
First equation is pivot equationa11 is pivot elementNow multiply second equation by a'32 /a'22
and subtract from third equation
222
323nn2
22
32n3323
22
3233
2nn2323222
1nn1313212111
ba
abxa
a
aaxa
a
aa
bxaxaxa
bxaxaxaxa
Gauss EliminationGauss EliminationRepeat the
elimination of
ai2 and get
Continue and get nnnn33n
3nn3333
2nn2323222
1nn1313212111
bxaxa
bxaxa
bxaxaxa
bxaxaxaxa
)()( 1nnn
1nnn
3nn3333
2nn2323222
1nn1313212111
bxa
bxaxa
bxaxaxa
bxaxaxaxa
Now we can perform back substitution to get {x}By simple division
Substitute this into (n-1)th equation
Solve for xn-1
Repeat the process to solve for xn-2 , xn-3 , …. x2, x1
)()(,
)(,
2n1nn
2nn1n1n
2n1n1n bxaxa
Back SubstitutionBack Substitution
)(
)(
1nnn
1nn
n a
bx
Back substitution: starting with xn
Solve for xn1 , xn2 , … , 3, 2, 1
Back SubstitutionBack Substitution
a
xab
x
a
bx
1iii
n
1ijj
1iij
1ii
i
1nnn
1nn
n
)(
)()(
)(
)(
for i = n1, n2, …, 1
0a 1iii )(
Naive Gauss EliminationNaive Gauss Elimination
Elimination of first columnElimination of first column
)()(
)()(
)()(
1f4
1f3
1f2
baaa0
baaa0
baaa0
baaaa
41
31
21
4444342
3343332
2242322
114131211
114141
113131
112121
444434241
334333231
224232221
114131211
aaf
aaf
aaf
baaaa
baaaa
baaaa
baaaa
/
/
/
Elimination of second columnElimination of second column
)()(
)()(
/
/
2f4
2f3
baa00
baa00
baaa0
baaaa
aaf
aaf
baaa0
baaa0
baaa0
baaaa
42
32
44443
33433
2242322
114131211
224242
223232
4444342
3343332
2242322
114131211
Elimination of third columnElimination of third column
Upper triangular matrix
)()(
/
3f4
ba000
baa00
baaa0
baaaa
aaf
aaa00
aaa00
baaa0
baaaa
43444
33433
2242322
114131211
33434344443
33433
2242322
114131211
1141431321211
2242432322
3343433
4444
a/)xaxaxab(x
a/)xaxab(x
a/)xab(x
a/bx
Back-SubstitutionBack-Substitution
Upper triangular matrix
0a,a,a,a 44332211
ba000
baa00
baaa0
baaaa
444
33433
2242322
114131211
ExampleExample
41
31
21
41
31
21
f14
f13
f12
5141020
24110
00420
13201
6f
0f
1f
14226
24110
13221
13201
)()(
)()(
)()(
Forward EliminationForward Elimination
42
32
42
32
f24
f23
5141400
24100
00420
13201
1f
1/2f
5141020
24110
00420
13201
)()(
)()(
Upper Triangular MatrixUpper Triangular Matrix
43
43
f34
3370000
24100
00420
13201
14f
5141420
24110
00420
13201
)()(
Back-SubstitutionBack-Substitution
13/70x3x21x
8/35x2x
4/352x4x
33/707033/x
431
32
43
4
33/70
4/35
8/35
13/70
x
3370000
24100
00420
13201
Print all factor and Aug
(do not suppress output)Eliminate first column
Eliminate second column
Eliminate third column Back-substitution
Aug = [A, b]
>> format short>> x = GaussNaive(A,b)m = 4n = 4Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1factor = -1Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 0Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 6 2 2 4 1factor = 6Aug = 1 0 2 3 1 0 2 4 0 0 0 1 1 4 2 0 2 -10 -14 -5
factor = 0.5000Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 2 -10 -14 -5factor = 1Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 -14 -14 -5
factor = 14Aug = 1 0 2 3 1 0 2 4 0 0 0 0 -1 4 2 0 0 0 -70 -33
x4
x3
x2
x1
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x = -0.1857 0.2286 -0.1143 0.4714
Algorithm for Gauss eliminationAlgorithm for Gauss elimination1. Forward elimination for each equation j, j = 1 to n-1
for all equations k greater than j
(a) multiply equation j by akj /ajj
(b) subtract the result from equation kThis leads to an upper triangular matrix
2. Back-Substitution (a) determine xn from
(b) put xn into (n-1)th equation, solve for xn-1
(c) repeat from (b), moving back to n-2, n-3, etc. until all equations are solved
)1n(nn
)1n(nn a/bx
Operation CountOperation Count
Important as matrix gets large
For Gauss elimination
Elimination routine uses on the order of O(n3/3) operations
Back-substitution uses O(n2/2)
Operation CountOperation Count
))(())((,
))(())((,
))(())((,
))(())((,
3121nn1n
2knkn1knknn1kk
n2n1n2nn32
1n1nn1nn21
flops
iontion/DivisMultiplica
flops
ubtractionAddition/S
i
Loop Inner
k
Loop Outer
Total operation counts for elimination stage = 2n3/3 + O(n2)
Total operation counts for back substitution stage = n2 + O(n)
Operation CountOperation Count Number of flops (floating-point operations) for
Naive Gauss elimination
%....
%.
%.
859910676106861000000106761000
539866666768155010000671550100
588766780510070510
nEliminatio to
Due Percentage
3
2nFlops
Total
onSbustituti
Back
nElimination
868
3
Computation time increase rapidly with n Most of the effort is incurred in the elimination step Improve efficiency by reducing the elimination effort
Partial PivotingPartial Pivoting
Problems with Gauss elimination division by zero
round off errors
ill conditioned systems
Use “Pivoting” to avoid this Find the row with largest absolute coefficient below the
pivot element
Switch rows (“partial pivoting”)
complete pivoting switch columns also (rarely used)
Round-off ErrorsRound-off Errors
A lot of chopping with more than n3/3 operations
More important - error is propagated
For large systems (more than 100 equations), round-off error can be very important (machine dependent)
Ill conditioned systems - small changes in coefficients lead to large changes in solution
Round-off errors are especially important for ill-conditioned systems
Consider
Since slopes are almost equal
22
21
22
212
12
11
12
112
2222121
1212111
a
bx
a
ax
a
bx
a
ax
bxaxa
bxaxa
22
21
12
11
a
a
a
a
Ill-Conditioned SystemIll-Conditioned System
0aa
aaD
2221
1211 Divided by small number
)1n(nn
n333
n22322
n1131211
a
aa
aaa
aaaa
U
)(detdet 1nnn332211 aaaaUA
DeterminantDeterminant
Calculate determinant using Gauss elimination
Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting
Forward elimination
for each equation j, j = 1 to n-1
first scale each equation k greater than j then pivot (switch rows) Now perform the elimination
(a) multiply equation j by akj /ajj
(b) subtract the result from equation
Partial (Row) PivotingPartial (Row) Pivoting
1x4x2x2x6
2x4xx
1x3x2x2x
1 x3x2x
4321
432
4321
431
14226
24110
13221
13201
bA
Forward EliminationForward Elimination
Interchange rows 1 & 4
41
31
21
41
31
21
f14
f13
f12
5/67/35/31/30
24110
5/67/37/37/30
14226
1/6f
0f
1/6f
13201
24110
13221
14226
)()(
)()(
)()(
Forward EliminationForward Elimination
No interchange required
42
32
42
32
f24
f23
5/72200
33/145000
5/67/37/37/30
14226
1/7f
3/7f
5/67/35/31/30
24110
5/67/37/37/30
14226
)()(
)()(
Back-SubstitutionBack-Substitution
Interchange rows 3 & 4
13/70/6x 2x 2x 41x
8/357/3/x 7/3x 7/35/6x
4/35/2x 25/7x
33/70/533/14x
2341
342
43
4
)(
)()(
)(
)(
33/70
4/35
8/35
13/70
x
0f
33/145000
5/72200
5/67/37/37/30
14226
43
MATLAB M-File: MATLAB M-File: GaussPivotGaussPivot
Partial Pivoting (switch rows)
[big,i] = max(x)
largest element in {x}
index of the largest element
Partial Pivoting
>> format short>> x=GaussPivot0(A,b)Aug = 1 0 2 3 1 -1 2 2 -3 -1 0 1 1 4 2 6 2 2 4 1big = 6i = 4ipr = 4Aug = 6 2 2 4 1 -1 2 2 -3 -1 0 1 1 4 2 1 0 2 3 1factor = -0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 1.0000 0 2.0000 3.0000 1.0000factor = 0.1667Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 1.0000 1.0000 4.0000 2.0000 0 -0.3333 1.6667 2.3333 0.8333
Aug = [A b]
Interchange rows 1 and 4
Find the first pivot element and its index
Eliminate first columnNo need to interchange
big = 2.3333i = 1ipr = 2factor = 0.4286Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 -0.3333 1.6667 2.3333 0.8333factor = -0.1429Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 0 5.0000 2.3571 0 0 2.0000 2.0000 0.7143big = 2i = 2ipr = 4Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571factor = 0Aug = 6.0000 2.0000 2.0000 4.0000 1.0000 0 2.3333 2.3333 -2.3333 -0.8333 0 0 2.0000 2.0000 0.7143 0 0 0 5.0000 2.3571
x = 0 0 0 0.4714x = 0 0 -0.1143 0.4714x = 0 0.2286 -0.1143 0.4714x =
-0.1857 0.2286 -0.1143 0.4714
Back substitutionSecond pivot element and index
Third pivot element and index
Eliminate second column
Eliminate third column
Interchange rows 3 and 4
No need to interchange
Save factors fij for
LU Decomposition
Statics: Force BalanceStatics: Force Balance
0FFFF
0FFF
0FFFF
0FFF
0FFF
0FVF
0FFFFF
100FFF
0FFHF
0FVF
4535255x
35255y
4524144x
24144y
35233x
3533y
252423122x
25242y
141211x
1411y
coscos
sinsin
coscos
sinsin
cos
sin
coscos
sinsin
sin
sin
,
,
,
,
,
,
,
,
,
,
Node 1
Node 2
Node 3
Node 4
Node 5
Example: Forces in a Simple TrussExample: Forces in a Simple Truss
0
0
0
0
0
0
0
100
0
0
F
F
F
F
F
F
F
V
H
V
1coscos0000000
0sinsin0000000
100cos0cos0000
000sin0sin0000
0cos00100000
0sin00000100
00coscos101000
00sinsin000000
00000cos1010
00000sin0001
45
35
25
24
23
14
12
3
1
1
function [A,b]=Truss(alpha,beta,gamma,delta)
A=zeros(10,10);A(1,1)=1; A(1,5)=sin(alpha);A(2,2)=1; A(2,4)=1; A(2,5)=cos(alpha);A(3,7)=sin(beta); A(3,8)=sin(gamma);A(4,4)=-1; A(4,6)=1; A(4,7)=-cos(beta); A(4,8)=cos(gamma);A(5,3)=1; A(5,9)=sin(gamma);A(6,6)=-1; A(6,9)=-cos(delta);A(7,5)=-sin(alpha); A(7,7)=-sin(beta);A(8,5)=-cos(alpha); A(8,7)=cos(beta); A(8,10)=1;A(9,8)=-sin(gamma); A(9,9)=-sin(delta);A(10,8)=-cos(gamma); A(10,9)=cos(delta); A(10,10)=-1;
b=zeros(10,1); b(3,1)=100;
Define Matrices A and b in script fileDefine Matrices A and b in script file
>> alpha=pi/6; beta=pi/3; gamma=pi/4; delta=pi/3;>> [A,b] = Truss(alpha,beta,gamma,delta)A = 1.0000 0 0 0 0.5000 0 0 0 0 0 0 1.0000 0 1.0000 0.8660 0 0 0 0 0 0 0 0 0 0 0 0.8660 0.7071 0 0 0 0 0 -1.0000 0 1.0000 -0.5000 0.7071 0 0 0 0 1.0000 0 0 0 0 0 0.7071 0 0 0 0 0 0 -1.0000 0 0 -0.5000 0 0 0 0 0 -0.5000 0 -0.8660 0 0 0 0 0 0 0 -0.8660 0 0.5000 0 0 1.0000 0 0 0 0 0 0 0 -0.7071 -0.8660 0 0 0 0 0 0 0 0 -0.7071 0.5000 -1.0000b = 0 0 100 0 0 0 0 0 0 0>> x = GaussPivot(A,b)x = 40.5827 0 48.5140 70.2914 -81.1655 34.3046 46.8609 84.0287 -68.6091 -93.7218
Gauss Elimination with Partial PivotingGauss Elimination with Partial Pivoting
Simple truss
•Banded Matrix
xw
vut
srq
pzk
nml
kji
hgf
edc
ba
0000000
000000
000000
000000
000000
000000
000000
000000
0000000ai,j= 0if j > i + HBor j< i - HB
HB: Half BandwidthB: Bandwidth
B = 2*HB + 1
In this exampleHB = 1 & B = 3
Tridiagonal MatrixTridiagonal Matrix
Only three nonzero elements in each equation (3n instead of n2 elements)
Subdiagonal, diagonal, superdiagonal
Row Scaling (not implemented in textbook)
-- scale the diagonal element to aii = 1
Solve by Gauss elimination
n
1n
i
3
2
1
n
1n
i
3
2
1
nn
1n1n1n
iii
333
222
11
r
r
r
r
r
r
x
x
x
x
x
x
fe
gfe
gfe
gfe
gfe
gf
Tridiagonal MatrixTridiagonal Matrix
Special case of banded matrix with bandwidth = 3
Save storage, 3 n instead of n n
Tridiagonal MatrixTridiagonal Matrix
Forward elimination
Back substitution
n32k
rf
err
gf
eff
1k1k
kkk
1k1k
kkk
,,,
1232n1nk f
xgrx
f
rx
k
1kkkk
n
nn
,,,,,
Use factor = ek / fk1
to eliminate subdiagonal element
Apply the same matrix operations to right hand side
Hand Calculations: Tridiagonal MatrixHand Calculations: Tridiagonal Matrix
53
2
5
3
x
x
x
x
2515000
50210
0152
0021
4
3
2
1
...
.
411
5053r
f
err
1501
50251g
f
eff
111
12r
f
err
111
12g
f
eff
131
25r
f
err
121
25g
f
eff
33
444
33
444
11
223
22
333
11
222
11
222
)(.
.
).(.
.
)(
)(
)(
)(
11
223
f
xgrx
21
311
f
xgrx
31
4501
f
xgrx
41
4
f
rx
1
2111
2
3222
3
4333
4
44
))((
))((
))(.(
(a) Forward elimination (b) Back substitution
» [e,f,g,r] = example
e = 0 -2.0000 4.0000 -0.5000 1.5000 -3.0000f = 1.0000 6.0000 9.0000 3.2500 1.7500 13.0000g = -2.0000 4.0000 -0.5000 1.5000 -3.0000 0r = -3.0000 22.0000 35.5000 -7.7500 4.0000 -33.0000
» x = Tridiag (e, f, g, r)x = 1 2 3 -1 -2 -3
Example: Tridiagonal matrixExample: Tridiagonal matrixfunction [e,f,g,r] = example
e=[ 0 -2 4 -0.5 1.5 -3];
f=[ 1 6 9 3.25 1.75 13];
g=[-2 4 -0.5 1.5 -3 0];
r=[-3 22 35.5 -7.75 4 -33];
33
4
75.7
5.35
22
3
x
x
x
x
x
x
133
375.15.1
5.125.35.0
5.094
462
21
6
5
4
3
2
1
Note: e(1) = 0 and g(n) = 0