CH 3073: Numerical Methods
CH 5500: Numerical Methods I
Autumn 2014-15
Instructor: Kishalay Mitra
Department of Chemical Engineering
Indian Institute of Technology Hyderabad
Tue / Thu: 11:30 AM โ 1 PM
Room # 132
Indian Institute of Technology Hyderabad
Overview
Topics
Linear Algebraic Equations (5 lecs)
Nonlinear Algebraic Equations (3 lecs)
Function Approximation (6 lecs)
Ordinary Differential EquationsโInitial Value Problems (ODE-IVPs)
(10 lecs)
Programming in MATLAB (10 classes)
Pre-requisites of the Course
BE / BTech in Chemical Engg / Chem Tech
Programming Skills (MATLAB)
Textbook for the Course
S. K. Gupta, Numerical Methods for Engineers, New Age Intl.
Publishers, New Delhi, 2nd Ed., 2009.
Indian Institute of Technology Hyderabad2
Examples
Topics
Linear Algebraic Equations
Nonlinear Algebraic Equations
Indian Institute of Technology Hyderabad3
Examples (contd.)
Topics
ODE - IVPs
ODE - BVPs
Indian Institute of Technology Hyderabad4
Evaluation
Indian Institute of Technology Hyderabad5
No. Type Final
Weightage
Tentative
Time
Remarks
1. Mid Semester Exam 30 End Sept. Absolute
Grading
2. End Semester Exam 30 End Nov. Absolute
Grading
3. Class Tests, Computer
Programming
15+15 Almost
each
class
Relative
Grading
4. Attendance & Class
Performance
10 85%
attendance
Total = 100
Programming Practices
Problems
Finding sum of a digit and reverse the order
In an array, store all the Fahrenheit values given Celsius values from -100C
to 100C
Sum the following series
Tabulate values of below function for integer values of k = 0 to 15
Use of subroutines (preferably)
Solution of a quadratic equation
Write program to evaluate any nth order polynomial
Fitting a straight line through number of given points
Write program to generate Fibonacci sequence
Given a sequence of numbers, sort them in ascending and descending order
Write programs to multiply two matrices taking their sizes, values of elements
as inputs
Indian Institute of Technology Hyderabad6
!12
1...!7!5!3
12753
n
xxxxxSum
nn
!/ kaeSum ka
Programming
Programming Review
EVALUATE S = 1 + 2 + 3 + โฆโฆโฆ. + n
and COMPARE WITH analytical results
S = n(n + 1)/2
GET FAMILIAR WITH C, FORTRAN,
MATLABยฎ
PLOT S vs. n
GET FAMILIAR WITH MATLABยฎ, MS
Excel
Indian Institute of Technology Hyderabad7
Chapter 1
LINEAR ALGEBRIC EQUATIONS
WHEN DO WE HAVE SOLUTIONS
GAUSS ELIMINATION
THOMASโ ALGORITHM
LU DECOMP
GAUSS โ SEIDEL, SOR, ETC.
Indian Institute of Technology Hyderabad8
Representing MENU
SET OF M EQUATIONS IN N UNKNOWNS
SIMULTANEOUS LINEAR ALGEBRAIC
EQUATIONS
a11x1 + a12 x2 + โฆโฆโฆโฆ..+ a1N xN = b1
a21x1 + a22 x2 + โฆโฆโฆโฆ..+ a2N xN = b2
.
.
.
.
aM1x1 + aM2 x2 + โฆโฆโฆโฆ..+ aMN xN = bM
e.g. x1 + x2 = 4
2x1 + 3x2 = 11
PDE ODE NLE SLE
Indian Institute of Technology Hyderabad9
Cramerโs Rule
Example 1.1
X1 + X2 = 4
2X1 + 3X2 = 11
Indian Institute of Technology Hyderabad10
Unique solution exists in case det(A) is nonsingular
Additionally, if b = 0, |Aj| = 0
all xj = 0 (trivial solution)
In case A is singular
First case: Identical equations
1 equation 2 unknown โ Infinite solutions
Second case: Incompatible and no solution exist
Reviewing Rank r
(AUGMENTED A โก Aug A)
Aug A โก [ A b]
One highest possible size non zero determinant
โด r = 2
โด r = 1
Indian Institute of Technology Hyderabad
1 1 4
2 3 11
1 10
2 3
11
Solution exists
IF and ONLY IF
RANK A = RANK (Aug A) COMPATABLE EQUATIONS
FOR M = N (FOR CHEMICAL ENGG. PROBLEMS)
IF r (A) = N UNIQUE SOLUTION
IF r (A) < N INFINITE SOLNS Assume N-r variables
Indian Institute of Technology Hyderabad12
Example
Example 1.3
X1 + X2 = 4
2 X1 + 3 X2 = 11
2 X1 + 3 X2 = 11
4 X1 + 6 X2 = 22
2 X1 + 3 X2 = 11
4 X1 + 6 X2 = 20
Indian Institute of Technology Hyderabad13
Homogeneous Case
ANxN xNx1 = bNx1
IF
Homogeneous equations
a11x1 + a12x2 = 0
a22x1 + a22x2 = 0; AND r(A) = 2 = N
Unique Solution Trivial solution
NON-TRIVIAL SOLUTIONS IF r(A) < N
Indian Institute of Technology Hyderabad14
4 2
Homogeneous Case (contd.)
NON-TRIVIAL SOLUTIONS IF r(A) < N
r(A) = 1
N โ r = 2 โ 1 = 1
ASSUME x1 = 5; THEN x2 = -10
x1 = 10; THEN x2 = -20
Indian Institute of Technology Hyderabad15
Review
Indian Institute of Technology Hyderabad16
๐ ๐ฑ = ๐ : FOCUS ON ๐ ๐ ร ๐ ONLY
SOLUTIONS ONLY IF RANK ๐ = ๐๐๐๐ ๐ โฎ ๐ โก ๐๐๐๐ ๐๐๐ ๐
IF RANK ๐ = ๐, UNIQUE SOLUTION
IF RANK ๐ < ๐, MANY SOLUTIONS
HOMOGENEOUS EQUATIONS (b = 0)
TRIVIAL SOLUTION x = 0, IF RANK ๐ = ๐
NONTRIVIAL SOLUTION, IF RANK ๐ < ๐
Direct Methods
Indian Institute of Technology Hyderabad17
๐๐๐ฎ๐ฌ๐ฌ ๐๐ฅ๐ข๐ฆ๐ข๐ง๐๐ญ๐ข๐จ๐ง (๐๐๐๐)
A x = b
RANK ๐ = ๐
Cramerโs Rule: Excessive Calculations (N2 ร N!)
Another route: A-1b
Equally expensive and suffers from additional problem of sensitivity
(round off errors)
0
G E Example
Indian Institute of Technology Hyderabad18
Example 1.4
0
Direct Methods
Indian Institute of Technology Hyderabad
PIVOT
19
๐๐๐ฎ๐ฌ๐ฌ ๐๐ฅ๐ข๐ฆ๐ข๐ง๐๐ญ๐ข๐จ๐ง (๐๐๐๐)
A x = b
RANK ๐ = ๐
Cramerโs Rule: Excessive
Calculaitons
G E (contd.)
Indian Institute of Technology Hyderabad
PIVOT
20
G E (contd.)
Indian Institute of Technology Hyderabad21
Product of diagonal terms gives you the value of determinant
Computational Complexity
Indian Institute of Technology Hyderabad22
Pivoting
Indian Institute of Technology Hyderabad
NOTE
23
Which one to prefer โ row or column shift? Which row / column to
shift? One that gives you largest absolute value in the pivot position
Process ends when zero cannot be removed from pivot position and
this occurs when set of equations does not have unique solutions
G E Example (Pivoting)
Indian Institute of Technology Hyderabad24
Example 1.5
Pivoting Practice
Indian Institute of Technology Hyderabad
25
Get correct result with pivoting and wrong result
without pivoting
Thomasโ Algorithm
Indian Institute of Technology Hyderabad
๐๐ ๐๐ ๐ ๐ ๐ ๐ โฆ ๐ ๐๐๐ ๐๐ ๐๐ ๐ ๐ ๐ โฆ ๐ ๐๐ ๐๐ ๐๐ ๐๐ ๐ ๐ โฆ ๐ ๐๐ ๐ ๐ ๐ ๐ ๐ โฆ ๐ ๐โฎ โฎ โฎ โฎ โฎ โฎ โฎ โฎ โฎ๐ ๐ ๐ ๐ ๐ โฆ ๐๐โ๐ ๐๐โ๐ ๐๐โ๐๐ ๐ ๐ ๐ ๐ โฆ ๐ ๐๐ ๐๐
๐ฑ๐๐ฑ๐๐ฑ๐๐ฑ๐โฎ๐ฑ๐โ๐๐ฑ๐
=
๐๐๐๐๐๐๐๐โฎ๐๐โ๐๐๐
26
Finite Difference BVP
Stage operations
Thomasโ Algorithm (contd.)
Indian Institute of Technology Hyderabad
27
LU Decomposition (Factorization)
Indian Institute of Technology Hyderabad
( )
( )
( ) ( )
( ) ( )
( )( ),
( ) ( )
,
. .
. .
.
. . . . . . .
. . . . . . .
. . . . . . .
. . . .
1
21
1
11
1 2
31 32
1 2
11 22
N 11N N 1N1
1 N 1
11 N 1 N 1
1 0 0 0 0
a1 0 0 0
a
a a1 0 0 0
a aL
aa1
a a
28
Cases where A does not
change but b changes
Multiplying factors are multiplied with the pivot row and added
with the other rows โ we take โve of multiplying factors
LU Decomp (contd.)
Indian Institute of Technology Hyderabad
We can show
A = L U; A x = L (U x) = b
L y = b SOLVE BY FORWARD /DOWNWARD SWEEP
โข Remember this b is original b, not the modified b as in case of
Gauss Elimination
U x = y THEN SOLVE BY REVERSE SWEEP
29
( )
( )
( ) ( )
( ) ( )
( )( ),
( ) ( )
,
. .
. .
.
. . . . . . .
. . . . . . .
. . . . . . .
. . . .
1
21
1
11
1 2
31 32
1 2
11 22
N 11N N 1N1
1 N 1
11 N 1 N 1
1 0 0 0 0
a1 0 0 0
a
a a1 0 0 0
a aL
aa1
a a
LU Decomp Example
Indian Institute of Technology Hyderabad30
Example 1.8
Solution: [-11 5 4]
Ly=b
Ux=y
LU Decomp Example
Indian Institute of Technology Hyderabad31
By R2 โ (1/7) R1 &
R3 โ (2/7) R1
35
14
15
x
x
x
932
151
217
3
2
1
15)7
2(35
15)7
1(14
15
x
x
x
2)7
2(91)
7
2(37)
7
2(2
2)7
1(11)
7
1(57)
7
1(1
217
3
2
1
7
2157
8315
x
x
x
7
59
7
190
7
5
7
340
217
3
2
1
M21 = (1/7)
M31 = (2/7)
LU Decomp Example
Indian Institute of Technology Hyderabad32
By R3 โ (19/34) R2
7
83
34
19
7
2157
8315
x
x
x
7
5
34
19
7
5900
7
5
7
340
217
3
2
1
7
2157
8315
x
x
x
7
59
7
190
7
5
7
340
217
3
2
1
M21 = (1/7)
M31 = (2/7)
M32 = (19/34)
238
191100
7
5
7
340
217
U
134/197/2
017/1
001
L
LU Decomp Example
Indian Institute of Technology Hyderabad33
238
57337
8315
x
x
x
238
191100
7
5
7
340
217
3
2
1 By GE, x1 = 1, x2 = 2, x3 = 3
35
14
15
y
y
y
134/197/2
017/1
001
Ly
3
2
1
By Forward Sweep, y1 =
15, y2 = 83/7, y3 = 5733/238
238
57337
8315
x
x
x
238
191100
7
5
7
340
217
Ux
3
2
1
By Backward Sweep, x1 =
1, x2 = 2, x3 = 3
Ly=b
Ux=y
LU โฆ Different b
Indian Institute of Technology Hyderabad34
77
35
45
y
y
y
134/197/2
017/1
001
Ly
3
2
1
By Forward Sweep, y1 =
45, y2 = 200/7, y3 =
11466/238
238
114667
20045
x
x
x
238
191100
7
5
7
340
217
Ux
3
2
1
By Backward Sweep, x1 =
4, x2 = 5, x3 = 6
LU Factorization
Indian Institute of Technology Hyderabad35
33
2322
131211
333231
2221
11
U00
UU0
UUU
LLL
0LL
00L
LU
932
151
217
A
7
5U1ULUL
7
34U5ULUL
;7
1L1UL
238
1911U9ULULUL;2U2UL
34
19L3ULUL;1U1UL
7
2L;2UL;7U7UL
1LLL
2323221321
2222221221
211121
33333323321331131311
3222321231121211
311131111111
332211
Dooli
ttle
โs M
eth
od
Row Column
multiplication:
9 equations,
12 unknowns โ
3 to be assumed
238/191100
7/57/340
217
134/197/2
017/1
001
A
U โ same as GE
LU Factorization (contd.)
Indian Institute of Technology Hyderabad36
33
2322
131211
333231
2221
11
U00
UU0
UUU
LLL
0LL
00L
LU
932
151
217
A
34
5U1ULUL
7
34L5ULUL
;1L1UL
238
1911L9ULULUL;
7
2U2UL
7
19L3ULUL;
7
1U1UL
2L;2UL;7L7UL
1UUU
2323221321
2222221221
211121
33333323321331131311
3222321231121211
311131111111
332211
Cro
utโ
sM
eth
od
Row Column
Multiplication:
9 equations,
12 unknowns โ
3 to be assumed
100
34/510
7/27/11
238/19117/192
07/341
007
A
LU Factorization (contd.)
Indian Institute of Technology Hyderabad37
T
33
2322
131211
332313
2212
11
UL
U00
UU0
UUU
UUU
0UU
00U
LU
210
121
012
A
Choleskyโs algorithm for symmetric positive definite A:
Take only the +ve roots
Twice as fast as GE
Row Column multiplication: 9 equations, 12 unknowns โ 3 to be assumed
3/200
3/22/30
02/12
3/23/20
02/32/1
002
A
LU Decomp (contd.)
Indian Institute of Technology Hyderabad
Benefit of LU Decomposition:
Solving Ax = b when only b is changing as in Finite difference
technique of ODE-BVP
FOR ONE A AND M bs:
# OF MULTIPLICATIONS/ DIVISIONS =
Substantially lower than GE being solved repetitively
WORKS IF A IS DIAGONALLY DOMINANT
|๐๐ข๐ข| โฅ
๐ฃ=๐;๐ฃโ ๐ข
๐
|๐๐ข๐ฃ|
OTHER LU DECOMPS: CHOLESKY PROB. 1.10 &
DOOLITTLE/CROUT PROB. 1.9
38
Gauss Jordon
Indian Institute of Technology Hyderabad
A, b & I (unit diagonal matrix) are written together
ELIMINATE TERMS IN A, BOTH BELOW AND ABOVE THE
DIAGONAL to convert it into I โ in turn, I is converted into A-1 & b
is converted into solution i.e. the values of x
Twice as time consuming as GE
39
GJ Example
Indian Institute of Technology Hyderabad40
Example 1.9
R2-R1, R3-2R1
R2/2
R2+R1
R3/-1
(1/2)R3+R2, (-3/2)R3+R1
Iterative Techniques
Indian Institute of Technology Hyderabad
GAUSS โ JACOBI / GAUSS โ SIEDEL (G S) / SOR (SUCCESSIVE OVER
RELAXATION)
11
k
NN1
k
313
k
21211k
1a
xaxaxabx
)()()()( .........
22
)k(
NN124
)k(
323
)1k(
1212)1k(
2a
xa.........axaxabx
NN
k
NNN
k
N
k
NNk
Na
xaxaxabx
)1(
11
)1(
22
)1(
11)1(.........
GUESS x1(1), x2
(1), โฆ, xN(1)
A x = b: a11x1 + a12x2 + a13x3 + --- + a1NxN = b1
41
For G-Jacobi, k
For G-Siedel,
k+1
Iterative Techniques (contd.)
Indian Institute of Technology Hyderabad
i.e., IF A IS DIAGONALLY DOMINANT (DD), WILL CONVERGE (IF A IS NOT
DD, MAY OR MAY NOT CONVERGE) โ sufficient, not necessary condition
# MULTIPLICATIONS + DIVISIONS = MN2 (M = # OF ITERATIONS)
CONVERGENCE CRITERION:
MOST COMMON:
TOL: USER SPECIFIED
42
GS Example
Indian Institute of Technology Hyderabad43
Example 1.10
Iterative Techniques (contd.)
Indian Institute of Technology Hyderabad
Solved Example: 1.11 โ Read
Reference: Press, Teukolsky, Vettering & Flannery - NUMERICAL RECIPES
Assignment: 1, 2, 3, 4, 6, 8, 10
Lab Assignment: Write a general program for G-Jacobi, G-Siedel, SOR to solve 13,
15, 18 (Keep a copy of your programs โ we will use them later in the course)
0 xi(k) xi,GS
(k+1) xi,SOR(k+1)
a a/2
SCHEMATICALLY
( ) ( ); , ,...., ( )
,k k 1
X X i 1 2 N ALLi i GS
( ) ( ) ( ) ( )
, , ; , ,....,k 1 k k 1 k
i SOR i i GS ix x w x x i 1 2 N
44
w > 1 over relaxation,
w < 1 under relaxation,
w = 1 same as GS;
w typically in between 1 & 2, but no rule
Purpose is to speed up the process,
hence we generally go for w > 1
SOR Example
Indian Institute of Technology Hyderabad45
Example 1.11
[1 2 1]
Comparison Iterative schemes
Indian Institute of Technology Hyderabad46
k
2
k
3
1k
3
k
1k
2
1k
2k
1
x4x
2
xx2x
2
x1x
1k
2
k
3
1k
3
1k
1k
2
1k
2k
1
x4x
2
xx2x
2
x1x
1k
3
k
GS,3
1k
3
k
3
1k
2
k
GS,2
1k
2
k
2
1k
1
k
GS,1
1k
1
k
1
k
GS,2
k
GS,3
1k
3
k
GS,1k
GS,2
1k
2k
GS,1
xx5.1xx
xx5.1xx
xx5.1xx
x4x
2
xx2x
2
x1x
GJ GS SOR
MATLAB Exercise
Indian Institute of Technology Hyderabad
Gauss Elimination
function x=gauss(a,b)
% A = [2 1 0; 1 2 1; 0 1 1]; B = [1 2 4]โ; p = gauss(A,B);
% a - (n x n) matrix; b - column vector of length n
Steps:
% Step 0: get number of rows in matrix a, get length of b & send
message in case they are not same
% Step 1: form (n,n+1) augmented matrix
% Step 2: Start from row 1 and do for all rows that Pivot row
divided by pivot element
% Step 3: For other rows convert all elements below pivot into 0
% Step 4: begin back substitution
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
Gauss Elimination
function x=gauss(a,b)
m=size(a,1); % get number of rows in matrix a
n=length(b); % get length of b
if (m ~= n)
error('a and b do not have the same number of rows')
end
% Step 1: form (n,n+1) augmented matrix
a(:,n+1)=b;
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
for i=1:n
%Step 2: Start from row 1 and do for all rows
% Pivot row divided by pivot element
a(i, i:n+1) = a(i, i:n+1) / a(i, i);
%Step 3: For other rows convert all elements below pivot into 0
for j=i+1:n
a(j, i:n+1) = a(j, i:n+1) - a(j, i)*a(i, i:n+1);
end
end
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
%Step 4: begin back substitution
for j=n-1:-1:1
a(j,n+1) = a(j,n+1) - a(j,j+1:n)*a(j+1:n,n+1);
end
%return solution
x=a(:,n+1)
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
for i=1:n
a(i, i:n+1)=a(i, i:n+1)/a(i, i);
for j=i+1:n
a(j, i:n+1) = a(j, i:n+1)
- a(j, i)*a(i, i:n+1);
end
end
for j=n-1:-1:1
a(j,n+1) = a(j,n+1) โ
a(j,j+1:n)*a(j+1:n,n+1);
end
a(1,1:4)=a(1, 1:4)/a(1,1);
a(2, 1:4) = a(2, 1:4)- a(2, 1)*a(1,1:4);
a(3, 1:4) = a(3, 1:4)- a(3, 1)*a(1,1:4);
i = 1
j = 2
j = 3
a(2,2:4)=a(2, 2:4)/a(2,2);
a(3, 2:4) = a(3, 2:4)- a(3, 2)*a(2,2:4);
i = 2
j = 3
a(2,4) = a(2,4) โ a(2,3:3)*a(3:3,4);j = 2
j = 1 a(1,4) = a(1,4) โ a(1,2:3)*a(2:3,4);
a(3,3:4)=a(3, 3:4)/a(3,3);i = 3
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
a(1,1:4)=a(1, 1:4)/a(1,1);
a(2, 1:4) = a(2, 1:4)- a(2, 1)*a(1,1:4);
a(3, 1:4) = a(3, 1:4)- a(3, 1)*a(1,1:4);
i = 1
j = 2
j = 3
a(2,2:4)=a(2, 2:4)/a(2,2);
a(3, 2:4) = a(3, 2:4)- a(3, 2)*a(2,2:4);
i = 2
j = 3
4110
2121
1012
4110
5.115.10
5.005.01
333.000
167.010
5.005.01
9100
167.010
5.005.01
a(3,3:4)=a(3, 3:4)/a(3,3);i = 3
MATLAB Exercise (contd.)
Indian Institute of Technology Hyderabad
a(2,4) = a(2,4) โ a(2,3:3)*a(3:3,4);j = 2
j = 1 a(1,4) = a(1,4) โ a(1,2:3)*a(2:3,4);
9100
167.010
5.005.01
x3 = 9
x2 + 0.67x3 = 1
x1+ 0.5x2 + 0x3 = 0.5
x3 = 9 = a(3,4)
x2 + 0.67 a(3,4) = 1
x2 = 1 - 0.67 a(3,4) = a(2,4)
x1+ 0.5 a(2,4) + 0 a(3,4) = 0.5
x1 = 0.5 - 0.5 a(2,4) - 0 a(3,4) = a(1,4)
9100
567.010
5.005.01
9100
567.010
305.01
G E Example using MATLAB
Indian Institute of Technology Hyderabad54
4110
2121
1012
4110
5.115.10
5.005.01
333.000
167.010
5.005.01
9100
167.010
5.005.01
9100
567.010
5.005.01
9100
567.010
305.01