Numerical Analysis – Numerical Analysis – Linear Equations(I)Linear Equations(I)

Hanyang University

Jong-Il Park

Linear equationsLinear equations N unknowns, M equations

where

coefficient matrix

Solving methodsSolving methods Direct methods

Gauss elimination Gauss-Jordan elimination LU decomposition Singular value decomposition …

Iterative methods Jacobi iteration Gauss-Seidel iteration …

Basic properties of matrices(I)Basic properties of matrices(I) Definition

element row column row matrix, column matrix square matrix order= MxN (M rows, N columns) diagonal matrix identity matrix : I upper/lower triangular matrix tri-diagonal matrix transposed matrix: At

symmetric matrix: A= At orthogonal matrix: At A= I

Diagonal dominance

Transpose facts

Basic properties of Basic properties of matrices(II)matrices(II)

Basic properties of matrices(III)Basic properties of matrices(III) Matrix multiplication

DeterminantDeterminant

C

Determinant facts(I)Determinant facts(I)

Determinant facts(II)Determinant facts(II)

Geometrical interpretation of Geometrical interpretation of determinantdeterminant

Over-determined/Over-determined/Under-determined problemUnder-determined problem

Over-determined problem (m>n) least-square estimation, robust estimation etc.

Under-determined problem (n<m) singular value decomposition

Augmented matrixAugmented matrix

Cramer’s ruleCramer’s rule

Triangular coefficient matrixTriangular coefficient matrix

SubstitutionSubstitution

Upper triangular matrix

Lower triangular matrix

Gauss eliminationGauss elimination

1. Step 1: Gauss reduction =Forward elimination Coefficient matrix upper triangular matrix

2. Step 2: Backward substitution

Gauss reductionGauss reduction

Gaussreduction

Eg. Gauss elimination(I)Eg. Gauss elimination(I)

Eg. Gauss elimination(II)Eg. Gauss elimination(II)

Troubles in Gauss eliminationTroubles in Gauss elimination Harmful effect of round-off error in pivot

coefficient

Pivoting strategy

Eg. Trouble(I)Eg. Trouble(I)

Eg. Trouble(II)Eg. Trouble(II)

Pivoting strategyPivoting strategy To determine the smallest such that

and perform

Partial pivotingdramatic enhancement!

Effect of partial pivotingEffect of partial pivoting

Scaled partial pivotingScaled partial pivoting

Scaling is to ensure that the largest element in each row has a relative magnitude of 1 before the comparison for row interchange is performed.

Eg. Effect of scalingEg. Effect of scaling

Complexity of Gauss eliminationComplexity of Gauss elimination

Too much!

Summary: Gauss eliminationSummary: Gauss elimination

1) Augmented matrix 의 행을 최대값이 1 이 되도록 scaling(=normalization)

2) 첫 번째 열에 가장 큰 원소가 오도록 partial pivoting

3) 둘째 행 이하의 첫 열을 모두 0 이 되도록 eliminating

4) 2 행에서 n 행까지 1)- 3) 을 반복5) backward substitution 으로 해를 구함

00

0

Gauss-Jordan eliminationGauss-Jordan elimination

Eg. Obtaining inverse matrix(IEg. Obtaining inverse matrix(I))

Eg. Obtaining inverse matrix(IIEg. Obtaining inverse matrix(II))

Backward substitution For each column

LU decompositionLU decomposition Principle: Solving a set of linear equations based on

decomposing the given coefficient matrix into a product of lower and upper triangular matrix.

Ax = b LUx = b L-1 LUx = L-1 bA=LU L-1

L-1 b=c U x = c

L

L L-1 b = Lc L c = b

(1)

(2) By solving the equations (2) and (1) successively, we get the solution x.

Upper triangular

Lower triangular

Various LU decompositionsVarious LU decompositions

Doolittle decompositionL의 diagonal element 를 모두 1 로 만들어줌

Crout decompositionU의 diagonal element 를 모두 1 로 만들어줌

Cholesky decomposition L 과 U 의 diagonal element 를 같게 만들어줌 symmetric, positive-definite matrix 에 적합

Crout decompositionCrout decomposition

Implementation of Crout methodImplementation of Crout method

Programming using NR in C(I)Programming using NR in C(I) Solving a set of linear equations

Programming using NR in C(II)Programming using NR in C(II) Obtaining inverse matrix

Programming using NR in C(III)Programming using NR in C(III) Calculating the determinant of a matrix

Homework #5 (Cont’)Homework #5 (Cont’)[Due: 10/22]

(Cont’) Homework #5(Cont’) Homework #5