Lecture 13

Inner Product Space & Linear

Transformation

Last Time- Orthonormal Bases:Gram-Schmidt Process

- Mathematical Models and Least Square Analysis

- Inner Product Space Applications

Elementary Linear AlgebraR. Larsen et al. (5 Edition) TKUEE翁慶昌 -NTUEE SCC_12_2007

13- 2

Lecture 12: Inner Product Spaces & L.T.

Today Mathematical Models and Least Square Analysis Inner Product Space Applications Introduction to Linear Transformations

Reading Assignment: Secs 5.4,5.5,6.1,6.2

Next Time The Kernel and Range of a Linear Transformation Matrices for Linear Transformations Transition Matrix and Similarity

Reading Assignment: Secs 6.2-6.4

What Have You Actually Learned about Projection So Far?

13- 3

13 - 4

5.4 Mathematical Models and Least Squares Analysis

Let W be a subspace of an inner product space V.

(a) A vector u in V is said to orthogonal to W,

if u is orthogonal to every vector in W.

(b) The set of all vectors in V that are orthogonal to W is

called the orthogonal complement of W.

(read “

perp”)

} ,0,|{ WVW wwvv

W W

Orthogonal complement of W:

0(2) 0(1) VV

Notes:

13 - 5

Direct sum:

Let and be two subspaces of . If each vector

can be uniquely written as a sum of a vector from

and a vector from , , then is the

direct sum of and , and you can write .

1W 2WnR nRx

1W1w

2W2w 21 wwx nR

21 WWRn

Thm 5.13: (Properties of orthogonal subspaces) Let W be a subspace of Rn. Then the following properties

are true.

(1)

(2)

(3)

nWW )dim()dim( WWRn

WW )(

1W 2W

13 - 6

Find by the other method:

bb

b

b

vbww

T1T)(

T1T

21

)(proj

)(

,,

AAAAAx

AAAx

Ax

A

Acs

13- 7

Thm 5.16: (Fundamental subspaces of a matrix)

If A is an m×n matrix, then

(1)

(2)

(3)

(4)

)())((

)())((

ACSANS

ANSACS

)())((

)())((

ACSANS

ANSACS

mT RANSACS )()(

nT RANSACS )()(

13 - 8

Ex 6: (Fundamental subspaces)

Find the four fundamental subspaces of the matrix.

000

000

100

021

A (reduced row-echelon form)

Sol: 4 of subspace a is0,0,1,00,0,0,1span)( RACS

3 of subspace a is1,0,00,2,1span)( RARSACS

3 of subspace a is0,1,2span)( RANS

13 - 9

4 of subspace a is1,0,0,00,1,0,0span)( RANS

0000

0010

0001

~

0010

0002

0001

RA

Check:

)())(( ANSACS

)())(( ANSACS

4)()( RANSACS T 3)()( RANSACS T

ts

13 - 10

Ex 3:

Let W is a subspace of R4 and .

(a) Find a basis for W

(b) Find a basis for the orthogonal complement of W.

)1 0, 0, 0,( ),0 1, 2, 1,( 21 ww

Sol:

21

00

00

10

01

~

10

01

02

01

ww

RA (reduced row-echelon form)

}),({ 21 wwspanW

13 - 11

1,0,0,0,0,1,2,1

)(

ACSWa

is a basis for

W

W

tst

s

ts

x

x

x

x

A

ANSACSWb

for basis a is 0,1,0,10,0,1,2

0

1

0

1

0

0

1

2

0

2

1000

0121

)(

4

3

2

1

Notes:

4

4

(2)

)dim()dim()dim( (1)

RWW

RWW

13 - 12

Least Squares Problem

Least squares problem:

(A system of linear equations)

(1) When the system is consistent, we can use the Gaussian

elimination with back-substitution to solve for x

bxA11 mnnm

(2) When the system is consistent, how to find the “best possible”

solution of the system. That is, the value of x for which the

difference between Ax and b is small.

13 - 13

Least squares solution:

Given a system Ax = b of m linear equations in n unknowns,

the least squares problem is to find a vector x in Rn that

minimizes with respect to the Euclidean inner

product on Rn. Such a vector is called a least squares

solution of Ax = b.

bx A

13 - 14

)(

)of subspace a is ()(

ACSW

RACSACSA

R

MA

m

n

nm

x

x

bx

xb

xb

xb

bx

AAA

AA

ANSACSA

ACSA

projA W

ˆi.e.

0)ˆ(

)())((ˆ

)()ˆ(

ˆLet

(the normal system associated with Ax = b)

13 - 15

Note:

The problem of finding the least squares solution of

is equal to he problem of finding an exact solution of the

associated normal system .

bxA

bx AAA ˆ

Thm:

For any linear system , the associated normal system

is consistent, and all solutions of the normal system are least

squares solution of Ax = b. Moreover, if W is the column space

of A, and x is any least squares solution of Ax = b, then the

orthogonal projection of b on W is

bxA

bx AAA ˆ

xb AW proj

13 - 16

Thm:

If A is an m×n matrix with linearly independent column vectors,

then for every m×1 matrix b, the linear system Ax = b has a

unique least squares solution. This solution is given by

Moreover, if W is the column space of A, then the orthogonal

projection of b on W is

bx AAA 1)(

bxb AAAAAW1)(proj

13 - 17

Ex 7: (Solving the normal equations)

Find the least squares solution of the following system

and find the orthogonal projection of b on the column space of A.

13 - 18

Sol:

the associated normal system

13 - 19

the least squares solution of Ax = b

23

35

x

the orthogonal projection of b on the column space of A

617

68

61

23

35

)(

31

21

11

proj xb AACS

13 - 20

Keywords in Section 5.4:

orthogonal to W: 正交於W orthogonal complement: 正交補集 direct sum: 直和 projection onto a subspace: 在子空間的投影 fundamental subspaces: 基本子空間 least squares problem: 最小平方問題 normal equations: 一般方程式

Application: Cross Product

Cross product (vector product) of two vectors

nsinABB

n

向量 (vector)

方向 : use right-hand rule

The cross product is not commutative:

The cross product is distributive:

BB

CBA)CB(

zyx

zyx

BBB

AAA

zyx

B

13 - 23

Parallelogram representation of the vector product

sinABCBAC

x

y

θ

A

B

Bsinθ Area

Application: Cross Product

13 - 24

向量之三重純量積

Triple Scalar product

)CBCB(A)CBCB(A)CBCB(A)CB(Axyyxzzxxzyyzzyx

)BA(C)AC(B

)CA(B)AB(C)BC(A

The dot and the cross may be interchanged :

C)BA()CB(A

zyx

zyx

zyx

CCC

BBB

AAA

純量 (scalar)

13 - 25

向量之三重純量積

Parallelepiped representation of triple scalar product

x

y

z

A

C

B

CB

)CB(A

Volume of parallelepiped defined by , , and A

B

C

13 - 26

Fourier Approximation

13 - 27

Fourier Approximation

The Fourier series transforms a given periodic function into a superposition of sine and cosine waves

The following equations are used

1000 )]sin()cos([)(

kkk tkbtkaatf

T

k dttkwtfT

a0

0 )cos()(2

T

k dttkwtfT

b0

0 )sin()(2

T

dttfT

a0

0 )(1

13 - 28

12- 29

Today Mathematical Models and Least Square Analysis (Cont.) Inner Product Space Applications Introduction to Linear Transformations The Kernel and Range of a Linear Transformation

6.1 Introduction to Linear Transformations

Function T that maps a vector space V into a vector space W:

spacevector :, ,: mapping WVWVT

V: the domain of T

W: the codomain of T

13 - 30

Image of v under T:

If v is in V and w is in W such that

wv )(T

Then w is called the image of v under T .

the range of T:

The set of all images of vectors in V.

the preimage of w:

The set of all v in V such that T(v)=w.

13 - 31

Ex 1: (A function from R2 into R2 )22: RRT

)2,(),( 212121 vvvvvvT

221 ),( Rvv v

(a) Find the image of v=(-1,2). (b) Find the preimage of w=(-1,11)Sol:

)3 ,3())2(21 ,21()2 ,1()(

)2 ,1( )(

TT

a

v

v

)11 ,1()( )( wvTb

)11 ,1()2,(),( 212121 vvvvvvT

11 2

1

21

21

vv

vv

4 ,3 21 vv Thus {(3, 4)} is the preimage of w=(-1, 11). 13 - 32

Linear Transformation (L.T.):

nnsformatiolinear tra to：:

spacevector ：,

WVWVT

WV

VTTT vuvuvu , ),()()( (1)

RccTcT ),()( )2( uu

13 - 31

Notes:

(1) A linear transformation is said to be operation preserving.

)()()( vuvu TTT

Addition in V

Addition in W

)()( uu cTcT

Scalar multiplicati

on in V

Scalar multiplicati

on in W

(2) A linear transformation from a vector space into itself is called a linear operator.

VVT :

13 - 34

Ex 2: (Verifying a linear transformation T from R2 into R2)

Pf:

)2,(),( 212121 vvvvvvT

number realany : ,in vector : ),( ),,( 22121 cRvvuu vu

),(),(),(

:addition(1)Vector

22112121 vuvuvvuu vu

)()(

)2,()2,(

))2()2(),()((

))(2)(),()((

),()(

21212121

21212121

22112211

2211

vu

vu

TT

vvvvuuuu

vvuuvvuu

vuvuvuvu

vuvuTT

13 - 35

),(),(

tionmultiplicaScalar )2(

2121 cucuuucc u

)(

)2,(

)2,(),()(

2121

212121

u

u

cT

uuuuc

cucucucucucuTcT

Therefore, T is a linear transformation.

13 - 36

Ex 3: (Functions that are not linear transformations)

xxfa sin)()(

2)()( xxfb

1)()( xxfc

)sin()sin()sin( 2121 xxxx )sin()sin()sin( 3232

22

21

221 )( xxxx

222 21)21(

1)( 2121 xxxxf

2)1()1()()( 212121 xxxxxfxf

)()()( 2121 xfxfxxf

nnsformatiolinear tra

not is sin)( xxf

nnsformatio tra

linearnot is )( 2xxf

nnsformatiolinear tra

not is 1)( xxf

13 - 37

Notes: Two uses of the term “linear”.

(1) is called a linear function because its graph is a line.

1)( xxf

(2) is not a linear transformation from a vector space R into R because it preserves neither vector addition nor scalar multiplication.

1)( xxf

13 - 38

Zero transformation:

VWVT vu, ,:

VT vv ,0)(

Identity transformation:

VVT : VT vvv ,)(

Thm 6.1: (Properties of linear transformations)

WVT :

00 )( (1) T

)()( (2) vv TT )()()( (3) vuvu TTT

)()()(

)()(Then

If (4)

2211

2211

2211

nn

nn

nn

vTcvTcvTc

vcvcvcTT

vcvcvc

v

v

13 - 39

Ex 4: (Linear transformations and bases)

Let be a linear transformation such that 33: RRT )4,1,2()0,0,1( T

)2,5,1()0,1,0( T

)1,3,0()1,0,0( T

Sol:)1,0,0(2)0,1,0(3)0,0,1(2)2,3,2(

)0,7,7(

)1,3,0(2)2,5,1(3)4,1,2(2

)1,0,0(2)0,1,0(3)0,0,1(2)2,3,2(

T

TTTT (T is a L.T.)

Find T(2, 3, -2).

13 - 40

Ex 5: (A linear transformation defined by a matrix)

The function is defined as32: RRT

2

1

211203

)(vv

AT vv

32 into formn nsformatiolinear tra a is that Show (b) )1,2( where, )( Find (a)

RRTT vv

Sol: )1,2()( va

036

12

211203

)( vv AT

)0,3,6()1,2( T

vector2R vector 3R

)()()()( )( vuvuvuvu TTAAATb

)()()()( uuuu cTAccAcT

(vector addition)

(scalar multiplication)13 - 41

13 - 42

Thm 6.2: (The linear transformation given by a matrix)Let A be an mn matrix. The function T defined by

vv AT )(

is a linear transformation from Rn into Rm.

Note:

nmnmm

nn

nn

nmnmm

n

n

vavava

vavavavavava

v

vv

aaa

aaaaaa

A

2211

2222121

1212111

2

1

21

22221

11211

v

vv AT )(mn RRT :

vectornR vector mR

Show that the L.T. given by the matrix

has the property that it rotates every vector in R2

counterclockwise about the origin through the angle .

Ex 7: (Rotation in the plane)22: RRT

cossinsincos

A

Sol:)sin,cos(),( rryxv (polar coordinates)

r ： the length of v

： the angle from the

positive x-axis

counterclockwise to the

vector v 13 - 43

13 - 44

)sin()cos(

sincoscossinsinsincoscos

sincos

cossinsincos

cossinsincos

)(

rr

rrrr

rr

yx

AT vv

r ： the length of T(v) +： the angle from the positive x-axis

counter-clockwise to the vector T(v)Thus, T(v) is the vector that results from

rotating the vector v counterclockwise through the angle .

is called a projection in R3.

Ex 8: (A projection in R3)

The linear transformation is given by

33: RRT

000010001

A

13 - 45

Show that T is a linear transformation.

Ex 9: (A linear transformation from Mmn into Mn m )

):( )( mnnmT MMTAAT

Sol:

nmMBA ,

)()()()( BTATBABABAT TTT

)()()( AcTcAcAcAT TT

Therefore, T is a linear transformation from Mmn

into Mn m.

13 - 46

Keywords in Section 6.1:

function: 函數 domain: 論域 codomain: 對應論域 image of v under T: 在 T映射下 v的像 range of T: T的值域 preimage of w: w的反像 linear transformation: 線性轉換 linear operator: 線性運算子 zero transformation: 零轉換 identity transformation: 相等轉換

13 - 47

13- 48

Today Mathematical Models and Least Square Analysis (Cont.) Inner Product Space Applications Introduction to Linear Transformations The Kernel and Range of a Linear Transformation

6.2 The Kernel and Range of a Linear Transformation

Kernel of a linear transformation T:

Let be a linear transformationWVT :

Then the set of all vectors v in V that satisfy is called the kernel of T and is denoted by ker(T).

0)( vT

} ,0)(|{)ker( VTT vvv

Ex 1: (Finding the kernel of a linear transformation) ):( )( 3223 MMTAAT T

Sol:

000000

)ker(T

13- 49

13 - 50

Ex 2: (The kernel of the zero and identity transformations)

(a) T(v)=0 (the zero transformation )WVT :

VT )ker(

(b) T(v)=v (the identity transformation )VVT :

}{)ker( 0T

Ex 3: (Finding the kernel of a linear transformation) ):( )0,,(),,( 33 RRTyxzyxT

?)ker( T

Sol:

}number real a is |),0,0{()ker( zzT