Seminar

Seminar

A Digital Image Stabilization Method Based

on the Hilbert–Huang Transform

Konstantinos Ioannidis and Ioannis AndreadisIEEE TRANSACTIONS on Instrumentation And Measurement,

VOL. 61, NO. 9, SEPTEMBER 2012

Presented by

Shibudas K Subhashdas

1/22

Introduction

An innovative technique for digital image stabilization

(DIS) based on the Hilbert Huang transform (HHT)

• Introduction

• Proposed method in a Nutshell

• Local Motion Vector (LMV) Es-

timation

• Hilbert Huang Transform( HHT)

• Empirical Mode Decomposi-

tion (EMD) process

• Hilbert Transform

• Detect the unwanted Camera

Motion

• Experimental Results

• Conclusion

2/22

Proposed method in a Nutshell

Local Motion Vector (LMV) Estimation

Hilbert Huang Transform ( HHT)

Empirical Mode Decomposition (EMD) process

Hilbert Transform

Detect the unwanted Camera Motion

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

3/22


LMV represents the offset of specific image regions be-

tween two consecutive images (displacement vector)

LMV includes intentional and unwanted motion of the

camera

Block based motion estimation used in LMV

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

4/22


The criteria for selecting the best match is sum of absolute

difference (SAD)

1 1

21 0 2 0

1( , ) [ ( 1, 2, ) ( 1 , 2 , 1)]

N N

n n

SAD i j s n n k s n i n j kN

S(0,0)S(0,1)S(1,0)S(1,1)

Current Frame

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

5/22

Previous Frame


Minimum SAD value defines the displacement vector

Full search (FS) algorithm is used in this method

[ 1, 2] arg min[ ( , )]d d SAD i j

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

6/22


Comparison of Full search (FS) algorithm with other

search method

three step search algorithm

Four step search algorithm

Diamond search algorithm

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

7/22


Using LMV estimation - x(t) (displacement vector)

[ 1, 2] arg min[ ( , )]d d SAD i j

1 1

21 0 2 0

1( , ) [ ( 1, 2, ) ( 1 , 2 , 1)]

N N

n n

SAD i j s n n k s n i n j kN

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

8/22


Empirical Mode Decomposition (EMD) process • Introduction



timation



tion (EMD) process



Motion


• Conclusion

9/22



11

( ) ( )( )

2

U t L tm t

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

10/22


Empirical Mode Decomposition (EMD) process Shifting Process

This will continue until value sum of difference (SD)

reaches between 0.2 and 0.3

1 1( ) ( ) ( )h t x t m t

11 1 11( ) ( ) ( )h t h t m t

12 11 12( ) ( ) ( )h t h t m t

1 1( 1) 1( ) ( ) ( )k k kh t h t m t

21

0

21

0

| ( ) ( ) |

( )

T

k kt

T

kt

h t h tSD

h t

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

11/22


Empirical Mode Decomposition (EMD) process If 0.2<= SD <=0.3

c1= h1k be the first IMF (Intrinsic Mode function)

c1 (t) is removed from the image to get residue

r1 is the residue

Residue is treated as the new data and subjected to same shifting process to give c2(t)

1 1( ) ( ) ( )r t x t c t

2 1 2( ) ( ) ( )r t r t c t

1( ) ( ) ( )w w wr t r t c t Cw(t) is the wth

IMF

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

12/22



When the residue rw becomes a monotonic function

from which no IMF can be extracted.

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

13/22



Sum of the IMFs and the residue recovers the original

signal.

1

( )w

j wj

x t c r

1 1

11 1

( )w w

j w w j wj j

x t c c r c r

2 1

1 1 21 1

w w

j w w j wj j

c c r c r

1

2 2 1 11

( )jj

c c r c r x t

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

14/22


Hilbert Transform

It is used to compute instantaneous frequencies and

amplitudes

Where P denotes Cauchy principal value

z(t) analytical signal is given by

Where α(t) – Instantaneous Amplitude

θ(t) – Instantaneous Phase

1 ( )( ( )) ( )

xH x t y t P d

t

( )( ) ( ) ( ) ( ) i tz t x t iy t t e

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

15/22


Hilbert Transform

Instantaneous Amplitude

Instantaneous Phase

Instantaneous Frequency

2 2( ) ( ) ( )t x t y t

1 ( )( ) tan

( )

y tt

x t

1 ( )( )

2

d tf t

dt

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

16/22



EMD process divides the initial signal into finite

number of sub signals based on their frequencies

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

17/22



The power of each IMF is proportional to the ampli-

tude of its sample.

Where αi – amplitude of a IMF’s

i = 1,2 ….. w+1

t – frame number

2

0

( )K

it

Pi t

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

18/22



IMF with lower indices include high frequency com-

ponent (jitter)

From lower to higher IMFs , the energy content is re-

duced

After a certain IMF , a sharp increase of the energy

occur due to intentional camera motion

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

19/22



IMF with higher index and lower energy content to

the last IMF which includes jitter components

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

20/22



Jitter and intentional camera motion

where

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

21/22

1

( ) ( )d

J ii

X t c t

( ) ( )w

G i wi d

X t c t r

arg min[ ]id P

Experimental Results

2 2

1

1( ) ( )

N

rms n n n nn

e x x y yN

Where N is the number of frames

Optimal camera motion

Resulting camera motion,( )n nx y

( , )n nx y

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

22/22

Conclusion

DIS method based on HHT has been presented

Jitter signal is defined based on its two principle feature

High frequency

Low energy

Experimental results have proven that the proposed

method can successfully decompose two camera motions

• Introduction



timation



tion (EMD) process



Motion


• Conclusion

23/22

Documents

Seminar