1/39

Motion Adaptive Search for Fast Motion Estimation

授課老師：王立洋 老師製作學生：M9535204

蔡鐘葳

2/39

Outline

▓ Introduction

▓ Motion Activity

▓ Motion Adaptive Search Algorithm

▓ Experimental Results

▓ Reference

3/39

Introduction

This paper presents the motion adaptive search (MAS) that effectively utilizes the motion activity information to improve

The speed

The accuracy of motion search

4/39

Motion Activity

Different sets of frames in a video have different motion activities corresponding to various scenes

We use the term

Global motion activity (GMA) to characterize motion activity over an entire video frame

Local motion activity (LMA) to characterize motion activity within a small region

5/39

LMA

The motion vectors of spatially neighboring macroblocks are highly correlated

Therefore, We compute the LMA at a macroblock position based on

The knowledge of the motion vectors of its neighboring macroblocks

6/39

Region of Support (ROS)

Let MBi, j be the (i, j)th macroblock in a frame

i and j indicate the horizontal and vertical positions

We define the region of support (ROS) of MBi, j , denoted by Ri, j

As the set of macroblocks to the left, above, and above right of MBi, j , respectively. That is

7/39

The City-block Lengths

We compute the LMA at MBi, j based on the information of city-block lengths of motion vectors of the macroblocks in Ri, j

Let Vi. j≡ (xi, j ,y i, j ) be the motion vector of MBi, j

The city city-block length of Vi, j, denoted by LVi j, is given by

8/39

The Grade of LMA

The grade of LMA at MBi, j , denoted by GLMA (i, j) , is defined as the maximum city-block length of the motion vectors of all macroblocks in Ri, j . Mathematically,

9/39

Example for LMA

Let LMA(i, j) denote the LMA at MBi, j . Based on GLM

A (i, j) , we categorize LMA(i, j) into two classes as follows

Fig. 1 depicts an example of distribution of GLMA

The LMA=Low at most of the macroblocks belonging to the background object

The LMA=High at most of the macroblocks belonging to the foreground object

10/39

Fig. 1

11/39

LMA and Motion Vector Bias

When LMA(i, j) is Low, all the neighborhood motion vectors of MBi, j are close to (0,0)

LMA(i, j)=High the motion vector of MBi, j is away from (0,0)

The motion vector histogram commonly observed in most video sequences reveals the link between local motion activity and motion vector bias

12/39

Fig. 2

13/39

CMV and SAD

Consider the candidate motion vector set, denoted by CMV given as follows

We define the predicted motion vector, denoted Vi, j

As the vector in CMV that yields minimum sum of absolute difference (SAD). Mathematically,

14/39

The Differential Motion Vector

Consider the differential motion vector

The differential motion vector dv is computed for each macroblock at which LMA=High

The histogram of dv is shown in Fig. 3

15/39

Fig. 3

16/39

LMA and Camera Motion

The camera motions can be divided into two categories

Nontranslational (e.g., zoom and rotation)

Translational (e.g., pan and tilt)

Translational camera motion is the most often encountered camera motion in the real world

17/39

Translational Camera Motion (1/3)

The translational camera motion introduces High LMA over the entire frame

Thus, the presence of translational camera motion is detected by first investigating

Whether the LMA is High at a majority of the macroblocks

Whether a high peak occurs at a point away from (0,0)

18/39

Translational Camera Motion (2/3)

Let H be the set of motion vectors of those macroblocks, at which LMA = High

Consider a subset of H, denoted by H1, whose city-block lengths are greater than one

Let C be the motion vector at which the histogram of H1 has a peak

The cardinalities of H and H1 are denoted by |H| and |H1|

19/39

Translational Camera Motion (3/3)

We declare that translational camera motion is present in the frame if the following two conditions are satisfied

|H| is greater than or equal to 25% of the total number of macroblocks

The peak value of H1 is greater than or equal to 0.1× |H1|

Then C is assigned as the candidate motion vector representing the camera motion

20/39

Global Motion Activity (GMA)

The GMA is estimated based on the motion vectors of all the macroblocks in a frame to characterize the overall motion activity in the entire frame

We define the grade of GMA, denoted by GGMA

21/39

Example for GMA (1/2)

An example of variation of GGMA obtained with FS over the 300 frames of the Foreman video sequence is shown in Fig. 4

22/39

Example for GMA (2/2)

From 0 to 150

An unstable camera is focused (a moderate value)

From 150 to 225

The camera pans to the right (the sudden increase due to the high motion activity caused by the camera motion)

From 225 to 300

The camera focuses on a building under construction (low value)

23/39

Motion Adaptive Search Algorithm (1/2)

The current frame are estimated from the motion vector field of the immediately previous predicted frame (P-frame)

The LMA and GMA information is used to adjust the search range for each macroblock

The estimated camera motion vector is used as a point on one of the search patterns

The search patterns are switched intelligently to suit the LMA

24/39

Motion Adaptive Search Algorithm (2/2)

Before the motion search begins, the current macroblock is first tested to determine if it is a stationary macroblock by comparing its SAD at (0,0)

If yes, the search is terminated immediately and its motion vector is assigned to be (0,0)

Otherwise, an initial motion vector is predicted based on the LMA and local search is performed to obtain the final motion vector

A detailed technical description of MAS is provided in the following

25/39

Estimation of Camera Motion Vector and Grade of GMA

Let C and GGMA denote the estimated values and the grade of GMA in the current frame

C' and G'GMA be the true values and the grade of GMA

If the current frame is the very first inter-frame

We set C equal to (0,0) and GGMA equal to 16 times the temporal distance between the current frame and its reference frame

Otherwise, GGMA = G'GMA and C = C'

26/39

Adaptation of Search Range

The maximum search range for each macroblock in the current frame srmax is given by

We adapt the search range for a as follows

sr i, j is the search range

LMA(i, j) is the LMA at MBi, j

27/39

Early Elimination of Search for Stationary Macroblocks

We compare the SAD of a macroblock MBi, j at (0,0), denoted by SADi, j (0,0) , with a threshold value T

If SADi, j (0,0) ≤ T

The motion search is terminated immediately

Assigning its motion vector as (0,0)

We call this process early elimination of search (EES)

28/39

Search Patterns

The two search patterns employed in the MAS are

The diamond search pattern (DSP)

The elastic search pattern (ESP) as shown in Fig. 5

29/39

Prediction of Initial Motion Vector

The initial motion vector at MBi, j , denoted by Ii, j, is expressed as follows

30/39

Intelligent Local Search (1/5)

We categorize the moves of DSP into three types, Type-1, Type-2 and Type-3

We make use of the vector cross product between ck −1ck and ckck +1 to determine the type

31/39

Fig. 6

32/39

Intelligent Local Search (2/5)

From vector algebra, we have

Where ‘ ’ represents the vector cross product⊗uz is the unit vector perpendicular to both ck −1ck and ck ck +1

33/39

Intelligent Local Search (3/5)

Another definition of cross product is

The value of sinθ is equal to zero when θ= 0o

The value of sinθ is equal to 1 when θ= 90o

The value of sinθ is equal to -1 when θ= 270o

Therefore, the value of Z distinguishes the types of moves of the DSP

Z=0 (Type-1), Z>0 (Type-2 ) and Z<0 (Type-3)

34/39

Intelligent Local Search (4/5)

If the move of Dk +1 is of Type-1

The other non-overlapping points are the two points on either side of dn

k+1

If the move of Dk +1 is of Type-2

The point next to dnk+1 is the other non-overlapping point

If the move of Dk +1 is of Type-3

The other non-overlapping points is the point previous to dn

k+1

35/39

Intelligent Local Search (5/5)

Thus, the sets of non-overlapping points of Dk +1 are

for Type-1 move

for Type-2 move

for Type-2 move

36/39

The intelligent local search algorithm (1/2)

37/39

The intelligent local search algorithm (2/2)

38/39

Experimental Results

39/39

Reference

[1] Prabhudev Irappa Hosur, Member, IEEE, “Motion Adaptive Search for Fast Motion Estimation,” IEEE Signal Processing Lett. Volume 49,  No. 4,  Nov. 2003 Page(s):1330 - 1340.