CCUCCUVISIONVISIONLABORATORYLABORATORY

Object Speed Measurements Using Motion Blurred Images

林惠勇中正大學電機系lin@ee.ccu.edu.tw

H.Y.Lin, CCUEE CCU Vision Lab 2

LL

CCVVImages…

H.Y.Lin, CCUEE CCU Vision Lab 3

LL

CCVVBlur Images…

Defocus blur: Motion blur:

H.Y.Lin, CCUEE CCU Vision Lab 4

LL

CCVVWhat Do They Tell Us?

Motion of Object Region of Interest:

H.Y.Lin, CCUEE CCU Vision Lab 5

LL

CCVVInformation from Blur

Images Two types of image blur: Defocus blur – due to the limitation of optical sensors

Image restoration Identification of region of interest Depth measurement

Motion blur – due to the relative motion between the camera and the scene Image restoration Motion analysis Increase still resolution from video Special effect Speed measurements? From the movie: “Chicken Run”

H.Y.Lin, CCUEE CCU Vision Lab 6

LL

CCVVDefocus Blur

q

f

p

z

FocusedPosition

d

ImageDetector

D

DefocusedPosition

3

Z

1

2

4

Blur circle

H.Y.Lin, CCUEE CCU Vision Lab 7

LL

CCVVMotion Blur

q

f

p

s

3

ImageDetector

D

1

2

x

4

H.Y.Lin, CCUEE CCU Vision Lab 8

LL

CCVVSpeed Measurements

Why measure speed? (motivation) Wind Experiments Sports (baseball, tennis ball), athletes Vehicle speed detection

How? RADAR (Radio Detection And Ranging) LIDAR (Laser Infrared Detection And Ranging) GPS Video-Based Analysis

H.Y.Lin, CCUEE CCU Vision Lab 9

LL

CCVVImage-Based Speed

Measurement Key idea: For a fixed camera exposure time:

Relative motion betweenobject and static camera

Motion blur appeared in the dynamic image region

H.Y.Lin, CCUEE CCU Vision Lab 10

LL

CCVVGeometric Formulation

Simple pinhole camera model:

Key components: Focal length, exposure time, CCD pixel size Object distance, blur length (blur extent)

L

fz

K

d

Tf

zKsv x

Tl

KLv

f

z

Ks

d

x

H.Y.Lin, CCUEE CCU Vision Lab 11

LL

CCVVImage Degradation

Image degradation – linear space invariant system Characterized by its point spread function (PSF) h(x,y)

Degradation under uniform linear motion (whole image)

How about space variant case? (partial blur & total blur)

ddfyxhyxg ),(),(),(

otherwise,0

tan,cos2

,1

),( xyR

xRyxh

H.Y.Lin, CCUEE CCU Vision Lab 12

LL

CCVVBlur Parameter

Estimation Edge detection ABC: Sharp edge step response Blur edge ramp response

How to use this fact to estimate blur extent?

H.Y.Lin, CCUEE CCU Vision Lab 13

LL

CCVVImage Deblurring

If H is linear, space invariant: Inverse filtering Wiener filter

Bad news: Our case is space variant Region segmentation

Degradationfunction

H

Degradationfunction

HRestoration

filter(s)

Restorationfilter(s)++f(x,y)

g(x,y)

Noise (x,y)

f(x,y)

Degradation Restoration

uajT tuxj euaua

TdtevuH

)sin(),(

0

)(2 0

H.Y.Lin, CCUEE CCU Vision Lab 14

LL

CCVVMore General Case – I

What if the object is not moving parallel to the image scanlines?

Motion direction estimation Image rectification

H.Y.Lin, CCUEE CCU Vision Lab 15

LL

CCVVMotion Direction

Estimation Fourier spectrum analysis:

It can also be implemented in spatial domain

H.Y.Lin, CCUEE CCU Vision Lab 16

LL

CCVVMore General Case – II

What if the object is not moving parallel to the image plane?

d

fz

Ll

H.Y.Lin, CCUEE CCU Vision Lab 17

LL

CCVVExtended Camera Model

]sin)(cos[ KPsfT

zKsv

x

x

cosTf

zKsv x

f

z

k

d

f

d

kp

cos

sin

sin)(cos kpf

zkd

H.Y.Lin, CCUEE CCU Vision Lab 18

LL

CCVVRequired Parameters

Intrinsic camera parameters Focal length, CCD pixel size, exposure time

Extrinsic camera parameters Distance to the object, camera orientation

Softball speed measurement Size of the softball (physical measurement)

Vehicle speed detection – “parallel case” Length of the vehicle (from manufacturer’s data sheet)

Vehicle speed detection – “non-parallel case” ? How to obtain the parameters z, , etc.?

H.Y.Lin, CCUEE CCU Vision Lab 19

LL

CCVVVehicle Speed Detection

Parameters: K = 22 pixels, sx = 11 m, f = 10 mm, T = 1/160 sec. l = 560 pixels, L = 4750 m

Detected speed – 104.86 km/hr Video-based speed – 106.11 km/hr, speed limit – 110 km/hr

H.Y.Lin, CCUEE CCU Vision Lab 20

LL

CCVVCamera Pose Estimation

Theorem: Given a parallelogram in 3-D space with known image projection

of four points, their relative depths can be determined.

To obtain the unknown scale factor: Absolute metric between two 3-D points License plate with standard size

P

a

B

A

b

dc

D

C

W

Icn

cn

ZYXd

fsysxZYXP

iiii

ii

iiiiiiiii

1

222

0

cos

/

),,(),,(

H.Y.Lin, CCUEE CCU Vision Lab 21

LL

CCVVVehicle Speed Detection

Parameters: K = 22 pixels, sx = 6.8 m, T = 1/400 sec., l = 560 pixels, L = 4750 m W = 320 mm, = 48.25, f = 26 mm

Detected speed – 112.97 km/hr Video-based speed – 110.22 km/hr

H.Y.Lin, CCUEE CCU Vision Lab 22

LL

CCVVFully Automated? How?

Intrinsic camera parameters? JPEG EXIF header

Target identification Motion blur analysis

Region segmentation Region growing Additional image capture

Robust blur extent estimation Image synthesis

Deblurred target region + static background region

H.Y.Lin, CCUEE CCU Vision Lab 23

LL

CCVVInitial Target

Segmentation

Horizontal ramp edge detection Run-length coding or projection Vertical continuity checking Multiple direction analysis

1230321

1230321

1230321

1230321

1230321

1230321

1230321

H.Y.Lin, CCUEE CCU Vision Lab 24

LL

CCVVSpherical Object in

Motion Problems on parameter estimation Accuracy, robustness, precision (subpixel resolution…)

Spherical object circular from any viewpoint Initial blur extent identification + circle detection

Circle fitting, Hough transform

More problems Motion blur due to rotation, three-dimensional translation,

shading, etc.

H.Y.Lin, CCUEE CCU Vision Lab 25

LL

CCVVSpeed Measurement

FlowchartMotion Blurred

Image

Target Identification

Image Segmentation

HorizontalMotion Blur

Initial Blur LengthEstimation

Image Rotation

Image Deblurring

Circle Fitting

Speed Measurement

Object Speed

no

yes

Two Images

Environment ParameterEstimation

H.Y.Lin, CCUEE CCU Vision Lab 26

LL

CCVVMotion Direction

Estimation Camera pose estimation – non-parallel case Two or more captures with fast shutter speed

Vertical projection Post-processing Fixed object size Could be blurred

Z

rP

f

x

z

12

2

p

O

I

Q

q

a

12

121tanxx

zz

H.Y.Lin, CCUEE CCU Vision Lab 27

LL

CCVVSoftball Speed

Measurement

Parameters: K = 26 pixels, T = 1/320 sec., l = 72 pixels, d = 97.45 mm

Detected speed – 40.5 km/hr Video-based speed – 40.9 km/hr

H.Y.Lin, CCUEE CCU Vision Lab 28

LL

CCVVConclusion

Object speed measurement using a single motion blurred image Vehicle speed detection Softball speed measurement

Advantages Low cost – off-the-shelf digital camera Passive device – can avoid anti-detection Passive device – no radiation, light Large measurement range – through adjustable shutter speed

Limitation Lighting condition Accuracy?

H.Y.Lin, CCUEE CCU Vision Lab 29

LL

CCVV

Thank you for your attention!

Any questions?