Image Transforms

Image Transforms

主講人：虞台文

Content Overview Convolution Edge Detection

– Gradients– Sobel operator– Canny edge detector– Laplacian

Hough Transforms Geometric Transforms

– Affine Transform– Perspective Transform

Histogram Equalization

Image Transforms

Overview

Image Transform Concept

T[]T[]

Image Transform Concept

T[]T[]

( , )I x y( , )I x y ( , )O x y( , )O x y

( , ) ( , )TO x y I x y

Image Transforms

Convolution

Image Convolution

( , )g x y

( , )f x y ( * )( , )f g x y*

g(x,y) is known as convolution kernel.

( * )( , ) ( , ) ( , )v u

f g x y f u v g x u y v

Image Convolution

( , )g x y

( , )f x y ( * )( , )f g x y*

g(x,y) is known as convolution kernel.

( * )( , ) ( , ) ( , )v u

f g x y f u v g x u y v

( * )( , ) ( , ) ( , )y h x w

v y h u x w

f g x y f u v g x u y v

height 2h + 1width 2w + 1

Image Convolution

g(x,y) is known as convolution kernel.

( * )( , ) ( , ) ( , )y h x w

v y h u x w

f g x y f u v g x u y v

height 2h + 1width 2w + 1

Some Convolution Kernels

OpenCV Implementation Image Filter

void cvFilter2D( const CvArr* src, CvArr* dst, const CvMat* kernel, CvPoint anchor=cvPoint(-1, -1));

void cvFilter2D( const CvArr* src, CvArr* dst, const CvMat* kernel, CvPoint anchor=cvPoint(-1, -1));

Deal with Convolution Boundaries

void cvCopyMakeBorder( const CvArr* src, CvArr* dst, CvPoint offset, int bordertype, CvScalar value=cvScalarAll(0));

void cvCopyMakeBorder( const CvArr* src, CvArr* dst, CvPoint offset, int bordertype, CvScalar value=cvScalarAll(0));

Image Transforms

Edge Detection

Edge Detection

Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels

Origin of Edges

depth discontinuity

surface color discontinuity

illumination discontinuity

surface normal discontinuity

Edges are caused by a variety of factors

Edge Detection

How can you tell that a pixel is on an edge?

Edge Types

Step Edges

Roof Edge Line Edges

Real Edges

Noisy and Discrete!

x

I

We want an Edge Operator that produces:– Edge Magnitude– Edge Orientation– High Detection Rate and Good Localization

Derivatives of Image in 1D

( )I x

2 ( ) ( )I x I x

( )I I x

Edges can be characterized as either:– local extrema of I(x)– zero-crossings of 2I(x)

1D image

gradient

Laplacian

2D-Image Gradient

( , ) ,T

I II x y

x y

,T

x yI I

2D-Image Gradient ( , ) , ,

TT

x y

I II x y I I

x y

Gives the direction of most rapid change in intensity

Gradient direction:

Edge strength:

,0T

xI I

0,T

yI I

,T

x yI I I

1tan /y xI I

2 2x yI I I

Classification of Points

To precisely locate the

edge, we need to thin.

Ideally, edges should be

only one point thick.

TI

Non-zeroedge width

I T

( , ) , ,T

T

x y

I II x y I I

x y

The Sobel Operators

( , ) , ,T

T

x y

I II x y I I

x y

-1 0 1

-2 0 2

-1 0 1

1 2 1

0 0 0

-1 -2 -1

x xI S y yI S

Sobel (3 x 3):

Sobel (5 x 5):

-1 -2 0 2 1

-2 -3 0 3 2

-3 -5 0 5 3

-2 -3 0 3 2

-1 -2 0 2 1

1 2 3 2 1

2 3 5 3 2

0 0 0 0 0

-2 -3 -5 -3 -2

-1 -2 -3 -2 -1

Good LocalizationNoise SensitivePoor Detection

Poor LocalizationLess Noise SensitiveGood Detection

OpenCV Implementation The Sobel Operators

( , ) , ,T

T

x y

I II x y I I

x y

void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);

void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);

OpenCV Implementation The Scnarr Operator

( , ) , ,T

T

x y

I II x y I I

x y

void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);

void cvSobel( const CvArr* src, CvArr* dst, int xorder, int yorder, int aperture_size = 3);

aperture_size

CV_SCHARR

Demonstration

( , )I x y

( , ) , ,T

T

x y

I II x y I I

x y

xI

yI2 2x yI I

Exercise

DownloadTest Program

DownloadTest Program

Effects of Noise

Where is the edge?Where is the edge?

Consider a single row or column of the image

Solution: Smooth First

f

g

*g f

*x g f

Solution: Smooth First

f

g

*g f

*x g f

Where is the edge?Where is the edge?

Derivative Theorem of Convolution

( , )g x y I *I g

2 2

2 2

1( , ) exp

2 2

x yg x y

Gaussian:

( , ) ( , )h x y g x y

2 2

4 2

2 2

4 2

( , )exp

2 2

( , )exp

2 2

g x y x x y

x

g x y y x y

y

Derivative Theorem of Convolution

saves us one operation. * *x xg f g f

x g

f

*x g f

Optimal Edge Detection: Canny

Assume: – Linear filtering– Additive iid Gaussian noise

An "optimal" edge detector should have:– Good Detection

Filter responds to edge, not noise.– Good Localization

detected edge near true edge.– Single Response

one per edge.

Optimal Edge Detection: Canny

Based on the first derivative of a Gaussian

Detection/Localization trade-off– More smoothing improves detection– And hurts localization.

Stages of the Canny algorithm

Noise reductionSize of Gaussian filter

Finding the intensity gradient of the image Non-maximum suppression Tracing edges through the image and hyste

resis thresholdingHigh thresholdLow threshold

Parameters of Canny algorithm

Noise reduction– Size of Gaussian filter

Finding the intensity gradient of the image Non-maximum suppression Tracing edges through the image and hyste

resis thresholding– High threshold– Low threshold

OpenCV Implementation The Canny Operator

void cvCanny( const CvArr* img, CvArr* edges, double lowThresh, double highThresh, int apertureSize = 3);

void cvCanny( const CvArr* img, CvArr* edges, double lowThresh, double highThresh, int apertureSize = 3);

Example: Canny Edge Detector

DownloadTest Program

DownloadTest Program

Review:Derivatives of Image in 1D

( )I x

2 ( ) ( )I x I x

( )I I x

Edges can be characterized as either:– local extrema of I(x)– zero-crossings of 2I(x)

1D image

gradient

Laplacian

Laplacian

2 22

2 2

( , ) ( , )( , ) ( , )

I x y I x yI x y f x y

x y

A scalar isotropic.

Edge detection: Find all points for which

2I(x, y) = 0

No thinning is necessary.

Tends to produce closed edge contours.

Laplacian

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

Discrete Laplacian Operators

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

010

141

010

111

181

111

121

242

121

OpenCV Implementation The Discrete Laplacian Operators

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

void cvLaplace( const CvArr* src, CvArr* dst, int apertureSize = 3);

void cvLaplace( const CvArr* src, CvArr* dst, int apertureSize = 3);

Example

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

2 ( , )I x y( , )I x y

Laplician for Edge Detection

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

2 ( , )I x y

Find zero-crossing on the Laplacian image.

Zero Crossing Detection

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

There is a little bug in the above algorithm.

Try to design your own zero-crossing detection algorithm.

Example:Laplician for Edge Detection

DownloadTest Program

DownloadTest Program

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

Laplacian for Image Sharpening

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

2 ( , )I x y( , )I x y

*w

Example:Laplacian for Image Sharpening

2 22

2 2

( , ) ( , )( , )

I x y I x yI x y

x y

( , )I x y Sharpened Image

Laplacian of Gaussian (LoG)

( , )g x y 2I 2*I g

2 2

2 2

1( , ) exp

2 2

x yg x y

Gaussian:

2( , ) ( , )h x y g x y

2 2 2 22

4 2 2

1( , ) ( , ) 1 exp

2 2

x y x yh x y g x y

Some LoG Convolution Kernels

2 2 2 22

4 2 2

1( , ) ( , ) 1 exp

2 2

x y x yh x y g x y

0 0 0 0 1 1 1 0 0 0 0

0 0 1 2 3 3 3 2 1 0 0

0 1 2 4 5 5 5 4 2 1 0

0 2 4 5 1 2 1 5 4 2 0

1 3 5 1 14 24 14 1 5 3 1

1 3 5 2 24 40 24 2 5 3 1

1 3 5 1 14 24 14 1 5 3 1

0 2 4 5 1 2 1 5 4 2 0

0 1 2 4 5 5 5 4 2 1 0

0 0 1 2 3 3 3 2 1 0 0

0 0 0 0 1 1 1 0 0 0 0

0 1 1 2 2 2 1 2 0

1 2 4 5 5 5 4 2 1

1 4 5 3 0 3 5 4 1

2 5 3 12 24 12 3 5 2

2 5 0 24 40 24 0 5 2

2 5 3 21 24 12 3 5 2

1 4 5 3 0 3 5 4 1

1 2 4 5 5 5 4 2 1

0 1 1 2 2 2 1 2 0

0 0 1 0 0

0 1 2 1 0

1 2 16 2 1

0 1 2 1 0

0 0 1 0 0

Example:LoG for Edge Detection

by LoG

by Laplacian

Image Transforms

Hough Transforms

Goal of Hough Transforms

A technique to isolate the curves of a given shape / shapes in a given image

Classical Hough Transform – can locate regular curves like straight lines,

circles, parabolas, ellipses, etc.

Generalized Hough Transform – can be used where a simple analytic

description of feature is not possible

HT for Line Detection

x

y

y m bx

m

b

(m, b)

A line in xy-plane is a point in mb-plane.

HT for Line Detection

x

y 1 1y m bx

m

b

( , )x y

(m1, b1)2 2y m bx

(m2, b2)

3 3y m bx

(m3, b3)

All lines passing through a point in xy-plane is a line in mb-plane.

A line in xy-plane is a point in mb-plane.

b my x

HT for Line Detection

x

y 1 1y m bx

m

b

( , )x y

(m1, b1)2 2y m bx

(m2, b2)

3 3y m bx

(m3, b3)

All lines passing through a point in xy-plane is a line in mb-plane.A line in xy-plane is a point in mb-plane.

b my x ( , )x y

Given a point in xy-plane, we draw a line in mb-plane.

b my x

HT for Line Detection

x

y

m

b

A line in xy-plane is a point in mb-plane.

A line in xy-plane is then transformed in to a set of lines in mb-plane, which intersect at a common point.

Given a point in xy-plane, we draw a line in mb-plane.

y m bx (m, b)

HT for Line Detection

x

y

m

b

A line in xy-plane is a point in mb-plane.

A line in xy-plane is then transformed in to a set of lines in mb-plane, which intersect at a common point.

Given a point in xy-plane, we draw a line in mb-plane.

y m bx (m, b)

How to implement?How to implement?

Is mb representation suitable?Is mb representation suitable?

HT Line Detection by -representation

x

y

( cos , sin )

( , )x y

cos sinx y (, )

A line in xy-plane is a point in -plane.

HT Line Detection by -representation

x

y

A line in xy-plane is a point in -plane.

( , )x y1

2

3 4

43

2

1

cos sinx y

All lines passing through a point in xy-plane is a curve in -plane.

HT Line Detection by -representation

x

y

A line in xy-plane is a point in -plane.

( , )x y1

2

3 4

43

2

1

cos sinx y

All lines passing through a point in xy-plane is a curve in -plane.

Given a point in xy-plane, we draw a curve in -plane.

( , )x y

cos sinx y

HT Line Detection by -representation

x

y

A line in xy-plane is a point in -plane.

Given a point in xy-plane, we draw a curve in -plane.

A line in xy-plane is then transformed in to a set of curves in -plane, which intersect at a common point.

(, )

HT Line Detection by -representation

A line in xy-plane is a point in -plane.

Given a point in xy-plane, we draw a curve in -plane.A line in xy-plane is then transformed in to a set of curves in -plane, which intersect at a common point.

OpenCV Implementation Hough Line Transform

CvSeq* cvHoughLines2( CvArr* image, void* line_storage, int method, double rho, double theta, int threshold, double param1 = 0, double param2 = 0);

CvSeq* cvHoughLines2( CvArr* image, void* line_storage, int method, double rho, double theta, int threshold, double param1 = 0, double param2 = 0);

Example:Hough Line Transform

DownloadTest Program

DownloadTest Program

Hough Circle Transform

2 2 2( ) ( )x ya b r Circle equation:

x

y

r

a

b

0 0( , )x y

image space

0 0( , )x y

parameter space

Hough Circle Transform

2 2 2( ) ( )x ya b r Circle equation:

x

y

r

a

b

0 0( , )x y

image space

0 0( , )x y

parameter space

Cost ineffective & time consuming

Hough Gradient Method

2 2 2( ) ( )x ya b r Circle equation:

x

y

image space

0 0( , )x y

( , )a b

Parametric form: 0

0

cos

sin

x

y

a r

b r

0

0

cos

sin

a r

b r

x

y

0 0tan tanxb ya

Hough Gradient Method

2 2 2( ) ( )x ya b r Circle equation:

x

y

image space

0 0( , )x y

( , )a b

Parametric form: 0

0

cos

sin

x

y

a r

b r

0

0

cos

sin

a r

b r

x

y

0 0tan tanxb ya

The value of can be obtained from the edge detection

process.

The value of can be obtained from the edge detection

process.

Hough Gradient Method

Quantize the parameter space for the parameters a and b.

Zero the accumulator array M(a, b). Compute the gradient magnitude G(x, y)

and angle (x, y). For each edge (x0, y0) point in G(x, y), incre

ment all points in the accumulator array M(a, b) along the line

Local maxima in the accumulator array correspond to centers of circles in the image.

Circle equation:

x

y

image space

0 0( , )x y

( , )a b

0 0tan tanxb ya

0 0tan tanxb ya

2 2 2( ) ( )x ya b r

OpenCV Implementation Hough Circle Transform

CvSeq* cvHoughCircles( CvArr* image, void* circle_storage, int method, double dp, double min_dist, double param1=100, double param2=100 int min_radius=0, int max_radius=0);

CvSeq* cvHoughCircles( CvArr* image, void* circle_storage, int method, double dp, double min_dist, double param1=100, double param2=100 int min_radius=0, int max_radius=0);

Example:Hough Circle Transform

DownloadTest Program

DownloadTest Program

Image Transforms

Geometric Transforms

Geometric Transforms Stretch, Shrink, Warp, and Rotate

Scaling , Rotation, Translation

0

0x

y

s

s

x x

y y

cos sin

sin cos

x x

y y

x x x

y y y

Scaling

Rotation

Translation

Scaling , Rotation + Translation

0

0x

y

s

s

x x

y y

cos sin

sin cos

x x

y y

x x x

y y y

Scaling

Rotation

Translation

+Translation

+Translation

x

y

x

y

Homogeneous Coordinate

xx

y

w

w

wy

1

xx

yy

Scaling , Rotation + Translation

0

0x

y

x x x

y y y

s

s

cos sin

sin cos

x x x

y y y

Scaling

Rotation

+Translation

+Translation

0

01

x

y

s

s

xx

yy

cos sin

sin cos1

xx

yy

2 3 matrix

2 3 matrix

Affine Transformation

An affine transformation is any transformation that can be expressed in the form of a matrix multiplication followed by a vector addition. – In OpenCV the standard style of representing such a tran

sformation is as a 2-by-3 matrix.

00 01 0

10 11 1

a a bx

a bya

x

y

00 01 0

10 11 1 1

a a b

a a b

x

y

2 3 matrix

Affine Transformation

00 01 0

10 11 1 1

xa a bx

ay

y a b

GetAffineTransform

00 01 0

10 11 1 1

xa a bx

ay

y a b

Get Affine Transform

00 01 0

10 11 1 1

xa a bx

ay

y a b

Get 2D Rotation Matrix

WarpAffine00 01 0

10 11 1 1

xa a bx

ay

y a b

GetQuadrangleSubPix

Example: Affine Transform

DownloadTest Program

DownloadTest Program

GetQuadrangleSubPix

Sparse Affine Transformation

Perspective Transform

Perspective Transform

Perspective Transform

Affine Transform vs. Perspective Transform

00 01 0

10 11 1 1

xa a bx

ay

y a b

00 01 0

10 11 1

0 0 11 1

a a bx x

by a ya

Affine Transform:

x w

y w

w

Perspective Transform:

00 01 0

10 11 1

20 21 1 1

a a b

a a b

a

y

w a

x x

y

/

/

x x w

y y w

Get Perspective Transform

00 01 0

10 11 1

20 21 1 1

x w x x

y

a a b

a a yb

a a

w y

w w

WarpPerspective

00 01 0

10 11 1

20 21 1 1

x w x x

y

a a b

a a yb

a a

w y

w w

Sparse Perspective Transformation

00 01 0

10 11 1

20 21 1 1

x w x x

y

a a b

a a yb

a a

w y

w w

Image Transforms

Histogram Equalization

Graylevel Histogram of Image

Goal of Histogram Equalization

Goal of Histogram Equalization

Image Enhancement

Method Graylevel Remapping

0 1

fX(x)

x 0 1

fY(y)

y

y

xX Y

Probability Theory

y

xX Y

( )Xf xpdf

( )XF xcdf( )XY F X ~ (0,1)Y U

( )Xy F x

Example: Gaussian

Example: Gaussian

Demonstration

OpenCV Implementation

Example

DownloadTest Program

DownloadTest Program