Chapter 9 Image Registration

Chapter 9Image Registration

Chuan-Yu Chang (張傳育 )Ph.D.

Dept. of Computer and Communication Engineering

National Yunlin University of Science & Technology

[email protected]

http://mipl.yuntech.edu.tw

Office: EB212

Tel: 05-5342601 Ext. 4337

2醫學影像處理實驗室 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.

Introduction Different medical imaging modalities provide specific information

about human physiology and physiological processes that are often complimentary in diagnosis.

To understand the physiological processes better, images obtained from different modalities need to be registered.

To study the variability of anatomical and function structures among the subjects, images from respective modalities can be registered to develop computerized atlases. Structural Computerized Atlas (SCA)

Represent the anatomical variations among subjects can be developed using registered image from the anatomical medical imaging modalities such as CT or MRI.

Functional Computerized Atlas (FCA) Represent the metabolic variations among subjects for a specific path

ology or function can be developed using registered images from the functional medical imaging modalities such as fMRI, SPECT or PET.


A schematic diagram of multi-modality MR-PET image analysis using computerized atlases.

Analysis

Functional Reference

(FCA)ReferenceSignatures

MR Image(New Subject)

PET Image(New Subject)

MR-PETRegistration

AnatomicalReference(SCA)


Introduction

Image registration methods and algorithms provide transformation of a source image space to the target image space. The target image may be an image of the same or any

other subject from any medical imaging modality. Registration methods

External markers and stereotactic frames based landmark registration.

Rigid-body transformation based global registration. Image feature-based registration

Boundary and surface matching based registration Image landmarks and features based registration


Image Registration Through Transformation

AB

f

g


Rigid-Body Transformation

Rigid-Body Transformation is based on translation and rotation operations.

Two images of equal dimensions are registered by applying a pixel-by-pixel transformation consistently throughout the image space.

A rigid transformation based mapping of a point vector x to x’ is defined by

where R is a rotation matrix and t is translation vector.

tRxx '


Rigid-body Transform

The translation and rotation operations of a 3-D rigid transformation.

Translation along x-axis by p

Translation along y-axis by q

Translation along z-axis by r

Translation of z

Translation of y Translation of x

Rotation by

Rotation by

Rotation by

zz

yy

pxx

'

'

'

zz

qyy

xx

'

'

'

rzz

yy

xx

'

'

'



The translation and rotation operations of a 3-D rigid transformation.

Rotation about x-axis by

Rotation about y-axis by

Rotation about z-axis by

Translation of z

Translation of y Translation of x

Rotation by

Rotation by

Rotation by

cossin'

sincos'

'

zyz

zyy

xx

cossin'

'

sincos'

zxz

yy

zxx

zz

yxy

yxx

'

cossin'

sincos'



The rotation matrix R for the x-y-z rotational order of operation can be given as

100

0cossin

0sincos

cos0sin

010

sin0cos

cossin0

sincos0

001

RRRR


Affine Transformation

Affine transformation is a special case of rigid-body transformation that includes translation, rotation and scaling operations.

If the two image volumes to be registered are not at the same scale, a scaling parameter in each dimension has to be added as

where a, b, and c are the scaling parameters along x, y, and z directions.

czz

byy

axx

'

'

'


Affine Transformation

The affine transformation can be expressed as

where A is the Affine matrix that includes the translation, rotation and scaling transformation with nine parameters.

The overall mapping can be expressed as

Axx '

11000

000

000

000

1000

0cossin0

0sincos0

0001

1000

0cos0sin

0010

0sin0cos

1000

0100

00cossin

00sincos

1000

100

010

001

1

'

'

'

z

y

x

c

b

a

r

q

p

z

y

x


Principal Axes Registration

Principal Axes Registration can be used for global matching of two binary volumes such as segmented brain volumes from CT, MR or PET images.

Let us represent a binary segmented B(x,y,z) as

Let the centroid of the binary volume B(x,y,z) be represented by (xg, yg, zg)T

object. in thenot is z)y,(x, if 0

object in the is z)y,(x, if 1z)y,B(x,

zyx

zyxg

zyx

zyxg

zyx

zyxg

zyxB

zyxzB

zzyxB

zyxyB

yzyxB

zyxxB

x

,,

,,

,,

,,

,,

,,

),,(

),,(

),,(

),,(

),,(

),,(


Principal Axes Registration The principle axes of B(x,y,z) are the eigenvectors of the inertia matrix I:

where

The method can resolve six degrees of freedom of an object including three rotations and three translations.

zzzyzx

yzyyyx

xzxyxx

III

III

III

I

zyxggzyyz

zyxggzxxz

zyxggyxxy

zyxggzz

zyxggyy

zyxggxx

zyxBzzyyII

zyxBzzxxII

zyxByyxxII

zyxByyxxI

zyxBzzxxI

zyxBzzyyI

,,

,,

,,

,,

22

,,

22

,,

22

),,())((

),,())((

),,())((

),,()()(

),,()()(

),,()()(



Let us define a normalized eigenvector matrix E as

Let R=RRR represent the rotation matrix as

where , , and are the rotation angles with respect to the x, y, and z axes.

333231

232221

131211

eee

eee

eee

E

cossin0

sincos0

001

cos0sin

010

sin0cos

100

0cossin

0sincos

RRR



By equating the normalized eigenvector matrix to the rotation matrix as

it can be shown that

RRRE

)cos/arcsin(

)cos/arcsin(

)arcsin(

32

21

31

e

e

e


Principal Axes Registration Given two volumes, V1 and V2, for registration, the

PAR method provides the following operations: Translate the centroid of V1 to the origin.

Rotate the principal axes of V1 to coincide with the x, y a

nd z axes. Rotate the x, y and z axes to coincide with the principal a

xes of V2.

Translate the origin to the centroid of V2.

The volume V2 is scaled to match the volume V1 using th

e scaling factor Fs.3

2

1

V

VFs


Principal axes Transformation

zyx

zyx

zyx

zyx

zyx

zyx

zyxV

zyxzV

z

zyxV

zyxyV

yzyxV

zyxxV

x

CentroidtheComputingStep

otherwise

boundarysurfacetheonzyxvoxelzyxV

,,

,,

,,

,,

,,

,,

),,(

),,(

),,(

),,(

,),,(

),,(

:2

0

),,(,1),,(

Step 1: Define the volume



Step 3: Computing the principal axes: The principal axes of V(x,y,z) are the eigenvectors of the inertia matrix I:

zyxzz

zyxyy

zyxxx

zzzyzx

yzyyyz

xzxyxx

zyxVzzI

zyxVyyI

zyxVxxI

where

III

III

III

I

,,

2

,,

2

,,

2

),,()(

),,()(

),,()(



zyxzx

zyxyz

zyxxy

zyxVxxzzI

zyxVzzyyI

zyxVyyxxI

,,

,,

,,

),,())((

),,())((

),,())((

The normalized eigenvector matrix E of I is then obtained with

333231

232221

131211

eee

eee

eee

E


Principal axes Transformation Step 4: Computing the rotation matrix

The E is expanded to a product of rotation matrix by

311

211

321

sin

cossin

cossin

cossin0

sincos0

001

cos0sin

010

sin0cos

100

0cossin

0sincos

e

e

e

RRRE


Principal axes Transformation Step 5: Computing the transform matrix

The registration of image 1 to image 2 can be obtained by a translation to the center of mass coordinate system followed by the transform matrix

TEE 21


A 3-D model of brain ventricles obtained from registering 22 MR brain images using the PAR method.


Rotated views of the 3-D brain ventricle model shown in Figure 9.3.


Iterative Principal Axes Registration The Iterative Principal Axes Registration method c

an be used with partial volumes. For registering MR and PET brain images. The IPAR algorithm allows registration of two 3D image d

ata sets in which one of the data set does not cover the entire volume but has the subvolume contained in the other data set.

Let V1 and V2 represent two volumes to be registered, the IPAR method can be implemented using the following steps:

1. Find the full dynamic range of PET data and select a threshold T, which is about 20% of the maximum gray-level value. Extract binary brain regions using a region growing method on the thresholded PET slice data.

2. Threshold and extract binary brain regions from the MR data using a region growing method.


Iterative Principal Axes Registration

3. Stack 2-D binary segmented MR slices and interpolate as necessary to obtain cubic voxel dimensions using a shape-based interpolation algorithm.(3-D binary MR data)

4. Stack 2-D binary segmented PET slices and interpolate as necessary to obtain cubic voxel dimension to match the voxel dimension of brain MR data using a shape-based interpolation algorithm. .(3-D binary PET data)

5. Define a Field of View box, FOV(0) as a parallelepiped from the slices of the interpolated binary PET data to cover the PET brain volume

6. Compute the centroid and principal axes of the binary PET brain volume.


Iterative Principal Axes Registration

7. IPAR algorithm

8. Interpolate the gray-level PET data to match the resolution of MR data to prepare the PET data for registration with MR data.

9. Transform the gray-level PET data into the space of the MR slices using the last set of MR and PET centroids and principal axes. Extract the slices from the transformed gray-level PET data that match the gray-level MR image.


Iterative Principal Axes Registration The IPAR algorithm

For i =n to 0 do Compute the centroid and principal axes of the current binary

MR brain volume. Transform the augmented FOV (i) box according to the space

of the MR slices The PET data are registered with the MR data by performing t

he required translations and rotations1. Translate the centroid of the binary PET data to origin.2. Rotate the principal axes of the binary PET data to coincide with t

he x, y and z axes.3. Rotate the x, y and z axes to coincide with the MR principal axes.4. Translate the origin to the centroid of the binary MR data.

Remove all voxels of the binary MR brain which lie outside the transformed FOV(i) box.


Three successive iterations of the IPAR algorithms for registration of vol 1 and vol 2: The results of the first iteration (a), the second iteration (b) and the final iteration (c). Vol 1 represents the MR data while the PET image with limited filed of view (FOV) is represented by vol 2.

Iteration 1 Iteration 2


Iteration 3


Sequential slices of MR (middle rows) and PET (bottom rows) and the registered MR-PET brain images (top row) of the corresponding slices using the IPAR method.






Image Landmarks and Features based Registration

Once the corresponding landmarks or features are identified from in source and target image spaces, a customized transformation can be computed for registering the source image into the target image space. Relationships of corresponding points Relationships of corresponding feature such as surface


Similarity Transformation for Point-Based Registration

Assume that x and y are the corresponding points in the source and target image spaces belonging to the source X and target Y images.

A non-rigid transformation T(x) for registering the source image into the target image space can be defined by a combination of rotation, translation and scaling operations to provide x’ from x as

such that the registration error E is minimized as

where r, s and t represent the rotation, scaling and translation operations.

tsrxx

yxTxE )()(



A transformation should be obtained with r, s and t values to minimize the error function as

where wis are the weighting factors representing the confidence in the specific landmark (point) or feature correspondence and N is the total number of landmarks.

N

iiii ytsrxw

1

22



To register the source image into the target image space

1. Set s=1

2. Find r through the following steps Compute the weighted centroid of the body representing th

e set of landmarks in each spaces as

Compute the distance of each landmark from the centroid as

N

ii

N

iii

N

ii

N

iii

w

yw

y

w

xw

x

1

2

1

2

1

2

1

2

,

yyy

xxx

ii

ii



Compute the weighted co-variance matrix as

with a singular value decomposition as

where UtU=VtV=Iand

Compute

N

i

tiii yxwZ

1

2

tVUZ

0 ,, 321321 diag

tUVUVdiagr det,1,1



3. Compute the scaling factor

4. Compute the translation factor

N

iiii

N

iiii

xxrw

yxrws

1

2

1

2

xrsyt


Point-Based Registration

N

jjj

xp

xj

pj

pxN

Td

NNN

NjxX

NjpP

tRpp

1

2)(

1)(

,...,1},{

,...,1},{

)(

R is a 3x3 rotation matrix,t is a 3x1 translation vector,p is a 3x1 position vector.

<=Orthogonal Procrustes


Surface-Based Registration

Outlining contours on the serial slices of each scan.

Head is a stack of disks or “prisms”, each of which has cross section determined by one of the contour.

Hat is represented as a set of independent points.


Weighted Feature Based Registration Different optimization functions can be designed to improve

the computation of parameters of transformation for registration of the source image into the target image space.

A disparity function can be designed as

where {Xi} for I = 1, 2, 3, …, N represents a set of corresponding data shapes in x and y spaces.

The transformation T must minimize the disparity function register the source image into the target space utilizing the correspondence of geometrical features.

s iXN

i

N

jijijij yxTwTd

1 1

22


Weighted Feature Based Registration Determine the parameters for a rigid or non-rigid b

ody transformation T. Initialize the transformation optimization loop for k

=1 as

For each shape Xi in the source space, find the closest points in the corresponding shape in the target space Yi as

where Ci is the corresponding function.

001

0

ijij

ijij

xTx

xx

ixi

kiji

kij NjYxCy ,,3,2,1,,


Weighted Feature Based Registration

Compute the transformation between {xij(0)} and {yij

(k)} with the weights {wij}. Use the transformation parameters for registration

of the corresponding points as

Compute the disparity measure difference d(T(k))-d(T(k+1)), if the convergence criterion is met, stop; otherwise increment k and go to step 3 for next iteration.

01ij

kkij xTx


Point-Based Registration

Determinate the coordinates of corresponding points in different images, and the estimation of the geometrical transformation using these corresponding point.

Intrinsic points: anatomic landmark Extrinsic points: artificially applied markers


Surface-Based Registration

),(

)(1

)(1

2

jj

N

jjj

p

pTCy

pTyN

Tdp

The general approach is to search iteratively for therigid-body transformation T that minimizes the cost function:

is a point on the surface X

C is a correspondence function.


Surface-Based Registration(2) Coarse registration: Principal axes Transfor

mation Fine registration: surface fitting A rigid body is determined by the position o

f its center of mass and its orientation with respect to its center of mass (principal axes)

Compute the centroid and the three principal axes for a 3-D volume data


Surface fitting based on (E-Distance Transformation) The DT is performed on the 1st surface imag

es (base image) base image 2nd surface image (match ima

ge) to determine the registration parameters

matrixtiontransformatheisM

pppp

Mpqzn

yn

xnn

nn

),,(


Surface fitting based on (E-Distance Transformation)

P

nnQ

zyx

pMDP

PQMF

functionObjective

RRRTM

1

2 )(1

),,(

1000

tmmm

tmmm

tmmm

z333231

y232221

x131211

DQ is the distance map of the base image Qpn is the nth point of match image P


ICP Registration algorithm Iterative Closest Point, ICP

1. assigning one shape to be the “data” shape

2. assigning other shape to be the “model” shape

3. The “data” shape is decomposed into a point set

4.The “data” shape is registered to the “model” shape by iteratively finding “model” points closest to the “data” primitives.

ICP registration method defines the “corresponding” point yj to be the “closest” point on the surface


Volume-Based Registration 1. Roger P. Woods, Simon R. Cherry, and Jo

hn C. Mazziotta, ”Rapid automated algorithm for aligning and reslicing PET images”, Journal of Computer Assisted Tomography, 16(4): 620-633, 1992.

2. Roger P. Woods, John C. Mazziotta, and Simon R. Cherry, ”MRI-PET registration with automated algorithm”, Journal of Computer Assisted Tomography, 17(4): 536-546, 1993.


Basic Assumption

If two image sets are accurately aligned, then the value of any voxel in one image set is related to the value of the corresponding voxel in the other image set by a single multiplicative factor,R.


Basic Assumption

令 ri=ai/bi

where ai and bi is the value of voxel i in reference study and the corresponding voxel in reslice study.

r: standard deviation of ri over all voxels within the brain.

rmean: mean value of ri over all voxels within the brain.

Objective:

Minimize the r / rmean



Image Alignment Algorithm

Step 1:使用者指定 Reference study 與 Reslice study.Step 2: 使用者初步評估六個 reslice parameters, (x-,y-,z-

的旋轉與平移 )Step 3: Linear interpolation in z-axis, 產生 3D reference v

olume.Step 4: 利用 Step 2 的 reslice parameters 以 Trilinear int

erpolation 法 ,產生 3D reslice volume.Step 5: 計算 ratio volumn ri=ai/bi

Step 6: 利用 thresholding法 ,只留下腦部區域 . 並將其他地區的 ratio volumn設為 0. ( 對 reference volume)


Image Alignment Algorithm

Step 7: 計算平均 ratio volumn rmean, 及正規化的標準差 r / rmean .(此比值越小 ,代表越 uniform)

Step 8: 調整 reslice parameters, 求出使 brain/ rbrain最小化的 reslice parameters參數值 .

Disadvantage:

1. 無法進行不同模組影像間的對準2. 計算複雜度很高


MRI-PET Registration

Two modifications: Edit the MR images to exclude nonbrain structures

prior to registration. Partitions the MR image into 256 separate compon

ents based on the value of the MR pixels. Seeks to maximize the uniformity of the PET pixel

values within each of these partition.


MRI-PET RegistrationStep 1:移除MRI中非腦部區域 .(人工 )

Step 2:使用者評估六個 reslice parameters, (x-,y-,z-的旋轉與平移 )

Step 3:Construct the 3D volume of MRI and PET

Step 4:計算MRI與 PET中相對應 voxels的平均值 a’j及標準差 j

Step 5: 計算

Step 6: 計算

Step 7: 調整 reslice parameters使‘’最小化

MRI為 referencePET為 reslice

''

j

jj

a

jj j

jj n

n'''


MRI-PET Registration Disadvantage

Segmentation是一件困難的事無法適用於所有類型的醫學影像 , (打藥前後 )


不同對準法的比較

2-D Registration Point-Based Registration

缺點 :必須找出欲對準影像的特徵點包括 Intrinsic points( anatomic landmark) 及 Extrinsic points( artificially applied markers)

Lvv 優點 :

不需對影像進行特徵擷取的動作 . 缺點 :

由於不同模組影像特性不同 ,造成 ridge image會存在許多差異 .



3-D Registration Surface-Based Registration

缺點 : 1.必須建立 3-D volume 2.必須找出每張影像的輪廓 ,以輪廓來進行 ,因此只適用於相同

modality的影像 . Volume-Based Registration (R. P. Woods ’92, ‘93)

缺點 : 1.無法進行不同模組影像間的對準 2.計算複雜度很高 3.不同模組影像的對準必須事先 segmentation,可是 segmentati

on是一件困難的事



Iterative Closest Point(ICP) 缺點 :

1.必須事先求取影像的幾何特徵 .(point sets, line segment sets, triangle sets, surfaces, …)

2.


Block diagram for the MR image registration procedure

MR ReferenceBrain Image

Data

Global RegistrationIPAR

Algorithm

MR NewBrain Image

Data

AnatomicalReference

Model

LandmarksLocalization and

VOICharacterization

Expert ViewerEditing andValidation

Low-ResolutionDeformation and

Matching

Spatial Relaxationand Constraint

Adapation

High-ResolutionDeformation and

Matching

Multi-Resolution DeformationBased

Local Registration andMatching

Documents

Chapter 9 Image Registration