Enhancement of X-ray Fluoroscopy Image Sequences using ...liu.diva-portal.org/smash/get/diva2:21841/FULLTEXT01.pdfAn algorithm based on recursive filtering, for temporal noise suppression,

Enhancement of X-ray Fluoroscopy Image Sequences using Temporal Recursive Filtering and

Motion Compensation

Anni Forsberg

2006-03-07

LiTH-IMT/MI20-EX--06/427--SE

Enhancement of X-ray Fluoroscopy ImageSequences using Temporal Recursive Filtering and

Motion Compensation

Examensarbete utfört i medicinsk bildbehandlingvid Tekniska högskolan i Linköping

av

Anni Forsberg

LiTH-IMT/MI20-EX- -06/427- -SE

Handledare: Mats Anderssonimt, Linköpings universitet

Bo WärnbergSwemac Medical Appliances

Examinator: Hans Knutssonimt, Linköpings universitet

Linköpings tekniska högskola Institutionen för medicinsk teknik Avdelningen för medicinsk informatik Department of Biomedical Engineering Division of Medical Informatics

Rapportnr:

LiTH-IMT/MI20-EX--06/427--SE Datum: 2006-03-07

Svensk titel

Förbättring av fluoroskopibildsekvenser med användande av temporal rekursiv

filtrering och rörelsekompensering

Engelsk titel

Enhancement of X-ray Fluoroscopy Image Sequences using Temporal Recursive

Filtering and Motion Compensation

Författare

Anni Forsberg

Uppdragsgivare:

Swemac Medical Appliances/

IMT

Rapporttyp:

Examensarbete Rapportspråk:

Engelska

Sammanfattning This thesis consider enhancement of X-ray fluoroscopy image sequences.

The purpose is to investigate the possibilities to improve the image

enhancement in Biplanar 500, a fluoroscopy system developed by

Swemac Medical Appliances, for use in orthopedic surgery.

An algorithm based on recursive filtering, for temporal noise

suppression, and motion compensation, for avoidance of motion

artifacts, is developed and tested on image sequences from the

system. The motion compensation is done both globally, by using

the theory of the shift theorem, and locally, by subtracting

consecutive frames. Also a new type of contrast adjustment is

presented, received with an unlinear mapping function.

The result is a noise reduced image sequence that shows no

blurring effects upon motion. A brief study of the result shows,

that both the image sequences with this algorithm applied and the

contrast adjusted images are preferred by orthopedists compared to

the present images in the system.

Nyckelord Keywords

Fluoroscopy, image enhancement, recursive filtering, motion compensation.

Bibliotekets anteckningar:

Abstract

This thesis consider enhancement of X-ray �uoroscopy image sequences. Thepurpose is to investigate the possibilities to improve the image enhancementin Biplanar 500, a �uoroscopy system developed by Swemac Medical Appli-ances, for use in orthopedic surgery.

An algorithm based on recursive �ltering, for temporal noise suppression,and motion compensation, for avoidance of motion artifacts, is developedand tested on image sequences from the system. The motion compensationis done both globally, by using the theory of the shift theorem, and locally,by subtracting consecutive frames. Also a new type of contrast adjustmentis presented, received with an unlinear mapping function.

The result is a noise reduced image sequence that shows no blurring e�ectsupon motion. A brief study of the result shows, that both the image se-quences with this algorithm applied and the contrast adjusted images arepreferred by orthopedists compared to the present images in the system.

i

Acknowledgements

First of all I want to thank the employees of Swemac Medical Appliances, forbeing helpful and friendly and for letting me do my thesis on image sequencesfrom their Biplanar 500 system. Special thanks are given to Bo Wärnbergfor supporting me with material and explaining the functions of the systemand to Lars Granath, who helped me get in contact with the orthopedists.

Secondly I want to thank my supervisor Mats Andersson for giving me guid-ance and many ideas. My examiner Hans Knutsson has also been availablefor valuable discussions.

Thanks are also given to the orthopedists at the University Hospital inLinköping, for letting me attend surgeries, and to the orthopedists at theHospital in Nyköping, who gave me their opinions on my results.

I also want to thank So�e Åberg for acting as opponent to my work andJenny Hammarfors for proofreading the report during development. Thankyou both for fruitful discussions and nice company during this period.

Finally I want to thank David Bäck for being supportive during my ups anddowns and for discussing and proofreading the thesis.

Anni ForsbergLinköping, March 2006

iii

Contents

1 Introduction 11.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Purpose and task description . . . . . . . . . . . . . . . . . . 11.3 Resources and limitations . . . . . . . . . . . . . . . . . . . . 21.4 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Overview of X-ray �uoroscopy imaging 52.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52.2 The �uoroscopy chain . . . . . . . . . . . . . . . . . . . . . . 6

2.2.1 X-ray generation and the X-ray tube . . . . . . . . . . 62.2.2 The image intensi�er . . . . . . . . . . . . . . . . . . . 72.2.3 The CCD-camera . . . . . . . . . . . . . . . . . . . . . 82.2.4 Image processing and presentation . . . . . . . . . . . 9

2.3 The �uoroscopy image . . . . . . . . . . . . . . . . . . . . . . 9

3 The Biplanar 500 system 113.1 System components . . . . . . . . . . . . . . . . . . . . . . . . 113.2 The acquisition into Matlab . . . . . . . . . . . . . . . . . . . 133.3 Input signal properties . . . . . . . . . . . . . . . . . . . . . . 14

3.3.1 Interlaced video . . . . . . . . . . . . . . . . . . . . . . 143.3.2 Spectrum of the input signal . . . . . . . . . . . . . . 163.3.3 Noise aspect . . . . . . . . . . . . . . . . . . . . . . . . 17

3.4 Present image processing . . . . . . . . . . . . . . . . . . . . . 17

4 Usability aspects 214.1 The human observer . . . . . . . . . . . . . . . . . . . . . . . 214.2 Attending surgery . . . . . . . . . . . . . . . . . . . . . . . . 224.3 The areas of interest . . . . . . . . . . . . . . . . . . . . . . . 224.4 Test image sequences . . . . . . . . . . . . . . . . . . . . . . . 24

5 Theory of the Fourier transform 315.1 The 2D Discrete Fourier Transform . . . . . . . . . . . . . . . 315.2 Convolution and the convolution theorem . . . . . . . . . . . 32

v

5.3 The shift theorem . . . . . . . . . . . . . . . . . . . . . . . . . 335.4 The phase correlation method . . . . . . . . . . . . . . . . . . 34

6 Theory of image enhancement 376.1 Interpolation methods . . . . . . . . . . . . . . . . . . . . . . 376.2 Thresholding . . . . . . . . . . . . . . . . . . . . . . . . . . . 386.3 Morphological operations . . . . . . . . . . . . . . . . . . . . 386.4 Contrast adjustment . . . . . . . . . . . . . . . . . . . . . . . 39

6.4.1 Gray scale mapping . . . . . . . . . . . . . . . . . . . 406.5 Noise reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 41

6.5.1 Averaging . . . . . . . . . . . . . . . . . . . . . . . . . 426.5.2 Median �ltering . . . . . . . . . . . . . . . . . . . . . . 426.5.3 Temporal averaging . . . . . . . . . . . . . . . . . . . 436.5.4 Temporal recursive �ltering . . . . . . . . . . . . . . . 44

7 Implementation and results 477.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477.2 Preprocessing algorithms . . . . . . . . . . . . . . . . . . . . . 48

7.2.1 RGB to gray scale transformation . . . . . . . . . . . . 487.2.2 Cropping . . . . . . . . . . . . . . . . . . . . . . . . . 487.2.3 Windowing . . . . . . . . . . . . . . . . . . . . . . . . 49

7.3 Contrast adjustment . . . . . . . . . . . . . . . . . . . . . . . 517.4 Deinterlacing . . . . . . . . . . . . . . . . . . . . . . . . . . . 527.5 Temporal recursive �ltering . . . . . . . . . . . . . . . . . . . 537.6 Temporal averaging . . . . . . . . . . . . . . . . . . . . . . . . 557.7 Global motion compensation . . . . . . . . . . . . . . . . . . . 56

7.7.1 Calculation of translation . . . . . . . . . . . . . . . . 567.7.2 Adapting the temporal impulse response . . . . . . . . 577.7.3 Shifting the impulse response . . . . . . . . . . . . . . 587.7.4 Width of the impulse response . . . . . . . . . . . . . 60

7.8 Local motion compensation . . . . . . . . . . . . . . . . . . . 607.9 An alternative method to detect global motion . . . . . . . . 63

7.9.1 Dividing frames into overlapping sections . . . . . . . 637.9.2 Representing shifts in vector plots . . . . . . . . . . . 637.9.3 The mean-std method . . . . . . . . . . . . . . . . . . 64

7.10 The combined algorithm . . . . . . . . . . . . . . . . . . . . . 65

8 Evaluation study 678.1 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 678.2 Manual contrast adjustment . . . . . . . . . . . . . . . . . . . 678.3 Comparison between di�erent mapping

functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 688.4 Noise reduction and blurring . . . . . . . . . . . . . . . . . . . 70

vi

9 Discussion 759.1 Contrast adjustment . . . . . . . . . . . . . . . . . . . . . . . 759.2 The proposed algorithm . . . . . . . . . . . . . . . . . . . . . 77

9.2.1 Global motion . . . . . . . . . . . . . . . . . . . . . . 779.2.2 Local motion . . . . . . . . . . . . . . . . . . . . . . . 789.2.3 The accuracy of shift calculation . . . . . . . . . . . . 799.2.4 Limitations . . . . . . . . . . . . . . . . . . . . . . . . 809.2.5 Time requirements . . . . . . . . . . . . . . . . . . . . 80

10 Conclusion 8110.1 Short summation . . . . . . . . . . . . . . . . . . . . . . . . . 8110.2 Recommended algorithm for Biplanar 500 . . . . . . . . . . . 8110.3 Future work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

10.3.1 Further investigation before usage of the algorithm . . 82

A The combined algorithm 91

vii

Abbreviations

CCD Charged Coupled DeviceDFT Discrete Fourier TransformFD Fourier DomainFFT Fast Fourier TransformFIR Finite Impulse ResponseFT Fourier TransformHP High PassIIR In�nite Impulse ResponseLP Low PassNN Nearest NeighborSD Spatial DomainVFM Vision for MatlabVCC Video Capture Cable

Glossary

�eld One frame is built up of 2 �elds, one consisting ofodd pixel rows and one of even pixel rows.

frame An image sequence is built up of a number of frames,i.e. images that correspond to a speci�c point of time.In this thesis, the frequency used is 25 frames per second.

interlaced A video format where the di�erent �elds of one frameare shown at di�erent times.

progressive A video format where the di�erent frames are shownconsecutively.

pixel The smallest picture element, with the position (x,y)in the image matrix. In this thesis, every pixel has a gray valuebetween 0 and 255, i.e. a depth of 8 bits.

ix

Chapter 1

Introduction

This chapter will introduce the reader to this thesis by giving a backgroundto and a description of the task. In the end an outline is found to guide thereader through the thesis.

1.1 Background

This thesis consider enhancement of the images from a Biplanar 500, a digitalX-ray �uoroscopy system, used in orthopedic surgery. The system is devel-oped by Swemac Medical Appliances AB, a company which produce and sellmedical equipment. Most of this work has been done at Swemac's o�ce inLinköping, Sweden. The thesis is done as a part of the Master of Science inApplied Physics and Electrical Engineering at Linköping University.

The Biplanar 500 system shows continuous image sequences, so that theorthopedic surgeon can get images of the area of interest in real-time dur-ing operation. Today the image enhancement in the system contains noisesuppression, edge enhancement (on 2D snapshots only) and contrast adjust-ment. The orthopedists using the system experience the images as noisy andthink that the contrast adjustment is poor.

1.2 Purpose and task description

The purpose of this thesis is to investigate the possibilities to improve theimage enhancement in Biplanar 500. It is interesting for Swemac to knowif better images can be received and in that case, how much it di�ers fromtoday's quality.

1

CHAPTER 1. INTRODUCTION 2

The task is to develop an algorithm that increase the contrast, reduces noiseand compensates for motion, with preservation of edges and lines, in a 3Dimage sequence.

To do this, the task has been divided into three major areas:

• Testing of the existing system and evaluation of the input signal

• Developing algorithms for continuous video (3D) in Matlab

• Evaluating the results in the meaning of image quality and time re-quirements

where the main focus is put on development of algorithms. Since Matlabcannot process in real-time, the image quality is of higher priority than afast algorithm.

1.3 Resources and limitations

A Biplanar 500 system, a PC with Windows XP and a VCC (Video CaptureCable) are provided. The image enhancement algorithms are developed apartfrom the current image processing in the system, since the source code is notknown. The algorithms are created in the calculation programme Matlabwith the toolboxes Signal Processing and Image Processing. Implementationof the algorithms in the Biplanar 500 system is not included. The imagesused are hip images, a common application for the Biplanar 500 system. Thealgorithms are not tested on other images.

1.4 Outline

In chapter 2 a brief overview of X-ray �uoroscopy imaging is given. Thefunction of di�erent components in the �uoroscopy chain are explained. Inchapter 3 the Biplanar 500 system is described, properties of the input signalare investigated and the results from system tests are presented. Chapter 4consider the human observer and why it is important to get an orthopedistsopinion about the images. The areas of interest in the images are speci�edand the test sequences are presented and motivated.

Chapter 5 consider the theory of the Fourier transform and the phase corre-lation method. In chapter 6 some basic image operations are described andthe image enhancement methods used in the algorithms are presented.

In chapter 7, a description is given of how the algorithms have been de-veloped. The theory from chapter 5 and 6 is here applied to the speci�c

CHAPTER 1. INTRODUCTION 3

problem. The results are presented as well as solutions to the problemsarising.

Chapter 8 presents an evaluation study of the results. The results are dis-cussed in chapter 9, in the meaning of image quality and time requirement.Finally, in chapter 10, conclusions are drawn and recommendations are givento Swemac. Also future work is considered.

Chapter 2

Overview of X-ray �uoroscopyimaging

In this chapter an overview of X-ray �uoroscopy is given. Some �elds ofapplication are mentioned and the basic theory of X-ray generation and theimage chain are presented.

2.1 Introduction

X-ray �uoroscopy is a diagnostic method used to get X-ray images of apatient in real-time. It is often used to get a quick overview of the patientand the X-ray �eld so that changes can be made to the position of the patientand the collimation of the �eld. [10]

The real-time aspect also makes it possible to study the motions of organs, forinstance the pulsations of the heart. Another application is in angiography,where a contrast agent spreads along the blood vessels. A foreign object like anail can be seen clearly and the fracture of a bone can easily be reponed. [10]This is why �uoroscopy often is used in orthopedic surgery to get informationabout a fractured bone's position when putting it together with nails.

Since the excitations of the X-ray tubes in �uoroscopy are longer than inconventional X-ray investigations, the dose to the patient is much higher.Therefore it is important to get images of good quality fast to get as lowdose as possible to the patient. Low dose implies noisy images and thus thisis often the main problem to solve in �uoroscopy image enhancement. [16]

To reduce the dose to the patient and the surgeon, pulsed �uoroscopy canbe used but this method is not further treated in this thesis.

5

CHAPTER 2. OVERVIEW OF X-RAY FLUOROSCOPY IMAGING 6

2.2 The �uoroscopy chain

General �uoroscopy systems consist of an X-ray tube that generates X-rays,an image intensi�er, a CCD-camera that works as a detector and a computerthat processes the image and presents it on a screen to the user. [10]

A schematic overview of the �uoroscopy chain is given in �gure 2.1. Thesubsections below cover the function of these di�erent parts.

Figure 2.1: Schematic overview of the �uoroscopy chain.

2.2.1 X-ray generation and the X-ray tube

The X-ray generation in a �uoroscopy system is made in the X-ray tube. Itis an evacuated tube with a high potential di�erence from one side to theother. [3, 10] An overview of the X-ray tube is given in �gure 2.2.

Figure 2.2: The X-ray tube and its components.

On one side there is a positively charged anode with a target media. The


target media is characterized by a high atomic number, to maximize the X-ray generation, and a high melting point, to manage the heat that will occuron the anode surface. To further cool the anode, it often rotates so that theheat get spread over a bigger area. [3, 10]

On the other side is the negatively charged cathode, surrounding a glowing�lament that gets heated up by a tube current. When the tube current is�owing through the �lament, electrons are emitted. The free electrons arethen focused in the electric �eld to the focusing spot on the anode. [3, 10]

When the electrons hit the target atoms on the anode, two things mayhappen. In 99 % of the cases, heat is generated from the kinetic energyof the electrons. The remaining part of the electrons, about 1 %, generatesX-rays. This is done via bremsstrahlung or characteristic radiation. [5, 3]For further reading of these processes, see for instance [5].

The X-rays generated are emitted in all directions but only the ones in aspeci�c angle, see �gure 2.2, are emitted out from the tube. There is a thinwindow, forming the X-ray beam that can be used in the �uoroscopy chain.[5]

The variable parameters of the X-ray tube are the voltage of the electrical�eld (kV), the time of excitation (s) and the tube current (mA). They arecorrelated as

E = mA · s · kV 2 (2.1)where E is the amount of emitted X-rays from the tube. When increasingthe voltage, the electrons have more energy and therefore hit the anode ina higher speed. This will cause a higher number of emitted X-rays, sincethe higher energy increase the probability of X-ray generation. The X-raysgenerated are of an increased energy. Increasing the current generates anincreased number of electrons emitted from the �lament. The X-ray gener-ation will then increase in the same amount. The excitation time of coursealso have an in�uence, the longer excitation, the more X-rays are generated.[5, 10]

2.2.2 The image intensi�er

The image intensi�er is an evacuated tube that transform X-rays into visi-ble light and intensi�es the image in a few steps. The potential di�erencebetween the input and output screen is approximately 25-30kV. [5]

In �gure 2.3 the components of the image intensi�er are shown.


Figure 2.3: The Image Intensi�er and its components.

The X-ray beam hits the input �uorescent screen, often made of CsI1, dopedwith Na. The �uorescent screen absorbs the X-rays photons and visiblelight photons are emitted in proportion to the incident X-rays. Next tothe �uorescent screen is a photocathode where electrons are pushed out viaphoto electrical e�ect. The photocathode is made of a material that matchwell with the spectral output from the CsI screen. [5]

The electrons are then accelerated throughout the tube in the electrical �eldcaused by the potential di�erence. The electrons are focused with positivelycharged electrodes (focusing elements) to the output �uorescent screen, oftenmade of ZnCdS2, doped with Ag. At the output �uorescent screen, electronsare transformed into photons. [5]

The acceleration of the electrons increases their energy and more light isthen emitted. The focusing makes the output image even more intensi�ed,since the output screen is much smaller than the input screen. The inputscreen and photocathode are convex to make sure that all the electrons hasthe same length of transportation to the output screen. The image receivedon the output screen is upturned and reversed in relation to the image onthe input screen. [5]

2.2.3 The CCD-camera

A digital detector often used in digital �uoroscopy systems is the CCD-camera. The advantages with CCD-cameras are their linear characteristicsand small dimensions. They also have a fast recovering after exposure. [12]

1Cesium Iodide2Zinc-Cadmium Sul�de


The CCD is a solid-state chip camera, consisting of an array with photodi-odes. The light incident to every photodiode excite valence electrons to theconduction band. This leads to an electric charge for every diode, propor-tional to the number of photons. Each diode correspond to one pixel in theimage, but is not read directly. Instead, the charges of the diodes are read,one row at a time, to an analogous video signal. [12]

The scanning of values can be done in di�erent ways, either progressive,or interlaced. See section 3.3.1 for further explanation of the two di�erentmethods. Often the signal intensity is adjusted by a logarithmic step, tobetter suit the visual system of the human observer. This is done in thecamera of the Biplanar 500 system.

2.2.4 Image processing and presentation

The image sequence from the CCD-camera is processed in a computer. Thisprocessing can for instance reduce noise, cut the important parts out orenhance the edges and lines. This is where the algorithms produced in thisthesis �t into the �uoroscopy chain. The processed images are then presentedto the human observer. The human observer will be discussed further insection 4.1.

2.3 The �uoroscopy image

The �uoroscopy image is essentially built up in the same way as an X-rayimage and has the same appearance. The X-ray image is a mapping ofthe objects ability to attenuate X-rays. An image occur since the densitydi�ers between di�erent tissues in the body. Bones for instance, have a highdensity, while soft tissue, e.g. fat or muscles, has a much lower density. Theconsequence is that the image gets dark in the bone-areas (low signal) andlighter in the soft tissue areas (high signal).

The appearance of the image can be changed by adjusting the X-ray tubeparameters to the current object examined. A high voltage gives a low doseto the patient but will cause a larger amount of scattering. This is often usedwhen bones are to be represented, su�cient contrast is given to the imageanyway. [5, 10]

When representing soft tissue, a much lower voltage is needed to visualizethe contrast between di�erent soft tissues, who's density di�erence is quitesmall. One can use a contrast agent, that have a di�erent density, to getbetter contrast with a higher voltage. This will decrease the dose, since more


photons are getting through the object. This is done in angiography wherethe subject of interest is the blood �ow. [5, 10]

In �uoroscopy, the thickness often vary in di�erent parts of the patient ex-amined. This may result in an image that is under- or overexposed in someareas. [10]

Chapter 3

The Biplanar 500 system

This chapter describes the Biplanar 500 system. The components, the prop-erties of the input signal, the present image processing and test results arehere accounted for.

3.1 System components

The G-stand of Biplanar 500, see �gure 3.2, include two sets of X-ray tubes,image intensi�ers and CCD-cameras to get images from two di�erent views(frontal and sagittal), without having to move the stand between the series.According to [10], at least two perpendicular images have to be taken to setthe exact position of a bone fracture and that is the main idea behind theconstruction of the G-stand.

During excitation, the output from the CCD-camera leads via a frame grab-ber, that digitizes the signal, to a computer. Here, the present image process-ing is done. The result is shown on two big screens, one for each view. Thereis also a user interface, where the orthopedist can write information aboutthe patient, control the excitation of the X-ray tubes, choose which imageenhancement to use and store images to �les.

Figure 3.1 gives a schematic overview of the Biplanar 500 system (for oneview). Figure 3.2 shows the Biplanar 500 system and its components. Sincesome parts of the Biplanar 500 system, including the image processing, arenot made by Swemac and thus unknown, a number of tests are done to geta brief view of the prerequisites of the task.

11

CHAPTER 3. THE BIPLANAR 500 SYSTEM 12

Figure 3.1: Schematic overview of the Biplanar 500 system (for one view).

Figure 3.2: The Biplanar 500 system. Left: The G-stand with X-raytubes, image intensi�ers and CCD-cameras marked. Right: The two viewingscreens and the user interface.


3.2 The acquisition into Matlab

To get the video signal into the computer a VCC is used. Grabbing of theframes into MatLab is done with the free software VFM (Vision For Matlab).In �gure 3.3 the arrangement of acquisition is shown.

Figure 3.3: Schematic overview of the acquisition into Matlab.

This arrangement has been used both when testing the system and for grab-bing the test sequences described in section 4.4.


3.3 Input signal properties

The resolution of the video from the CCD-camera is 576x720 pixels and theframe rate is 25 frames per second. An example of an input signal, an imageof a hip with a nail inserted, is shown in �gure 3.4.

Figure 3.4: Example of an input image: A hip with a nail. kV=79, mA=2.5

As one can see, the image has rather similar gray values in di�erent parts,both in the bone tissue and in the soft tissue. This is because the surgeonneeds to see the di�erence between the compact and the spongy bone tissue.Thus the X-ray voltage and current has been chosen so that these bordersare as clear as possible.

3.3.1 Interlaced video

In interlaced video, the odd and even pixel rows in one frame are drawn atdi�erent times. Two �elds, one consisting of the odd and one of the evenrows, build up one frame, see �gure 3.5. [8, 12]

This format is often used when the video is to be shown on a TV monitor.The advantage with interlaced video, is that you can get the impressionof 50 frames/s with only 25 frames/s. The human eye do not notice thetime di�erence, thus experience the video as normal. When displaying video


Figure 3.5: The principle of interlaced video.

on a computer monitor, often progressive video is used. Progressive videodraws the whole frame simultaneously, with the frequency of 25 frames/s forinstance. [8, 12]

In the Biplanar 500 system, the image sequences are in interlaced format butare presented progressive, i.e. frame-wise, on the two viewing monitors. Thisleads to interlacing artifacts upon motion. To investigate this phenomenon,a test was done in the system with a lead disc as test object. The discwas moved vertically and horizontally in the X-ray �eld during acquisitioninto Matlab. A distinct interlacing pattern was seen when the disc wasmoved horizontal in the X-ray �eld. See �gure 3.6 for the interlacing pattern.This will lead to problems during image enhancement, since the artifactsintroduces high spatial frequencies in the wrong directions (vertical in �gure3.6).

380 390 400 410 420 430 440

250

255

260

265

270

275

280

285

290

295

Figure 3.6: Distinct interlacing pattern when moving the disc from right toleft. A high spatial frequency is introduced in vertical direction.

In the Biplanar 500 system the interlaced video draws the even �elds before


the odd ones. Since the frame rate is 25 frames/second, an update of �eldsis made every 1/50 second.

3.3.2 Spectrum of the input signal

To analyze the input signal, an image and its corresponding spectrum isstudied (the interesting part of the frame is cut out and windowed as will bedescribed in section 7.2.3). See �gure 3.7.

Figure 3.7: An input image and its corresponding spectrum.

In the plot of the spectrum the intensity is concentrated to lower frequencies,which is the common case for most images. There is also a distinct whitehorizontal line, that corresponds to the high contrast nail in the image, whichhas a well de�ned direction.

The representation of the frequencies is not fully circular and maybe thatdepends on the limitation in bandwidth in the CCD-camera. It is moredi�cult for the camera to handle high frequencies horizontally than verticallybecause of the time and scanning aspect. Two consecutive pixels in the samerow are scanned after each other, while two consecutive pixels in the samecolumn are drawn with one row in between (or in interlaced video in thenext �eld). This result in limited frequency in the x-direction, seen as thespread in the y-direction of the spectrum.

Clear dark vertical borders are also seen. Probably the VCC is the source ofthis error. In the next section we will see that the borders are not presentin the spectrum of the output signals from the system.


3.3.3 Noise aspect

The noise in the image chain can appear in the CCD-camera when the pho-tons are collected, in the image intensi�er or in the cables. [12] The noiseappearing in the camera is assumed to have Poisson distribution and theother types of noise are random.

Testing the properties of the input signal from the CCD-camera shows thatthere is a lot of noise, even when the X-ray tube is not excited. Since no lightis coming from the image intensi�er unless there is X-rays incident on theinput screen, this noise is only derived from the camera. To get a picture ofhow this noise looks, frames are captured when the X-ray tube is not excited.The noise image and its corresponding spectrum are shown in �gure 3.8.

Figure 3.8: The image without X-ray excitation that describes the noise fromthe CCD-camera and its corresponding spectrum (DC-component removed).

The noise is spread in the spectrum and not concentrated to lower frequenciesas images usually are. The white dots might imply that there is some sortof periodic part in the noise.

3.4 Present image processing

To get a hint of how the image processing is made today, a test was done.By reading images into Matlab and at the same time save the correspondingoutput image in the user interface, two images were received that could becompared. The test was done using an image of a hip with a nail, similar tothe one presented previously in this chapter.

A cutout of the important part was made in the Matlab image, similar to


the cutout made previously. Then the 1024x1024 image from the systemwas downsampled to 512x512 so that the images could be compared. Thedi�erent images are shown in �gure 3.9. The image saved in the user interfacewas rotated 90 degrees compared to the input signal and could simply berotated back when comparing them. The spectrums are shown in �gure 3.10.

Figure 3.9: The input image, the original output image and the edge enhancedoutput image.

Figure 3.10: The spectra of the input image, the output image with no en-hancements and the output image with edge enhancement.

It is interesting that the spectrum of the output signal does not have the darkareas on the sides as the input spectrum has. This leads to the conclusionthat the stripes originate from the VCC and not from the camera.

Contrast adjustment Unfortunately, a contrast enhanced image can notbe saved in the system. The quality of the contrast adjusted image on thescreen is not good. The contrast reinforcement is too drastic and the imagegets to dark.

Edge enhancement To the right in �gure 3.9, the edge enhancement re-inforces the noise. This is probably the reason why the surgeons are notsatis�ed with the images. In the corresponding spectrum, �gure 3.10, thehigher frequencies are reinforced.

Noise suppression The noise suppression is done by averaging a number


of images in the sequence. When no contrast or edge enhancement is used incombination with the noise suppression, the images are not very noisy. Themore images used in the averaging, the more blurring occurs when an objectis moving.

Late in this thesis there was a discussion with the creator of the present imageenhancement. It turned out that the contrast adjustment is a linear grayscale transformation inside a speci�ed window. The limits of the windowwas computed in a function, that used the lowest and highest gray values.See chapter 6 for further reading about contrast adjustment.

The present edge enhancement is done with unsharp masking. It is a methodoften used in X-ray imaging. First, the image to enhance is LP �ltered. Toget the mask, the blurred image is subtracted from the original. The resultis then computed as the mask added to the original. [15] This is equal toadd he HP �ltered image to the original, the high frequent components ofthe image will be reinforced. The method of unsharp masking will not befurther treated in this thesis.

There is also a pixel based motion detector available in the Biplanar 500system. If the intensity value in a pixel change a lot from one frame to thenext, it will be considered as motion. This motion detection is not used inthe system today, since it is too sensitive for noise. When using it, motionis detected much to often and therefore the system never suppresses noise.

Chapter 4

Usability aspects

Di�erent people interpret images in di�erent ways, depending on the back-ground of the observer. This chapter describes how this is taken into accountin this thesis, to get the best result possible. The areas of interest in the imageare speci�ed. Finally the chosen test sequences are presented and motivated.

4.1 The human observer

We all se di�erent things when we look at an image. An X-ray physicianknows exactly where too look for disorders, which lines and areas that areof importance. An image analyst has, on the other hand, a di�erent relationto noise and contrast and often think that the most important in the imageare the clear edges and lines. An orthopedic surgeon have yet another rela-tionship to the image, since his or her speciality is to operate and to placenails and drills in the right position. Thus the areas of interest in the imagedepend on the observer.

In �uoroscopy, the human observer is of big importance, since the imagesare interpreted in real-time. The human observer spatially and temporallyprocesses the image to reduce the e�ective noise in his or her visual system.For instance, when looking at a noisy image sequence, often a low contrastobject can be seen until the display stops. This is because the temporal�ltering of noise then stops and the object disappear in noise. [16]

As mentioned in section 2.2.3, images received with a CCD camera are linear,meaning that the intensity values are proportional to the incident photons.The human visual system, on the other hand, is logarithmic, i.e. the eyeresponds logarithmical to changes in light intensity. [12, 5]

This fact is used in some image processing methods that will not be imple-

21

CHAPTER 4. USABILITY ASPECTS 22

mented in this thesis. For further reading of the human visual system, seefor instance [16, 7, 1, 14].

Since the subjective judgement is very important in this application it is ofbig importance that the result is evaluated by the users. In the evaluation ofthe algorithm later on, the resulting image sequences are shown to a numberof orthopedists, see chapter 8.

4.2 Attending surgery

To get a result that is satisfactory in this thesis it is important to achieveknowledge about what parts of the images that are of interest in the eye ofan orthopedist. Otherwise perhaps some important structures are removedfrom the images during processing. To achieve this knowledge and to seehow the Biplanar 500 system is used in reality, an orthopedic surgery at theUniversity Hospital in Linköping, was attended.

It showed that both the patient and the Biplanar system are moved a lotduring operation, mostly to get a good overview and to �nd the speci�c anglefrom which the images give the best information. When drilling a hole for anail the patient is still and only the drill is moving. This is useful informationwhen constructing the motion compensator.

4.3 The areas of interest

After discussion with Swemac and after attending surgery, the conclusionis drawn that the most important areas in the Biplanar 500 images are thebone structures. It is desired to improve the edges and lines that separatethe bone from other tissue and the borders between the compact and thespongy bone tissue. Also an increased contrast is wanted of the nail and drillborders to distinguish them from the patient.

In the images the joint area are of great importance, since the nail has toreach the hard parts of the bone but not as far as to the joint area. Animportant line which is wanted clearer is called the caput line. The areas ofinterest are shown in �gure 4.1.


Figure 4.1: The important lines and areas in the input images (frontal view).


4.4 Test image sequences

To get a good result that can be of interest for Swemac, it was important toget test image sequences that as much as possible looked like the ones duringoperation.

A phantom of a human, see �gure 4.2, was used to get image sequences ofthe most important area for which Biplanar 500 is used, the hip. Nails anddrills were stuck into the phantom to get a more realistic image, with darkobjects in the main area.

Figure 4.2: The phantom used in the test sequences.

The phantom gives better images than in reality, since the patients oftenare old and their body therefore is not as exemplary as the phantom. Thelegs are also missing on the phantom, whereas in a real patient there is a lotof adjustments of the legs before the images cover the right area. However,since the phantom can be used many times during testing, regardless of dose,it was chosen to be the best test object.

Test image sequences were grabbed into Matlab at di�erent times during thiswork. At �rst only still image sequences were grabbed, to see if the signalwas ok. Later on, two di�erent kinds of sequences were grabbed, sequenceswith local motion and sequences with global motion. This was to see howthe algorithms handled motions. During testing of the algorithms, even moresequences were grabbed.

The test sequences are in both frontal and sagittal view. In table 4.1 below,the main test sequences, used during development, testing and evaluation,


Seq. View Frames Time DescriptionB Frontal 65 2.6s Alternating between still phantom

and fast global motion.L Sagittal 55 2.2s First still phantom,

then slow global motion.F Frontal 61 2.44s No motion followed by

slow local motion.C Sagittal 31 1.24s Fast local motion.

Table 4.1: Facts about the test sequences used.

are described. The sequences are cropped to the size 256x256 pixels, to getthe algorithms less computationally demanding during development.

Figure 4.3-4.6 show some chosen frames of the sequences. It is relativelydi�cult to render image sequences on paper, but hopefully it is su�cient toclarify the examples later on in this report.


Figure 4.3: The B test sequence, frontal view. Alternating between still andfast moving phantom.


Figure 4.4: The L test sequence, sagittal view. No motion followed by slowglobal motion.


Figure 4.5: The F test sequence, frontal view. No motion followed by slowlocal motion.


Figure 4.6: The C test sequence, sagittal view. Fast local motion.

Chapter 5

Theory of the Fouriertransform

In this chapter the theory of the Fourier transform is presented. The di�erentuseful theorems are discussed and the phase correlation method is explained.

5.1 The 2D Discrete Fourier Transform

The Fourier transform is often used in image processing, since importantfeatures of the image, such as orientation, frequency and phase, are ofteneasier to describe in the Fourier Domain (FD) than in the Spatial Domain(SD). An image is a 2D discrete and truncated signal, i.e. the signal is onlyde�ned in a �nite number of points. [7] To transform a discrete image intothe FD, the discrete fourier transform (DFT) is used, de�ned as

F{f(x, y)} = F (u, v) =N−1∑

x=0

M−1∑

y=0

f(x, y)e−j2π(xuN

+ yvM

) (5.1)

where f(x, y) is the image in the SD and F (u, v) is the DFT of the image inthe FD, where u, v are frequencies. In this thesis the fast fourier transform(FFT) will be used to calculate the DFT. It is done by separating the suminto smaller sums that are not so demanding to compute. The complexitygoes from O(N2) with DFT to O(NlogN) with FFT for N samples. [6]

The DFT is complex and therefore often the power spectrum, |F (u, v)|2,is used instead. The spectrum shows the frequency contents of the image.The intensity in the center of the spectrum correspond to low frequenciesand further out, higher frequencies are represented. If, for instance, an

31

CHAPTER 5. THEORY OF THE FOURIER TRANSFORM 32

image of a vertical line is transformed, the corresponding spectrum will showa horizontal line, due to the higher frequency of the borders of the linehorizontally in the image.

When using DFT, artifacts will occur, called wrap-around-errors. They occursince, in the FD, the image is treated as a periodic signal, meaning that at theborders, the image is repeated. If the content of the image di�er a lot fromthe left to the right side, for instance, this will appear as a sharp horizontalline in the spectrum. [7]

To avoid these artifacts a window can be used; i.e. a weighting function,that has a unit value in the middle and softens the image down to zero atthe borders. When using a window like this, the representing of the imagecontent will also be a�ected but this is often negligible. This windowed DFTis often the image meant, when referring to a Fourier transformed image. [7]

5.2 Convolution and the convolution theorem

Convolution is an operation performed through shifting a kernel across animage. The discrete convolution used on images is de�ned as

h(x, y) = f(x, y) ∗ g(x, y) =N−1∑

α=0

M−1∑

β=0

f(x− α, y − β)g(α, β) (5.2)

where f(x, y) is the image and g(x, y) is the kernel in the SD. When the kernelgets outside of the image at the borders, either zero padding or repetition ofthe border pixels is used to �ll the empty spaces.

The convolution theorem says that convolution in the SD correspond tomultiplication in the FD. Hence,

f(x, y) ∗ g(x, y) ⇐⇒ F (u, v)G(u, v) (5.3)

where F (u, v) and G(u, v) are the DFTs of f(x, y) and g(x, y) respectively. Sowhen �ltering an image there is always two ways to do the same operation.Depending on in what situation the �ltering is to be performed, the bestmethod can be chosen. It is easier to control the locality using convolutionwith a small kernel in the SD. When the �lter kernel is large, the FFT methodoften is to prefer. The operation in the FD is then less computationallydemanding and therefore require less time than the corresponding operationin the SD.


5.3 The shift theorem

The shift theorem says that a shift in the SD corresponds to a shift of phasein the FD. If the image f(x, y) is shifted the distance a in the x-directionand b in the y-direction, the shift theorem can be written as below.

F{f(x− a, y − b)} = e−j2π(au+bv)F (u, v) (5.4)

In equation 5.4 it can be seen that there is no change of magnitude in theFD, only the phase is a�ected. All components will be delayed in phase,proportional to the frequencies u and v. For higher frequencies the shift inphase angle will be greater. [2, 7] To get a picture of the connection betweenthe SD and the FD, a graphical example for a 1D signal is presented, see�gure 5.1.

Figure 5.1: An 1D example to illustrate the shift of phase in the FD.Top: The signal f(x) in the SD, original to the left and the shifted version tothe right. Bottom: The FT, F(u), to the left and the FT to the shifted signalto the right. Inspired of [2].

If the signal f(x) is shifted the distance a = 12 , the corresponding phase shift

in the FD will be 2πau. This is subject to a uniform twist of π radians perunit of u.


5.4 The phase correlation method

The phase correlation method [9] uses the shift theorem to �nd the shiftof an image in the SD, by looking at the corresponding phase shift in theFD. An example is made out of two �uoroscopy images, see �gure 5.2, toillustrate the method.

Figure 5.2: To the left: Original image i1. To the right: Image i2, translated20 pixels upwards and 15 pixels to the right, compared to i1.

The di�erence between the images is that the image i2 has been shiftedcompared to the image i1. To get the distance and direction of the shift,�rst the FT for both images, I1(u, v) and I2(u, v), are computed. Then thephase correlation matrix, Q, is computed with the expression

Q =I2(u, v)I1(u, v)∗

|I2(u, v)I1(u, v)|γ = e−j2π(au+bv) (5.5)

The pattern seen in Q can be connected to the phase twist that was shown in�gure 5.1. The parameter γ controls the equalization of magnitude. Equal-ization of magnitude is wanted, since only the phase is of interest. A toolarge di�erence in magnitude will a�ect the result. Since the plotting ofphase will be wrapped, a saw-tooth pattern is seen. In this example, everyperiod means another translated pixel in the SD, since the phase shift is pro-portional to the spatial shift for every unit of frequency. Hence, the directionand frequency of the saw-tooth pattern give information about the spatialshift. To get this information, Q is inverse transformed back to the SD. Amore or less distinct point will then be seen in the SD, since


q = F−1{Q} = δ(x− a, y − b) (5.6)

for a continuous signal. The discrete nature of the image will make the diracspread some to the surrounding pixels, depending on if the shift is an integerof pixels or not. The vector from the origin to (x− a, y − b) in q equals thetranslation, both in direction and distance. [9] Figure 5.3 shows the phasecorrelation matrix Q and its inverse FT.

Figure 5.3: To the left is the phase correlation matrix, Q (γ = 0.8), and tothe right, its inverse FT, q.

This method can be used for instance when detecting motions, from oneframe to another in an image sequence. If the exact shift is known, it canbe compensated for.

Chapter 6

Theory of image enhancement

When processing images, basic operations are often used to simplify problemsand to visualize the image properties. Some of these operations will be de-scribed in this chapter. The theories of image enhancement, used in di�erentshapes in this thesis, are presented.

6.1 Interpolation methods

Interpolation is used when resampling an image. This is done for instancewhen scaling or shifting an image, or when reconstructing an image from adata set, e.g. in ultrasound. Interpolation is used to get the underlying con-tinuous signal and from that calculate the values in the new sample points.[6]

First three commonly used interpolation methods applied in the SD arepresented. They all take the original image samples and the correspondingintensity values as input. Then the corresponding values in the new samplesare calculated in di�erent ways. At last, an interpolation method in the FDwill be mentioned.

Nearest Neighbor interpolationIn the nearest neighbor (NN) method, the neighbor that is closest to the newsample will decide the value. The output image will with this method have avery edgy appearance. The method is simple and not very computationallydemanding. [6, 12]

Bilinear interpolationBilinear interpolation is done using the four surrounding pixel values. Firstthe linear interpolation is done in one direction. The calculated values from

37

CHAPTER 6. THEORY OF IMAGE ENHANCEMENT 38

the �rst direction is then used when interpolating in the second direction.[12] This will result in a much nicer appearance, the edges are not as edgyas in the NN-case. This method also preserves locality.

Bicubic interpolationThe bicubic interpolation method uses a cubic spline function to calculatethe new values. It takes into account for instance 4x4 neighbors with aweighting function. The result will have an even more isotropic appearancebut it is also more computationally demanding. [12]

Adjusting of phase in the FDAccording to the shift theorem in chapter 5, a shift in the SD correspondsto a phase shift in the FD. Hence, to interpolate an image in the SD, thephase can be adjusted in the FD. In the SD this correspond to interpolatingwith a sinc function on a repeated image. This may result in artefacts andthe result will be more smoothed than with bilinear interpolation.

6.2 Thresholding

Thresholding is a simple way to segment an image into di�erent objects. Athreshold is set on a speci�c gray value and all pixels above that value areset to white (for instance the object pixels) and all pixels with lower grayvalue are set to black (for instance the background pixels). The result is abinary image. This method is often applied when deciding an objects shapeor location, or when counting objects in an image.

6.3 Morphological operations

When thresholding an image, the object segmented often is not perfect forfurther treatment. For instance there can be noise or di�erent intensities inthe object that is segmented. Then morphological operations can be appliedto shape the object. A binary structure element, a, is applied on the imageand depending on operation, di�erent results are received.

DilationThe center pixel of the structure element a is placed at every 1-set pixelin the binary image b. In every position, the structure elements pixels arecopied to the resulting image, see �gure 6.1. This operation expand theobject, depending on the chosen a. [4]


ErosionThe structure element a is now placed at every pixel in b. For the positionswhere the elements are entirely covered by 1-set pixels in b, the current pixelis set to 1 in the resulting image, see �gure 6.1.

This operation shrink the object, depending on the form of the structureelements. A combination of these two operations can open (erosion+dilation)or close (dilation+erosion) objects. [4]

Figure 6.1: Top: An example of dilation of image b with the structure elementa. The result is shown with the added pixels in gray. Bottom: An exampleof erosion of image b with structure element a. The result is shown with theremoved pixels in gray.

6.4 Contrast adjustment

The contrast in an area of an image can be de�ned as

C =S1 − S2

S1 + S2(6.1)

where S1 and S2 is the signal intensity in two close objects in the image.The purpose of contrast adjustment is to increase C, so that objects can beclearly separated and to get an increased wealth of details. In the followingsection two methods of gray scale mapping are described.


6.4.1 Gray scale mapping

Each pixel in an image has a speci�c gray value. The histogram of an imageshows the total amount of pixels for every gray value in stacks. The stacksare often divided into 256 di�erent gray levels between 0 and 1. See �gure6.2 for an example of an image and its histogram.

Figure 6.2: The original test image Lena and its corresponding histogram.

To change the contrast in an image, usually a gray scale transformation isdone. This is done by mapping the gray levels with a mapping function. Theidea is to represent the image content with more gray levels. Two pixels thatbefore the transformation are close to each other in gray value, will now berepresented by values more apart. [4]

Histogram stretching

The simplest mapping function is a linear function between the minimumand maximum gray value in the image. This will stretch the histogram sothat all the 256 levels are represented in the image. In �gure 6.3, this methodis applied. Compared with �gure 6.2, the contrast is higher.


Figure 6.3: The histogram after linear mapping between the lowest and high-est pixel values. The histogram shows that the values has been spread so thatall gray levels are used. This result in better contrast in the image.

Histogram windowing

In medical imaging it can be interesting to increase the contrast for somevalues and decrease it for others. This is done by creating a window contain-ing the important gray values and set values below or above these values to0 and 1 respectively. Then the contrast will be even better for the choseninterval. The other information is lost and therefore knowledge about theimportant parts is necessary. This method is used in the Biplanar 500 sys-tem today. The window is computed from a function that uses the minimumand maximum values of the image. A discussion of this method comparedto the proposed method is found in chapter 9.

6.5 Noise reduction

There are di�erent kinds of noise that can appear in images. The most com-mon are signal dependent (multiplicative) or signal independent (additive)noise. To measure noise in images the signal-to-noise ratio (SNR) is oftenused. It is expressed in dB and gives the relation between the signal varianceand the noise variance in the image, see equation 6.2.

SNR = 10logσ2

signal

σ2noise

(6.2)

The main di�culty in noise reduction is to preserve lines and edges. Inthe following sections, some di�erent noise reduction methods are presented


which take care of this in di�erent ways.

6.5.1 Averaging

One way to suppress noise in a 2D image is averaging, or Low Pass (LP)�ltering. It is a linear method that can be applied either in the FD or inthe SD. The noise will be e�ciently smoothed, since it is often high fre-quent. Unfortunately also detailed, high frequent structures in the imageget smoothed. Hence, this is not a suited method for X-ray images. See�gure 6.4 in the next section for averaging applied to a 1D signal.

6.5.2 Median �ltering

This method is often used to reduce so called salt and pepper noise in images.The principle of median �ltering is to look at the neighboring pixels of eachpixel. The median value for every neighborhood is then put in that pixelsposition in the output image.

The idea with median �ltering is to remove sharp and small tops in theimage, such as noise, but to preserve the edges. [4] It is not easy to showthe meaning of median �ltering in 2D, but in 1D the principle can be shown,see �gure 6.4. Median �ltering is a non-linear method, there is no equivalentmethod to median �ltering in the FD.

Figure 6.4: The e�ect of median �ltering on a 1D signal. At the top is theoriginal signal, in the middle the median �ltered signal is shown and at thebottom is the averaged signal, shown as a reference. Inspired by [4].

The size of the neighborhood used is often the smallest possible, 3x3 pixels.


A bigger neighborhood tends to smooth the images. In noise reduction,iteration of �ltering is often applied. This preserves the edges and reducesnoise but thin lines, narrower than half of the kernel, will be lost. After someiterations also homogenous areas will occur, with the same gray level. [12, 4]

This method is not suitable for X-ray images, since thin lines are to bepreserved and the median �ltered images looks constructed because of thetoo homogenous areas between the edges. It is also very computationallydemanding.

6.5.3 Temporal averaging

The principle of temporal averaging is to sum a number of consecutive framesfrom an image sequence and then divide the result with the number of frames.It is one example of a FIR-�lter, where FIR stands for Finite Impulse Re-sponse. [13] The algorithm is shown in equation 6.3,

Iout =1n

n∑

k=1

framek (6.3)

where n is the number of frames. The result is one single, noise reducedimage, Iout. This method preserves the structures that are constant in time.Other parts that di�er from one frame to another, such as noise, will besuppressed. Thus the more frames used, the more e�cient noise reduction.This method often gives very good result unless there are moving objects inthe image sequence. In case of motion the image gets blurred around themoving object.

This method can be applied continuously on image sequences. A lot ofdata then have to be saved, n frames. When n frames have been collectedthough, only one frame has to be replaced with a new one in the next pointof time. The advantage with this method is that when n number of framesare available, the di�erent frames can be weighted di�erently in the outputimage.


6.5.4 Temporal recursive �ltering

Recursive �ltering is a temporal noise reduction method, often used on imagesequences. [1, 11, 16] The recursive �lter is often called IIR-�lter, where IIRstands for In�nite Impulse Response. For every new input frame also theprevious output frames are taken into account when creating the new outputframe. The output from a �rst order recursive �lter can be written as

Iout[t] = αIin[t] + (1− α)Iout[t− 1] (6.4)

i.e. it uses one previous output frame to compute the present one. The timeis set in brackets, since the image sequence is discrete in time. Figure 6.5shows the corresponding block diagram. [11, 13]

Figure 6.5: Block diagram of recursive �ltering.

T is a time delay element, giving the previous output image. The parameterα controls the amount of recursive �ltering and can be tuned between 0 and1. The output is a sum of di�erent frames back in time and the compositionof frames at every point of time can be described with the impulse response.To get the impulse response, �rst the Z-transform [13] of equation 6.4 iscomputed to

Y [z]− (1− α)Y [z]z−1 = αX[z] (6.5)where X and Y are the Z-transform of Iin and Iout respectively. The transferfunction H is then given as

H =Y

X=

α

1− (1− α)z−1=

zα

z − (1− a)(6.6)

and the inverse Z-transform gives the impulse response h, according to

h = Z−1{H} = α(1− α)t (6.7)


In �gure 6.6 the impulse response is shown for di�erent α. When α is set to0 the �lter totally ignores the present frame and for α=1 only the presentframe is used.

0 2 4 6 8 100

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

t

Figure 6.6: Impulse response α(1−α)t of the temporal IIR-�lter for di�erentα; α =0.8: dotted line, α =0.5: solid line and α=0.2: dashed line.

The longer the impulse response of the �lter stretches back in time, thebetter noise reduction is received. Thus a low α is wanted and that worksvery well for image sequences that does not change in time. If an object ismoving from one frame to another though, there will be blurring, motionartifacts, around the moving object when α is low. [11]

The method of recursive �ltering is often used when the time aspect is impor-tant. The input frames do not have to be saved, since the impulse responseis always in the last output signal. This can be compared with the temporalaveraging, that needs n frames to build up one output frame.

Chapter 7

Implementation and results

The theory from chapter 5 and 6 is here applied to the speci�c problem to-gether with some new ideas. First there is a short method description thatexplains how the algorithms have been developed. Then the implementationincluding preprocessing, contrast adjustment, noise suppression and motioncompensation is described.

7.1 Method

The algorithms have been developed in Matlab. Test image sequences havebeen acquired, as described in chapter 4, and saved to disk. The wholeimage sequences are thus accessible when starting the operations, saved in 4Dmatrices (see next section for an explanation). By testing di�erent existingtheories and new ideas separately, the best combination could be chosenfor every subproblem. The algorithms are implemented in di�erent Matlabfunctions. The implementation is made in a number of steps, see the listbelow.

• Preprocessing algorithms

• Deinterlacing

• Contrast adjustment

• Temporal recursive �ltering

• Global motion compensation

• Local motion compensation

• A combined algorithm

The following sections describe the implementation and results of these steps.

47

CHAPTER 7. IMPLEMENTATION AND RESULTS 48

7.2 Preprocessing algorithms

The raw data image sequence that is acquired into Matlab is not optimizedfor image processing. Thus some preprocessing steps are made to simplifythe image enhancement. These steps are considered in this section.

7.2.1 RGB to gray scale transformation

The VCC used gives an output signal in the image format RGB, consistingof three channels; one for red, one for green and one for blue contents. Thisformat is often used when processing color images. The raw data imagesequence is thus a 4D signal (RGB+time).

Since the video from the CCD-camera consists only of gray values, all theRGB channels are equal. A transformation is made into gray scale format,that is 3D (x, y, t). This is done by only using one of the three color dimen-sions. The gray scale image sequence is then ready for further treatment.

7.2.2 Cropping

The original image sequence is built up of frames with the size 576x720pixels. The �uoroscopy image only occupy about 512x512 pixels and theimage is circular. Thus a cutout is made around the circular image, to makethe future processing more speci�c and less computationally demanding.

The cutout is done by �rst thresholding the image to get a binary maskthat tells where the circular image is located. The size of the cutout isspeci�ed depending on the speci�c operation and it is made from the middleof the mask to make it symmetric. For instance, in the recursive �lteringand motion compensation algorithms, the size 256x256 pixels is used.


7.2.3 Windowing

When transforming an image into the FD, it is important to window theimage, as discussed in section 5.1. Otherwise very distinct artifacts wouldbe seen in the FD, corresponding to the sharp borders of the image.

The image is circular and thus a circular window is wanted, that softens theborders of the image. A circular symmetric cos2 is used in this case, that ismultiplied with the same mask as was described in the previous section, seethe left image in �gure 7.1 for the window.

A quadratic window is constructed as well, see the image to the right in �gure7.1. It is used in the algorithm for recursive �ltering, where the cutout is256x256 pixels and the image has the shape of a square. This window isbuilt up of a cos2 in both x- and y-direction this time and is not circularsymmetric.

Figure 7.1: Left: The circular symmetric cos2 window. Right: The quadraticcos2 window.

The window is then multiplied with the image. The FT of the windowedimage will then be received after transformation, according to

F (u, v) = F{Wf(x, y)} (7.1)

See �gure 7.2 for the e�ect of windowing.


Figure 7.2: Top left: The original image. Top right: The windowed image.Bottom left: FFT of the original image, with distinct artifacts. Bottom right:FFT of the windowed image. The artifacts are gone.



The idea of contrast adjustment in this thesis is to use a smoother mappingfunction than in the present method, which only expands the values withina certain window. A mapping function that can be adjusted to get di�er-ent slope and position is wanted. A mapping function that comply withthese demands is the h-mapping function, see equation 7.2. In [7], a similarfunction is used to map tensors.

Iout =Iin(1− a)b

(Iin(1− a))b + (a(1− Iin))b(7.2)

The h-mapping function is nonlinear and can be adjusted to the desiredshape. The parameter a controls the position of the curve and b controls theslope. Examples of the h-mapping function for di�erent a and b can be seenin �gure 7.3 below. The smoothness is better for the eye to look at and thelower and higher values are not set to 0 or 1 but is decreasing or increasingall the way.

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0 0.2 0.4 0.6 0.8 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Figure 7.3: The h-mapping function for di�erent a and b. The gray valueson the x-axes in the �gures above, are transformed to the values on the y-axes. Left: b=2 and a is changed. a=0.3 (dashed), a=0.5 (solid) and a=0.8(dotted). Right: a=0.5 and b is changed. b=1 (dashed), b=2 (solid) and b=4(dotted).

The h-mapping function from equation 7.2 is implemented. In �gure 7.4, anexample of mapping result is shown.


Figure 7.4: A frame from test sequence F in original (left) and contrastadjusted with the h-mapping function, a=0.63 and b=2.82 (right).

7.4 Deinterlacing

The original image sequence is in interlaced format but is presented as pro-gressive on the screen, i.e. frame by frame. That makes the image blur andshow interlacing artifacts when an object is moving. See section 3.3.1 for thetheory of interlacing.

To avoid this, an algorithm is developed that compensate for the interlacingpattern. To separate the �elds that the frame is built up of, every secondrow is set to zero. This striped image is then convolved with the kernel [0.51 0.5]T , that will interpolate the image from the odd/even rows to build upa new frame, from only one point of time. See �gure 7.5 for the originalimage, one �eld and the resulting image.

Figure 7.5: The method of deinterlacing. Left: The original frame withinterlacing artifacts. Middle: The �eld of odd lines. Right: The interpolatedresult.


When implementing the deinterlacing based on the odd �eld, the last pixelrow will get lower pixel values than what is wanted, because of the limit ofthe image. To avoid this, the second last row is copied to the last low. Themethod of deinterlacing is used both in the global and in the local motioncompensation.

7.5 Temporal recursive �ltering

The implementation of pure recursive �ltering is quite easy. The outputimage is computed according to equation 6.4, using one previous outputframe. Depending on the parameter α, a reduction of noise is received.Figure 7.6 shows the recursive �ltering applied to the B test sequence withboth global motions and constant position, where α is set to 1, 0.6 and 0.2.

As expected the noise is reduced more and more down the left column. Inthe right column, it is seen that the small α result in blurred images. Toavoid the blurring that occurs when there is motion, a motion compensatoris developed. The method of recursive �ltering will underlie the algorithmsthat are described later on.


Figure 7.6: Recursive �ltering of test sequence B. Top: α = 1 for frame20 (still phantom) and frame 37 (moving phantom). Middle: α = 0.6 forframe 20 and 37. Bottom: α = 0.2 for frame 20 and 37. Deinterlacing isnot applied in this �gure.


7.6 Temporal averaging

In opposite to the recursive �ltering, averaging in time does not use previousoutput images when computing the present one. Instead a speci�ed num-ber, n, of input frames back in time are used, see section 6.5.3. Since thepresent system uses averaging with di�erent number of frames, as found outin section 3.4, this is reconstructed to get something to compare the resultswith. Figure 7.7 shows temporal averaging applied to the test sequence C,compared to the original sequence.

Figure 7.7: Temporal averaging of test sequence C. Top: Original frame 11(moving drill) to the left and frame 20 (still drill) to the right. Bottom: Thecorresponding temporal averaged frames, frame 11 to the left and frame 20to the right, with n=6.


7.7 Global motion compensation

The basic idea of the global motion detection and compensation is the shifttheorem; when a spatial shift is made of an object in the SD, a shift of phasewill occur in the FD. The example from section 5.4 cover the theory and theimplementation is straightforward from that.

7.7.1 Calculation of translation

To detect weather something in the scene has been moved from one frameto the next, the phase correlation method has been used. Two consecutiveframes are Fourier transformed and the phase correlation matrix, Q, seeequation 5.5, is calculated. Tests showed that a γ equal to 0.82 gives thebest result. After inverse transforming Q back to the SD, an image, q, isreceived, with a dirac in the pixel that correspond to the shift. This is onlyexact if the shift made is an exact number of pixels. Since that is not oftenthe case, the dirac will spread out to the surrounding pixels. For an exampleof shift calculation, see �gure 7.8.

To get the exact translation, �rst the maximum value of q is located. Thenthe mass center is calculated in an area around that point, according to

r =∑

x

∑y xyI∑

x

∑y I

(7.3)

where I is the intensity in every coordinate (x, y). The calculated shift willthen be more exact when using it in the motion compensator. In the ex-ample in �gure 7.8, the calculated shift is 2.57 pixels to the right and 5.59downwards.

If the size of the shift is equal to or more than 2 pixels, the deinterlacingalgorithm, described in section 7.3, is used, to avoid interlacing artifacts.


Figure 7.8: Top left: Frame no 34 of test sequence B. Top right: Frame no35. Bottom left: The phase correlation matrix, Q, (γ = 0.82). Bottom right:The inverse FT of Q, q.

7.7.2 Adapting the temporal impulse response

For every input frame, the shift between the current frame and the previousframe is computed, as described in the previous section. The shift is given invector-form, with direction and magnitude. If there is no shift, the recursive�ltering can proceed as usual. If there is a shift, α is set to 1 and the impulseresponse is cut. This means that the recursive �ltering is only operating whenthere is no motion.

When a motion is detected, the input frame is used as output. The bonestructures in the sequence will then be more prominent during movement,even though there is a lot of noise that has not been reduced. When thephantom is still, the image will be very clear. In �gure 7.9 the result of thismethod is shown for a frame with no motion and for a frame with motion.This �gure can be compared to �gure 7.6 in the previous section.


Figure 7.9: Cutting the impulse response upon motion in the test sequenceB. Left: Frame 20 (still phantom). Right: Frame 37 (moving phantom).Deinterlacing is here applied.

In [11], Jostschulte et al use the magnitude of the shift to set α according toa look-up table. For small shifts, α can stay low and the larger shifts, thehigher α is used. Since the exact shift is known in this case, it is chosen tocompensate for the motion instead.

7.7.3 Shifting the impulse response

For every new frame, the impulse response (or the previous output frame) isshifted to the position of the current frame. The shift is done with bilinearinterpolation in the SD, as described in section 6.1.

When this is done, the recursive �ltering can be applied to the compensatedframes. The shifted frames will have dark edges, since the frames havelimited boarders. This leads to that the resulting output frame is built upof a di�erent number of frames in di�erent parts. This problem is solved bynormalizing the image sequence with a normalization sequence that followin the process. See �gure 7.10 for the results. The parts of the image thatare build up of only a few frames become more noisy. This is a small e�ect,compared to the enhancement thanks to the global motion compensationwhen the phantom is moved.


Figure 7.10: Top: Result of the global motion compensation before nor-malization in test sequence B. Middle: The corresponding normalization se-quence. Bottom: Normalized result of the global motion compensation.


7.7.4 Width of the impulse response

The global motion compensation combined with the recursive �ltering willresult in a LP e�ect in the frames that originate from distant points of time.Since the translation is calculated exactly, from the mass center formula insection 7.6.1, and the images are discrete, the value is split between thenearest pixels when the translation in pixels is not an integer. In the worstcase, each new sample is in the middle of four pixels. The interpolationcauses a LP e�ect that will increase back in time, since the whole impulseresponse is shifted during compensation. To get a picture of how big thislow-pass e�ect is, �gure 7.11 shows the e�ect for 4, 8 and 16 frames back intime in the worst case.

Figure 7.11: The LP e�ect at 4, 8 and 16 frames back in time, in the worstcase.

Another compensation for global motion was tested, to see how big thedi�erence can be. If every frame is only interpolated once, the LP e�ect willbe much lower. For every new frame, the output frame is instead composedof the 16 previous frames, all of them compensated the distance that di�erfrom the present frame.

This algorithm gets much more time demanding, since the shift of framesis done up to 16 times/frame instead of only one when shifting the impulseresponse. This was done with both uniform distribution and a cos4 function.When this method was compared to the motion compensation describedearlier, no big di�erence could be seen. Thus no further work was put intothis idea.

7.8 Local motion compensation

The other type of motion that can occur in the Biplanar 500 image sequencesis a local motion, for instance when the patient is still and only the drillis moving. To handle this kind of motion, a local motion compensator isdeveloped.


By subtracting the previous frame from the present one, an image is receivedthat can tell where the motion is. If there is no motion, the di�erence imageis all zero, except from the noise. If there is a local motion, the exact positionof this will be shown. See �gure 7.12 for an example of two consecutive inputframes that are used to get a di�erence image.

Figure 7.12: Frame no 8 (left) and 9 (right) of the C test sequence.

The magnitude of the di�erence image is thresholded to get a mask thatshows the area of local motion. After that the mask is opened with a 2x2kernel to get rid of the small objects referring to noise. This is done viaerosion followed by dilation, as described in section 6.3. LP-�ltering is thenapplied in the SD with a gaussian �lter, to make the mask smooth with nodistinct edges. Figure 7.13 shows the di�erent steps of adjusting the mask.

Figure 7.13: From left to right: The thresholded di�erence image, the openedmask and the LP-�ltered mask.

The resulting mask, m, is used to cut the moving parts (i.e. the drill in thiscase) in the present frame into the recursive �ltered frame, according to

Iout = mIin + (1−m)Irec (7.4)


In �gure 7.14 the result of the local compensation is shown, compared to theresult from pure recursive �ltering.

Figure 7.14: The result using pure recursive �ltering with α = 0.3 (left)compared to the local motion compensated result (right).

Deinterlacing, described in section 7.3, is also applied to the mask in thecutting-in step, to avoid interlacing patterns. See �gure 7.15 for an exampleof the result with and without deinterlacing.

Figure 7.15: Output frame 8, with interlacing artifacts to the left and withdeinterlacing applied to the right.


7.9 An alternative method to detect global motion

In the two previous sections, the global and local compensation methodshave been described. Both of the algorithms work well on test sequences withglobal and local motions respectively. In the Biplanar 500 system however,it is wanted to compensate for both global and local motions. Therefore analgorithm is developed that can distinguish between local and global shiftsand choose between these two methods, see the next section.

The global detection then has to be done di�erently than before, since testsshowed that big local motions are treated as global motions. Another ap-proach is then tested, mostly based on dividing the frames into smaller sec-tions.

7.9.1 Dividing frames into overlapping sections

The frames are divided into a number of overlapping sections. See �gure7.16 for the division principle of one frame. The number of sections has beenchosen to 7x7=49 sections, i.e. the side of every section is a quarter of thewhole image side.

Figure 7.16: An image divided into sections. The sections are cut out in thisway. In this case the total number of sections will be 3x3=9.

7.9.2 Representing shifts in vector plots

The shift between two frames is now computed, in the same way as describedearlier, for each section, to detect weather there has been a shift in the speci�c


section or not. The results from this operation are then analyzed in di�erentways.

In theory, when there is a global motion, the vectors for all the sections willpoint in the same direction and have the same magnitude. When there is alocal motion or no motion, most of the vectors will have a zero magnitude,except for in the sections where the local motion occur.

It showed that the local shifts were not outstanding from the false, highermagnitude shifts occurring, probably because of noise. The non motion andlocal motion cases could therefore not be separated. The global motion onthe other hand could be separated and that was the main idea. In �gure7.17 the resulting vectors for no motion and global motion is shown.

0 1 2 3 4 5 6 7 80

1

2

3

4

5

6

7

8

0 1 2 3 4 5 6 7 80

1

2

3

4

5

6

7

8

Figure 7.17: The shifts computed in each section at no motion (left) andglobal motion (right).

First, the number of sections that registered shifts were used to sort out ifthere was a global motion or not. It functioned quite well but to increasethe robustness, also another method was developed.

7.9.3 The mean-std method

The vectors were plotted in a (x-y)-coordinate system, one point representingone vector. Figure 7.18 shows that a global motion corresponds to pointsspread at the global shifts representation in the plot. If no motion is presentor the motion is local, a large amount of points will be tightly clustered tothe origin.

This lead to a classi�cation based on the mean value and the standard devia-tion of the (x,y)-plot. If the mean value is close to the origin and the standarddeviation is low, probably there is no motion. If the points are concentrated


Figure 7.18: A plot of the translations computed, at no motion (left) and atglobal motion (right).

to another point instead, and the points are more spread, there is a globalmotion. This mean-std method function well and it is more robust than themethod of counting the number of translations. This method is thereforeused in the combined algorithm, described in the next section.

7.10 The combined algorithm

The global and local compensation is now put together in one algorithm.This algorithm will be referred to as the combined algorithm from now on.The parameter global is set to true or false, depending on the result from theglobal detection, described in the previous section. If it is true, the globalcompensation and the recursive �ltering is carried through. Otherwise therecursive �ltering goes on as usual and will be followed by the local motioncompensation. Hence, the algorithm now function for both global and localshifts. The combined algorithm is described in detail in the block schemesin Appendix A.

Chapter 8

Evaluation study

In this chapter, an evaluation study is presented. First, the method is brie�ydescribed. Then the di�erent excersices are presented one at a time, withprerequisites, results and conclusions drawn.

8.1 Method

The study was carried through at the department for orthopedic surgeryat the Nyköping Hospital. This hospital was chosen, since the orthopediststhere often use the Biplanar 500 system during hip operations. The ortho-pedists are therefore used to work with hip image sequences, matching thetest sequences used during development of the algorithms.

A laptop was used to show the image sequences to the orthopedists, one at atime. Notes were taken and their opinions and thoughts were written down.The images and image sequences were rotated so that the orthopedists wouldrecognize the scenes as they appear in the Biplanar 500 system. I got to talkto 4 orthopedists and their answers are presented in the following sections.Each person needed about 5 minutes to look at and discuss the images.

8.2 Manual contrast adjustment

At �rst, the orthopedists were to adjust the contrast parameters a and bin the h-mapping function on a typical hip image, until they were satis-�ed. By using the matlab function mumap, the parameters were appliedcontinuously to the image. The result is shown in table 8.1 below.

67

CHAPTER 8. EVALUATION STUDY 68

Orthopedist a b

1 0.60 1.842 0.56 1.603 0.53 3.964 0.55 2.90

Table 8.1: Result of manual contrast adjustment of the h-mapping functionparameters.

This �rst contrast adjustment exercise came out with a widespread result,as expected. Some of the orthopedists liked brighter images and some muchdarker and with a higher reinforcement of contrast. They liked the bars andunderstood the meaning of adjusting the parameters. One orthopedist askedif it could be possible to adjust the contrast in a similar way in the X-raysystem.

8.3 Comparison between di�erent mappingfunctions

In the second contrast exercise, two pairs of images were shown, comparingthe h-mapping function with the linear mapping function used in the systemtoday. See �gure 8.1 and �gure 8.2.

Figure 8.1: The images used for comparing the result of the h-mapping func-tion (left) to the result of linear mapping (right). This is for low contrastreinforcement, a=0.51, b=2.7 in the h-mapping function, corresponding to alinear mapping of the values between 75-175.


Figure 8.2: The images used for comparing the result with linear mapping(left) to the result using the h-mapping function (right). This is for highcontrast reinforcement, a=0.54, b=3.96, corresponding to a linear mappingof the values between 113-175.

The di�erence between those two sets were the gradient and location of themapping curves. Between the two sets, the order was shifted, to assure thatthe orthopedists choose the best image independent of order. The result ofthe second contrast exercise is shown in table 8.2 and 8.3 below.

Orthopedist Choice Comment1 1A The other one is too light in the

surroundings.2 1A The white surroundings are disturbing.3 1A The other one takes away the borders

of the trochanter of femur.4 1A This one is better in the caput line.

Table 8.2: The result of comparison between the linear and the h-mappingfunction. Low gradient of the mapping functions, see the images in �gure8.2.


Orthopedist Choice Comment1 2A This one is better in the main area and

the caput line.2 2B The other one is too white in the outer

parts of the image.3 2B The contours remain even in the outer

parts of the image.4 - No di�erence in the main area.

Table 8.3: The result of comparison between the linear and the h-mappingfunction. High gradient of the mapping functions, see the images in �gure8.3

In this second contrast exercise, the comparison of images adjusted with alinear mapping function versus a h-mapping function, the result was moreunanimously. For the �rst set of images, with a low contrast reinforcement,all of the orthopedists preferred the new h-mapping function. 3 of 4 weredisturbed by the white areas in the outer parts of the linear mapped image,that made the structures of the trochanter of femur invisible. One thoughtthat the caput line got more prominent. For the second set of images theopinion of the too sharp light areas remained but one orthopedist preferredthe linear one, due to better contrast around the caput line, and one couldnot see the di�erence.

8.4 Noise reduction and blurring

In this exercise the orthopedists were asked to rank di�erent image sequencesin the meaning of noise, blurring and sharpness and what would be thepreferred image during operation.

The sequences with global and local motion were shown in di�erent sets.For both of these sets the original sequence, an averaged sequence (n = 6)and a recursive �ltered sequence (α = 0.3) were shown at �rst. Then threedi�erent motion compensated sequences were shown.

Table 8.4 and 8.5 explain the di�erences between the sequences used in thestudy. For the global compensation, the test sequence B was used and forthe local compensation, the test sequence C.

The result from the sequences with global and local motion is shown in table8.6 and 8.7 respectively.


Sequence Description3A Temporal averaging, using n=6.3B Recursive �ltering, using α=0.3.3C Upon motion, the impulse response is cut (α=1/0).3D Global motion compensation, α=0.2.3E Global motion compensation, α=0.4.

Table 8.4: The global motion compensated test sequences used in the evalua-tion study.

Sequence Description4A Temporal averaging, using n=6.4B Recursive �ltering, using α=0.3.4C Local motion compensation, α=0.4.4D Local motion compensation, α=0.2.4E Local motion compensation, the di�erence

between the present frame and the lastoutput frame is used as a mask.

Table 8.5: The local motion compensated test sequences used in the evaluationstudy.

Orthopedist Comment1 3B is better than 3A. 3C is better than 3A and 3B since

the contours remain during motion.3D and 3E are similar and the best sequences.

2 Maybe 3A is a little bit better than 3B.3D and 3E are the best sequences. Often Ionly use still images and therefore itis not that relevant.

3 It is most important to get a sharp imagewhen there is no motion.3D and 3E are similar and the best ones.Maybe the compensation can imply that stillimages gets better if the patient is movedduring excitation.

4 No di�erence between 3A and 3B.3C is a little bit better but 3D and 3E are even better.Perhaps 3D is a little bit better than 3E.

Table 8.6: The opinions of the di�erent versions of test sequence B (globalmotion). See table 8.4 for a description of the sequences 3A-E.


Orthopedist Comment1 No big di�erence between 4A and 4B. 4C-E is

much better! It is important that the imageis not delayed when drilling.

2 4A is better than the original, the noise ise�ciently removed. No big di�erence between4A and 4B. Since I only use still images, theblurring is no problem. 4C-D are bettersequences but I do not know if it is useful.

3 The 4C sequence is the best one. It does notloose in quality but the blurring is gone.It depends on how experienced the orthopedistis, how much the continuous excitation is used.

4 No di�erence can be seen between 4A and 4B.4C-E are better, no di�erence can be seenbetween those.

Table 8.7: The opinions of the di�erent versions of test sequence C (localmotion). See table 8.5 for a description of the sequences 4A-E.


The result from the global and local sequences show that in almost everycase, no di�erence could be seen between the averaged image sequences andthe recursive �ltered ones. Two exceptions from this opinion was one whopreferred the averaged sequence and one who preferred the recursive �lteredsequence. The conclusion drawn from this is that averaging of a sequencewith n = 6 corresponds quite well to recursive �ltering with α=0.3.

When looking at the results from the compensated sequences, it turned outto be too small di�erences to get a good view of which parameters thatwere the best ones. In the global sequence, where the impulse responsewas cut in sequence 3C, the main opinion was that it was better than thenon compensated ones. Then, when showing the compensated sequences 3Dand 3E they were even better. A lot of comments on the usefulness of thecompensation were given.

The same thing was said about the local motion compensated sequences, nobig di�erence could be seen between 4C-E but the general opinion was thatmotion compensation was good. Some questioned the usefulness, while somewhere thrilled that the quality was so good and emphasized the importanceof no blurring during drilling.

This study was performed before the algorithm for both local and globalcompensation in combination was developed. Hence, that algorithm couldnot be evaluated. However, the results from the combined algorithm is sim-ilar to the results shown to the orthopedists, since the combined method isbased on the same algorithms. The results of this study is therefore alsovalid for the proposed algorithm.

Chapter 9

Discussion

In this chapter the results are discussed in terms of accuracy, image qualityand time requirements.


When using the h-mapping function, the desired contrast and intensity inthe image can be adjusted, independent of pixel values. The smoothnessof the curve will not result in the black and white areas in the image, thatoccur today. An example is given, in �gure 9.1, of the present linear map-ping function compared to the proposed h-mapping function. The resultingimages are shown in �gure 9.2.

Figure 9.1: A comparison between the linear (dashed) and the h-mappingfunction (solid).

75

CHAPTER 9. DISCUSSION 76

Figure 9.2: An example of contrast reinforcement with the di�erent mappingfunctions used. Linear mapping (left) and h-mapping (right).

Contrast will with the h-mapping function be preserved also in the less im-portant areas of the image. See for instance the bone structure in the leftupper corner in �gure 9.2.

In the present system, the lowest and highest pixel values in the histogramare used to compute the window width in the linear mapping of gray values.That implies that an image will change its contrast, depending on if there isa dark object present, e.g. a nail, or not. This will not be the case with theproposed mapping function, see �gure 9.3.

Figure 9.3: Mapping of images from the F test sequence, using a=0.63 andb=2.82. Mapped image without dark objects (left) and the mapped image witha drill inserted (right).


9.2 The proposed algorithm

The global detection with the mean-std method is robust and therefore thecombined algorithm functions for both global and local shifts. The proposedalgorithm is therefore the combined algorithm. Below, the algorithm is ap-plied to test sequences with global motions, B and L, and with local motions,C and F.

9.2.1 Global motion

The proposed algorithm functions well for both the B and the L test se-quences. In �gure 9.4 and 9.5 comparisons are made between the originalsequence, averaging with n=6 (the present method), and the sequence withthe proposed algorithm applied.

Figure 9.4: A comparison of the proposed method to the present averagingmethod on the test sequence B, at global motion. Left: the original sequence.Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =0.3.

Figure 9.5: A comparison of the proposed method to the present averagingmethod on the test sequence L, at global motion. Left: the original sequence.Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =0.3.


9.2.2 Local motion

The proposed algorithm also works for both the C and the F test sequences.See �gure 9.6 and 9.7 below for comparisons to the original and the averagedsequences.

Figure 9.6: A comparison of the proposed method to the present averagingmethod on the test sequence C, at local motion. Left: the original sequence.Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =0.3.

Figure 9.7: A comparison of the proposed method to the present averagingmethod on the test sequence F, at local motion. Left: the original sequence.Middle: the averaged sequence, n = 6. Right: the proposed algorithm, α =0.3.

The C sequence has a fast local motion and the compensation can handlethis. In the F test sequence, the result is not as good as in the C testsequence. This is because the drill is moving in its own direction, meaningthat the di�erence mask will only cover a small area, just at the end of thedrill. A thin blurring can be seen to the right in �gure 9.7. Compared to thepresent method, averaging with n=6, the proposed method is still better,since the position of the drill is more prominent.


9.2.3 The accuracy of shift calculation

The accuracy of shift calculation can be estimated by adding noise to a testsequence and see on what noise levels the shift calculated is wrong. This wasdone to the B test sequence, by adding random noise to the averaged resultfrom frames 16-20 (still phantom). To get the shifted image, the cutout wasmade di�erently. A small shift, (-5,-3) and a large shift (15,-20) was tested.See table 9.1 and 9.2 below for the results.

SNR[dB] x y9.46 -5.029 -3.0476.64 -5.082 -3.0864.77 -5.083 -3.1343.23 -5.110 -3.1681.82 -5.181 -3.1230.42 -5.183 -3.161

Table 9.1: The result of shift calculation for the true shift (-5,-3) when noiseis added.

SNR[dB] x y9.46 14.95 -19.936.64 14.94 -19.944.77 14.90 -19.953.23 14.82 -19.941.82 14.70 -19.940.42 28.70 -11.82

Table 9.2: The result of shift calculation for the true shift (15,-20) whennoise is added.

It showed that the shift calculation is very robust to noise. Down to anSNR of 1.82dB, the shift calculated is in the right integer pixel. This isindicative of that the algorithm can handle even noisier image sequencesthan today, but still calculate the true shift. This is interesting, since noisyimages implies a low dose to the patient.


9.2.4 Limitations

In the proposed algorithm, the mean-std method cannot sort out local mo-tions from no motions. This is probably based on two major things. Thesections have to be quite small to detect a local motion and if the sectionsare that small, false shifts are given in the homogenous areas since the noiseis the big di�erence between the two compared frames.

The case of global and local shift at the same time has not been included inthe proposed algorithm. If a robust method for local shift detection couldbe found also this case could be implemented. On the other hand maybethe application for this is not that useful, since the patient is hopefullyanaesthetic and still during drilling or nailing.

9.2.5 Time requirements

To estimate the time required for compiling the combined algorithm, theMatlab function pro�le is used. When applying the combined functionto the C test sequence, i.e. 31 frames, it takes 45 seconds. Of this time,32.8 s is used for shift calculations. Other functions that occupy time isthe windowing and FFT calculations. 45 s is not very much, relative toprocessing of 31 frames.

To improve the functions included in the combined algorithm, in the meaningof computational cost, some code parts can be done more e�ciently. Sincethe functions have to be rewritten into C if implemented in the system, notmuch time has been put into time e�ciency.

Chapter 10

Conclusion

The conclusions drawn from this work are summarized in this chapter and arecommendation to Swemac is done. Future work, including further investi-gation before implementation, is considered.

10.1 Short summation

In this thesis, an e�ective algorithm for enhancement of the Biplanar 500system image sequences is developed. It uses temporal recursive �ltering fornoise suppression and compensate for both local and global motions. A newgray scale mapping function is also presented, that makes the contrast moresmooth in the lower and upper parts of the histogram.

10.2 Recommended algorithm for Biplanar 500

As a �rst step it is suggested that a change of the contrast adjustment isdone. A change of mapping function to the proposed one should be quiteeasy to carry through. The evaluation of the results show that orthopedistsprefer the proposed mapping function compared to the present one. It is alsosuggested that the parameters of the curve shall be adjustable and availableto the user, to further comply with the orthopedists opinions.

The algorithm that is recommended to Swemac is the combined algorithm,that can handle both global and local motions. The results show that themotion artifacts, seen as blurring in the averaged image sequences of today,are essentially reduced with this algorithm applied.

81

CHAPTER 10. CONCLUSION 82

The method of shift computing, with the phase correlation method, is robustto noise and is very accurate. Compared to pixel-based motion compensa-tion, the proposed compensation is to prefer. The results from the accuracytest of shift calculation, speaks for that high quality image sequences maybe received even if the dose is lower than today.

10.3 Future work

• The proposed algorithm cannot yet be used on the circular, full-scaleimage sequences, because of the global normalization. This could besolved by instead of a full intensity square use the circular mask of theimage and let it follow the algorithm. For the local motion compensa-tion this is not a problem since the whole image is subtracted.

• Perhaps the image sequences can be improved further by applying highpass �ltering. The resulting sequences are that noiseless and sharp,that the grainy e�ect of unsharp masking should not appear.

• The algorithm has to be converted into C-code to �t in the Biplanar500 system. Some calculations are in Matlab easy to carry through,since some operations are prede�ned in functions. In C, the FFT op-eration for instance does not have a predetermined function so it hasto be constructed. The time requirement will be reduced a lot whenimplemented in C code. Probably the algorithm can be applied inreal-time, as desired.

10.3.1 Further investigation before usage of the algorithm

• A lot more testing is required before using the algorithm in the system.Also a more extensive study should be done, to more e�ciently respondto the orthopedists opinions.

• The possibilities to use a progressive camera instead of an interlacedshould be investigated. There is no reason to use interlaced formatwhen using computer monitors for presentation. This could improvethe result even more and make the processing less computationallydemanding.

• The possibilities to use a square image, with the size of a second power,for instance 512x512, should be discussed. That is another way toreduce time requirements.

List of Figures

2.1 Schematic overview of the �uoroscopy chain. . . . . . . . . . . 62.2 The X-ray tube and its components. . . . . . . . . . . . . . . . 62.3 The Image Intensi�er and its components. . . . . . . . . . . . 8

3.1 Schematic overview of the Biplanar 500 system. . . . . . . . . 123.2 The Biplanar 500 system. . . . . . . . . . . . . . . . . . . . . 123.3 Schematic overview of the acquisition into Matlab. . . . . . . 133.4 Example of an input image. . . . . . . . . . . . . . . . . . . . 143.5 The principle of interlaced video. . . . . . . . . . . . . . . . . 153.6 Interlacing artefact when moving a lead disc in the X-ray �eld. 153.7 An input image and its corresponding spectrum. . . . . . . . . 163.8 The noise from the CCD-camera and the corresponding spec-

trum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173.9 A comparison between input and output images in the Bipla-

nar system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.10 A comparison between the spectrums of the input and output

images in the Biplanar system. . . . . . . . . . . . . . . . . . 18

4.1 The important lines and areas in the input images. . . . . . . 234.2 The phantom used in the test sequences. . . . . . . . . . . . . 244.3 The B test sequence. . . . . . . . . . . . . . . . . . . . . . . . 264.4 The L test sequence. . . . . . . . . . . . . . . . . . . . . . . . 274.5 The F test sequence. . . . . . . . . . . . . . . . . . . . . . . . 284.6 The C test sequence. . . . . . . . . . . . . . . . . . . . . . . . 29

5.1 An 1D example to illustrate the shift of phase in the FD. . . . 335.2 Original image i1 and the translated image i2. . . . . . . . . . 345.3 The phase correlation matrix Q and its inverse FT, q. . . . . 35

6.1 An illustration of dilation and erosion. . . . . . . . . . . . . . 396.2 The original test image Lena and its corresponding histogram. 406.3 The Lena histogram after linear mapping. . . . . . . . . . . . 416.4 The e�ect of median �ltering on a 1D signal. . . . . . . . . . 426.5 Block diagram of recursive �ltering. . . . . . . . . . . . . . . . 44

83

LIST OF FIGURES 84

6.6 The impulse response of the recursive �lter for di�erent α. . . 45

7.1 The quadratic and the circular cos2 windows. . . . . . . . . . 497.2 The e�ect of windowing before transformation. . . . . . . . . . 507.3 The h-mapping function for di�erent a and b. . . . . . . . . . 517.4 Contrast adjusted image using the h-mapping function. . . . . 527.5 The method of deinterlacing. . . . . . . . . . . . . . . . . . . . 527.6 Recursive �ltering of test sequence B. . . . . . . . . . . . . . . 547.7 Temporal averaging of test sequence C. . . . . . . . . . . . . . 557.8 An example of q and Q, used in the shift calculation. . . . . . 577.9 Cutting the impulse response upon motion in test sequence B. 587.10 The normalization of a global motion compensated sequence. . 597.11 The LP-e�ect that will occur in the impulse response, in the

worst case. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607.12 Frame no 8 and 9 of the C test sequence. . . . . . . . . . . . . 617.13 The thresholded di�erence image, the opened mask and the

LP-�ltered mask. . . . . . . . . . . . . . . . . . . . . . . . . . 617.14 Comparison between pure recursive �ltering and with the local

motion compensation applied. . . . . . . . . . . . . . . . . . . 627.15 The local motion compensation with and without deinterlacing. 627.16 The division of a frame into sections. . . . . . . . . . . . . . . 637.17 The shifts computed in each section at no motion and global

motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647.18 An (x,y) plot of the translations at no motion and at global

motion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

8.1 Comparison between linear- and h-mapping. Low contrast. . . 688.2 Comparison between linear- and h-mapping. High contrast. . . 69

9.1 A comparison between the linear (dashed) and the h-mappingfunction (solid). . . . . . . . . . . . . . . . . . . . . . . . . . 75

9.2 An example of contrast reinforcement with the di�erent map-ping functions used. Linear mapping (left) and h-mapping(right). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

9.3 Mapping of images from the F test sequence, using a=0.63and b=2.82. Mapped image without dark objects (left) and themapped image with a drill inserted (right). . . . . . . . . . . 76

9.4 A comparison of the proposed method to the present averagingmethod on the test sequence B. . . . . . . . . . . . . . . . . . 77

9.5 A comparison of the proposed method to the present averagingmethod on the test sequence L. . . . . . . . . . . . . . . . . . 77

9.6 A comparison of the proposed method to the present averagingmethod on the test sequence C. . . . . . . . . . . . . . . . . . 78

LIST OF FIGURES 85

9.7 A comparison of the proposed method to the present averagingmethod on the test sequence F. . . . . . . . . . . . . . . . . . 78

A.1 Block scheme of the detection of global motion. . . . . . . . . 91A.2 Block scheme of the local motion compensation. . . . . . . . . 92A.3 Block scheme of the global motion compensation. . . . . . . . 93

List of Tables

4.1 Facts about the test sequences used. . . . . . . . . . . . . . . . 25

8.1 Result of manual contrast adjustment of the h-mapping func-tion parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . 68

8.2 Result of the comparison between the linear and the h-mappingfunction. Low contrast. . . . . . . . . . . . . . . . . . . . . . . 69

8.3 Result of the comparison between the linear and the h-mappingfunction. High contrast. . . . . . . . . . . . . . . . . . . . . . 70

8.4 The global motion compensated test sequences used in the eval-uation study. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

8.5 The local motion compensated test sequences used in the eval-uation study. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

8.6 The opinions of the di�erent versions of test sequence B. . . . 718.7 The opinions of the di�erent versions of test sequence C. . . . 72

9.1 The result of shift calculation when noise is added. Small shift. 799.2 The result of shift calculation when noise is added. Large shift. 79

87

Bibliography

[1] Aishy Amer and Hartmut Schröder. A new video noise reduction al-gorithm using spatial subbands. In Electronics, Circuits, and Systems,1996. ICECS '96.

[2] Ronald N. Bracewell. The Fourier transform and its applications.McGraw-Hill Book Co., 2 edition, 1986.

[3] Eastman Kodak Company. The Fundamentals of Radiography. HealthSiences Markets Division, Rochester, New York, 12 edition, 1980. Pub-lication No. M1-18.

[4] Per-Erik Danielsson, Olle Seger, Maria Magnusson Seger, and IngemarRagnemalm. Bildanalys, TSBB52, 2004 � Kompendium. Departmentof Electrical Engineering, Linköping University, 2004.

[5] P P Dendy and B Heaton. Physics for diagnostic radiology. Instituteof physics publishing, 2 edition, 1999. ISBN 0-7503-0590-8.

[6] Per-Erik Danielsson et al. Bilder och gra�k, 2003 � Kompendium. De-partment of Electrical Engineering, Linköping University, 2003.

[7] Gösta H. Granlund and Hans Knutsson. Signal Processing for ComputerVision. Kluwer Academic Publisher, 1 edition, 1995. ISBN 0-7923-9530-1.

[8] Luke's Video Guide. Interlacing, Used on the 1st of March, 2006.http://neuron2.net/LVG/interlacing.html.

[9] William Scott Hoge. A Subspace Identi�cation Extension to the PhaseCorrelation Method. 2003. IEEE.

[10] Bertil Jacobson. Medicin och teknik. Studentlitteratur, 1995. ISBN91-630-3338-0.

[11] K. Jostschulte, A. Amer, M. Schu, and H. Schröder. Perception adap-tive temporal tv-noise reduction using contour preserving pre�lter tech-niques. In IEEE Transactions on Consumer Electronics, volume 44,pages 1091�1096, Phoenix, AZ, USA, August 1998. IEEE.

89

BIBLIOGRAPHY 90

[12] John C. Russ. Image processing handbook. CRC Press LLC, 4 edition,2002.

[13] Sune Söderkvist and Lars-Erik Ahnell. Tidsdiskreta signaler och system.Sune Söderkvist, 2 edition, 1994.

[14] Gerard J. Tortora and Sandra Reynolds Grabowski. Principles ofAnatomy and Physiology. John Wiley & Sons, Inc., 10 edition, 2003.ISBN 0-471-41501-4.

[15] Keith Wiley. How Unsharp Masking and Laplacian Sharpening Work,Used on the 23rd of February, 2006.http://www.unm.edu/ keithw/astroPhotography/imageSharpening.html.

[16] Jabri Kadri N. Wilson, David L. and Aufrichtig Richard. Preceptionof temporally �ltered x-ray �uoroscopy images. IEEE Transactions onmedical imaging, 18(1), January.

Appendix A

The combined algorithm

Figure A.1: Detection of global motion. Depending on the result, the algo-rithm carry the global or the local compensation through. T is a time delayelement.

91

APPENDIX A. THE COMBINED ALGORITHM 92

Figure A.2: Block scheme of the local motion compensation. T is a timedelay element.

APPENDIX A. THE COMBINED ALGORITHM 93

Figure A.3: Block scheme of the global motion compensation. T is a timedelay element.

Documents

Enhancement of X-ray Fluoroscopy Image Sequences using ...liu.diva-portal.org/smash/get/diva2:21841/FULLTEXT01.pdfAn algorithm based on recursive filtering, for temporal noise suppression,