Embed Size (px)
Citation preview
Chapter 5Image Restoration
國立雲林科技大學 資訊工程研究所張傳育 (Chuan-Yu Chang ) 博士Office: EB 212TEL: 05-5342601 ext. 4337E-mail: [email protected]
2
Chapter 5 Image Restoration Image Degradation/Restoration Process
The objective of restoration is to obtain an estimate of the original image.
will be close to f(x,y).
),(ˆ yxf
),(ˆ yxf
Restoration 的目的在於獲得原始影像的估計影像，此估計影像應儘可能
的接近原始影像。
3
Image Degradation/Restoration Process
The degraded image is given in the spatial domain by
The degraded image is given in the frequency domain by
),(),(*),(),( yxyxfyxhyxg
),(),(),(),( vuNvuFvuHvuG
(5.1-1)
(5.1-2)
Degradation function
noise
4
Noise Models: Some Important Probability Density Functions The principal sources of noise
Image acquisition transmission
Gaussian noise (normal noise)
Rayleigh noise
Erlang (Gamma) noise
22 2/)(
2
1)(
zezp
4
)4(
4/
0
)(2
)(
2
/)( 2
b
ba
azfor
azforeazbzp
baz
22
1
00
0)!1()(
a
ba
b
zfor
zforeb
zazp
azbb
5
Noise Models
Exponential noise
Uniform noise
Impulse (salt and pepper) noise12
)(
2
0
1)(
22 ab
ba
otherwise
bzaifabzp
otherwise
bzforP
azforP
zp b
a
0
)(
22 1
1
00
0)(
a
a
zfor
zforaezp
az
6
Some important probability density function
偏離原點的位移，向右傾斜
7
Example 5.1Sample noisy images and their histograms 下圖為原始影像，將在此圖中加入不同的雜訊。
8
Example 5.1 (cont.)Sample noisy images and their histograms
9
Example 5.1 (cont.) Sample noisy images and their histograms
10
Periodic Noise
Periodic Noise Arises typically from electrical or electromechanical
interference during image acquisition. It can be reduced via frequency domain filtering.
Estimation of Noise Parameters Estimated by inspection of the Fourier spectrum of the
image. Periodic noise tends to produce frequency spikes that often
can be detected by visual analysis. From small patches of reasonably constant gray level. The heights of histogram are different but the shapes are
similar.
11
Example影像遭受sinusoidal
noise 的破壞
影像的spectrum
有規則性的亮點
12
Fig 5.4(a-c) 中的某小塊影像的histogram
Histogram 的形狀幾乎和 Fig.4 (d,e,k) 的形狀一樣但高度不同。
13
Periodic Noise
The simplest way to use the data from the image strips is for calculating the mean and variance of the gray levels.
The shape of the histogram identifies the closest PDF match.
Sz
iii
zpz
Sz
iii
zpz 22
(5.2-15)
(5.2-16)
14
Restoration in the presence of noise only-spatial filtering When the only degradation present in an image is
noise, Eq. (5.1-1) and (5.1-2) become
The noise terms ((x,y), N(u,v)) are unknown, so subtracting them from g(x,y)or G(u,v) is not a realistic option.
In periodic noise, it is possible to estimate N(u,v) from the spectrum of G(u,v).
),(),(),(
),(),(),(
vuNvuFvuG
yxyxfyxg
(5.3-1)
(5.3-2)
15
Restoration in the presence of noise only-spatial filtering Mean Filter
Arithmetic mean filter Let Sxy represent the set of coordinates in a rectangular
subimage windows of size mxn, centered at point (x,y). The arithmetic mean filtering process computes the average
value of the corrupted image g(x,y) in the area defined by Sxy.
This operation can be implemented using a convolution mask in which all coefficients have value 1/mn.
Noise is reduced as a result of blurring
xySts
tsgmn
yxf),(
),(1
),(ˆ
16
Mean Filter (cont.)
Geometric mean filter Each restored pixel is given by the product of the pixels in
the subimage window, raised to the power 1/mn.
A geometric mean filter achieves smoothing comparable to the arithmetic mean filter, but it tends to lose less image detail in the process.
mn
Sts xy
tsgyxf
1
),(
),(),(ˆ
17
Example 5.2Illustration of mean filters
被平均值被平均值 00 ，變異數，變異數400400 的加成性高斯雜的加成性高斯雜
訊破壞的結果訊破壞的結果
18
Restoration in the presence of noise only-spatial filtering
Harmonic mean filter
Contra-harmonic mean filter
xySts tsg
mnyxf
),( ),(1
),(ˆ
xy
xy
Sts
Q
Sts
Q
tsg
tsg
yxf
),(
),(
1
),(
),(
),(ˆ
可濾除 salt noise,但對 pepper noise 則失
敗
若 Q>0 可濾除 pepper noise,
若 Q<0 可濾除 salt noise,若 Q=0 為算數平均若 Q=-1 為 Harmonic mean
19
Chapter 5Image Restoration
Chapter 5Image Restoration
被機率被機率 0.10.1 的的 saltsalt 雜雜訊破壞的結果訊破壞的結果
被機率被機率 0.10.1 的的 pepperpepper雜訊破壞的結果雜訊破壞的結果
The positive-order filter did a better job of cleaning the background.In general, the arithmetic and geometric mean filters are well suited for random noise.The contra-harmonic filter is well suited for impulse noise
Contraharmonic Contraharmonic mean filter, Q=1.5mean filter, Q=1.5
濾波的結果濾波的結果
Contraharmonic Contraharmonic mean filter, Q= -mean filter, Q= -1.5 1.5 濾波的結果濾波的結果
20
Results of selecting the wrong sign in contra-harmonic filtering
The disadvantage of contraharmonic filter is that it must be known whether the noise is dark or light in order to select the proper sign for Q. The result of choosing the wrong sign for Q can be disastrous.在 contra-harmonic filter 中選取錯誤正負號所導致的結果
21
Order-Statistics Filters The response of the order-statistics filters is based
on ordering (ranking) the pixels contained in the image area encompassed by the filter.
Median filter Replaces the value of a pixel by the median of the gray
levels in the neighborhood of that pixel.
Median filter provide excellent noise-reduction capabilities, with considerably less blurring than linear smoothing filters of similar size.
Median filters are particularly effective in the presence of both bipolar and unipolar impulse noise.
tsgmedianyxfxySts
,),(ˆ),(
(5.3-7)
22
Order-Statistics Filters (cont.) Max filter
This filter is useful for finding the brightest points in an image.
It reduces pepper noise
Min filter This filter is useful for finding the darkest points in an image
It reduces salt noise.
tsgyxfxySts
,max),(ˆ),(
tsgyxfxySts
,min),(ˆ),(
可濾除 pepper noise
可濾除 salt noise
(5.3-8)
(5.3-9)
23
Order-Statistics Filters (cont.) Midpoint filter
This filter works best for randomly distributed noise, such as Gaussian or uniform noise.
Alpha-trimmed mean filter We delete the d/2 lowest and the d/2 higest gray-level values
of g(s,t) in the neighborhood Sxy.
tsgtsgyxf
xyxy StsSts,min,max
2
1),(ˆ
),(),(
xySts
r tsgdmn
yxf),(
),(1
),(ˆ(5.3-11)
(5.3-10)
先刪除 0.5d 最大與最小的灰階值，再求剩下的灰階平均值當 d=0 ，則為 mean filter當 d=(mn-1)/2 時，為 median filter
24
Example 5.3Illustration of order-statistics filtersImage corrupted by salt
and pepper noise with probabilities Pa=Pb=0.1
Result of one pass with a median filter of size 3x3, several noise points are still visible.
Result of processing (b) with median filter again
Result of processing (c) with median filter again
25
Example 5.3Illustration of order-statistics filters
Result of filtering with a max filtering
Result of filtering with a min filtering
26
Example 5.3 Illustration of order-statistics filters
Result of filtering (b) with a 5x5 arithmetic
mean filter
Result of filtering (b) with a geometric
mean filter
Result of filtering (b) with a median filter
Result of filtering (b) with a alpha-
trimmed mean filter with d=5
Image corrupted by additive uniform
noise (variance 800 and zero mean)
Image corrupted by additive salt-and-
pepper noise (Pa=Pb=0.1)
27
Adaptive Filter Adaptive Filter
The behavior changes based on statistical characteristics of the image inside the filter region defined by the m x n rectangular windows Sxy.
The price paid for improved filtering power is an increase in filter complexity.
Adaptive, local noise reduction filter The mean gives a measure of average gray level in the
region. The variance gives a measure of average contrast in that
region. The response of the filter at any point (x,y) on which the
region is centered is to be based on four quantities: g(x,y): the value of the noisy image. , the variance of the noise corrupting f(x,y) to form g(x,y) mL, the local mean of the pixels in Sxy. , the local variance of the pixels in Sxy.
2L
2
28
Adaptive local noise reduction filter The behavior of the adaptive filter to be as follows:
If the variance of noise is zero, the filter should return simply the value of g(x,y).
If the local variance is high relative to the variance of noise, the filter should return a value close to g(x,y).
If the two variances are equal, return the arithmetic mean value of the pixels in Sxy.
An adaptive expression for obtaining estimated based on these assumptions may be written as
LL
myxgyxgyxf ,),(),(ˆ2
2
(5.3-12)
The only quantity thatNeeds to be known
yxf ,ˆ
29
Example 5.4 Illustration of adaptive, local noise-reduction filtering
Gaussian noise
Arithmetic mean 7*7
geometric mean 7*7
Adaptive filter
30
Adaptive median filter
Adaptive median filtering can handle impulse noise, it seeks to preserve detail while smoothing nonimpulse noise.
The adaptive median filter changes the size of Sxy during filter operation, depending on certain conditions.
Consider the following notation: zmin: minimum gray level value in Sxy. zmax: maximum gray level value in Sxy. zmed: median of gray levels in Sxy. zxy: gray level at coordinates (x,y). Smax: maximum allowed size of Sxy.
31
Adaptive median filter (cont.) The adaptive median filtering algorithm Level A: A1=zmed-zmin
A2=zmed-zmax
if A1>0 and A2<0, goto level Belse increase the window sizeif window size <=Smax, repeat level Aelse output zxy
Level B: B1=zxy-zmin
B2=zxy-zmax
if B1>0 and B2 <0, output zxy
else output zmed
判斷 zmed 是否為impulse noise
32
Adaptive median filter (cont.)
The objectives of the adaptive median filter Remove the slat-and-pepper noise Preserve detail while smoothing nonimpulse noise Reduce distortion
The purpose of level A is to determine in the median filter output, zmed is an impulse or not.
33
Example 5.5 Illustration of adaptive median filtering
Corrupted by salt-and pepper noise with probabilities
Pa=Pb=0.25
Result of filtering with a 7x7 median filter
Result of adaptive median filtering with
Smax=7
The noise was effectively removed, the filter caused significant loss of detail in the image
Preserved sharpness and detail
34
Periodic Noise Reduction by Frequency Domain Filtering Bandreject Filter
Remove a band of frequencies about the origin of the Fourier transform.
2),(1
2),(
20
2),(1
),(
0
00
0
WDvuDf
WDvuD
WDif
WDvuDif
vuH
n
DvuDWvuD
vuH 2
20
2 ),(),(
1
1),(
220
2
),(
),(
2
1
1),(
WvuD
DvuD
evuH
Ideal Bandreject filter
n order Butterworth filter
Gaussian Bandreject filter
(5.4-1)
(5.4-2)
(5.4-3)
bandband 的寬度的寬度
35
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
36
Image corrupted by sinusoidal noise
Butterworth bandreject filter of order 4
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
37
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Bandpass Filters A bandpass filter performs the opposite operation of a
bandreject filter. The transfer function Hbp(u,v) of a bandpass filter is
obtained from a corresponding bandreject filter with transfer function Hbr(u,v) by
),(1),( vuHvuH brbp (5.4-4)
38
以帶通濾波器所獲得的圖 5.16(a) 影像的
雜訊圖樣
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Bandpass filtering is quit useful in isolating the effect on an image of selected frequency bands.
This image was generated by(1) Using Eq(5.4-4) to obtain
the bandpass filter.(2) Taking the inverse
transform of the bandpass-filtered transform
39
Periodic Noise Reduction by Frequency Domain Filtering (cont.) Notch Filters
Rejects frequencies in predefined neighborhoods about a center frequency.
Due to the symmetry of the Fourier transform, notch filters must appear in symmetric pairs about the origin
212
02
02
2
12
02
01
0201
2/2/),(
2/2/),(
1
),(),(0),(
vNvuMuvuD
vNvuMuvuD
otherwise
DvuDorDvuDifvuH
(5.4-5)
(5.4-6)
(5.4-7)
因為對稱性的關係
40
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
order n Butterworth notch filter
Gaussian notch reject filter
These three filters become highpass filters if u0=v0=0.
n
vuDvuDD
vuH
),(),(1
1),(
21
20
(5.4-8)
20
21 ),(),(
2
1
1),( D
vuDvuD
evuH (5.4-9)
41
Ideal notch
order 2 Butterworth notch filter
Gaussian notch filter
若 u0=v0=0 ，上述三種濾波器，則退化成高通濾波器
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
42
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Notch pass filters We can obtain notch pass filters that pass the frequencies
contained in the notch areas. Exactly the opposite function as the notch reject filters.
Notch pass filters become lowpass filters when u0=v0=0.
vuHvuH nrnp ,1, (5.4-10)(5.4-10)
43
佛羅里達州和墨西哥灣的衛星影像
( 存在水平掃描線 )
Spectrum image
Notch filter
空間域的雜訊影像
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
44
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Optimal Notch filtering Clearly defined interference patterns are not
common. Images obtained from electro-optical scanner are
corrupted by coupling and amplification of low-level signals in the scanners’ electronic circuitry.
The resulting images tend to contain significant, 2D periodic structures superimposed on the scene data.
45
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Image of the Martian terrain taken by the Mariner 6 spacecraft. The interference pattern is hard to detect. The star-like components were caused by the interference, and
several pairs of components are present. The interference components generally are not single-frequency
bursts. They tend to have broad skirts that carry information about the interference pattern.
46
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Optimal Notch filtering minimizes local variances of the restored estimate image.
The procedure contains three steps: Extract the principal frequency components of the
interference pattern. Subtracting a variable, weighted portion of the
pattern from corrupted image.
47
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
The first step is to extract the principal frequency component of the interference pattern Done by placing a notch pass filter, H(u,v) at the
location of each spike. The Fourier transform of the interference noise
pattern is given by the expression
where G(u,v) denotes the Fourier transform of the corrupted image.
vuGvuHvuN ,,,
48
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Formation of H(u,v) requires considerable judgment about what is or is not an interference spike. The notch pass filter generally is constructed
interactively by observing the spectrum of G(u,v) on a display.
After a particular filter has been selected, the corresponding pattern in the spatial domain is obtained from the expression
vuGvuHyx ,,, 1
49
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Because the corrupted image is assumed to be formed by the addition of the uncorrupted image f(x,y) and the interference, if (x,y) were know completely, subtracting the pattern from g(x,y) to obtain f(x,y) would be a simple matter. This filtering procedure usually yields only an approximation of
the true pattern. The effect of components not present in the estimate of (x,y)
can be minimized instead by subtracting from g(x,y) a weighted portion of (x,y) to obtain an estimate of f(x,y).
The function w(x,y) is to be determined, which is called as weighting or modulation function.
The objective of the procedure is to select this function so that the result is optimized in some meaningful way.
yxyxwyxgyxf ,,,,ˆ (5.4-13)
50
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
To select w(x,y) so that the variance of the estimate f(x,y) is minimized over a specified neighborhood of every point (x,y).
Consider a neighborhood of size (2a+1) by (2b+1) about a point (x,y), the local variance can be estimated as
where
a
as
b
bt
yxftysxfba
yx2
2 ,ˆ,ˆ1212
1,
a
as
b
bt
tysxfba
yxf ,ˆ1212
1,ˆ
(5.4-15)
(5.4-14)
51
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
Substituting Eq(5.4-13) into Eq(5.4-14) yield
Assuming that w(x,y) remains essentially constant over the neighborhood gives the approximation
This assumption also results in the expression
in the neighborhood.
),(, yxwtysxw
22 ,,),(,,,1212
1, yxyxwyxgtysxtysxwtysxg
bayx
a
as
b
bt
yxyxwyxyxw ,,,, (5.4-18)
(5.4-17)
(5.4-16)
52
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
With these approximations Eq5.4-160 becomes
To minimize variance, we solve
for w(x,y)
The result is
22 ,,),(,,,1212
1, yxyxwyxgtysxtysxwtysxg
bayx
a
as
b
bt
0
,
,2
yxw
yx
),(),(
),(,),(,,
22 yxyx
yxyxgyxyxgyxw
(5.4-21)
(5.4-20)
(5.4-19)
53
圖 5-20(a) 的傅立葉頻譜
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
54
N(u,v) 的傅立葉頻譜
對應的雜訊圖樣
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
55
處理後的影像
Periodic Noise Reduction by Frequency Domain Filtering (cont.)
56
Linear, Position-Invariant Degradations
)],([)],([)],(),([
1
)],([)],([)],(),([
)],([),(
0),(
),()],([),(
2121
2121
yxfHyxfHyxfyxfH
baIf
yxfbHyxfaHyxbfyxafH
iflinearisH
yxfHyxg
yxthatassume
yxyxfHyxg
AdditivityIf H is a linear operator, the response to a sum of two inputs is equal to the sum of the two response
(5.5-1)
(5.5-2)
(5.5-3)
57
Linear, Position-Invariant Degradations
Homogeneity The response to a constant multiple of any input is equal
to the response to that input multiplied by the same constant.
),(),(
)2.5.5.(,0),(
11
2
yxfaHyxafH
becomesEqyxfwith
(5.5-4)
58
Linear, Position-Invariant Degradations
Position (space) invariance The response at any point in the image depends only on the
value of the input at that point, not on its position.
f(x,y) 可用連續脈衝函數表示成
假設 (x,y)=0, 則將 Eq(5.5-6) 代入 Eq(5.5-1) 可得
因為 H 為線性運算子 , 利用加成性的性質
),(),( yxgyxfH (5.5-5)
ddyxfyxf
,,,
ddyxfHyxfHyxg ,,,,
(5.5-6)
(5.5-7)
ddyxfHyxfHyxg
,,,, (5.5-8)
59
Linear, Position-Invariant Degradations (cont.)
又因為 f(,) 和 x,y 無關 , 因此利用 Homogeneity 可得
其中 ,H 為脈衝響應 (impulse response),h(x,,y,) 為點展開函數 (point spread function, PSF)
將 Eq(5.5-10) 代入 Eq(5.5-9) 可得
ddyxHfyxfHyxg
,,,,
yxHyxh ,,,,
(5.5-9)
(5.5-10)
ddyxhfyxg
,,,,, (5.5-11)
60
Linear, Position-Invariant Degradations (cont.) 若 H 為位置不變 , 則 Eq(5.5-5) 可知
則 Eq(5.5-11) 可變成
上式為 convolution integral( 同 Eq(4.2-30)) 若在有加成性雜訊下 , Eq(5.5-11) 可表示成
若 H 是位置不變 , 則 Eq(5.5-14) 會變成
yxhyxH ,,
ddyxhfyxg
,,,
yxddyxhfyxg ,,,,,,
yxddyxhfyxg ,,,,
(5.5-12)
(5.5-13)
(5.5-14)
(5.5-15)
61
Linear, Position-Invariant Degradations (cont.) Summary
因為雜訊項 (x,y) 是隨機 , 與位置無關的 , 可將 Eq(5.5-15) 改寫成
A linear, spatially invariant degradation system with additive noise can be modeled in the spatially domain as the convolution of the degradation function with an image, followed by the addition of noise.
),(),(),(),(
),(),(*),(),(
vuNvuFvuHvuG
yxyxfyxhyxg
(5.5-16)
(5.5-17)
62
Estimating the Degradation Function
There are three principal ways to estimate the degradation function for use in image restoration: Observation Experimentation Mathematical modeling
63
Estimating the Degradation Function
Estimation by image observation When a given degraded image without any knowledge about the
degradation function H. To gather information from the image itself.
Look at a small section of the image containing simple structures. Look for areas of strong signal content. Gs(u,v) Construct an unblurred image as the observed subimage. Fs(u,v) Assume that the effect of noise is negligible, thus the degradation
function could be estimated by Hs(u,v)=Gs(u,v)/Fs(u,v) To construct the function H(u,v) by turns out the Hs(u,v) to have the
shape of Butterworth lowpass filter.
64
Estimating the Degradation Function
Estimation by experimentation A linear, space-invariant system is described completely
by its impulse response. A impulse is simulated by a bright dot of light, as bright as
possible to reduce the effect of noise. 把一個已知的函數加以模糊以便得到近似的 H(u,v)
A
vuGvuH
vuFvuHvuG
),(),(
1)y)(x,(constant aisimpulseanoftransformFourierthe
),(),(),(
(5.6-2)
65
An impulse of light ( 光脈衝 )
Degraded impulse( 影像脈衝 )
Estimating the Degradation Function
66
Estimating the Degradation Function Estimation by modeling
Degradation model based on the physical characteristics of atmospheric turbulence:
This model has a familiar form
where k is a constant that depends on the nature of the turbulence.
With the exception of the 5/6 power on the exponent, this equation has the same form as the Gaussian lowpass filter.
In fact, the Gaussian LPF is used sometimes to model mild uniform blurring.
6/522 )(),( vukevuH (5.6-3)
67
Estimating the Degradation Function
68
Remove the degradation of planar motion An image has been blurred by uniform linear motion
between the image and the sensor during image acquisition. Suppose that an image f(x,y) undergoes planar motion and
that x0(t) and y0(t) are the time varying components of motion in the x- and y-directions.
The total exposure at any point of the recording medium is obtained by integrating the instantaneous exposure over the time internal during which the imaging system shutter is open.
Assuming that shutter opening and closing takes place instantaneously, and that the optical imaging process is perfect, isolates the effect of image motion.
69
Remove the degradation of planar motion (cont.)
If T is the duration of the exposure, it follows that
where g(x,y) is the blurred image. From Eq(4.2-3), the Fourier transform of Eq(5.6-4) is
dttyytxxfyxgT
0 00 )(),(),(
(5.6-7)
(5.6-6)
(5.6-5)
(5.6-4)
T tvytuxj
tvytuxjT
T vyuxj
vyuxjT
vyuxj
dtevuF
dtevuF
dtdxdyetyytxxf
dxdyedttyytxxf
dxdyeyxgvuG
0
)]()([2
)]()([2
0
0
)(200
)(2
0 00
)(2
00
00
),(
),(
)(),(
)(),(
),(),(
Reversing the order of integration
The Fourier transform ofthe displaced function.
F(u,v) is independent of t
70
Remove the degradation of planar motion (cont.) By defining
Eq(5.6-7) may be expressed in the familiar form
If the motion variables x0(t) and y0(t) are known, the transform function H(u,v) can be obtained from Eq(5.6-8).
Suppose that the image in question undergoes uniform linear motion in the x-direction only, at a rate given by
When t=T, the image has been displaced by a total distance a. With y0(t)=0, Eq(5.6-8) yields
T tvytuxj dtevuH0
)]()([2 00),(
uajT TuatjT tuxj euaua
TdtedtevuH
)sin(),(
0
/2
0
)(2 0
),(),(),( vuFvuHvuG
Tattx /)(0
(5.6-10)
(5.6-9)
(5.6-8)
71
Remove the degradation of planar motion (cont.)
If we allow the y-component to vary as well, with the motion given by
Then the degradation function becomes
)()](sin[)(
),( vbuajevbuavbua
TvuH
Tbtty /)(0
(5.6-11)
72
將左圖的 Fourier Transform 乘上 (5.6-11)的 H(u,v) 後，取其反
Fourier Transform 的結果。a=b=0.1, T=1
Remove the degradation of planar motion (cont.) Example 5.10:
Fig. (b) is an image blurred by computing the Fourier transform of the image in Fig. (a), multiply the transform by H(u,v) from Eq.(5.6-11), and taking the inverse transform.
The images are of size 688x688 pixels, and the parameters used in Eq(5.6-11) were a=b=0.1 and T=1
73
Inverse Filtering Direct inverse filtering
Compute an estimate, ,of the transform of the original image simply by dividing the transform of the degraded image, G(u,v), by the degradation function:
Even if we know the degradation function we cannot recover the undegraded image exactly because N(u,v) is a random function whose Fourier transform is not known.
If the degradation has zero or very small values, then the ratio N(u,v)/H(u,v) could easily dominate the estimate .
),(ˆ vuF
(5.7-1)
(5.7-2)),(
),(),(),(ˆ
),(),(),(),(
),(
),(),(ˆ
vuH
vuNvuFvuF
vuNvuFvuHvuG
vuH
vuGvuF
),(ˆ vuF
74
Inverse Filtering (cont.)
One way to get around the zero or small-value problem is to limit the filter frequencies to values near the origin. We know that H(0,0) is equal to the average value of
h(x,y) and that this is usually the highest value of H(u,v) in the frequency domain.
Thus, by limiting the analysis to frequencies near the origin, we reduce the probability of encountering zero values.
In general, direct inverse filtering has poor performance.
75
直接G(u,v)/H(u,v)
Cutoff H(u,v) a radius of 40
Cutoff H(u,v) a radius of 70
Cutoff H(u,v) a radius of
85
Inverse Filtering (cont.)
76
Minimum Mean Square Error (Wiener) Filtering Incorporated both the degradation function and statistical
characteristics of noise into the restoration process. The objective is to find an estimate f of the uncorrupted image f
such that the mean square error between them is minimized. The error measure is given by
The minimum of the error function is given in the frequency domain by
),(),(
),(
),(
1
),(),(/),(),(
),(
),(
1
),(),(/),(),(
),(
),(),(),(),(
),(),(),(ˆ
2
2
2
2
2
*
2
*
vuGKvuH
vuH
vuH
vuGvuSvuSvuH
vuH
vuH
vuGvuSvuSvuH
vuH
vuGvuSvuHvuS
vuSvuHvuF
f
f
f
f
(5.8-1)
(5.8-2)
(5.8-3)
通常為未知，因此以 K來估計
22 f̂fEe
77
Example 5.12 Fig. (a) is the full inverse-filtered result shown in Fig.
5.27(a). Fig. (b) is the radially limited inverse filter result of Fig.
5.27(a). Fib. (c) shows the result obtained using Eq(5.8-3) with
the degradation function used in Example 5.11.
78
Example 5.13
From left to right, the blurred image of Fig.
5.26(b) heavily corrupted by additive Gaussian noise of zero mean and variance of 650.
The result of direct inverse filtering
The result of Wiener filtering.
79
Constrained Least Squares Filtering
The difficulty of the Wiener filter: The power spectra of the undegraded image and noise
must be known A constant estimate of the ratio of the power spectra is not
always a suitable solution. Constrained Least Squares Filtering
Only the mean and variance of the noise are needed.
80
Constrained Least Squares Filtering We can express Eq(5.5-16) in vector-matrix form, as
Suppose that g(x,y) is of size M x N, then we can form the first N elements of the vector g by using the image elements in first row of g(x,y), the next N elements from the second row, and so on.
The resulting vector will have dimensions MN x 1. these are also the dimensions of f and .
The matrix H then has dimensions MN x MN Its elements are given by the elements of the convolution given in Eq(4.2-30).
Central to the method is the issue of the sensitivity of H to noise. To alleviate the noise sensitivity problem is to base optimality of restoration
on a measure of smoothness, such as the second derivation of an image.
(5.9-1)ηHfg
81
Constrained Least Squares Filtering (cont.) To find the minimum of a criterion function, C, defined as
subject to the constraint
where is the Euclidean vector norm, and is the estimate of the undegraded image.
The frequency domain solution to this optimization problem is given by the expression
where is a parameter that must be adjusted so that the constraint inEq(5.9-3) is satisfied.
21
0
1
0
2 ),(
M
x
N
y
yxfC (5.9-2)
(5.9-3)
(5.9-4)
22ˆ fHg
),(),(),(
),(),(ˆ
22
*
vuGvuPvuH
vuHvuF
www T2
f̂
82
Constrained Least Squares Filtering (cont.)
P(u,v) is the Fourier transform of the function.
This function is the same as the Laplacian operator. Eq.(5.9-4) reduces to inverse filtering if is zero.
010
141
010
),( yxp (5.9-5)
83
were selected manually to yield the best visual results.
Constrained Least Squares Filtering (cont.)
84
Constrained Least Squares Filtering (cont.) It is possible to adjust the parameter interactively until
acceptable results are achieved. If we are interested in optimality, the parameter must be
adjusted so that the constraint in Eq(5.9-3) is satisfied. Define a “residual” vector r as
Since, from the solution in Eq(5.9-4), is a function of , then r also is a function of this parameter. It can be shown that
is a monotonically increasing function of . What we want to do is adjust gamma so that
fHgr ˆ
2rrr T
f̂
(5.9-6)
(5.9-7)
a 22ηr (5.9-8)
85
Constrained Least Squares Filtering (cont.) Because () is monotonic, finding the desired
value of is not difficult. Step 1: specify an initial value of Step 2: Compute ||r||2
Step 3: Stop if Eq(5.9-8) satisfied; otherwise return to Step 2 after increasing ifor decreasing ifUse the new value of in Eq(5.9-4) to recompute the optimum estimate
a 22ηr
a 22ηr
vuF ,ˆ
86
Constrained Least Squares Filtering (cont.) In order to use the algorithm, we need the quantities and
. To compute , from Eq(5.9-6) that
From which we obtain r(x,y) by computing the inverse transform of R(u,v).
Consider the variance of the noise over the entire image, which we estimate by the sample-average method:
where
is the sample mean.
vuFvuHvuGvuR ,ˆ,,,
1
0
1
0
22,
M
x
N
y
yxrr
1
0
21
0
2 ,1 M
x
N
y
myxMN
1
0
1
0
,1 M
x
N
y
yxMN
m
2r
22
r
(5.9-9)
(5.9-10)
(5.9-11)
(5.9-12)
87
Constrained Least Squares Filtering (cont.) With reference to the form of Eq(5.9-10), the double
summation in Eq(5.9-11) is equal to This gives us the expression
We can implement an optimum restoration algorithm by having knowledge of only the mean and variance of the noise.
2
mMN 22(5.9-13)
88
Constrained Least Squares Filtering (cont.) The initial value used for was 10-5, the correction
factor for adjusting was 10-6, the value for a was 0.25.
89
Geometric Mean Filter The generalized Wiener filter is the form of the so-
called geometric mean filter.
where and being positive, real constants. When =1 this filter reduces to the inverse filter With =0, the filter becomes the so-called parametric
Wiener filter. If =0.5, the filter becomes the geometric mean filter. With =1, as decreases below 0.5, the filter tend more
toward the inverse filter. When increases above 0.5, the filter will behave more
like the Wiener filter. When =0.5 and =1, the filter is referred to as spectrum
equalization filter.
),(
),(
),(),(
),(
),(
),(),(ˆ
1
2
*
2
*
vuG
vuS
vuSvuH
vuH
vuH
vuHvuF
f
(5.10-1)
90
Geometric Transformations Geometric transformations modify the spatial relationships
between pixels in an image. Often be called rubber-sheet transformations. They may be viewed as the process of printing an image on a
sheet of rubber and then stretching this sheet according to some predefined set of rules.
A geometric transformation consists of two basic operations: Spatial transformation
Defines the “rearrangement” of pixels on the image plane Gray-level interpolation
Deals with the assignment of gray levels to pixels in the spatially transformed image.
91
Geometric Transformations Spatial Transformations
Suppose that an image f with pixel coordinates (x,y) undergoes geometric distortion to produce an image g with coordinates (x’, y’). This transformation may be expressed as
where r(x,y) and s(x,y) are the spatial transformations that produced the geometrically distorted image g(x’,y’).
To formulate the spatial relocation of pixels by the use of tiepoints, which are a subset of pixels whose location in the input (distorted) and output (correct) images is known precisely.
),(
),('
'
yxsy
yxrx
(5.11-1)
(5.11-2)
92
Geometric Transformations (cont.) Figure 5.23 shows quadrilateral regions in a distorted and
corresponding corrected image. The vertices of the quadrilaterals are corresponding tiepoints. Suppose that the geometric distortion process within the
quadrilateral regions is modeled by a pair of bilinear equations so that
Then, from Eqs.(5.11) and (5.11-2)
8765
4321
'
'
cxycycxcy
cxycycxcx
8765
4321
),(
),(
cxycycxcyxs
cxycycxcyxr
(5.11-3)
(5.11-4)
93
Geometric Transformations (cont.)
Since there are a total of eight known tiepoints, these equations can be solved for the eight coefficients ci, i=1, 2,…,8.
The coefficients constitute the geometric distortion model used to transform all pixels within the quadrilateral region defined by the tiepoints used to obtain the coefficients.
In general, enough tiepoints are needed to generate a set of quadrilaterals that cover the entire image, with each quadrilateral having its own set of coefficients.
If we want to find the value of the undistorted image at any point (x0,y0), we simply need to know where in the distorted image f(x0, y0) was mapped. Substituting (x0,y0) into Eqs.(5.11-5) and (5.11-6) to obtain the
geometrically distorted coordinates (x’0,y’0). The restored image value by letting
The procedure continues pixel by pixel and row by row until an array whose size does not exceed the size of image g is obtained.
',',ˆ00 yxgyxf
94
Geometric Transformations (cont.) Depending on the values of the coefficients ci,
Eqs(5.11-5) and (5.11-6) can yield noninteger values for x’ and y’.
Because the distorted image g is digital, its pixel values are defined only at integer coordinates.
Thus using noninteger values for x’ and y’ causes a mapping into locations of g for which no gray levels are defined.
What the gray-level values at those locations should be? The technique used to accomplish this is called gray-level
interpolation
95
Geometric Transformations (cont.) The simplest scheme for gray-level interpolation is
based on a nearest neighbor approach. Zero-order interpolation (1) mapping the integer coordinates (x, y) into fractional
coordinates (x’, y’) by Eqs.(5.11-5) and (5.11-6). (2) select the closest integer coordinates neighbor to (x’, y’). (3) assign the gray level of this nearest neighbor to the pixel
located at (x,y).
(1)
(2)
(3)
96
Geometric Transformations (cont.)
The drawback of nearest neighbor interpolation is producing undesirable artifacts.
Smoother results can be obtained by using more complex techniques, such as Cubic convolution interpolation
Which fits a surface of the sin(z)/z type through a much larger number of neighbors to obtain a smooth estimate of the gray level at any desired point.
Bilinear interpolation Uses the gray levels of the four nearest neighbors to estimate
interpolation value. Because the gray level of each of the four integral nearest
neighbors of a nonintegral pair of coordinates (x’, y’) is known, the gray-level value at these coordinates, denoted v(x’, y’) can be interpolated from the values of its neighbors by using
dycxbyaxyxv '''')','( (5.11-7)
97
Geometric Transformations (cont.) (a) Image with 25 regularly
spaced tiepoints. (b) a simple rearrangement of
the tiepoints to create geometric distortion.
98
Geometric Transformations (cont.)
