L. Yaroslavsky
FAST SIGNAL SINC-INTERPOLATIONAND ITS APPLICATIONS IN IMAGE
PROCESSING
OUTLINE
Topicality and principles of signal/imageresamplingSinc-interpolation as a gold standard forconvolution based interpolationFast SDFT based sinc-inerpolationExamples of applicationsSinc-interpolation in DCT domainSinc-interpolation in sliding window
Signal/Image Resampling Applications
Signal/image sampling standard conversionSignal “fractional” delayImage co-ordinate conversion; nonuniform-to-uniformraster resamplingSignal/image localization and alignment with sub-pixelaccuracyRadon Transform and tomographic reconstruction3D imaging
Image resampling: basic principle
OutputimageInput
image
Resampling
Signal interpolation computationalmethods
• Nearest neighbor (zero order) interpolation:
( ) ( ) ( )[ ]Lones,,akronconva Lint δrr=0 ; ( ) [ ]
→←=
L,..,,,L 0001δ
• Linear (bilinear) interpolation: ( ) ( ) ( )[ ] ( )[ ]Lones,Lones,,akronconvconva Lint δrr
=1 .
• R-th order spline interpolation: ( ) ( ) ( )[ ] ( )[ ][ ]Lones,Lones,,akronconvconvconva LR
int δrLr
= .
• Sinc-interpolation:
( ) ( )[ ]x/xkxxak
k ∆∆−= ∑∞
−∞=
πα sinc
TWO APPROACHES:Convolution based and Fourier based
For a long time, sinc interpolation has been the holy grail of geometrical operations. ... The sinc-function provides error-free interpolation of bandlimited functions. There are two difficultiesassociated with this statement. The first one is that the class of bandlimited functions represents buta tiny fraction of all possible functions; moreover, they often give a distorted view of the physicalreality – think of transition air/matter in CT scan: ... this transition is abdrupt and cannot beexpressed as a bandlimited function.... The second difficulty is that the support of the sinc-functionis infinite. An infinite support is not too bothering, provided an efficient algorithm can be found toimplement interpolation with another equivalent basis function that has a finite support. This isexactly the trick we used with B-splines and o-Moms” (Ph. Thevenaz, Th. Blu, M. Unser,Interpolation Revisited, IEEE Trans. on Medical Imaging, V. 19, No. 7, July 2000)
Sinc-interpolation versus spline interpolation: an opinion
The opinion that sinc-interpolation assumes band-limited functions is afallacy. The meaning of the sampling theorem is this:
Given samples of a continuous function , there is no betterapproximation of the function with convolution bases functions than bya bandlimited function obtained by sinc-interpolation of its samples.
Sinc-interpolation: the gold standard for discretelinear interpolation of sampled data
Discrete Sampling Theorem: Let a signal of LN samples is sampled with sampling interval of L samples. For
101 −= N,...,k ; 102 −= L,...,k the sampled signal can be written as ( )21kak δ .
Compute its DFT:
( ) ( ) ( )=
+
→← ∑ ∑− −
=
1 1
0
2122
2 111
21 L
k
N
kk
DFTk r
LNkLkika
LNka πδδ exp
( ) ( )=
+∑ ∑
−
=
−
=
1
0
1
0
212
1 21
21 N
k
L
kk r
LNkLkiexpka
LNπδ
( ) Nmodr
N
kk L
rNkiexpa
LNαπ 121 1
0
1
11
=
∑
−
=
This means that sampling discrete signal results in periodical replication of its spectrum with the number of replicas equal to sampling interval.
Discrete sinc-interpolation: the gold standard forconvolution based interpolation of sampled data
Signal interpolation by (periodical) digital convolution:
( ) ( )∑ ∑−
=
−
=
+−=1
0
1
0212
1 21
N
k
L
kNLkk kNkkhkaa modint
~ δ
In DFT domain:{ }( ) ( )( ) { }( )kKrrk hDFTaDFT ⋅==
11 mod~~ αα
Therefore, the only way to avoid aliazing and signal distortions issignal ideal low pass filtering. For N-odd number:
( )( ) ( )( )∑
−
=
→←
−−+−
−1
01
11211
N
kn
IDFTNr La
LNLNNrrect 1mod k-kN;N;sincd/ α
( ) ( )( )NxN
NKx/sin/sinxN;K;sincd
ππ
=
Problem: bounary effects due to the periodical convolution
Discrete sinc-interpolation: a gold standard forinterpolation of sampled data
50 100 150 200 2500
2
4
6
8
10
50 100 150 200 2500
2
4
6
8
10
Low pass filtering
5 0 1 0 0 1 5 0 2 0 0 2 5 0
-1 . 5
-1
-0 . 5
0
0 . 5
1
1 . 5
2
0 50 100 150 200 250
-2
-1
0
1
2
Sincd-Interpolated signal
5 10 15 20 25 30
-1.5
-1
-0.5
0
0.5
1
1.5
2
5 10 15 20 25 300
1
2
3
4
5
6
7
8
9
10
Discrete signal
Its DFT spectrum
Nearest neighbor, Bilinear, Bicubic spline andSinc-image interpolation
Nearest neighborinterpolation
Bicubic splineinterpolation
Bilinearinterpolation
Sinc-interpolation
Computationl algorithms for discteresinc-interpolation: Zero Padding Method
Interpolation functions:
( )( )∑−
=
=1
01
1 N
k1kk Lk-kN;M;sincda
La~
1;
( ) ( )( )N/xsinN
N/KxsinxN;K;sincdπ
π= ; 110 −= LN,...,,k
Shifted Discrete Fourier Transforms as discreterepresentations of the Fourier integral:
x
a(x) Sampled signal
x∆u
f
( )fαSampled signal spectrum
f∆
v
( ) ( )( )xukxaxaN
kreconstr_signk ∆+−= ∑
−
=
1
0ϕ
( ) ( )( )fvrffN
kreconstr_spnr ∆+−= ∑
−
=
1
0ϕαα
Continuos signal
Continuous signalspectrum
( ) ( ) ( )dxfxiexpxaf ∫∞
∞−
= πα 2 ( )( )∑−
=
++
=1
021 N
kk
v,ur N
vrukiexpaN
πα
Fourier integral Shifted DFT (canonic form)
fx/N ∆∆= 1
Signal and spectrum sampling
( )∑−
=
+
=
1
0221 N
kk
v,ur N
rukiexpNkviexpa
Nππα ( )∑
−
=
+
−=
1
0221 N
kk
v,ur N
vrkiexpNruiexpa
Nππα
Direct and inverse Shifted DFTs (reduced form)
L.P. Yaroslavsky, Shifted Discrete Fourier Transforms, In: Digital Signal Processing, Ed. by V. Cappellini, and A. G.Constantinides, Avademic Press, London, 1980, p. 69- 74.
Signal resampling using SDFTs
( )=
+
−
−= ∑
−
= Nqrniexp
Nrpiexp
Na~
K
r
v,uq/v,p/un r
ππα 221 1
0
( )( ) ×
+
+
−∑−
=
1
021N
kk p-un-kN;K;sincdv
NKkiexpa π
( )
−
−
+
−− pu
NKiexpnq
NKiexp 121 ππ
Signal SDFT(u,v) ISDFT(p,q) Sinc-intrerpolated signalshifted by (u-p)
L. Yaroslavsky, M. Eden, Fundamentals of Digital Optics, Birkhauser, Boston, 1996
An algorithm of SDFT based signal sinc-interpolationL. P. Yaroslavsky, Signal sinc-interpolation: a fast computer algorithm, Bioimaging, 4, p. 225-231, 1996
Discrete sinc interpolation versus otherconvolution based interpolation methods
SeparableSeparable & nonseparable2-Dinterpolation
Requires analytical and numericaloptimization
Simple: direct spectrumshaping in DFT domain
Design
Non-invertibleCompletely invertibleInvertibility ofresampling
Zero interpolation error
O( log(Size of the signal frame))
SDFT based discrete sincinterpolation
Non-zero interpolation error:1/ O(Interpolation function support)
Interpolationaccuracy
O(Interpolation function support)Computationalcomplexity (peroutput sample)
Convolution basedinterpolation methodsPh. Thevenaz, T. Blu, M. Unser, Interpolation Revisited,IEE Trans. on Medical Imaging, v. 19, No. 17, July 2000
SDFT versus zero padding sinc-interpolation
No intermediate buffer isrequired
Requires an intermediate buffer for NLsamples
Memory usage
Arbitrary signal shift;Rational zoom-factor
Power of 2 for the most widely used FFTalgorithms
Zoom factor
O(logN)Arbitrary shifts are possible
O(LlogNL)unless FFT pruned algorithms are used.Shift only by (power of 2)-th fraction of thediscretization interval are possible when themost wide spread FFT algorithms are used.
Computational complexity (generaloperations, per output sample) forsignal shift by a fraction of thediscretization interval
O(N)O(NLlogNL)unless FFT pruned algorithms are used
Computational complexity (generaloperations, per output sample) of L-fold zooming signal of N samples inthe vicinity of an individual sample
O(logN)O(logNL)Computational complexity (generaloperations, per output sample) ofL-fold zooming signal of N sampleswith the use of FFT
SDFT based methodZero padding method
Applications:Spectrum analysis with sub-pixel resolution
Applications:Image rotation: a three-pass algorithm
Image rotation by an angle θ as a geometrical transformation of signal co-
ordinates can be described as a multiplication of signal coordinates ( )y,x vector
by a rotation matrix:
( )
−=
θθθθ
θcossinsincos
ROT
In order to simplify computations, one can factorize rotation matrix ( )θROT intoa product of three matrices each of which modifies only one co-ordinate:
(X-shearing Y-shearing X-shearing)
( ) ( ) ( )
−
−=
−=
1021
101
1021 /tan
sin/tan
cossinsincos
ROTθ
θθ
θθθθ
θ
Fast signal sinc-interpolation algorithm is ideally suited for signal translationneeded for image shearing
M. Unser, P. Thevenaz, L. Yaroslavsky, Convolution-based Interpolation for Fast,High-Quality Rotation of Images, IEEE Trans. on Image Processing, Oct. 1995, v. 4, No. 10, p. 1371-1382
Applications: Three pass image rotation withsinc-interpolation
Initial image First pass
Second pass Third pass: rotated image
Applications:Radon Transform and tomosynthesis:Filtered back projection reconstruction
Radon transform: rotationand directional summation
Tomographicreconstruction: ramp-filtering projections, backprojecting, rotation andsummation
Applications:Radon Transform and tomosynthesis:Fourier reconstruction method
Object
Projections
Projection spectra
Interpolated 2-Dobject spectrum
Reconstructedobject
Image geometrical transformations by means ofsinc-interpolated image zooming (oversampling)
Sinc-interpolatedsubsampling
Initial image
Magnified image
Transformed image copies
Image geometrical transformations with sinc-interpolation
Radius
Angle
Cartesianto polar
Polar toCartesian
Angle
Radius
Measuring image profile with sub-pixel resolution
Sincd-interpolation in DCT domain
Purpose: minimization of boundary effects
( )=
+
−= ∑−
=
12
0 2212
21 N
rrk N
rkiN
b /exp πβ
( ) ( )
+
+
+
−+ ∑−
=
∗1
100
212121 N
rrr
DCTr
DCT rN
kirN
kiN
/exp/exp πηπηαηα
−+=
−=
−= ∑
−
=
1210
11021 1
0
NNNk
NkNksi
NhN
ss
k
,...,;
,...,;exp πϕ
For u-shift( )( )
−+==
−==
∗− 112
2212102
NNsNsNus
NsNusi
sN
s
,...,/;/;/cos
/,...,,;/exp
ϕππ
ϕ
=
Nurir πη 22 exp
( )=
+
−= ∑∑
−
=
−
=+
1
0
1
012 2
122221 N
k
N
ssr N
rkiNksi
Nππϕη expexp ( )∑∑
−
=
−
=
+−1
0
1
0 21222
21 N
k
N
ss N
srkiN
πϕ exp
Pruned DFT is useful: first half of the sequence is zero, only odd terms ofDFT are to be computed
Symmetricallyextended signal
Interpolationfilter PSF
50 100 150 200 250
0
5
10
15
20
25
30
35FFT sinc-interpolationDCT sinc-interpolation
Sincd-interpolation in DCT domain
10 20 300
5
10
15
20
25
30
Initial signal8x sinc-interpolated signals
2-D separable sincd-interpolation in DCT domain
FFT interpolation DCT interpolation
Signal resampling in sliding window:Summary
Good approximation to sincd-interpolationSimple design in DFT domainIrregular -to-regular sampling raster resampling is possibleCan naturally be combined with signal denoising and
restorationLocal adaptive interpolation is possibleImplementation of inseparable interpolation kernel is easyComputational complexity O(WindowSize) per output
signal sample (comparable to that of spline interpolation)
Signal sinc-interpolation in sliding window:Impulse and frequency responses
1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0
- 0 . 1
0
0 . 1
0 . 2
0 . 3
0 . 4
0 . 5
0 . 6
0 . 7
0 . 8
0 . 9
1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0
- 0 . 1
0
0 . 1
0 . 2
0 . 3
0 . 4
0 . 5
0 . 6
0 . 7
0 . 8
0 . 9
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
2.5
3
3.5
Window11 Window15
Interpolation functions for window size 11 and 15 samples Interpolation filter frequency responses: signal 3x zoom
Image zoom: global versus sliding window sinc-interpolation
Global sinc-interpolation Sliding window sinc-interpolation
Arbitrary image mapping in sliding window
lcmapping_arbitr(len,(mapX+i*mapY)/1.5,8,8)
MapX
MapY
100 200 300 400 500 600 700 800 900 1000
0
0.2
0.4
0.6
0.8
1
Ada ptive ve rs us non a da ptive s inc a nd ne a re s t ne ighbor inte rpola tion
non a da ptivene a re s t ne ighbora da ptive
Sliding window sinc-interpolation:non-adaptive vesrus adaptive
10 20 30 40 50 600
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Initial signal Interpolated signal (8xZoom)
Sliding window sinc-interpolation:non-adaptive vesrus adaptive
non-adaptive adaptive
Irregular-to-regular raster image resampling anddenoising in sliding window
Additional bibliography on FFT based and spline basedinterpolation
• D. Fraser, Interpolation by the FFT revisited – An experimental evaluation, IEEE Trans. ASSP-37, p. 665-676, May1989
• T.J. Cavichi, DFT time domain Interpolation, Proc. Inst.Elec.Eng.-F, vol. 139, pp. 207-211, 1992
• M. Unser, A. Aldroubi, M. Eden, Plynomial spline signal approximations: filter design and asymptotic equivalencewith Shannonäs sampling theorem, IEEE Trans. IT, v. 38, pp. 95-103, Jan. 1992
• Ph. Thevenaz, Th. Blu, M. Unser, Interpolation Revisited, IEEE Trans. on Medical Imaging, V. 19, No. 7, July 2000