A Lagrange Interpolation Based Error Correction Coding for the Images

Hossein Ajorloo Mohammad-Taghi Manzuri-ShalmaniDepartment of Computer Engineering * Computer Engineering Faculty

Sharif University of Technology, Tehran, Iran Sharif University of Technology, Tehran, Iranajerlou@ce.sharif.edu t IPM School of Computer Science, Tehran, Iran

manzuri@ sharif.edu

Abolfazl LakdashtiRouzbahan Institute of Higher Education

Sari, Iranlakdashti @rouzbahan.ac.ir

Abstract VLC coded streams can also cause effective erasure errorssince a single bit error can lead to many following bits

An extended version ofLagrange interpolation is devel- get undecodable, and useless. The effect of erasure errorsopedfor coding of images in real field in a way that at the (including those due to random bit errors) is much morereceiver; one can restore the lostportions ofthe images. The destructive than random bit errors due to the loss or damageproposed solution is similar to channel coding techniques, of a contiguous segment of bits.but it is done before source coding at the transmitter and One can categorize error control mechanisms devisedafter decoding of the compressed data at the receiver Our for video transport into three groups: i) those introducedproposed solution is faster than the other one reported in at the network layer, that require the network to affordthe literature when the size ofdamaged blocks is large. some type of guaranty for Quality of Service (QoS),

such as Differentiated Services (DiffServ) in InternetProtocol (IP) or Multi Protocol Label Switching (MPLS)

1. Introduction networks [14, 23]; ii) those exercised at the transportlevel, including channel coder, packetizer/multiplexer, andtransport protocol such as Forward Error Correction (FEC)Effetiv vieocmmuicaion equresto hndl erors and Automatic Repeat reQuest (ARQ) [5, 9, 10, 16, 19]; and

and losses in a communication network. In contrast to ...data communications, which are not usually subject to finally iii) those related to the application layer. The third

strict delay constraints and can therefore be handled using category can be itself divided into three classes: 1) thoseinvoked at the decoder upon detection of errors, to concealnetwork protocols that use retransmission to ensure error-. the effect of errors [1, 2, 6, 18]; 2) those introduced at the

free delivery, real-time video is delay sensitive and cannoteasily makeuseofetrans s.Te e e u

source encoder, to make the bit-stream more resilient toeasily make use of retransmission. The extensive useof predictive coding and Variable Length Coding (VLC) potential errors [3,8,11,22]; and finally 3) those that requireinvideo coding renders compressed.video, especiay,, interventions of both source encoder and decoder [4,13].

iunevideo cod trendmi ersomrs, essed video,especiall Each of these approaches has its own benefits and draw-vommulnerablento transmissin err errorsd sucessful video backs. The first category, although can lead to satisfactorycommunicaftion in thepresence ofherrors requires cavrefu results, it needs special networks other than IP or modifica-designs~~~of th enodr decde,an teytmlyr. tion in the IP and hence costs more than other approaches.

Transmission errors can be roughly classifiedl into twocateoris: rndo bi errrs nd easue erors Radom In the second category, the proposed approaches are not

bit rror arcaued y th imprfetion ofphyscal responsible for recovery of several adjacent lost packets... . . ....bi due to real time nature of the applications. Using FECchannels which results in bit inversion, bit insertion and ned bueamuto eunac ocpewt oso

deletion. Erasure errors, on the other hand, can be caused seea daetpces oee,we h mutoby packet loss in packet networks such as the Internet, reudac is sufcet hs ppoce a eoeburst errors in storage media due to physical defects, or

system failurs for a shor time. Randombit errors i the exact values of lost samples. Error concealment has,

Proceedings of the 5th International Symposium on image and Signal Processing and Analysis (2007) 293

the advantage of not employing any additional bit rate, RawDigitalbut adds computational complexity at the decoder and Video Signal |usually did not result in acceptable qualities. Error resilient Sr'lInsertion ofsystems only prevent from the propagation of the error and Serosdforalso avoid from completely compressing the source video. ,We believe that the less expensive approach that leads to Souce Coder Inverseacceptable results is the third class from the third category. STherefore, we proposed a novel technique for recovery of Transmitlarge sized lost video blocks in this paper that lies in thethird class.

There are many efforts in the literature to re- Received video streamconstruct damaged blocks of videos. For example, - some blocks may be

see [7, 12, 15, 17, 21] for recently proposed solutions. Allof these algorithms try to reconstruct blocks of size 8 x 8 Video decoder(or 16 x 16) in corrupted frames. But, how can the decoderdeal with the lost blocks of large sizes, about 64 x 64 or , ' , -larger. The problem gets more difficult when a large lost lost pix available pxelblock has a significant content that has no correlation withthe other regions of the frame. Compute

In [20] a coding technique based on the Reed-Solomon T and q)

codes is proposed to solve this problem. In this method, thetransmitter expands the size of images (or video frames) N m oewhich makes sufficient redundancy to restore a damaged valuepixlxelblock at the receiver. We used the same idea for encodingpurpose, but proposed a Lagrange interpolator based es Nodecoding solution which requires less computational time lostrpixels? cambled FFTat large sizes of damaged blocks.

Therefore, the problem we are going to solve in this nversepaper can be stated as follows: because of a congestion in Scrambled FFT e

)packet networks, some parts of images may be not received Recoveredor because of errors in wireless channels or storage devices video streamsuch as compact discs, some portion of the images or video (b)frames may be decoded erroneously; and the solution wepropose here is to add some redundancies in the images or Figure 1. Overall structure; (a) Transmitter;video frames before source coders to enable the receiver torecover the lost portions. The details are discussed in the (b) Receiver.next sections.

The remainder sections of the paper is organized as fol- unique polynomial pn (z) C Pn for whichlows: In section 2 our proposed solution is presented. Sec-tion 3 discusses the experimental results and section 4 con- Pn(i) Wi, i 0,1, , . (1)cludes the paper.

This theorem can be easily extended to 2-D signals. For2. Proposed 2-D Lagrange Interpolation De- the 2-D case, we define Pm1(x, Y) as the polynomial of

coding variables x and y of degrees m and n, respectively, whichsatisfy Pmri(XJi, yj) = i for values wij, iz 0, 1, ,. m,and j =0, 1,. , n. The form of polynomial pmm(xJ, y)

In this section, we extend the Lagrange interpolation given above is not very convenient. A better representationmethod to 2-D signals and then use it as a decoding so- oficabebtndasolw:Lt(i,j)beitntadlution for our purpose. We begin by a theorem. introduce the following polynomials of degrees m. and n:

Theorem 1 (Lagrange interpolation theorem) Givennm+ k _Hi#k(x -Xi) HjXl-(Y (2)1 distinct (real or complex) points, zo, zi. , zn and n + 1 qz-zY))H= (Y2)Yj(real or complex) values, w0, W,, *,:W , there exists a il(x-i) 7 Y-Y)

294 Proc. ISPA07

where k =,1, ..., m and I 0, 1,... , n. It is clear that manipulations we get:

(\;; J ~~~~0 ifk#zorl#j (3) WMm( )WnL1)Tnqkl(Xi7yj) = /6kidl61 =

lk j (3) wm( l)N(L1qkl~x,yj)UiUlj 1, ifkitandlIK-1iL-1E S `(k, I)wT(K+k-1)wn(L+±-1)+

For given values wij the polynomial k=o l=0K-1 L-1

z B*(kl)wTjKkiw§!Li+Pmn(X, Y) = Wklqkl(X,Y) (4) E( ) N

k= 1=0 K-1 N-11: -E(k 1)m(K±k-1) m(L±l-1)

is in Pmn and takes on these values at the points (xi, yj): >2 m N(k,l)wM WN+k=0 1=N-L+l

Pmn(Xi,yj) =wjj i= 0 , m; j 0, ,n. (5) K N (kl)w iKkl) m(LlI)WN

Since the interpolation problem (5) has a unique solu- k=1 1=N-L+ltion, all other representations of the solution must, upon re- 2K-2 2L-2

arrangement of terms, coincide with (4). f, M N(Now, Let us assume that in an image, we have a set of k=° 1=0

M x N real pixels {f (mn, m= 0,... , M - 1 and where f(B, B*) is a function of B and *. Equation (7)n = O, N - 1} that is a low-pass image in the DFT shows that modified samples WMWN(K -nm(Ll)mn are de-sense. This implies that we inserted some zeros in the DFT rived from a polynomial with respect to WM and WN of de-domain such that B(k, 1) = 0, k = K + 1,... ,M - K + 1 grees 2k-2 and2L-2. Hence (2K-i) x (2L-1) samplesORI = L + 1,... , N - L + 1 to create a low-pass sig- should be sufficient to determine(. The Lagrange interpo-nal (B is the 2-D DFT transform of (). Now, we will show lation for the exponential polynomial represented in (7) isthat this set ofM x N samples is derived from an exponen-tial polynomial of order 2K - 2 and 2L - 2. This implies m(-I) - Xthat (2K - 1) x (2L - 1) samples of the set ( would be Wm(K-1) wn(L-1)sufficient to reconstruct all the M x N samples of (. The 2K1 2L1(2K - 1) x (2L - 1) samples could be uniform or nonuni- 5 z (rk, )Wk(K-1)Ws (L-1)

kml

form. If we suppose that the original image has the size m N

of U x V pixels, we developed a (U x V,M x N) code (8)which is capable of correcting a block of matrix p x T wherep = M - U and T = N - V. This interpolation formula in which rk and sl are the coordinates of available samplewill be derived below. positions and

Since (is real, its DFT B has Hermitian symmetry, i.e., (Wn Wsi)B(k,l) = *(M - k,N -1) for k = 1,k ,MW-1 and k Hikwk _WS) HIn sl s)I=1,.. ,N-1. Since ( is a low-pass signal, we can write -Hi.ik( wM-WM ) Hi7 (WN-w )

(9)K-1 L-15 5 B(k, l)w kwmkW + For making the algorithm stable, one approach is to usek=O 1=0 the scrambled kernel of the DFT as WN = exp(-j2wq/N)K-1 L-1 instead of WN = exp(-j2w/N), where q is a natural num-5 5 B(M - k, N -)W (M-k)Wn(N-1)+ ber prime with respect to N. All tasks of our techniquek=1 1=l can be done with this kernel and transformation. The trans-K-1 N-1 formation is equivalent to a permutation of DFT as shownN7 N -E (k, AI)MWNT+ below:k=0 I=N-L+1±i [qk]N (10)

N N - -kmlII) n(N-1) where [I]N denotes modulo operator with respect to N and> B(M - k, -l)w7jM WN k and ks denote the indices of DFT and scrambled DFTk=i I=N-L±i frequency bins, respectively. Note that the Hermitian sym-

(6) metry is still preserved after scrambling the DFT points in

where WN =exp(j2w/N) is the kernel of the DFT. Multi- thssne icplying both sides of (6) by wlj,(Kil)wjijL l) and after some X([qk]N) =X~([-qk]f). (1 1)

Proc. ISPA07 295

(a) (b) (c) (d)

(e) (f) (g) (h)

Figure 2. Experimental results: (a) Original image; (b) the corrupted image corresponding to theblock loss in (f); (c) the recovered image using the proposed Lagrange interpolation based methodin which the padded zeros are omitted; (d) the recovered image using method of [20] in which thepadded zeros are omitted; (e) the image after the padded zeros in the SDFT domain; (f) the blockloss of the image shown in (e); (g) the recovered image containing the padded zeros in the SDFTdomain using the proposed method; (h) the recovered image containing the padded zeros in theSDFT domain using method of [20].

Another important point to be noted here is that the dc losses into isolated losses by proper choices of qi and q2.value in our correlating operation using the scrambled DFT Therefore, to determine the coefficients qi and q2, twois not made zero, since X([q x O]N) = X(0) and the DFT points should be considered. Firstly, qi and q2 have tofrequency bins made zero are lie in the range ks C Li, be prime with respect to M and N, respectively. Sec-which the dc point is outside this range. ondly, based on the size of the block losses, the qi and

To determine suitable values for qi and q2 and their ef- q2 values must be chosen such that the locations of the ze-fects on the behaviors of the algorithm, at first we choose ros of T4km-.lj are approximately distributed symmetricallyq, = q2 = 1. For this choice, the SDFT is identical to DFT. around the unit sphere. For example, for an even numberIn many problems, the number of pixels are large, so that of M and N, when the block size of losses is equal tobursty losses produce large dynamic range in TS"kmDlm The (M/2, N/2), the best choices for qi and q2 are M/2 - 1large dynamic ranges are due to the concentration of zeros and N/2 - 1, respectively.on one side of the semi-sphere. T4km'.l, values are zero This completes our solution. A block diagram for thefor si, rj = 1,.. , 32 and very large around si, r = 48. overall structure of the proposed solution is shown in Fig-This can be verified by calculating the value of the T4km'.l, ure 1. In Figure 1-(a) the required modules for the trans-which is equal to the product of all the 32 x 32 vectors em- mitter side is depicted in which a certain number of zerosanating from the position si, 48 on the upper half of is inserted in the SDFT domain in the filtering module andthe unit sphere. after source coding it is transmitted to the receiver. At the

For the case of bursty losses, 4'kmbJlm has a large dy- receiver, first the received signal is decoded and then basednamic range that creates a large computational error in (8). on the proposed algorithm, its damaged slices is recovered.Therefore, the implementation of the algorithm for a largeblock size of losses is impossible. Since the coefficients4'kmbJlm have a very small dynamic range in the case of iso-lated losses, it would be beneficial to transform the bursty

296 Proc. ISPAO7

3. Experimental Results10O2 j Method of [20]

The experimental results are shown in Figure 2. The .. ..

original standard 256 x 256 image of barbara is shown in 1o..01Figure 2(a). This image is padded with enough zeros tocreate an scaled image of 512 x 512 which is shown in Fig- N =ure 2(e). Then a block of size 256 x 256 in this image .-,is made lost (Figure 2(f)) which a corrupted image corre- N=25

sponding to it iS shown in Figure 2(b). Uslng our proposedLagrange interpolation based method, this corrupted part is 7

recovered and the resultant image is shown in Figure 2(g) 2 N =-where after removing the padded zeros from the SDFT do- Xmain, the image of Figure 2(c) is obtained. Figures 2(d) N

and 2(h) are the similar images resulted from the method3 :..................of [20]. As can be seen, there does not exists any difference 10° 101 102 103between the final results in both methods, but only the re- L (Size of lostblock)

quired time is different.Figure 3. Required time for various values

For effective computation of (8) we can separate it as of image sizes versus the size of the lostfollows: block for running the proposed solution and

1 method of [20]; Our method:-; method; m(K-1) n(L-1) of [20]:

2K-1 2L-1

rW (K-1)km 5 X(rk, Sl)WV(Ll )lnk=O 1=o the literature which our experimental results show that our

(12) proposed solution is faster than it when the size of lost blockis large. Specially, in some cases, the proposed method can

If we first compute 4Ekm andnnt h for all required values of be used in real-time applications, while the previous onek, 1, m and n, and store them in memory, then the number cannot.of multiplications required for recovery of a block of L x Lin an image ofN x N will be 5. Acknowledgment

16L(N - L)2 + (N - L + 1)(3N - 3L + 7). (13)The IPM school of computer science financial support

Figure 3 shows the required time for running each method under grant No. CS 1385-4-03 is acknowledged.versus L for some typical values of N on a processor withthe frequency of 400 MHz, if each multiplication of two 32 Referencesbit numbers requires 32 cycles. As can be seen, our pro-posed algorithm requires much smaller time for recovery in [1] C. Adsumilli, M. Farias, S. Mitra, and M. Carli. A robustany image sizes when the lost block is a large one. For ex- error concealment technique using data hiding for image andample, for recovery of a 65 x 65 block in a 256 x 256 image, video transmission over lossy channels. IEEE Transactionsthe overall time will be about 0.3 seconds, whereas for the on Circuits and Systemsfor Video Technology, 15 (11):1394method of [20] will be about 14 seconds. Therefore, in most - 1406, November 2005.real-time applications, our proposed solution is applicable, [2] I. Bajic. Adaptive MAP error concealment for dispersivelywhile the method of [20] is not useful. packetized wavelet-coded images. IEEE Transactions on

Image Processing, 15 (5):1226 - 1235, May 2006.[3] H.-T. Chan, C.-M. Fu, and C.-L. Huang. A new error re-

4. Conclusion silient video coding using matching pursuit and multiple de-scription coding. IEEE Transactions on Circuits and Sys-

Inti*ae,w rpsda oe arneitroa tems for Video Technology, 15 (8):1047 - 1052, AugustIn ths paer, w proosed novl Lagangelnterola-2005.tion based error correction coding for the recovery of a lost [4] K. Chawea, Z. Jiang, X. Qiu, and A. Reibman. Trans-block in an image. In the proposed method, we add some mission of streaming video over an EGPRS wireless net-redundancy in an image which enlarges it. Using this redun- work. In IEEE International Conference on Multimedia anddancy, we can reconstruct a lost block. The proposed solu- Expo (ICME 2000), volume 1, pages 275 -278, July-Augusttion is compared with another similar method reported in 2000.

Proc. ISPAO7 297

[5] M. Chen and G. Wei. Multi-stages hybrid ARQ with condi- on Communications (ICC 2005), volume 2, pages 1190 -tional frame skipping and reference frame selecting scheme 1194, May 2005.for real-time video transport over wireless LAN. IEEE [20] S. Tayeb and F. Marvan. A novel method for burst errorTransactions on Consumer Electronics, 50 (1): 158 - 167, recovery of images. In SampTA 99, Loven, Norway, AugustFebruary 2004. 1999.

[6] M.-J. Chen, C.-S. Chen, and M.-C. Chi. Temporal error con- [21] D. Zhang and Z. Wang. Image information restoration basedcealment algorithm by recursive block-matching principle. on long-range correlation. IEEE Transactions on CircuitsIEEE Transactions on Circuits and Systems for Video Tech- and Systems for Video Technology, 12(5):331 - 341, Maynology, 15 (11):1385 - 1393, November 2005. 2002.

[7] F. De Natale, C. Perra, and G. Vernazza. Dct informa- [22] J. Zheng and L.-P. Chau. Error-resilient coding of h.264tion recovery of erroneous image blocks by a neural pre- based on periodic macroblock. IEEE Transactions ondictor. IEEE Journal on Selected Areas in Communications, Broadcasting, 52 (2):223 - 229, June 2006.18(6):1111 - 1121, June 2000. [23] X. Zhu, M. Chen, and Y Yu. Video transmission based on

[8] T. Fang and L.-P. Chau. An error-resilient GOP structure for diffserv over IP networks. In International Conference onrobust video transmission. IEEE Transactions on Multime- Communication Technology Proceedings (ICCT 2003), vol-dia, 7 (6):1131 - 1138, December 2006. ume 2, pages 1754 - 1757, April 2003.

[9] M. Freytes. New ARQ protocol for interactive video overdata networks. In Proceedings of the 2003 Joint Confer-ence of the Fourth International Conference on Information,Communications and Signal Processing, volume 3, pages1967 - 1971, December 2003.

[10] Z. Guo, Y Nishikawa, R. Omaki, T. Onoye, and I. Shi-rakawa. A low-complexity FEC assignment scheme for mo-tion JPEG2000 over wireless network. IEEE Transactionson Consumer Electronics, 52 (1):81 - 86, February 2006.

[11] A. Jerbi, J. Wang, and S. Shirani. Error-resilient region-of-interest video coding. IEEE Transactions on Circuits andSystems for Video Technology, 15 (9):1175 - 1181, Septem-ber 2005.

[12] Z. Kachouh and M. Bellanger. Efficient restoration tech-nique for missing blocks in images. IEEE Transactions onCircuits and Systems for Video Technology, 13(12):1182 -1186, December 2003.

[13] C. Kodikara, S. Worrall, A. Kondoz, and S. Fabri. Linkadaptation for streaming video in EGPRS mobile networks.In IEEE 59th Vehicular Technology Conference (VTC 2004-Spring), volume 5, pages 2763 - 2767, May 2004.

[14] G.-S. Kuo and C. Lai. A new architecture for transmission ofMPEG-4 video on MPLS networks. IEEE CommunicationsMagazine, 40(12): 114-119, December 2002.

[15] W. Li, D. Zhang, Z. Liu, and X. Qiao. Fast block-based im-age restoration employing the improved best neighborhoodmatching approach. IEEE Transactions on Systems, Manand Cybernetics, Part A, 35(4):546 - 555, July 2005.

[16] Y. Ohtani, N. Kawahara, T. Tomaru, K. Maruyama,K. Onoye, I. Shirakawa, and T. Chiba. Error correction blockbased ARQ protocol for wireless digital video transmission.In IEEE International Symposium on Circuits and Systems(ISCAS 2002), volume 1, pages 1-605 - 1-608, May 2002.

[17] J. Park, D.-C. Park, R. Marks, and M. El-Sharkawi. Recov-ery of image blocks using the method of alternating projec-tions. IEEE Transactions on Image Processing, 14(4):461 -474, April 2005.

[18] G. Schuster and A. Katsaggelos. Motion compensated shapeerror concealment. IEEE Transactions on Image Processing,15 (2):501 - 510, February 2006.

[19] H. Seferoglu, Y. Altunbasak, 0. Gurbuz, and 0. Ercetin.Rate distortion optimized joint ARQ-FEC scheme for real-time wireless multimedia. In IEEE International Conference

298 Proc. ISPAO7