Paradigm Shift in Turbo Processingfrom P2P to Network –‐ from P2P to Network –

Slepian Wolf and CEO Problem Viewpoints

Tad Matsumoto* ** and Khoirul Anwar*

Slepian Wolf, and CEO Problem Viewpoints

Tad Matsumoto*, ** and Khoirul Anwar*Information Theory and Signal Processing Lab.

* Japan Advanced Institute of Japan Advanced Institute of Science and Technology (JAIST), Japan

** CWC, University of Oulu, Finland, y ,

April 19, 2013

* This Material to be published in IEEE Trans. on Signal Processing

Preliminaries (1)

Japan experienced huge disaster in March 2011, and still a lot of people are in refugee’s shelter/and still a lot of people are in refugee s shelter/temporary houses.

- Lost lives: 15,854, data effective as of March 8, 201218,183, Sept 30, 2012

- Unfound: 3,203 2,700- Injured: 26,992 6,114

Preliminaries (2)7000 5007000

6000

500

400Tohoku Electric Power Supply

April 7 the next earthquake (M6 0)

5000

300

NTT

ations

Dead of Ele

April 7, the next earthquake (M6.0)Dead Base Stations:NTT DoCoMo: 1200 BTSKDDI (au) : 500 BTSSoftbank : 2200 BTS

4000

3000200

SoftBank Mobile

KDDI (au)ead Ba

se Staectric Pow

er

EMOBILE: 200 BTS

2000

100

KDDI (au)

EMOBILE

De r Supply

1000

0

100

011 12 13 14 15 16

March 201111 12 13 14 15 16 …  1 2 3 4 5 6 7 8 9…  25 26 28     2    6

April 2011 May 2011

Our Networks are Fragile!Our Networks are Fragile! What can we do?

S

Q

S

A

B

P Q

RZ

X

CD

E

Z

T

Lossy Link

E

F

U

VY

Devastated Area Ordinary Areay

Lossless Link

- The key is “Accept Distortion less than Specified” CEO Problem- Very high energy and spectrum efficiencies required.Very high energy and spectrum efficiencies required.-Seek for beyond the P2P Shannon Limit. Achieve the limit of network

as a whole!

Bounds

C1X

Correlated Joint Dec.

X~RX

~,Y

C2Y RY

Berger Tung’s Bounds

Starting Point: System Model+

H11

C1 Π1 D1Π1−1

Π1

Hb1

+‐

1 Π1 D1Π1

IMO

C-M

MSE

H21

H12

1

C2 Π2

MI

FD/S

C

D2Π2−1

H22b2 2 2 D2Π2

Π2 +‐

2

Block at m‐th transmit antenna: 

Bl k f ll iBlock from all transmit antennas: 

FD/SC MMSE: Frequency Domain Soft Cancellation Minimum Mean Square ErrorFD/SC‐MMSE: Frequency Domain Soft Cancellation Minimum Mean Square Error

6

Multiuser MIMO SC/MMSE/

H11U 1

+Π 10

0

H11

H21C1 Π

1

User 1:

d1 C1-1Π1

−1

MSE

1 ‐

1d̂

10‐1

H22

H12

C2 Π

User 2:

d

MIM

OFD

/SC‐MM

1 d̂10

‐2

ERC2 Π2

d2

C2-1Π2

−1

Π2

+‐

2d

10‐3Av

erage BE

MIMO SC/MMSE jointly decode information from each user

The decoding is performed separately for  10‐4 ?

If sources are correlated

each user.Encoder: NSNRCC 4(17,15) , FFT=512, 2x2 

MIMO, Rayleigh 64‐path, Decoder: BCJR Log‐8 6 4 2 0 2 4 6 8

10‐5

7

MAP‐8 ‐6 ‐4 ‐2 0 2 4 6 8

Per‐antenna receive SNR (dB)

T. Matsumoto and K. Anwar ‐MIMO Spatial Turbo Coding with Iterative Equalization

Idea: Vertical Iteration (STC)+

H11Π1

b ˆ

+

‐ ‐

xC1 Π1 D1

Π1−1

SE

H21b b

+‐

x

Π0Π0

MIM

OFD

/SC

-MM

S

Π0−1

H12

C2 Π2

F

D2Π2−1

H22 +

‐

yd

Π2 +‐ ‐

Vertical iteration is expected improve the performance because of space diversity utilization from antenna 1 and 2 and coding gain [Mariela et al., ‘06].

This design is called as Spatial Turbo Code (STC) since the output of second decoder

8

This design is called as Spatial Turbo Code (STC) since the output of second decoder is not multiplexed but transmitted in parallel over the space. 

STC vs. Turbo Code  

C ΠMUX

Spatial Turbo Code +

C -1Π1−1

Π1‐

Π0

C1 Π1MUX

Π0

C11Π1

MIM

OSC

-MM

SE

Π0−1

+

‐

C2 Π2MUX

FD/

C2-1

Π2−1

+

‐

T b C d

+Π2

‐

C1

C1-1

+DEMUX

‐

Turbo Code

Π0

C1

MUX Π0Π0

−1‐

Π0

9

C2C2

-1+

‐

STC vs. Turbo Code  

C ΠMUX

Spatial Turbo Code +

C -1Π1−1

Π1‐

Π0

C1 Π1MUX

Π0

C11Π1

MIM

OSC

-MM

SE

Π0−1

+

‐H=UDVH

C2 Π2MUX

FD/

C2-1

Π2−1

+

‐

T b C d

+Π2

‐

C1

C1-1

+DEMUX

‐

Turbo Code

Π0

C1

MUX Π0Π0

−1‐

Π0

10

C2C2

-1+

‐

Contibution of Vertical Iterations

0.9

1

Vertical Iteration converts CC Turbo0 7

0.8

converts CC  TurboStuck point is shifted to the right side.

Convolutional code0.6

0.7

+

C1-1Π1

−1

Π1‐

0.4

0.5

Π0

1

MIM

OD

/SC

-MM

SE

Π0−1

+

‐Turbo code

0.2

0.3

200 channel Realizations

FD

C2-1

Π2−1

+

‐

0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 10

0.1 FFT: 512, SNR=‐3.5dB, Channel: Rayleigh 64‐path, Encoder: SRCC 4(17,15)17

11

+Π2

‐0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Is BER 0 Always Guaranteed?Fact: Conditioning Reduces Entropy

)|()( YXHXH ≥Turbo Feedback works  as Conditioning

L),,|(),|()|()( 211 LLYXHLYXHYXHXH ≥≥≥Mutual Information between Transmitted Coded Bit and LLR:

)|()();( LXHXHLXI −=N i i i d t LNow, giving index to L:

)|()();( 11 LXHXHLXI −=

),;(),|()()|()();( 212122 LLXILLXHXHLXHXHLXI =−≤−= ),;(),|()()|()();( 212122 LLXILLXHXHLXHXHLXI ≤

),,;(),,|()()|()();( NNNN LLXILLXHXHLXHXHLXI LL 11 =−≤−=M

This means that:1),;(),;();( 21211 →≤≤≤ NLLLXILLXILXI LL

if0),,|( 1 →NLLXH L

BER Performance of STC

Parameters:100

500 channel realizations

Transmitter:Encoder: SRCC 4(17 15) 17

10-1

4(17,15),17Interleaver=1024 (random)10-2

BER

Channel:MIMO 2x2Equal Power 64-10-3Av

erag

e B

w/o verticaliteration

w/ verticaliteration

2x2 Equal Power 64path

Receiver:10-4

iteration2 MIM

O AW

GN 5 Gain: 6dB

Decoder: BCJR Log-MAPFFT=512

10-5

N Capacity/dim

(H1V5)

13

-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6 710

Average Per-Antenna SNR [dB]

m

How to Model Correlated Sources?

peP == )1(Bit flipping e

Π1+

peP −== 1)0(

C1 Π1 C1‐1Π1−1

Π1

XX

‐

R1 τ

Π0

1

MO

‐MMSE

Π0−1

‐

d

fc

MI

FD/SC ‐

C 1

e

‐

mod

fc

C2 C2‐1Π2−1

Π2

Π2Π0

‐+

R2eXY ⊕=

Probability Update (fc)

Joint Decoding+

)0()1()1()1P(x)1()0()1(0)P(x

=+=−===+=−==

xpPxPpxpPxPp

oo

oo

How to estimate p at the receiver?

Probability Update (fc)

K1

14

{ }∑=

==+===K

kkokokoko yPxPyPxP

Kp

110011 )()()()(ˆ

L

L

update peppeplnLLR

+−+−

=)()(

11

Slepian‐Wolf TheorempD

R

C1X RX

SC2Y

Correlated Joint Dec.

X~

R

Relay

RY

C2Y RY

Slepian‐Wolf Theorem:

H(Y)RX > H(X|Y)

( | )H(Y) RY > H(Y|X)RX+RY > H(X,Y)

RX

H(Y|X) RX+RY=H(X,Y)

15

XH(X|Y) H(X)

Effect of Weak Correlation

Π +0 9

1SNR= -3.5dB

C1‐1Π1

−1

Π1

X

‐

0.8

0.9 Interleaver=1024SRCC-4 (17,15)Iterations=5x

Π0

MIM

OSC

‐MMSE

Π0−1

‐fc

0.6

0.7

MFD

/S

C2‐1Π2−1

‐fc

0.4

0.5

2

Π2Joint Decoding

‐+

0.2

0.3

0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 10

0.1

L

L

update peppeplnLLR

+−+−

=)()(

11

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

16

pep +)(1

STC‐SW: BER PerformanceParameters:10

0

200 channels realizations

Transmitter:Encoder: CC 4(17,15),17Interleaver 1024

10‐1

Interleaver=1024 (random)Correlation Model: Bit-flipping

10‐2

BER

pp g

Channel:MIMO 2x2Equal Power 64

10‐3Av

erage B

M Equal Power 64-path

Receiver:10‐4

MIM

O 2x2 C

Decoder: BCJR Log-MAPFFT=512

10‐5

p=0

Capacity

17

‐6 ‐4 ‐2 0 2 4 6 810

SNR (dB)

SW with Doped Accumulatorp

H Π1+L 0

a1;EQL a1;EQH11

C1 Π1 D1Π1

−1

‐

ΜP DA-1

0bê1

x x0 x00

τs1 1

Q +

M-1H12 +

‐Correlation

D-ACC

ú(p)0

L 0e1;EQ

s1 L e1;EQ

Π0

ultiu

ser

EH21e

Π0

Mu

D2Π2−1

H22 +‐

C2 Π2 P

D ACC

DA-1

L 0e2;EQy00y y0

Μs2

L e2;EQ

Π2 +‐

D-ACC

L 0a2;EQ

L a2;EQ

1818T. Matsumoto and K. Anwar ‐MIMO Spatial Turbo Coding with Iterative Equalization

Partial/Doped Accumulatorp

U

CU

UUUCUUUC UUU_UUU_

U: UncodedC: Coded _ _ _ C _ _ _ CPA PA-1

(a) (c)

UUUUUUUU UUU0UUU0U U U U U U U U

(a)

_ _ _ 0 _ _ _ 0PA0U U U C U U U C

(b) (d)

19

19

Problem of p and its Solution (SRCC) 

0 9

1SNR= -3.5dBI t l 1024

Parameters:

0 7

0.8

0.9 Interleaver=1024SRCC-4 (17,15)Iterations=5x with PAcc, P=16

Transmitter:Encoder: CC 4(17,15),17

0.6

0.7( )

Interleaver=5000 (random)Correlation Model: Bit flipping

0.4

0.5 Bit-flipping

Channel:MIMO 2x2

0.2

0.3 Equal Power 64-path

Receiver:

0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 10

0.1Receiver:

Decoder: BCJR Log-MAPFFT=5120 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

20

STC‐SW: with PAcc

100

200 channels realizationsParameters:

10‐1

200 channels realizations

Transmitter:Encoder: CC 4(17,15),17

10‐2

ER

Interleaver=1024 (random)Correlation Model: Bit flipping

10‐3

Average BE Bit‐flipping

Channel:MIMO 2x2

10‐4

10A p=0.01

Equal Power 64‐path

Receiver:Decoder: BCJR Log‐

10‐5

10 Decoder: BCJR Log‐MAPFFT=512

21

‐6 ‐4 ‐2 0 2 4 6 810

SNR (dB)T. Matsumoto and K. Anwar ‐MIMO Spatial Turbo Coding with Iterative Equalization

SpCC‐SW: Freq. Selective Fading MIMO 2x2p q g010

Parameters:

10-1

p = 0.49T = 3

Transmitter:Encoder: CC 4(17,15)Interleaver=10000 

10-2

BER

T 3

p = 0.30T = 3

p = 0.10T = 3

(random)Correlation Model: Bit‐flipping

10-3

Aver

age

B 3p = 0.01T = 6 Channel:

MIMO 2x2Equal Power 64‐path

10-4

Interleaver: 5120

known punknown p

p = 0.00T = 6

64-path FadingReceiver:

Decoder: BCJR Log‐MAP

10-5

Interleaver: 5120NSNRCC-4 (17,15)Iteration: 50(H1V5)

MAPFFT=512

-5 -4 -3 -2 -1 010

Average SNR (dB)22

Approaching Shannon/Slepian‐Wolf Limit

23