EC 515 LinearBlockCodes

7/29/2019 EC 515 LinearBlockCodes

1/29

Linear Block Codes

EC 515: Coding Theory


2/29

Introduction to block codes

Message bit-stream split into blocks of k-bits; eachblock considered as a message.

Therefore, every message is a k-tuple mi total 2k

messages possible.

k-bit message block mi mapped to an n-blockcodewordci by adding nk redundant bits.

All codewords are distinct: one-to-one mapping.

One form of block codelinear block code simpleencoding.


3/29

Linear block code

Definition: A block code of length n and 2kcodewords is called a linear (n, k) code if and only ifits 2k codewords form a k-dimensional subspace ofthe vector space of all the n-tuples over the field

GF(2). Every codeword ci is a vector in the vector space

Vn of all the binary n-tuples (and so is also calledcodevector).

Linearity: a binary code is linear if and only ifmod-2 sum of two codewords is also a codeword.

The above also implies closure property.


4/29

Linear block code

Since an (n, k) linear code C is a k-dimensional

subspace of the vector space Vn of all the binary n-

tuples, we can always find a set of k linearly

independent codewords such that every codeword is

a linear combination of these codewords.

Gm

g

g

g

gggc

i

k

kiii

kkiiii

mmm

mmm

1

1

0

1,1,0,

11,11,00, ............


5/29

Generator matrix

G is the k n generator matrix for C the rows ofGgenerate all the codevectors and hence span thesubspace C.

G may not be unique any k linearly independent

vectors may be used to form C.

A linear code is completely specified by the k rows ofG encoder needs to store only these rowscodeword generated by linear combination of these

rows based on the input message.

Check that ci + cj = (mi + mi)G = mkG = ck; whichagain is the closure property


6/29

Parity check matrix

For the k-dimensional subspace C containing all thecodewords, we can find its (nk)-dimensional dualspace Cd.

Cd is called the dual code for the linear code C.

All 2nk 1 number of vectors in Cd are orthogonalto each of the codewords in C and vice-versa;vector0 is common to both C and Cd.

We can now find a set of (nk) linearly independentvectors h0, h1, .., hnk1 that span Cd.

We now define a matrix H of size (nk) n whoserows are these vectors.


7/29

Parity check matrix

It follows that G.HT = [0] and ci.HT = 0; [0] is an all-

zero matrix of size k (nk) and vector0 is an (nk)-

tuple.

Any received vectorv conforming to v.HT

= 0 is acodeword, if not then v is not a codeword implying

transmission error.

So, H may be used for detecting transmission error.

Accordingly, H is called parity check matrix.

H forms the generator matrix for the vectors in the

dual code Cd.


8/29

Syndrome

Let el be the error vector; elements in el are 0 at

positions where there is no bit-flipping and are 1 at

positions where there is bit-flipping due to

transmission error.

This (nk)-tuple sl is called the syndrome.

This syndrome is used forerror detection. Ifsyndrome is non-zero then there is error.

lT

lT

liT

li sHeHecHvecv ...


9/29

Syndrome

If syndrome is zero then el is either equal to 0 (thereis no error) or another codeword; ifel equals toanother non-zero codeword, error will beundetectable.

Error correction possible only when el can bedetermined.

There are 2n number of possible error vectorsincluding 0 and all other codewords.

But, there are only 2nk possible syndromes.

It follows that 2k number of error vectors yield thesame syndrome syndrome-based error-correction may not be always correct.


10/29

Minimum distance of block code

Hamming distance is a metric function that satisfies

the triangle inequality: d(u, v) + d(v, x) d(u, x)

d(u, v) = w(u + v); w(.) denotes Hamming weight.

Theorem: The minimum distance of a linear blockcode is equal to the minimum weight of its nonzero

codewords and vice versa.

Theorem: For each codeword of Hamming weight l,

there exist l columns of the parity-check matrix H

such that the vector sum of these l columns is equal

to zero vector.


11/29

Minimum distance of block code

If there exist l columns ofH that add to 0, then there

exist a codeword of weight l.

If no d 1 or fewer columns ofH add to 0, the code

has min. weight at least d. The min. distance (or min. weight) of a code is equal

to the smallest number of columns ofH that add to 0.


12/29

Error-detecting capability

Error is detected if received vectorris not a codeword.

Random error-detecting capability is dmin 1; i.e. error

detection is always guaranteed if the weight t of the

error-pattern is

Error-patterns of weight dmin or more are not always

detectable.

Detectable error patterns = all 2n 2k vectors that

are not codewords.

1min dt


13/29

Error-detecting capability

If p is the probability of bit-flipping duringtransmission over a BSC (transition prob.), then theprobability that a particular error-pattern of weight ioccurs is pi(1p)ni.

Undetectable error patterns = all 2k 1 non-zerocodewords.

Therefore, total prob. of error being undetected is

where Ni is the number of codewords of weight i.

n

di

inii

n

i

iniiu ppNppNPmin

111


14/29

Error-correcting capability

If p < 0.5, then pi(1p)ni > pj(1p)nj for i < j.

Since p is generally very small in a BSC, it follows

that error-patterns of less weight is more probable.

Accordingly, for a received vectorr, which is not acodeword, the most probable transmitted codeword is

the one that is nearest to r.

Max. likelihood decoding: error-correction based on

the above, i.e. nearest-codeword mapping.


15/29


Random error-correcting capability is

i.e. error correction is guaranteed if the weight of the

error-pattern is t or less.

If the weight of an error-pattern is more than t, then

error correction still possible if the codeword nearestto the received vectorris still the transmitted

codeword.

2

1mindt


16/29


No. ofcorrectable error-patterns =

Prob. that the decoder commits an erroneous

decoding is upper bounded by

Perfect code: Only error-patterns of weight t or less

are correctable. Examples Hamming code, Golaycode.

inin

ti

e ppi

nP

1

1

max,

t

i

kn

i

n

0

2

t

i

kn

i

n

0

2


17/29

Standard array

Array of 2nk rows and 2k columns.

First row contains all codewords.

Each subsequent row contains all codewords + one

particular error-pattern, not contained in any rowabove.

Check

Every row is a coset of the code C.

All n-tuples in a row, say ith row, form a set of error-patterns that yield the same syndrome si.

A codeword corrupted by any of the error-patternsin a row forms the received vector also contained in

the same row.


18/29

Standard array

knkknknkn

k

k

k

k

j

iijii

j

j

j

222222

22

323323

222222

221

ececece

ececece

ececece

ececece

ccc0c


19/29

Standard array

It follows, all the 2n entries in the array are all the

possible error-patterns as well as the received vectors.

In a particular row, say ith row,

Find the n-tuple having the min. weight. This istaken as ei.

Other entries in the row are recalculated.

Check that the entries in the row do not change butsimply get shuffled.

This ei is the coset-leaderwhich is the most probable

error-pattern corresponding to the syndrome si.


20/29

Standard array

So, for the received vectorris in the ith row and jthcolumn of the array

The most probable error-pattern (error with min.weight) is ei which is the coset-leader or the left-

most entry in that coset (row).

The most probable transmitted codeword is cjwhich is the top-most entry in that column.

Prob. of decoding error:

where Ki is the no. of coset-leaders of weight i.

n

i

iniie ppKP

0

11


21/29

Standard array

Note:

The first row of the array contains all the

undetectable error-patterns (including 0).

Other rows contains all the detectable error-patterns.

The first column contains all the correctable error-

patterns (including 0).


22/29

Syndrome decoding procedure

For a received vectorr, calculate the syndrome rHT

= si (say)

This indicates that the received vector lies in the ith

row of the standard array. Hence, the corresponding error-vector is

determined as ei.

Obtain the decoded codeword as r+ ei.


23/29

Theorems

No two n-tuples in the same row of a standard array

are identical. Every n-tuple appears in one and only

one row.

Every (n, k) linear block code is capable of correcting2nk error-patterns.

All the 2k n-tuples of a coset have the same

syndrome. The syndrome of different cosets are

different.


24/29

Theorems

For an (n, k) linear code C with minimum dist. dmin, all

the n-tuples of weight t or less can be used as coset-

leaders of a standard array of C, where

If all the n-tuples of weight t or less are used as

coset-leaders, there is at least one n-tuple of weight

(t + 1) that cannot be used as a coset-leader.

This implies that the error-correcting capability is

exactly t, no more no less.

2

1mindt


25/29

Systematic code

Systematic block codes: a desired code structure

as follows

The nk redundant bits on the left are calledparity-check bits formed by linear sum of the

message bits.

The k (nk) coefficient matrix P concatenated

with k k identity matrix forms G, i.e.

kiiii IPmmbc

kIPG


26/29

Systematic code

The parity-check matrix, therefore, is of the form:

Check that G.HT = [0]

Examples of simple systematic block-codes:

Parity-check code:

n = k+1

dmin = 2; so can reliably

detect upto 1-bit error;

no correction possible.

Tkn PIH

1111

1001

0101

0011

H

G


27/29

Systematic code

Repetitive code: k = 1; two possible 1-bitmessages 0 or 1.

Two possible codewords (0 0 . 0) and (1 1 . 1)obtained by repeating message bit n times (n is

generally taken to be odd).

dmin = n

Error-correction is

simply to decide infavor of majority no.

of 0 or 1 in the

received vector.

1100

1010

1001

1111

H

G


28/29

Hamming code

For any pos. integer m

We desire error-correction for at least 1-bit error:

All columns in H must be non-zero

All columns in H must be distinct.

Each column in H is (nk)-tuple; so 2nk 1 numberof non-zero combinations possible.

Check that number of columns in H is n = 2nk 1.

mknmkn mm ,12,12


29/29

Hamming code

That means, all possible combinations are to be usedin forming different columns ofH.

This means, two columns ofH add up to form a thirdcolumn ofH there exist 3 columns ofH that add to

0

On the other hand, since all columns are distinct, notwo columns ofH can add to 0 dmin = 3.

Error-correcting capability t = 1.

No. of error-patterns having weight t (excluding 0)is n = 2nk 1 So no error-pattern having weightmore than t can be correctedperfect code.

Documents

EC 515 LinearBlockCodes