Image and Video Coding - UPM

Institut fürNachrichtentechnik

Image and Video CodingDr.-Ing. Henryk Richter

Institute of Communications EngineeringPhone: +49 381 498 7303, Room: W 8220

EMail: [email protected]

mailto:[email protected]

mailto:[email protected]

MAD 2015 © 2009 UNIVERSITÄT ROSTOCK | FAKULTÄT INFORMATIK UND ELEKTROTECHNIK H. Richter - Image/Video Processing and Coding


Literature / Referencesl Gonzalez, R.; Woods, E. : „Digital Image Processing“, Prentice Hall 2008l Jähne, B.: Digitale Bildverarbeitung, 5. Auflage, Springer-Verlag, 2002l Tönnies, K. D.: Grundlagen der Bildverarbeitung, Pearson Studium, 2005l Ohm, J.-R.: Digitale Bildcodierung, Springer-Verlag, 1995l Rao K.R.: Techniques & Standards for Image, Video & Audio Coding, Prentice Hall

1996l Ohm, J.-R.: Multimedia Communications Technology, Springer-Verlag 2004l Wang, Y. et al.: Video Processing and Communications, Prentice Hall 2002l Rao K.R. et al.: The transform and data compression handbook, CRC Press 2001l Watkinson: MPEG-2, Focal Press 1999l Pennebaker, W.B. et al.: JPEG still image compression standard, NY 1993l Mitchell J. L. et al.: MPEG Video Compression Standard. Chapman and Hall 1997l Taubman, D.S. et al.: JPEG2000, Kluwer Academics Publishers, 2002l Richardson I.: H.264 and MPEG-4 Video Compression, Wiley & Sons 2003l Gersho A. and Gray R. M.: Vector Quantization and Signal Compression, Kluwer

Academics Publishers 1992

2



Significant Conferences, Journalsl Conferences

l IEEE International Conference on Image Processing (ICIP)l Picture Coding Symposium (PCS)l International Conference on Visual Communications and Image Processing (VCIP)

l Journalsl IEEE Transactions on Circuits and Systems for Video Technologyl IEEE Signal Processing Magazinel IEEE Communications Magazinel IEEE Transactions on Image Processing

3

source source encoding

channel coding (error protection) modulation

Sender

sink source decoding error correction demodulation

Receiver

channel(transmission/storage)

Noise, Fading, InterferenceErrors

MAD 2015


© 2009 UNIVERSITÄT ROSTOCK | FAKULTÄT INFORMATIK UND ELEKTROTECHNIK H. Richter - Image/Video Processing and Coding

General Transmission Model

4

source coding ⇔ channel coding



Scope

5

l Separation of source coding and channel coding l allows independent adaptation to

- properties of information source and sink- properties of transmission channel

l direct reusability of source coding output and transmission channel for other meansl Drawbacks

- joint source/channel coding enables balanced optimization between coding and error protection (graceful degradation)

- unexploited redundancy by separate coding

l Scope of image compression part of lecture:

data compression = source coding

PAL 720x576@25 Hz

HDTV 720p 1280x720@60 Hz

HDTV 1080p 1920x1080@ 30 Hz

CIF 352x288

QCIF176x144

NTSC 720x480@30 HzDVD/DVB (4:3 format)

ITU-R BT.709 (16:9 format)

UHDTV 2160p 3840x2160@120 Hz (~4k)

MAD 2015


© 2009 UNIVERSITÄT ROSTOCK | FAKULTÄT INFORMATIK UND ELEKTROTECHNIK H. Richter - Image/Video Processing and Coding 6

CIF = common interchange format (5-30 Hz)QCIF = quarter common interchange format (10-30 Hz)

MAD 2015



CD 700 MB DVD 8.5 GB Blu-ray 50 GB

22 s ⇔ 4.5 s 4.5 min ⇔ 54 s 26 min ⇔ 5 min 35 h ⇔ 7.1 h

Capacity

Storage time

ADSL2+ 16 MBit/s

11 d ⇔ 55 d 1 d ⇔ 4.8 d 0.37 h ⇔ 1.8 h

Bandwidth

Transmission 90 min film

HDD 4 TB

Ethernet 1GEUMTS 1.4 MBit/s

720 • 576 • 3 • 25 = 31.1 MB/s1920 • 1080 • 3 • 25 = 155.5 MB/s

Width Height RGB FPS Data rate

Necessity of data compression

VDSL 50 MBit/s

7.4 h ⇔ 1.5 d

MAD 2015



Aspect ratio and non square pixels

8

➠ Representation of digital video commonly requires appropriate resampling

2,3

5 :

1

1,8

5 :

1

1,3

3 :

1

(4:3

)

Cinematic recordingstraditionally recorded on 35 mm film horizontal image contraction on film (anamorphic representation)typical frame aspect ratios

2,35:1 (21:9, Cinemascope)1,85:1 (100:54)

Application to non-widescreen standardse.g. DVD 720x576 pixels (1,25:1)

Cropping (Pan/Scan)Letterbox Anamorphic

MAD 2015



General data compression tools

9

data compression

decorrelation data reduction coding

• reversible• concentration of signal energy

• reduction of symbol count• increased information per symbol

• irreversible• removal of irrelevancy

• reversible• removal of redundancy

optimization / adaptation

MAD 2015



Decoder

Encoder

General lossy codec

10

•decorrelation• prediction• transformation

•data reduction• quantization

•coding• precoding• entropy coding

•inverse transformation• prediction

•data reconstruction• inverse quantization

•decoding

input

output

transmission /storage

WS 2013




11

data compression

decorrelation data reduction coding

reversibleconcentration of signal energy

reduction of symbol countincreased information per symbol

irreversibleremoval of irrelevancy

reversibleremoval of redundancy

WS 2013 © 2009 UNIVERSITÄT ROSTOCK | FAKULTÄT INFORMATIK UND ELEKTROTECHNIK H. Richter - Image/Video Processing and Coding


12

l Removal or reduction of irrelevancy

l Pre-assumption: l relevant and irrelevant parts are separated from each other as far as possible

l Quantization: l number of symbols or samples keeps constantl precision decreasesl reduced symbol set results in lower entropy

l Sub-sampling: l precision keeps constantl number of symbols or samples decreases

Characteristic Features

WS 2013



Example: spatial resolution

13

256x224 pixels

128x112 pixels

64x56 pixels 32x28 pixels 16x14 pixels

WS 2013



Nea

rest

Nei

ghbo

r

Orig

inal

Bi-c

ubic

Sub

sam

plin

g E

xam

ple:

(red

uctio

n to

30%

plu

s en

larg

emen

t for

dis

play

)

Win

dow

ed S

inc

(Lan

czos

)

0 1 ... W-1W-2...

W Columns

01

...H

-2H

-1...

H R

ows

Pixel at position x,ys[x,y]

Address in linearframe buffery ⋅ W + x

MAD 2015



The image matrix

15

n1

n2

Column

Row

Row Pitch

Column Pitch Pixel

Black

White

Medium Gray

Light Gray

Dark Gray

255

0

128

S =�s [n1, n2]

s [n1, n2] = g 2 G = {0, 1, . . . , 255}

MAD 2015



Image Matrix addressing and level scale

16

Gray-level image(2D signal)

with for 8 Bit per pixel

l row-column addressing natural for most framebuffer types but beware of Matlab®,where coordinates are swapped in the default toolsets and run from 1 (matrix/mathematics) notation instead of 0 (absolute offsets)

MAD 2015



packed YCbCr 4:2:2, YVYU (Philips)

Organization of color components

17

planar RGBpacked RGB 24 Bit

packed RGB 32 Bit (RGBA)

packed YCbCr 4:2:2, UYVY (Abekas)

RGB as 3D matrix(logical)

➠ only a few common examples shown, RAW data organized to the needs of actual applications➠ beware of Endianess issues when dealing with data accesses >8 Bit

WS 2013




18

data reduction

subsampling quantization

reduction of spatial resolutionreduction of temporal resolution

reduction of signal amplitude precision

mapping of similar values/amplitudes to a common representative value

scalar quantizationvector quantization

mapping of similar vectors to a common representative vector

Documents

Image and Video Coding - UPM