View
219
Download
4
Category
Preview:
Citation preview
112/04/18 MC 2009 1
多媒體編碼技術導論Introduction to Multimedia
Coding
Text Book Iain E.G. Richardson, “H.264 and MPEG-4
Video Compression,” John Wiley & Sons, 2003.
References1. Yao Wang, Jorn Ostermann, and Ya-Qin Zhang,
“Video Processing and Communications,” Prentice Hall Signal Processing Series, 2002.
2. Khalid Sayood, “Data Compression,” Morgan Kaufmann, 2nd edition, 2000.
3. M.S. Drew and Z.-N. Li, "Fundamentals of multimedia," Prentice-Hall, 2003.
112/04/18 MC 2009 2
ContentIntroduction
Video Formats and Quality
Video Coding Concept H.261 Variable length coding, DCT, Motion
estimation
The MPEG-4 and H.264 Standards H.264/AVC, Main/High/SVC/Multi-ViewWavelets: Image and Video Coding
Image Coding Standard: JPEG-2000, HD Photo
Voice/Audio Coding Standards: iLBC (Skype) MP3, and AAC Applications and Directions
112/04/18 MC 2009 3
Bernd Girod, Stanford Universityhttp://www.stanford.edu/class/ee398
/
Entropy and lossless coding techniquesRun-length coding, fax standardsArithmetic codingRate-distortion limits and quantizationLossless and lossy predictive codingTransform coding, JPEG standard Subband coding, wavelets, JPEG-2000Motion compensated coding, MPEG standards
112/04/18 MC 2009 4
Learn and PurposeProvide students with a global overview of state-of-the-art in the area of VIDEO and VOICE COMPRESSION
Video: current standards, H.264 and MPEG-4
Voice/Audio: current standards, iLBC (for VoIP), MP3, G.729
Purpose: inspire the students to develop value-added supplementary on H.264 or iLBC
112/04/18 MC 2009 5
Schedule on Experiments
3th-4th weeks: Trace H.261 code5th-12th weeks: Trace H.264 code…
112/04/18 MC 2009 6
評分標準 Midterm Exam.: 35%
Homework: 15%
H.264 program trace, Report: 20% Presentation and Discussion: 30%
112/04/18 MC 2009 7
Applications on H.264/AVC standardsBroadcast over cable, satellite, cable modem, DSL, terrestrial, DVB-H, etc.Interactive or serial storage on optical and magnetic devices: Blu-ray DVD, HD-DVD, etc.Conversational services over ISDN, Ethernet, LAN, DSL, wireless and mobile networks, modems, etc. or mixtures of these.Video-on-demand or multimedia streaming services over ISDN, cable modem, DSL, LAN, wireless networks, etc.Multimedia messaging services (MMS) over ISDN, DSL, Ethernet, LAN, wireless and mobile networks, 3G, etc.
112/04/18 MC 2009 8
Chronological Table of Video Coding Standards
H.261
(1990)
MPEG-1
(1991)
H.263
(1995/96)
H.263+
(1997/98)
H.263++
(2000)
H.264
( MPEG-4
Part 10 )
(2002)MPEG-4 v1
(1998/99)MPEG-4 v2
(1999/00)MPEG-4 v3
(2001)
1990 1992 1994 1996 1998 2000 2002 2003
MPEG-2
(H.262)
(1993/94)ISO/IEC
MPEG
ITU-TVCEG
112/04/18 MC 2009 9
Position of H.264
112/04/18 MC 2009 10
H.264/AVC Profiles (1)
112/04/18 MC 2009 11
H.264/AVC Profiles (2)
112/04/18 MC 2009 12
H.264/AVC Profiles (3) -Scalable Video Coding (SVC)
Functionality of SVC Graceful degradation when “right” parts
of the bit-stream are lost Bit-rate adaptation to match the
channel throughput Format adaptation for backwards
compatible extension Power adaptation for trade-off between
runtime and quality
2007/8 Kai-Chao Yang, NTHU, Taiwan 12
112/04/18 MC 2009 13
Then …MPEG 3D Audio-visual encoding – built on
AVC ? Workshops on future directions in
video compression
Multi-view Video Coding ITU-T VCEG Reduced complexity H.264 ? H.265 with 50% better compression by
2010
112/04/18 MC 2009 14
Terminology, Chapter 1 Sampling: spatial and temporal Frame and field; progressive and raster scan Color space: RGB, YIQ, YCbCr, CMY, … Video format
Analog: NTSC, PAL, SECAM Digital: CCIR601, Sub-QCIF, QCIF, CIF, BT.601(720480/576),
SMPTE295M(19201080) 4:2:0, 4:2:2, 4:4:4
112/04/18 MC 2009 15
Terminology, Chapter 1
112/04/18 MC 2009 16
Terminology, Chapter 1
112/04/18 MC 2009 17
Terminology, Chapter 1Digital Video:BT.601
112/04/18 MC 2009 18
Terminology, Chapter 1
112/04/18 MC 2009 19
Terminology, Chapter 1
112/04/18 MC 2009 20
Subjective Quality Measurement
Visual scene: complex interaction between the Human Visual System, the eye, and the brain
ITU-R BT.500, Double Stimulus Continuous Double Stimulus Continuous Quality ScaleQuality Scale (DSCQS) Mean Opinion Score (MOS) The assessor is presented with a pair of images A
and B, one after the other, and is asked to give A and B a ‘quality score’ with five intervals ranging from ‘Excellent’ to ‘Bad’. The selection of which sequence is A (source) and which is B (impaired) is randomized.
112/04/18 MC 2009 21
Objective Quality MeasurementPeak Signal to Noise Ratio (PSNR)
Other objective quality metrics, ITU-T Video Quality Experts Group (VQEG) Currently, no objective measurement
system is able to replace subjective testing, no one objective model outperforms the others in all cases.
MSEPSNR
n
dB
2
10
)12(log10
112/04/18 MC 2009 22
Motion Vector
Frame i-1 Frame i
Current frame(to be encoded)
Reference frame(existed)
Occlusion
matched
Motion vector
Block matching
112/04/18 MC 2009 23
Motion Compensation & Residuals
motion Residualsocclusion
112/04/18 MC 2009 24
Encoder
MotionCompensation
DCT + Q
iDCT + iQ
MV = (dx, dy)①
② ③
④
Residuals
PreviousFrame Buffer
Encoder (DCT Quantization Binary coding)Residual only
112/04/18 MC 2009 25
Decoder
CodedBitstream VLD
1Q IDCT
MotionCompensation
PreviousFrame memory
Reconstructedframe
MV
Residuals
112/04/18 MC 2009 26
DCT (1)
8
8
139
148
150
149
155
164
165
168
98 115
130
135
143
146
142
147
89 110
125
128
129
121
104
106
96 116
128
132
134
132
113
109
111
125
127
131
137
137
120
110
122
126
126
131
133
131
126
112
133
134
136
138
140
144
141
139
138
139
139
139
140
146
148
147
112/04/18 MC 2009 27
DCT
IDCT
Pixel values in spatial domain
DCT coefficients in transform domain
DC term:The coefficient of the basis vector (0,0)
DCT (2)
112/04/18 MC 2009 28
Does DCT Transform Really Make Sense ?
Energy compaction De-correlation: dependency elimination
112/04/18 MC 2009 29
112/04/18 MC 2009 30
112/04/18 MC 2009 31
Natural Audio Coder
Quality
Cellular
Telephone
AM
FM
CD
2 4 8 16 32 64 kbit/s
Parametric speech(HVXC)
High quality speech(CELP)
General audio(AAC, TwinVQ)
Parametric audio(HILN)
From Olivier Dechazal
Recommended