Scalable ROI Algorithm for H.264/SVC-Based Video Streaming

Scalable ROI Algorithm for H.264/SVC-Based Video Streaming

Jung-Hwan Lee and Chuck Yoo, Member, IEEE

Overviews

• Introduction• H.264/SVC• Region of Interests• System Architecture• Experimental Results• Conclusion

Introduction

Introduction

• Why SVC?

• What is ROI?

• How to combine SVC and ROI?

• Using FGS for example

Bandwidth Variation

Packet-losses

ROI

Combine SVC and ROI

Use real-time face-detection algorithms

Combine SVC and ROI (cont.)

Introduction

• The authors propose a scalable ROI algorithm, which can support fine-grained scalability in region of interests with low computing complexity.

H.264/SVC

H.264/SVC

• H.264/SVC made adaptive bit rate control according to network condition and resolution control according to device capability possible.

H.264/SVC Enhancement

• Spatial • Temporal • SNR (Quality)

Spatial Enhancement

Reference : H.264 and MPEG-4 VIDEO COMPRESSION, Iain E.G. Richardson

Temporal Enhancement

Reference : Overview of The Scalable Video Coding Extension, SCHWARZ et al

Quality Enhancement

• The multilayer concept for quality scalable coding allows a few selected bit rates to be supported in a scalable bit stream.

Quality Enhancement (cont.)

Texture FDCT Quant Encode coefficient

s

Rescale

Encode each

bitplane

Baselayer

Enhancementlayer

Figure FGS encoder block diagram (simplified)

ROI • Passive Setting of ROI• Active Setting of ROI

Passive Setting of ROI

• The aim of ROI coding is to set a high resolution in ROI and low resolution in nonROI.

• Methods of setting ROI• Passive setting of ROI

• Define regions of interest beforehand

• Active setting of ROI• Constantly change according to environment or

contents

Active Setting of ROI

System Architecture

• Method of structuring ROI• Scalable ROI Algorithm

System Architecture

System Architecture (cont.)

• Two processes need to be defined beforehand.• H.264/SVC video file is encoded with the SNR

enhancing MGS method.• QoE monitor is needed to regularly check the network

status.

• Through this process, the algorithm controls the enhancement layers and the range of ROIs.

Method of structuring ROI

• Passive method of setting ROI is used in this study.

• Center of the screen is set as ROI and areas far from the screen are non-ROIs.

• FMO Box-Out method is applied and ROIs are divided into three stages (Slice Group) .

Method of structuring ROI (cont.)

• After the steps mentioned, the scalable ROI layers are extracted in three different forms.

Scalable ROI algorithm

• The scalable ROI algorithm is applied to the existing bit stream extractor functions.

Reference : JSVM 9_18 software manual

Scalable ROI algorithm (cont.)

• As shown in Fig. 6, ROI algorithm extracted models needs the elements in Table below.

Scalable ROI algorithm (cont.)

• Bw() must (not) be more than the total sum of the basic layer, the layer without SR (Scalable ROI layer) application and the upper layer with SR application.

1. Basic layer.2. The SNR level

range that is not set as ROI.

1. Upper layers with SR application.

2. Enhanced layers set as ROI.

Scalable ROI algorithm (cont.)

• is the SNR dividing coefficient of selected layers, and has the maximum value of MGS division.

• has a different value according to MGS quality in regions with SR application, calculated with eq(2),(3),(4)

1. Basic layer.2. The SNR level

range that is not set as ROI.

1. Upper layers with SR application.

2. Enhanced layers set as ROI.

Scalable ROI algorithm (cont.)

• (2) try to extract ROI from the overall screen. But ROI method cannot applied because the number of quality flags does not meet the minimum value for which video improvement is possible after extraction.

Scalable ROI algorithm (cont.)

• (3) is the case where the quality flags are applied most to the top layer of the overall screen. Since the layer with the highest quality flag value in the overall screen changes in quality flag number in layers according to the j value, this indicates that the size of the ROI screen changes.

Scalable ROI algorithm (cont.)

• (4) is when the quality flag value is half or more of the overall screen. The SR is applied differently to different screen sizes according to the size of the bandwidth and number of quality flags on the screen.

Experimental Result

Experiment Environment

• The JVSM version 9.13 is used.

Experiment Environment (cont.)

Experiment Environment (cont.)

Experiment Environment

• Figure shows PSNR between comparison between traditional and proposed methods.

• Proposed method confirms ROI areas have higher PSNR than non-ROI areas.

Conclusion

Conclusion

• Traditional CGS cannot provide high video quality when the network condition is unstable.

• Proposed method support high subjective quality with FGS by applying ROI to H.264/SVC.