1

Design and Implementation of an Efficient MPEG-4 Interactive Terminal on

Embedded Devices

Yi-Chin Huang, Tu-Chun Yin, Kou-Shin Yang, Yan-Jun Chang, Meng-Jyi Shieh, Wen-Chin ChenIEEE International Conference on Multimedia and ExpoVolume 1,  27-30 June 2004 Page(s):715 - 718

Reporter：陳憲章Date： 2006.4.11

2

Outline

Introduction System Architecture Synchronization of Heterogeneous

media Implementation and Optimization Experimental Results Conclusions

3

Introduction

This paper present an efficient MPEG-4-based interactive player for PDA-like embedded devices.

There are many technical issues need to be tackled: lower computing power less hardware acceleration constrained memory size lack of built-in multimedia APls

4

Introduction (con’t)

To overcome these constraints, two novel mechanisms were introduced: Adaptive Frame Rate (AFR) Scene Cache Graph Management

(SCGM) A semi-pull model was also designed

for the synchronization of heterogeneous media objects.

5

System Architecture

The MPEG-4 Media Process Flow

6

System Architecture (con’t)

7

Scene Cache Graph Management (SCGM)

8

Synchronization of

Heterogeneous media

9

Categories of Media

1. Instant Time Media The media data that can be computed

immediately. 2. Discrete Time Media

The media whose sampling lengths are long enough so that computers can accomplish the processing. (ex: video & animation)

3. Continual Time Media Those media whose sampling time is very short.

(ex: auditory)

10

Semi-Pull Model

A new decoding thread, is created for decoding. Each decoded data sample is associated with a composition timestamp.

A composition timestamp is created with respect to a global reference clock generated by scene graph manager.

11

Intra-media Synchronization

For instant time media, synchronization can be achieved if the Event Monitoring Procedure (EMP) execution rate is kept high enough.

For other media types, semi-pull model ensures that the presented data sample is the most updated and rendered on time.

12

Inter-media Synchronization

Synchronization between time-variant (discrete and continual) media Forcing all media streams synchronized

to the global clock. Currently in MPEG-4, only audio streams

have this problem. Audio streams are rendered by a sound card, which has a local clock.

13

Inter-media Synchronization (con’t)

Synchronize the audio stream to the global clock by adjusting the amount of waveform data.

14

Implementation and Optimization

15

Adaptive Frame Rate Mechanism

The job of visual render thread are divided into two procedures: Event Monitoring Procedure (EMP) Scene Render Procedure (SRP).

16

Adaptive Frame Rate Mechanism (con’t)

In this mechanism, a scene runs in either active state or inert state.

1. Inert state render thread adopts lazy-render

strategy. with which the only periodic routine task is to execute EMP.

17

Adaptive Frame Rate Mechanism (con’t) 2. Active state

To continually render the active movie textures, render thread executes SRP in every specified period Pa.

18

2D/3D Graphic Engine

Hardware acceleration for 3D rendering and floating-point number processing unit are not present on the target platform.

To overcome these problems, the engine applies fixed-point arithmetic and performs all 3D rendering functionalities by software emulation.

19

Buffer Management

This module maintains two buffers to hold decoded data.

While a decoding thread writes data to one buffer, the render thread reads data from the other one.

To avoid the race condition, both threads may lock the buffers.

20

Script Engine

To enhance the interactivity of BlFS scene, the system supports JavaScript mechanism specified in MPEG-4.

The script engine parses all the script codes in the scene and transforms them into a syntax tree for speeding up the script execution.

21

Experimental Results

When the rendering frame rate is fixed at 12 fps, the average CPU load is 98.65% without SCGM.

When SCGM is enabled, the CPU load is improved toward 6.37%.

The 3D model placed in the middle →

22

Experimental Results (con’t)

There are three objects, each of which is bound with a movie texture. To measure the performance of AFR, we

consider the lifetime of the battery with power consumption varies from 100% to 20%.

The experimental result shows that AFR gains 118% improvement of the lifetime in the first scene.

23

Conclusions

This paper have successfully developed a MPEG-4 player on a resource constrained PDA.

The system adopts four novel mechanisms: Adaptive Frame Rate Scene Cache Graph Management Media Decoding Framework Semi-pull Model

24

Conclusions (con’t)

Resolving the problems including: computational resource reduction low power consumption heterogeneous media synchronization

For further, this system can be developed by new functionalities.

25

Seminar

Thanks for comment !