OpenVG Architecture & Programming Tips...(3) Transform (4) Rasterization (6) Paint Generation (7) Image Interpolation (8) Blend/Anti-Aliasing CPU (Context) (5) Clip & Mask Fragment

© Copyright Khronos Group, 2012 - Page 1

OpenVG Architecture & Programming Tips

OpenVG架構與程式編製技巧

Tommy Asano

Director of Advanced Technologies

高級技術主任

TAKUMI Corporation


Contents 內容

• Benefit of using Khronos Standards ・使用 Khronos 標準的好處

• What is OpenVG? ・什麼是 OpenVG 呢？

- Benefit of OpenVG in real applications - OpenVG 在實際應用上的優勢

- OpenVG Architecture - OpenVG 架構

• OpenVG Programming Tips ・OpenVG 程式編寫的技巧

• Demonstration ・示範

© Copyright Khronos Group, 20102- Page 3

TAKUMI - super-sophisticated skill 超精密技巧

A leading provider of 3D/2D/VG Graphics accelerator IP cores

for embedded systems and mobile devices…

Established: July, 2003

Headquarters: Tokyo, Japan Main Business: Graphics Accelerator IP Core licensing Track Record: 30+ license cases including NEC and Samsung mobile handsets

Mobile Telecom Smart Phone/MID

DSC/Camcorder

Car Navigation

MFP

DTV / STB


Graphics Accelerator 圖像加速器

Graphics Khronos Standard APIs 圖像 Khronos 標準 API

Easy to port middleware on top of Khronos standard API 使用 Khronos 標準 API，可輕鬆接口到中間軟體

Allow users the option to select the best development environment for them 可供使用者為自己選用開發最好的環境

Reduce development cost for both users and middleware developers 為使用者與中間軟體開發商降低開發成本

ＯＳ（ITRON / Linux / Android etc.）

Scalable Font

可縮放字體

Ｆｌａｓｈ® Middleware GUI Engine

中間軟體 GUI 引擎

Graphics Application 圖像應用

GUI, MAP ・・


OpenVG Applications OpenVG 應用程式

ＦＬＡＳＨ® GUI © TAKUMI GUI - coverflow © TAKUMI

Scalable Font © TAKUMI

© Rightware © TAKUMI © Rightware


AntiAlias + Image Rendering 防鋸齒＋圖像輸出

Photo Viewer 照片檢視器

Edge AntiAliasing 邊緣防鋸齒

Alias-free high quality image 沒有鋸齒的高品質圖像

Low cost compared to MSAA or FSAA

與 MSAA 或 FSAA 相比，成本較低

Perspective transformation 透視轉換 Transform image arbitrarily by image matrix

用圖像矩陣轉換圖像變化

AntiAlias ON 開啟防鋸齒

AntiAlias OFF 關閉防鋸齒


Bezier Curve + AntiAlias + Transform 貝茲曲線 + 防鋸齒＋轉換

Outline Font & Map 字體與地圖的輪廓

Bezier Curve 貝茲曲線

High quality smooth curves 高品質的平滑曲線

Edge AntiAliasing 邊緣防鋸齒

Alias free high quality shape edge

高品質的完整邊緣、無鋸齒

Avoiding uncomfortable jitter 避免不舒服的抖動

Low cost compare to MSAA or FSAA

與 MSAA 或 FSAA 相比，成本較低

Transform 轉換

Smooth Zoom In/Out, Rotation and Moving around

平順地推進/拉遠、旋轉和移動


Image Filter 圖像過濾器

Photo Viewer 照片檢視器

Image Filtering 圖像過濾

Special effect for images (GaussianBlur)

圖像特效 (GaussianBlur)

Gaussian Blur Filter

Gaussian 模糊過濾器

One of the most expected effect

最令人期待的效果之一

Gaussian Filter

is extremely heavy

Gaussian 過濾器負載非常重 Use smaller image 使用較小的圖像

Use appropriate standard deviation

使用適當的標準偏差

Use blurred image for next frame

下一個圖框使用模糊的圖像

256x256 Std.Dev 4.0 64x64 Std.Dev 4.0 64x64 Std.Dev 2.0 64x64 BilinearFilter

Original 256x256

64x64 64x64 64x64


Understanding Architecture

•Khronos provides common APIs,

But implementations are different

- Wide variety of implementation exists in the market

•How to improve your application performance?

- Understand Architecture (Character of the device)

- Understand where is the Bottleneck

- Understand what is happening in API and GPU


OpenVG Pipeline OpenVG 管線

OpenVG Pipeline Stages OpenVG 的各管線階段


OpenVG Pipeline in Hardware

(1) Path Generation

(2) Stroke Generation

(3) Transform

(4) Rasterization

(6) Paint Generation

(7) Image Interpolation

(8) Blend/Anti-Aliasing

CPU (Context)

(5) Clip & Mask

CPU Context and data management

Pixel Engine Alpha Mask, Color conversion, Anti-

Aliasing, Gradient, Blending

Rasterizer Generate pixels with parameters

EdgeGen Generate edges, Determine render region

Vector Processor Path/Stroke Generation with Cap/Join/Dashing, Tessellation,

Transform, Image Filtering

Setup Edge parameter calculation

Curve Tessellation

Har

dwar

e P

ipel

ine

TAKUMI GV330/GV550/G3000


OpenVG Pipeline in Software

(1) Path Generation


(3) Transform

(4) Rasterization




CPU (Context)

(5) Clip & Mask

Curve Tessellation








CPU

All process is done

by

CPU

(1) Path Generation


(3) Transform

(4) Rasterization




CPU (Context)

(5) Clip & Mask


OpenVG Pipeline in Software & Hardware

(1) Path Generation


(3) Transform

(4) Rasterization




CPU (Context)

(5) Clip & Mask




Curve Tessellation

Har

dwar

e P

ipel

ine








CPU

Path Generation

Curve Tessellation

Stroke Generation

Transform

by

CPU

……


OpenVG Pipeline in 3D Hardware

(1) Path Generation


(3) Transform

(4) Rasterization




CPU (Context)

(5) Clip & Mask

Fragment Shader

Pixel Pipeline


Vertex Shader Transform & Lighting


Stroke/Path Generation

Curve Tessellation

Triangle Tessellation

Primitive Assembly

Curve Tessellation

Har

dwar

e P

ipelin

e


Deep PIPELINE in Hardware









Har

dwar

e P

ipel

ine

TAKUMI GV330/GV550/G3000

Segment N-2

Pixel N-m

Pixel N-7

Pixel N-6

Segment N-1

Segment N

Raster Cmd N-5

Raster Cmd N-4

Raster Cmd N-3

…

Hundreds of commands are processing in the

pipeline stages simultaneously


Critical Requirements for Embedded System 嵌入式系統的必要條件

• Minimize CPU load (CPU is no only for Graphics) 將 CPU 的負載最小化（CPU 不只要處理圖形）

- High-End market: CPU with FPU in 1.5 GHz or dual core and more 高檔市場： CPU with FPU in 1.5 GHz 或雙核心以上

- Low-End market: CPU without FPU in 200 MHz or less 低階市場：CPU without FPU in 200 MHz 以下

• Use memory efficiently 有效地使用記憶體

- Generating Graphics = Processing Power + Memory Access

Display resolution (Mobile Phone/DSC/DVC…): VGA, WideVGA, XGA, FullHD 產生圖形＝處理用電＋記憶體使用

顯示解析度（手機 /DSC/DVC…）：VGA、WideVGA、XGA、FullHD

- Minimize memory footprint 將記憶體使用量最小化

• Minimize LSI footprint (Hardware requirement) 將LSI 使用量最小化

- Minimize Footprint = Low power consumption & Low LSI cost 將使用量最小化＝低用電消耗與低 LSI 成本


General Programming Tips

•Various useful functions in API, use them effectively!!

•Introducing general programming tips

- Better to know timing to use the function

- Better to know what happens when you call the function

•Your platform has your own issues need to be considered

- There are wide variety of implementations

Wide variety of CPU, FPU, GPU architecture, memory …


Programming Tips 程式編製技巧

(Fundamentals) （基本原則）

• Keep object number in small 縮小物件數量

- Share (Cache) objects as much as possible 盡量共享物件

- Delete unnecessary objects

刪除不必要的物件

- Reduce malloc cost/Error check cost 降低 malloc 成本/ 錯誤檢查成本

• Avoid unnecessary vgSet, vgSetparameter 避免不必要的 vgSet、vgSetparameter

- Do not set same parameter over and over to reduce CPU load 不要一再重複設置相同的參數以避免 CPU 負載



(Fundamentals) （基本原則）

• Minimize matrix update 將矩陣更新最小化

- Set pre-calculated matrix with vgLoadMatrix 用 vgLoadMatrix 設置預先計算的矩陣

- Minimize floating calculation by CPU 將 CPU 的浮點計算最小化

• Avoid unnecessary vgFinish 如非必要，別使用 vgFinish

Avoid unnecesarry vgReadPixels, vgWritePixels, vgGetImageSubData 如非必要，別使用 vgReadPixels、vgWritePixels、vgGetImageSubData

Minimize use of vgImageSubData 將 vgImageSubData 使用次數降到最低

- Avoid unnecesarry rendering pipeline sync 避免輸出管道同步



(VGPath)

• Set geometry data with vgAppendPathData at once 立刻用 vgAppendPathData 設置幾何數據

- Avoid unnecessary realloc operation 避免不必要的 realloc 運作

• Avoid modifing path with vgModifyPathData, vgAppendPath, vgClearPath 避免用 vgModifyPathData、vgAppendPath、vgClearPath 修改路徑

vgInterpolatePath, Avoid vgPathBounds, vgTransformedPathBounds 避開 vgInterpolatePath, vgPathBounds, vgTransformedPathBounds

- Could be CPU load 可成為 CPU 負載



(VGPath)

• Recycle path object 回收路徑物件

Avoid vgCreatePathvgAppendPathDatavgDestroyPath cycles

in each frame 在每個圖框中避免出現 vgCreatePathvgAppendPathDatavgDestroyPath 的循環

- Avoid unnecessary malloc/realloc/free 避免不必要的 malloc/realloc/free



(Others)（其他）

• Avoid variable type conversion 避免變量的類型轉換

Use vgSetf(v) for VGfloat type parameter

VGfloat 類型參數就使用 vgSetf(v)

Use vgSeti(v) for VGint/VGboolean type parameter

VGint/VGboolean 類型參數就使用 vgSeti(v)

- Avoid unnecessary CPU load 避免不必要的 CPU 負載

• Use small kernel and image for blur operation (ImageFiltering)

with vgConvolve, vgSeparableConvolve, vgGaussianBlur 模糊運作 (ImageFiltering) 要用小核心和小圖像，使用 vgConvolve、vgSeparableConvolve、

vgGaussianBlur

- Image filtering operation is extremely heavy, even for GPU 圖像過濾運作的負載非常重，甚至是 GPU 也很吃力



(Others) （其他）

• Avoid using VGU functions 避免使用 VGU 功能

- Useful, but heavy load for CPU 有用但 CPU 的負載很重

• Avoid changing Context frequently 避免經常修改內容

Avoid updating VG_RENDERING_QUALITY frequently Avoid 經常更新 VG_RENDERING_QUALITY

- Avoid unnecessary rendering pipeline sync, register set load and store 避免輸出管線同步，暫存設置負載並儲存


Thank you 謝謝

Documents

OpenVG Architecture & Programming Tips...(3) Transform (4) Rasterization (6) Paint Generation (7) Image Interpolation (8) Blend/Anti-Aliasing CPU (Context) (5) Clip & Mask Fragment