55
www.kandroid.org 운영자 : 양정수 (yangjeongsoo at gmail.com), 닉네임: 들풀 2012. 10. 26. The Gate of the AOSP #3 : Externals & Extras LLVM(Low Level Virtual Machine), Clang and GDK 10 th Kandroid Conference

LLVM(Low Level Virtual Machine), Clang and GDK · 2012. 10. 30. · LLVM(Low Level Virtual Machine), Clang and GDK 10th Kandroid Conference . 10th Kandroid Conference ... (Dalvik

  • Upload
    others

  • View
    7

  • Download
    0

Embed Size (px)

Citation preview

  • www.kandroid.org 운영자 : 양정수 (yangjeongsoo at gmail.com), 닉네임: 들풀

    2012. 10. 26.

    The Gate of the AOSP #3 : Externals & Extras

    LLVM(Low Level Virtual Machine), Clang and GDK

    10th Kandroid Conference

  • 10th Kandroid Conference

    LLVM(Low Level Virtual Machine), Clang and GDK

    1. Background History

    • LLVM and RenderScript

    • NDK : Current State and Issues(limitations)

    2. LLVM(Low Level Virtual Machine)

    • Why/What is LLVM?

    • LLVM IR(Intermediate Representation)

    • Lifetime long optimization / Multi-stage optimization

    3. Clang : LLVM Compiler Frontend

    • Why new Compiler Frontend : Motivation and Features

    • Clang Architecture

    • Integrate clang into IDEs – Clang and EClipse CDT

    4. Clang/LLVM vs. GCC Issues and GDK git branch

    • Issue #1 : LLVM vs. GCC Optimization

    • Issue #2 : Clang vs. GCC Static Analysis

    • GDK : What is GDK?

    • GDK : Why GDK uses Clang/LLVM?

  • 3 10th Kandroid Conference - www.kandroid.org

    RenderScript offers

    a high performance 3D graphics rendering and compute API

    at the native level, which you write in the C (C99 standard).

    The main advantage :

    • Portability

    • Performance

    • Usability

    The main disadvantage :

    • Development complexity

    • Debugging visibility

    • Less features

    The general guideline for OpenGL and RenderScript

    • Android Framework OpenGL API

    • OpenGL and NDK

    • RenderScript

    LLVM and RenderScript

  • 4 10th Kandroid Conference - www.kandroid.org

    • Develop a solution to providing a solid 60fps for our application

    frameworks

    • Has to work well with Dalvik

    • Has to work with existing hardware

    • Has to scale to future hardware without requiring developers to rewrite

    their applications

    • Developed applications must be portable across a variety of hardware.

    ARM v5, v7, NEON, x86, SSE, GPUs, DSPs

    • Good performance across all devices instead of peak performance for one

    device at the expense of others

    • Build a forward looking 3D graphics and compute API

    Source : http://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

    LLVM and RenderScript : RenderScript Goals

    http://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

  • 5 10th Kandroid Conference - www.kandroid.org

    Native Layer

    (RenderScript)

    Native Layer

    (LLVM Code)

    Reflected Layer

    (C99)

    helloworld.rs

    helloworld.bc

    ScriptC_

    helloworld

    .java

    ScriptField_

    xxxxxxxxx

    .java

    llvm-rs-cc

    App Java

    Sources

    App Java

    Sources

    LLVM and RenderScript

  • 6 10th Kandroid Conference - www.kandroid.org

    APK file

    Native Layer

    (LLVM Code)

    Reflected Layer

    helloworld.bc

    ScriptC_

    helloworld

    .java

    Framework Layer

    App Java

    Sources

    App Java

    Sources

    App Java

    Sources Dalvik

    JIT Compiler

    libbcc

    LLVM based

    Jit compiler

    System lib.

    .bc files

    Multicore CPUs

    GPUs/DSPs

    LLVM and RenderScript

  • 7 10th Kandroid Conference - www.kandroid.org

    Android Runtime

    (Dalvik VM)

    RenderScript Runtime

    (LLVM)

    Reflected Layer

    helloworld.bc

    ScriptC_

    helloworld

    .java

    ScriptField_

    xxxxxxxxx

    .java

    App Java

    Sources

    memory

    Memory

    Allocation

    Read/Write

    Read/Write

    Memory

    Bind

    LLVM and RenderScript

  • 8 10th Kandroid Conference - www.kandroid.org

    LLVM and RenderScript

  • 9 10th Kandroid Conference - www.kandroid.org

    Library Linking option 3 4 5,6,7 8 9(10) 14

    c default ○ ○ ○ ○ ○ ○

    math default ○ ○ ○ ○ ○ ○

    c++ default ○ ○ ○ ○ ○ ○

    log -llog ○ ○ ○ ○ ○ ○

    zlib -lz ○ ○ ○ ○ ○ ○

    Dynamic Linker -ldl ○ ○ ○ ○ ○ ○

    OpenGL ES 1.x

    -lGLESv1_CM.so ○ ○ ○ ○ ○

    OpenGL ES 2.0

    -lGLESv2.so ○ ○ ○ ○

    jnigraphics -ljnigrahics ○ ○ ○

    OpenSL/ES -lOpenSLES ○ ○

    Android Native

    Application API -landroid ○ ○

    OpenMAX AL -lOpenMAXAL ○

    NDK : Current State and Issues(limitations)

  • 10 10th Kandroid Conference - www.kandroid.org

    Hardware Platform

    User Applications

    Kernel

    System Call Interface

    glibc

    Architecture Dependent Kernel Code

    User

    Space

    Kernel

    Space

    NPTL

    (pthread)

    bionic

    libc

    lib

    cutils

    NDK : Current State and Issues(limitations) – C libc

  • 11 10th Kandroid Conference - www.kandroid.org

    Hardware Platform

    User Applications

    Kernel

    System Call Interface

    glibc

    Architecture Dependent Kernel Code

    User

    Space

    Kernel

    Space

    NPTL

    (pthread)

    bionic

    libc

    lib

    cutils

    gnu-

    libstdc++ libstdc++

    gabi++

    stlport

    gnustl

    libutils

    libbinder

    NDK : Current State and Issues(limitations) – C++ libc

  • 12 10th Kandroid Conference - www.kandroid.org

    C++ Exceptions C++ RTTI Standard Library

    system No No No

    gabi++ No Yes No

    stlport No Yes Yes

    gnustl Yes Yes Yes

    The Android platform provides a very minimal C++ runtime support library

    (/system/lib/libstdc++) and corresponding headers for it in the NDK.

    By default, this 'system' runtime does *not* provide the following:

    - Standard C++ Library support (except a few trivial headers).

    - C++ exceptions support

    - RTTI support

    The only headers provided here are the following:

    cassert cctype cerrno cfloat climits cmath csetjmp csignal cstddef cstdint

    cstdio cstdlib cstring ctime cwchar new stl_pair.h typeinfo utility

    NDK : Current State and Issues(limitations) – C++ libc

  • 10th Kandroid Conference

    LLVM(Low Level Virtual Machine), Clang and GDK

    1. Background History

    • LLVM and RenderScript

    • NDK : Current State and Issues(limitations)

    2. LLVM(Low Level Virtual Machine)

    • Why/What is LLVM?

    • LLVM IR(Intermediate Representation)

    • Lifetime long optimization / Multi-stage optimization

    3. Clang : LLVM Compiler Frontend

    • Why new Compiler Frontend : Motivation and Features

    • Clang Architecture

    • Integrate clang into IDEs – Clang and EClipse CDT

    4. Clang/LLVM vs. GCC Issues and GDK git branch

    • Issue #1 : LLVM vs. GCC Optimization

    • Issue #2 : Clang vs. GCC Static Analysis

    • GDK : What is GDK?

    • GDK : Why GDK uses Clang/LLVM?

  • 14 10th Kandroid Conference - www.kandroid.org

    What is LLVM?

    • LLVM Project History : This project started in 2000 at the University of Illinois at

    Urbana-Champaign, under the direction of Vikram Adve and Chris Lattner.

    • LLVM Compiler Infrastructure

    “a design and implementation of a compiler infrastructure which supports a

    unique multi-stage optimization system. This system is designed to support

    extensive interprocedural and profile-driven optimizations, while being efficient

    enough for use in commercial compiler systems”

    - In 2002, Chris Lattner stated a definition of the LLVM in his master‟s thesis

    “a compiler framework that aims to make lifelong program analysis and

    transformation available for arbitrary software and in a manner that is

    transparent to programmers”

    - In 2004, Vikram Adve and Chris Lattner provided a further description of the LLVM

  • 15 10th Kandroid Conference - www.kandroid.org

    Knowledge Base : Compiler Infrastructure (1)

    source program

    modified source program

    target assembly program

    relocatable machine code

    target machine code

    Preprocessor

    Compiler

    Assembler

    Linker library files

    relocatable object files

    Loader (+ fork & exec)

    Memory

  • 16 10th Kandroid Conference - www.kandroid.org

    gcc binutils glibc

    gcc

    cpp

    cc1

    as

    collect2

    ld

    (Soft-Link)

    ld.gold

    ld.bfd link script

    CRT*

    library

    Source (.c and .h)

    Preprocessed source (.i or .ii)

    Asm (.s)

    Relocatable (.o)

    Binary

    (Executable)

    Compiler driver vs. compiler proper

    The executable gcc is not a compiler proper, but a compiler driver

    Knowledge Base : Compiler Infrastructure (2)

    gcc --save-temp

  • 17 10th Kandroid Conference - www.kandroid.org

    Knowledge Base : Compiler Infrastructure (3)

    Lexical Analyzer

    Parser (syntax

    & semantic analyzer)

    Intermediate Code

    Generator

    Code Optimizer

    Code Generator

    token stream

    Intermediate

    representation

    Intermediate

    representation

    Symbol

    Table

    Name + Attribute

    Attribute

    • Type

    • Scope

    • Offset

    machine

    independent

    machine

    dependent

    assembly code

    character stream

    frontend

    backend

    frontend

    middle-end

    backend

    CFG

    Language

    Expression

    yacc,

    bison

    Parsing

    table

    Yacc : LALR(1) Parser Generator

    lex,

    flex

    Regular

    Expression &

    Action Code

  • 18 10th Kandroid Conference - www.kandroid.org

    What is LLVM?

    $ svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm

    $ cd llvm/tools ; svn co http://llvm.org/svn/llvm-project/cfe/trunk clang

    $ cd ../projects ; svn co http://llvm.org/svn/llvm-project/compiler-rt/trunk compiler-rt

    $ svn co http://llvm.org/svn/llvm-project/test-suite/trunk test-suite

    $ cd ../.. ; mkdir build ; cd build/ ; ../llvm/configure ; make

    $ export PATH=/Release+Asserts/bin/:$PATH:.

    $ cat > hello.c

    #include

    int main(void) {

    printf("Hello LLVM\n"); return 0;

    }

    $ clang -emit-llvm -O0 -c hello.c -o hello.bc

    $ llc hello.bc # result : hello.s

    $ lli hello.bc

    Hello LLVM

    $ gcc hello.s -o hello

    $ ./hello

    Hello LLVM

    $ llvm-dis hello.bc # result : hello.ll

    $ vi hello.ll

    # [12 x i8] c"Hello LLVM

    => [13 x i8] c"Hello Clang

    # [12 x i8]* @.str

    => [13 x i8]* @.str

    $ llvm-as hello.ll # result : hello.bc

    $ lli hello.bc

    Hello Clang

  • 19 10th Kandroid Conference - www.kandroid.org

    What is LLVM ? - LLVM Tools (1)

    • Basic Commands

    – llvm-as : LLVM assembler (LLVM ASM (.ll) → bitcode (.bc))

    – llvm-dis : LLVM disassembler (bitcode (.bc) → LLVM ASM (.ll))

    – opt : LLVM optimizer

    – llc : LLVM static compiler (bitcode (.bc) → Machine ASM (.s))

    – lli : directly execute programs from LLVM bitcode

    – llvm-link : LLVM linker (bitcodes (.bc) -> bitcode (.bc))

    – llvm-ar : LLVM archiver

    – llvm-ranlib : Generate index for LLVM archive

    – llvm-nm : list LLVM bitcode and object file‟s symbol table

    – llvm-config : Print LLVM compilation options

    – llvm-sterss : generate random .ll files

    – llvm-diff : LLVM structural „diff‟

    – llvm-prof : print execution profile of LLVM program

    – llvm-cov : emit coverage information

  • 20 10th Kandroid Conference - www.kandroid.org

    • Debugging Tools

    – bugpoint : automatic test case reduction tool

    – llvm-extract : extract a function from an LLVM module

    – llvm-bcanalyzer : LLVM bitcode analyzer

    • Developer Tools

    – FileCheck : Flexible pattern matching file verifier

    – tblgen : Target Description To C++ Code Generator

    – lit : LLVM Integrated Tester

    – llvm-build : LLVM Project Build Utility

    What is LLVM ? - LLVM Tools (1)

  • 21 10th Kandroid Conference - www.kandroid.org

    0

    100000

    200000

    300000

    400000

    500000

    600000

    700000

    800000

    900000

    Commits

    Insertions

    Deletions

    % of changes

    What is LLVM?

    Chris Lattner Shih-Wei Liao

  • 22 10th Kandroid Conference - www.kandroid.org

    Upstream Major Releases

    • 3.1 (May 2012) Includes AddressSanitizer, a fast memory error detector at runtime

    • 3.0 (Dec 2011) GCC frontend(llvm-gcc) is no longer supported, and not included in the release

    Supports MIPS target

    • 2.1 (Sep 2007) Includes its own native C and Objective-C front-end (clang)

    • 1.8 (Aug 2006) Supports DWARF debugging, ARM target

    • 1.6 (Nov 2005) Supports Mac OS X

    • 1.4 (Dec 2004) Supports PowerPC

    • 1.0 (Oct 2003) : The first public release - Includes stable C compiler, beta C++ compiler

    - Supports X86, Sparc

    android/external/llvm git branch

    • 368 line diff from LLVM upstream!

    • Mostly patches for legacy JIT path (no longer used - essentially dead code)

    • Stripped down a bit to fit on current tablet/smartphone

    • No debugging support currently

    (since we don't emit DWARF from llvm-rs-cc today)

    source : http://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

    What is LLVM?

    http://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

  • 23 10th Kandroid Conference - www.kandroid.org

    LLVM Life Long Time Optimization & IR

    • LLVM enables life long time optimization

    • Why is the whole optimization possible with LLVM?

    – LLVM is well-defined and modular architecture.

    – It is easy to make a new compiler/optimizer with LLVM.

    – Powerful IR(LLVM Virtual Instruction Set and Type system)

    and IR’s long lifetime

    Source Code

    Object Code

    Executable Code

    Running Code

    Profile

    Compile-time Optimization

    Link-time Optimization

    Run-time

    Optimization

    Off-line Optimization Install-time

    Optimization

    AOT vs. JIT

  • 24 10th Kandroid Conference - www.kandroid.org

    C

    Frontend C

    Code

    LLVM

    Optimizer

    Machine

    Code

    Generator

    ARM

    Code

    Bitcode

    (LLVM IR)

    A C Compiler based on LLVM

    LLVM Life Long Time Optimization & IR

  • 25 10th Kandroid Conference - www.kandroid.org

    • LLVM IR(Intermediate Representation)

    – LLVM 기반 컴파일러가 소스 코드를 최적화하거나 기계어 코드를 생성할 때, 내부적으로 유지하는 중간 코드

    – A Low-level RISC-like Virtual Instruction Set

    (Example) %a = add i32 4, %b (i32 : b+4의 결과는 32비트 정수 타입)

    • LLVM IR Features

    – Language-independent Instruction Set

    – Instruction in SSA(Static Single Assignment form)

    • SSA : Each variable is assigned exactly once.

    (Example of SSA transformation) X=1; X=2; Y=X ⇒ X1=1; X2=2; Y=X

    • It‟s feasible to do fast analysis of bitcode.

    – Type System

    • Supporting types such as integer, float, pointer, array, vector, structure,

    function

    LLVM Life Long Time Optimization & IR

  • 26 10th Kandroid Conference - www.kandroid.org

    • IR Example

    < C Code >

    < LLVM IR Assembly Code >

    LLVM Life Long Time Optimization & IR

  • 27 10th Kandroid Conference - www.kandroid.org

    LLVM : A Collection of Libraries

    • LLVM Design

    – It is designed as a set of libraries, rather than a monolithic command line

    compiler like GCC or an opaque virtual machine like the JVM

    • The Number of LLVM Libraries : 82 (LLVM 3.1)

    ※ LLVM은 라이브러리 형태로 모듈화가 잘 되어 있어서, 필요한 형태의 최적화, 새로운 최적화 및 코드 생성 기법 추가 등을 빠르게 할 수 있음

  • 28 10th Kandroid Conference - www.kandroid.org

    LLVM Optimization (Compile-Time)

    Source

    Code

    Compile-Time

    Optimizer Frontend

    Object

    Code

    Bitcode

    (LLVM IR)

    1. Perform language-specific optimization

    (ex, optimizing closures in languages with higher-order functions)

    2. Translate source program to LLVM code : required

    (synthesizing as much useful LLVM type information as possible,

    especially to expose pointer, structures, and arrays)

    3. Invoke LLVM passes for global or interprocedural optimizations at module level

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

  • 29 10th Kandroid Conference - www.kandroid.org

    LLVM Optimization (Link-Time)

    Object Code

    Object Code Link-Time

    Optimizer

    Object Code

    ∙∙∙

    Executable

    Code

    Bitcode

    (LLVM IR)

    LLVM link-time optimizer can use the optimized information to improve its performance of

    interprocedural optimizations at link-time.

    The design of the compile- and link-time optimizers in LLVM permit the use of a well-known

    technique for speeding up interprocedural analysis. At compile-time, interprocedural

    summaries can be computed for each function in the program and attached to the

    LLVM bytecode. The link-time interprodecural optimizer can then process these

    interprocedural summaries as input instead of having to compute result from scratch.

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

  • 30 10th Kandroid Conference - www.kandroid.org

    The job of linker is to combine the object files generated by compiler and libraries to form an

    executable program. The GOLD is a new linker written in C++, which is developed by Ian Lance

    Taylor and was added to the GNU Binary Utilities(binutils) in 2008. The goal of the gold linker

    is to be faster than the GNU linker, and also enable link-time (global) optimization (LTO).

    The gold linker can only be used for ELF operating systems such as GNU and Linux.

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

    LLVM Optimization (Link-Time)

    gcc

    cpp

    cc1

    as

    collect2

    ld

    (Soft-Link)

    ld.gold

    ld.bfd link script

    CRT*

    library

    Source (.c and .h)

    Preprocessed source (.i or .ii)

    Asm (.s)

    Relocatable (.o)

    Binary

    (Executable)

    gcc --save-temps

  • 31 10th Kandroid Conference - www.kandroid.org

    LLVM Optimization (Runtime)

    Executable

    Bitcode

    Machine

    Code

    Generator

    Executable

    Machine Code

    Hot

    Spot

    Runtime

    Optimizer

    Hot

    Spot

    optimized

    hot spot

    bitcode after machine code

    generation, replaced

    after first

    running,

    hot spot

    found

    Running

    1

    2

    3

    4

    5

  • 32 10th Kandroid Conference - www.kandroid.org

    Some “hot” functions may not be able to defined and optimized at compile-time,

    except the program is running. The main purpose of run-time optimizer is to

    overcome this disadvantage. Run-time optimizer monitors the running program in real-

    time, if the “hot” functions are detected, run-time optimizer will reoptimize the function.

    “When a hot loop region is detected at runtime, a runtime instrumentation library

    instrument the executing native code to identify frequently-executed paths within that

    region. Once hot paths are identified, we duplicate the original LLVM code into a trace,

    perform LLVM optimization on it, and then regenerate native code into a software-

    managed trace cache. We then insert branches between the original code and the new

    native code.”

    Chris Lattner and Vikram Adve, “LLVM: A Compilation Framework for Lifelong Program

    Analysis & Transformation”, 2004, Proc. of the 2004

    International Symposium on Code Generation and Optimization (CGO'04), Palo

    Alto, California.

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

    LLVM Optimization (Runtime)

  • 33 10th Kandroid Conference - www.kandroid.org

    LLVM Optimization (Idle time)

    “Some transformations are too expensive to perform at run-time, even given an

    efficient representation to work with. For these transformations, the run-time optimizer

    gathers profile information, serializing it to disk … when idle time is detected on the user‟s

    computer, on offline reoptimizer is used to perform the most aggressive profile-

    driven optimizations to the application”

    Compared to traditional profile-guided optimizers, the different of the LLVM offline, idle-

    time reoptimizer is that it can use profile information gather from run-time optimizer with

    the LLVM bytecode to reoptimize and recompile the application, and perform aggressive

    profile-driven interprocedural optimizations without competing with the application for

    processor cycles.

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

  • 34 10th Kandroid Conference - www.kandroid.org

    LLVM Performance (Execution Time)

    • Benchmark Program : SPEC 2000

    source : http://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdf

    Optimization Level

    http://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdfhttp://llvm.org/pubs/2008-10-04-ACAT-LLVM-Intro.pdf

  • 10th Kandroid Conference

    LLVM(Low Level Virtual Machine), Clang and GDK

    1. Background History

    • LLVM and RenderScript

    • NDK : Current State and Issues(limitations)

    2. LLVM(Low Level Virtual Machine)

    • Why/What is LLVM?

    • LLVM IR(Intermediate Representation)

    • Lifetime long optimization / Multi-stage optimization

    3. Clang : LLVM Compiler Frontend

    • Why new Compiler Frontend : Motivation and Features

    • Clang Architecture

    • Integrate clang into IDEs – Clang and EClipse CDT

    4. Clang/LLVM vs. GCC Issues and GDK git branch

    • Issue #1 : LLVM vs. GCC Optimization

    • Issue #2 : Clang vs. GCC Static Analysis

    • GDK : What is GDK?

    • GDK : Why GDK uses Clang/LLVM?

  • 36 10th Kandroid Conference - www.kandroid.org

    What is Clang? (in Wikipedia)

    • Clang

    – A compiler front-end for the C, C++, Objective-C, and Objective-C++

    programming languages. It uses LLVM as its back-end and has been

    part of its releases since LLVM 2.6

    • Why not GCC for LLVM front-end ?

    – GCC is a large and somewhat cumbersome system to develop

    – Apple software makes heavy use of Objective-C, but the Objective-C

    front-end in GCC is a low priority

    – GCC does not fit smoothly into Apple‟s IDE

    – GCC is GPL V3 licensed, but LLVM has a BSD-like license which

    permits including the source into proprietary software

    – So Apple chose to develop a new compiler front-end from scratch. And

    the Clang project was open-sourced in July 2007.

  • 37 10th Kandroid Conference - www.kandroid.org

    Structure of Clang Compiler

    C/C++, Objective C/C++

    Code

    Frontend

    LLVM Compile-Time Optimizer

    LLVM Link-Time Optimizer

    Machine Code Generator

    Linker

    Executable Machine Code

    LLVM

    IR

    Clang

    LLVM

    GCC linker

    What is Clang?

  • 38 10th Kandroid Conference - www.kandroid.org

    0

    100000

    200000

    300000

    400000

    500000

    600000

    700000

    800000

    900000

    Commits

    Insertions

    Deletions

    % of changes

    What is Clang?

  • 39 10th Kandroid Conference - www.kandroid.org

    What is Clang?

    Clang is a native front-end of the LLVM compiler, a new generation development tool,

    which supports C, C++ and Objective-C languages. As a new compiler front-end, Clang has

    many new features, such as fast compile time, low memory use, providing better

    support for Apple’s IDE, expressive diagnostics (which provide very useful error and

    warning message, clearly point out where exactly the problem is and highlight the errors

    code with colorful font), modular design, easy to understand and maintain.

    The fast compile time and low memory use are two of major purpose of the Clang, a testing

    result published on the Clang performance testing website shows that the time taken by

    clang is 2.5 times faster than GCC(version 4.0) on Mac OS/X when parsing “Carbon.h”: the

    Abstract Syntax Tree(AST) memory use of Clang is 5 times less than GCC syntax trees on

    the Space testing : http://clang.llvm.org/features.html#performance

    Source : Getting to know the LLVM compiler, Guobin YE, August 19, 2011

    android/external/clang

    • 32 line diff from Clang upstream!

    • Support for RGBA vector selection (28 lines)

    • Support flag to force "long" to 64-bit (4 lines)

    • Ready to upstream now

    Source : http://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

    http://clang.llvm.org/features.htmlhttp://clang.llvm.org/features.htmlhttp://clang.llvm.org/features.htmlhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdfhttp://llvm.org/devmtg/2011-11/Hines_AndroidRenderscript.pdf

  • 40 10th Kandroid Conference - www.kandroid.org

    Structure of Clang (Frontend)

    C/C++ Source Code

    Lexer & Parser

    Semantic Analyzer

    LLVM Code Generator

    Bitcode

    Preprocessor

    AST

    (Abstract

    Syntax Tree)

    • Syntax checking

    • Semantic analysis

    • Type checking

    • Processing C/C++

    macros (#...)

    Static Code Analyzer • Finding runtime

    bugs

  • 41 10th Kandroid Conference - www.kandroid.org

    Clang Supports Expressive Diagnostics

    • Example

    < Sample code with compile-time error >

    < Error messages in clang compiler >

    < Error messages in GCC >

  • 42 10th Kandroid Conference - www.kandroid.org

    Clang Performance (Compile Time)

    • Test Environment

    – Benchmark Program : 176.gcc (in SPECINT 2000)

    – Including 65 source files, LOC : 205,760

    – Compile option : -O0 –g

    – Tested at 2009.06.26

    source : http://clang.llvm.org/performance.html

    http://clang.llvm.org/performance.htmlhttp://clang.llvm.org/performance.htmlhttp://clang.llvm.org/performance.html

  • 43 10th Kandroid Conference - www.kandroid.org

    Clang Static Analyzer

    • Clang Static Analyzer

    – 컴파일 시간에 실행 시간에 발생할 수 있는 오류를 미리 예측

    • 실행시간 분석보다 정확도는 떨어지지만 빠르게 분석할 수 있는 장점이 있음

    • Clang Command

    – clang –analyze

    • Example

    < foo.c >

    < clang과 gcc 수행 결과 >

  • 44 10th Kandroid Conference - www.kandroid.org

    Eclipse CDT for Clang/LLVM

    • Eclipse CDT 홖경에서 Clang/LLVM 빌드 홖경 지원

    • A Bachelor‟s Thesis in Finland (Mar. 2011)

    – Refer http://code.google.com/p/llvm4eclipsecdt/

    < A snap shot >

    http://code.google.com/p/llvm4eclipsecdt/

  • 10th Kandroid Conference

    LLVM(Low Level Virtual Machine), Clang and GDK

    1. Background History

    • LLVM and RenderScript

    • NDK : Current State and Issues(limitations)

    2. LLVM(Low Level Virtual Machine)

    • Why/What is LLVM?

    • LLVM IR(Intermediate Representation)

    • Lifetime long optimization / Multi-stage optimization

    3. Clang : LLVM Compiler Frontend

    • Why new Compiler Frontend : Motivation and Features

    • Clang Architecture

    • Integrate clang into IDEs – Clang and EClipse CDT

    4. Clang/LLVM vs. GCC Issues and GDK git branch

    • Issue #1 : LLVM vs. GCC Optimization

    • Issue #2 : Clang vs. GCC Static Analysis

    • GDK : What is GDK?

    • GDK : Why GDK uses Clang/LLVM?

  • 46 10th Kandroid Conference - www.kandroid.org

    LLVM and GCC

    • LLVM은 단일 IR로 다양한 형태(compile-time, link-time, runtime, offline)의 최적화 지원 가능하지만 GCC는 compile-time 과 gold linker 사용시에 한정된 link-time에서의 최적화만 가능

    • LLVM은 BSD License로 소스 코드 공개의 의무가 없으나, GCC는 GPL로 소스 코드 공개 의무 있음

    • LLVM, GCC 누가 더 성능 좋은 코드를 만들어 내는 것은 논의 대상이 아니며, 대신 새로운 최적화 기법을 추가하는 데 있어서, LLVM이 GCC 보다 쉽게 접근할 수 있는 구조인 것은 명확함

    • GCC는 1987년 이후로 다양한 응용과 타겟 지원을 통하여 검증된 컴파일러이지만, LLVM은 10년 정도의 역사라 아직 검증된 컴파일러라고 볼 수는 없음

    • LLVM은 GCC와 같은 범용적인 응용이 아니라, 응용의 범위를 좁혀서 시험 검증한다고 하면 완성도가 높은 컴파일러에 쉽게 도달 가능함

    The LLVM Instruction Set and Compilation Strategy GCC Internals, Diego Novillo

    [email protected], November 2007

  • 47 10th Kandroid Conference - www.kandroid.org

    Clang and GCC

    • Clang은 LLVM의 frontend로 시작되었지만, LLVM backend를 사용하여 GCC와 유사한 명령 형태를 지원함

    • Clang도 LLVM 처럼 라이브러리 형태로 모듈화가 잘되어 있어서, frontend에 새로운 기능을 추가, 삭제가 GCC에 비하여 용이함

    • Clang도 BSD License로 소스 코드 공개의 의무 없음

    • Clang은 컴파일 시간에 GCC에 비하여 다양한 오류, 경고를 제공하여 개발자가 디버깅을 쉽게 할 수 있음

    • Clang은 컴파일 시간에 오류, 경고에 대한 메시지를 GCC 보다 세밀하게 표현하여 IDE와 통합할 때 유용함

    Basic

    Lex

    Parse

    AST

    Sema

    Analysis

    Rewrite

    LLVMGen

    Core Libraries App Libraries

  • 48 10th Kandroid Conference - www.kandroid.org

    GDK : What is GDK?

    Hooray! We done the GDK building system.

    This building system is independent on NDK.

    It builds what is should (bitcodes), and then transfer the control

    to NDK building system, doing the remaining building.

    This code is still ugly. Need cleanup.

    gdk git commit id : edde771d8940a6f1b00fd68bcca1486b575e6d9e

    Author : Nowar Gu

  • 49 10th Kandroid Conference - www.kandroid.org

    LOCAL_PATH := $(call my-dir)

    include $(CLEAR_VARS)

    LOCAL_MODULE := hello_llvm

    LOCAL_CFLAGS := -D NUM=7788

    LOCAL_SRC_FILES := hello_llvm.c test.cpp

    LOCAL_C_INCLUDES := jni/test-include

    include $(BUILD_BITCODE)

    include $(CLEAR_VARS)

    LOCAL_MODULE := test2

    LOCAL_SRC_FILES := test2.c

    include $(BUILD_BITCODE)

    Android-portable.mk

    $ cd ~/android-4.1.1_r1/gdk/samples/hello-llvm/jni

    $ export OUT=~/android-4.1.1_r1/out/target/product/generic

    $ ../../../gdk-build --ndk-root=~/android-ndk-r8b

    GDK : What is GDK? : hello-llvm Sample (1)

  • 50 10th Kandroid Conference - www.kandroid.org

    $ ../../../gdk-build --ndk-root=/home/jsyang/android-ndk-r8b

    Compile Bitcode : hello_llvm

  • 51 10th Kandroid Conference - www.kandroid.org

    GDK : What is GDK? : bitmap-plasma-llvm Sample (1)

    bccReadBC(script_ref, "libplasma_portable.bc",

    (const char*)script_ptr, length, 0);

    bccLinkFile(script_ref, "/system/lib/libclcore.bc", 0);

    bccPrepareExecutable(script_ref,

    "/data/data/com.example.plasma.llvm/",

    "plasmaLLVM", 0);

    native_function

    = (pPlasmaType)bccGetFuncAddr(script_ref, "root");

    native_function(info.width, info.height,

    info.stride, time_ms,

    palette, pixels, angle_sin_tab);

    plasmaLLVM.cpp

    crash

  • 52 10th Kandroid Conference - www.kandroid.org

    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***

    Build fingerprint: 'google/yakju/maguro:4.1.1/JRO03C/398337:user/release-keys'

    pid: 16063, tid: 16063, name: ple.plasma.llvm >>> com.example.plasma.llvm

  • 53 10th Kandroid Conference - www.kandroid.org

    GDK : What is GDK? : bitmap-plasma-llvm Sample (3)

    crash

    LLVM Frame : ??

  • 54 10th Kandroid Conference - www.kandroid.org

    GDK : Why GDK uses Clang/LLVM?

    Frame Rate Drawing Performance Comparison - LLVM vs. GCC

    Enable the use of cache.

    With this check-in, end-to-end LLVM milestone achieved.

    For NDK samples, Plasma, running on Nexus-S, 800x480 screen:

    • LLVM Time Run: 12.60ms per frame, Avg: 12.57

    • GCC Time Run: 50.89ms per frame, Avg: 52.34

    LLVM can achieve 80 fps,

    while GCC yields 19 fps.

    GCC is visibly much slower on my Nexus-S.

    gdk git commit ID :

    4a395f0e19b3e3f5790ef95fd7b72694cd4296b6

    Shihwei Liao

  • 10th Kandroid Conference

    www.kandroid.org

    Q & A