Андрей Володин — Как подружиться с роботом

Making friends with robots

1

Agenda

• Building hello-world for Android

• Building Fiber2D project for Android

• Swift + Java = <3

• Overview

2

Hello world

• Linux environment

• Ubuntu 15.10 is the only officially supported

• 16.04 is used in Fiber2D

• NDK (r13 and later)

• Android API Version 21+

• Android device (meh…) with remote debugging

Prerequisites

3

Main header

• Installing curl, autoconf, automake, libtool, git + CMake (!)

• Getting Android NDK r13b (and later)

• Getting SwiftAndroid/libiconv-libicu-android

Building dependencies

4

Hello world

Building Swift compiler

$ utils/build-script -R \ --android \ --android-ndk $ANDROID_NDK_HOME \ --android-api-level 21 \ --android-icu-uc $ANDROID_LIBICONV/armeabi-v7a \ --android-icu-uc-include $ANDROID_LIBICONV/armeabi-v7a/icu/source/common \ --android-icu-i18n $ANDROID_LIBICONV/armeabi-v7a \ --android-icu-i18n-include $ANDROID_LIBICONV/armeabi-v7a/icu/source/i18n/

Export $ANDROID_NDK_HOME and $ANDROID_LIBICONV

5

Hello world

Before building the executable

sudo ln -s \ $ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ld.gold \ /usr/bin/armv7-none-linux-androideabi-ld.gold

Let Swift compiler find the correct linker (gold)

From docs:

Also needed:sudo ln -s \ $ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ld.gold \ /usr/bin/armv7-none-linux-android-ld.gold

6

Hello world

Notes: compiler flags

7

• -l[foo] : Link to libfoo[.so][.dylib]XCode: Other Link Flags

• -L[/path/foo] : Search path for linked libraries XCode: Library Search Paths

• -I[/path/foo] : Search path for headers (C/C++) or .swiftmodule (Swift) XCode: Header Search Paths/Import Paths

• -D[FLAG] : Preprocessor flag (#if FLAG) XCode: Preprocessor macros

Hello world

Building the executable

$ build/Ninja-ReleaseAssert/swift-linux-x86_64/bin/swiftc \ -target armv7-none-linux-androideabi \ -sdk $ANDROID_NDK_HOME/platforms/android-21/arch-arm \ -L $ANDROID_NDK_HOME/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a \ -L $ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9 \ hello.swift

8

Hello world

Deploying the executableStandalone Swift executable requires this to run:

• libswiftCore.so • libswiftGlibc.so • libswiftRemoteMirror.so • libswiftSwiftOnoneSupport.so

and this:• libc++_shared.so

finally, push the executable:

adb push hello /data/local/tmp

9

adb shell LD_LIBRARY_PATH=/data/local/tmp \/data/local/tmp/hello

Hello world

10

Compiling Fiber2D

11

Compiling Fiber2D

The goal

• Swift Package Manager at the heart of build process • Compile Fiber2D framework • Use it in the real world (.apk) Android application

13

Compiling Fiber2D

Project structure

Fiber2D

Cpng CChipmunk2DSwiftMath

SwiftBGFX

Cbgfx

bgfx

14

Compiling Fiber2D

Project structure

Fiber2D

Cpng CChipmunk2DSwiftMath

SwiftBGFX

Cbgfx

bgfx

Legend: • C++ • C • Swift

15

Compiling Fiber2D

SwiftPM limitations

• Does not allow mixing languages within one target • No Bridging-Headers • No custom source code layouts • No custom local config (-l/-L/-I/-D flags) • No custom scripts/makefiles

16

Compiling Fiber2D

Notes: module maps

module CChipmunk2D { header "chipmunk_private.h" header "chipmunk.h" header "cpHastySpace.h" link "CChipmunk2D" export * }

17

Compiling Fiber2D

NDK• Android apps are Java-based • Android NDK lets you implement parts of your app using

native-code languages such as C and C++ • Java to C calls are handled by Java Native Interface (JNI)

public class MyActivity extends Activity {  /**  * Native method implemented in C/C++  */  public native void computeFoo();}

18

Compiling Fiber2D

NDK

• NDK project must contain jni folder at its root • ndk-build command recursively traverses it and

compiles every Android.mk it can find • Compiled binaries are stored in libs/<ARCH> folders

19

Compiling Fiber2D

SDL

• SDL - low-level abstraction layer for audio, keyboard, mouse, joystick, and graphics hardware

• Provides Android template project with JNI shim • Wants you to add your C/C++ files to its main lib’s

Android.mk:# Add your application source files here... LOCAL_SRC_FILES := $(SDL_PATH)/src/main/android/SDL_android_main.c

20

Compiling Fiber2D

What SDL does not tell us?

#define main SDL_main // And then extern C_LINKAGE int SDL_main(int argc, char *argv[]);

• SDL secretly redefines main function with its own macro on certain platforms:

• We can’t use it in Swift

21

Compiling Fiber2D

What we want to do?

• Compile SDL and boilerplate code only once • Compile our project as dynamic library • Put it into Android project • Load it during the runtime • Make SDL call our SDL_main implementation

22

Compiling Fiber2D

Notes: Swift name mangling

23

class Shape { func numberOfSides() -> Int { return 5 } }

_TFC9swifttest5Shape17simpleDescriptionfS0_FT_Si

JFYI: You can use swift-demangle

Compiling Fiber2D

Implementing SDL_main

• Make sure you don’t have main.swift • Use @_silgen_name or @_cdecl attributes to overcome

Swift symbol mangling

@_cdecl("SDL_main")public func SDL_main(argc: Int32, argv: OpaquePointer) -> Int32 { …}

• Function must be top-level, public and can’t throw

24

Compiling Fiber2D

Creating the build script

• To build SwiftPM package you do swift build • swift != swiftc • Use -Xlinker, -Xcc and -Xswiftc to forward flags • Declarations with spaces requires multiple -X* flags, i.e.: -Xlinker -framework -Xlinker AppKit

25

Compiling Fiber2D

Creating the build script

First, add flags that we used to build hello-world:-Xswiftc -target -Xswiftc armv7-none-linux-androideabi \ -Xswiftc -sdk -Xswiftc $ANDROID_NDK_HOME/platforms/android-21/arch-arm \ -Xlinker -L$ANDROID_NDK_HOME/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a \ -Xlinker -L$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/

26

Compiling Fiber2D

Creating the build script

Add Fiber2D include paths:-Xswiftc -I../../../../.build/debug \-Xswiftc -I../../../../.build/checkouts/Cpng-8308068493617107876/Cpng/include \-Xswiftc -I../../../../.build/checkouts/CChipmunk2D--4671306517288557973/CChipmunk2D/include \-Xswiftc -I../../../../external/SwiftBGFX/.build/debug \-Xswiftc -I../android-project/jni/SDL2-2.0.5/include \

Android-specific ones:-Xswiftc -I$ANDROID_NDK_HOME/sources/android/native_app_glue/ \-Xcc -I$ANDROID_NDK_HOME/platforms/android-21/arch-arm/usr/include \

And some for Foundation:-Xswiftc -I$ANDROID_SWIFT_SOURCE/build/Ninja-ReleaseAssert/foundation-linux-x86_64/Foundation \-Xswiftc -I$ANDROID_SWIFT_SOURCE/swift-corelibs-foundation \-Xswiftc -I$ANDROID_SWIFT_SOURCE/swift-corelibs-foundation/closure \

27

Compiling Fiber2D

Creating the build script

Link to libraries:-Xlinker -lgcc -Xlinker -lc++ -Xlinker -ldispatch \-Xlinker -lFoundation -Xlinker -latomic -Xlinker -lFiber2D \-Xlinker -licui18n -Xlinker -licuuc \

Add library search paths for libDispatch and libFoundation:-Xlinker -L/usr/local/lib/swift/android/ \-Xlinker -L$ANDROID_SWIFT_SOURCE/build/Ninja-ReleaseAssert/foundation-linux-x86_64/Foundation/ \

Add library search paths for Android stuff:-Xlinker -L$ANDROID_NDK_HOME/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a \-Xlinker -L$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/ \-Xlinker -L$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/lib/armv7-a/ \-Xlinker -L$ANDROID_LIBICONV/armeabi-v7a \

And for Fiber2D:-Xlinker -L../../../../.build/debug \-Xlinker -L../../../../external/SwiftBGFX/.build/debug \

28

Compiling Fiber2D

Creating the build script

Compile libSwiftMath without SIMD:-Xswiftc -DNOSIMD \

Add cross-compilation flags:-Xcc -B -Xcc $ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ \-Xcc --sysroot=$ANDROID_NDK_HOME/platforms/android-21/arch-arm/ \-Xlinker --sysroot=$ANDROID_NDK_HOME/platforms/android-21/arch-arm/ # This does not work for now and requires a hack for clang

29

Compiling Fiber2D

Build script:cd f2dc swift build \ -Xswiftc -I$ANDROID_NDK_HOME/sources/android/native_app_glue/ \ -Xcc -I$ANDROID_NDK_HOME/platforms/android-21/arch-arm/usr/include \ -Xswiftc -I../../../../.build/debug \ -Xswiftc -I../../../../.build/checkouts/Cpng-8308068493617107876/Cpng/include \ -Xswiftc -I../../../../.build/checkouts/CChipmunk2D--4671306517288557973/CChipmunk2D/include \ -Xswiftc -I../../../../external/SwiftBGFX/.build/debug \ -Xswiftc -I../android-project/jni/SDL2-2.0.5/include \ -Xswiftc -I$ANDROID_SWIFT_SOURCE/build/Ninja-ReleaseAssert/foundation-linux-x86_64/Foundation \ -Xswiftc -I$ANDROID_SWIFT_SOURCE/swift-corelibs-foundation \ -Xswiftc -I$ANDROID_SWIFT_SOURCE/swift-corelibs-foundation/closure \ -Xswiftc -target -Xswiftc armv7-none-linux-androideabi \ -Xswiftc -sdk -Xswiftc $ANDROID_NDK_HOME/platforms/android-21/arch-arm \ -Xswiftc -DNOSIMD \ -Xcc -target -Xcc armv7-none-linux-androideabi \ -Xcc -B -Xcc $ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ \ -Xcc --sysroot=$ANDROID_NDK_HOME/platforms/android-21/arch-arm/ \ -Xlinker -L/usr/local/lib/swift/android/ \ -Xlinker -L$ANDROID_SWIFT_SOURCE/build/Ninja-ReleaseAssert/foundation-linux-x86_64/Foundation/ \ -Xlinker -L$ANDROID_NDK_HOME/sources/cxx-stl/llvm-libc++/libs/armeabi-v7a \ -Xlinker -L$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/lib/gcc/arm-linux-androideabi/4.9.x/ \ -Xlinker -L$ANDROID_NDK_HOME/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/lib/armv7-a/ \ -Xlinker -L$ANDROID_LIBICONV/armeabi-v7a \ -Xlinker -L../../../../.build/debug \ -Xlinker -L../../../../external/SwiftBGFX/.build/debug \ -Xlinker -L../android-project/libs/armeabi-v7a \ -Xlinker -lgcc -Xlinker -lc++ -Xlinker -ldispatch \ -Xlinker -lFoundation -Xlinker -latomic -Xlinker -lFiber2D \ -Xlinker -licui18n -Xlinker -licuuc \ -Xlinker --sysroot=$ANDROID_NDK_HOME/platforms/android-21/arch-arm/ # This does not work for now and requires a hack for clang

30

Compiling Fiber2D

clang hack

#!/bin/bash

# This is a proxy hack until it gets fixed in SPM # You have to add chmod +x to this file /usr/bin/@clang -target armv7-none-linux-androideabi \ -B $(ANDROID_NDK_HOME)/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/arm-linux-androideabi/bin/ \ —sysroot=$(ANDROID_NDK_HOME)/platforms/android-21/arch-arm/ $*

• Linking C/C++ with Swift is done by clang • SwiftPM does not pass platform flags for cross-

compilation (-B/—sysroot/-target) to clang during linkage stage (even when passed via -Xlinker)

• So we had to do a proxy clang script • mv /usr/bin/clang /usr/bin/@clang • chmod +x /usr/bin/clang

31

Compiling Fiber2D

Compiling libSDL2 and libmain

• Put freshly compiled libf2dc.so lib to jni/dependncies

• Add -lf2dc -Ljni/dependencies to main lib’s Android.mk

• Run ndk-build in the project root

32

Compiling Fiber2D

SDL hack + pure NDK

• As we wanted to use custom renderer we had to hack in SDL to prevent it from managing the window’s context

• We create CAndroidAppGlue package to wrap pure NDK API into Swift Module

• It is possible to create pure Swift + Java app

33

Compiling Fiber2D

Putting Swift binaries into .apk

libCbgfx.so libCChipmunk2D.so

libCpng.so libcurlll.so

libdispatch.so libf2dc.so

libFiber2D.so libFoundation.so libscudata.so libscui18n.so libscuuc.so

libSwiftBGFX.so libswiftCore.so libswiftGlibc.so

libswiftRemoteMirror.so libswiftSwiftOnoneSupport.so

libxml222.so libc++_shared.so

Copy these libraries to libs/armeabi-v7a:

34

Compiling Fiber2D

Notes: executable file formats

35

• PE (Windows) • Mach-O (Mac/iOS) • ELF (Linux/Android)

Contain executable format, architecture type and program header

Compiling Fiber2D

Notes: executable file formats

36

Compiling Fiber2D

Putting Swift binaries into .apk

Note that Android 5 manages sub-dependencies on its own, but on Android 4.4 you have to put all libs to

System.load section of SDLActivity.java

protected String[] getLibraries() { return new String[] { "SDL2", "main" }; }

// Load the .so public void loadLibraries() { for (String lib : getLibraries()) { System.loadLibrary(lib); } }

37

Compiling Fiber2D

Deploying the app

• ant debug install • Run the app from Android launcher • Use logcat to debug (NSLog() instead of print())

38

Compiling Fiber2D

libicu hack

• Android linker ignores LD_LIBRARY_PATH when loading with System.loadLibrary()

• Android is shipped with its own outdated version of libicu

• We have to ship our own libraries • And hack into compiled binaries…

39

Compiling Fiber2D

libicu hack

• We have to fool Android linker to load our version of libicu

• Conventional hack: rename libi***.so to libs***.so

• Rename binary links as well: rpl -R -e libicu libscu lib*.so

• Note: keep the strings you change the same length

40

Compiling Fiber2D

libxml/libcurl hackreadelf -d libFoundation.so

41

Compiling Fiber2D

libxml/libcurl hackreadelf -d libFoundation.so

42

Compiling Fiber2D

libxml/libcurl hack• Android can’t load libraries via symlinks or custom

names • We have to ship our own versions of libxml and libcurl

• You can find initial binaries in $ANDROID_NDK_HOME/sysroot/src/libxml2/.libs

• Rename: libxml2.so.2 to libxml222.solibcurl.so.5 to libcurlll.so

43

Compiling Fiber2D

Results

44

Compiling Fiber2D

Results

45

Swift + Java = <3

46

Swift + Java

Calling from Java into C

Compile your C code as dynamic library, load it at runtime

System.loadLibrary("fooLib");

Define method with native keyword

public native Int intFromJNI();

Java side:

47

Swift + Java

Calling from Java into C

C side:JNIEXPORT jint JNICALL Java_com_example_app_HelloJNI_intFromJNI( JNIEnv* env, jobject thiz )

• JNI naming rule:Java_[package_name]_[class_name]_[exported_function_name]

• JNIEnv* is the pointer to the VM, and jobject is a pointer to the implicit this object passed from the Java side.

48

Swift + Java

Calling from Java into Swift• Compile your Swift code as shared object library (.so),

load it at runtime • Use @_silgen_name or @_cdecl to create JNI

declaration • Call your top-level public Swift func whatever you want

internally

@_silgen_name(«Java_com_example_app__OurHello_intFromJNI") public func fooWhatever(env: UnsafeMutablePointer<JNIEnv>, jobj: jobject, x: jint, y: jint) -> jint { return x + y }

49

Swift + Java

Calling from Swift into Java

Hint: create dummy XCode project and add «jni.h» to Bridging Header

50

Swift + Java

Calling from Swift into Java

private func foo(env: UnsafeMutablePointer<JNIEnv>, jobj: jobject, value: jint) { let jni = env.memory.memory let methodName = "foo" let methodSignature = "(I)V" let javaClass = jni.GetObjectClass(env, jobj) let methodID = jni.GetMethodID(env, javaClass, methodName, methodSignature) let valueAsJValue = jvalue(i: value) var methodArgs: [jvalue] = [valueAsJValue] jni.CallVoidMethodA(env, jobj, methodID, &methodArgs) }

51

Swift + Java

Decoding calling from Swift into Java

… func foo(env: UnsafeMutablePointer<JNIEnv> …

env is passed by Java.

env JNIEnv JNINativeInterface

let jni = env.memory.memory

52

Swift + Java

Decoding calling from Swift into Java

let methodName = "foo" let methodSignature = "(I)V" let javaClass = jni.GetObjectClass(env, jobj) let methodID = jni.GetMethodID(env, javaClass, methodName, methodSignature)

Next we have to get an object and a method signature to be called according to Oracle JNI Docs

(I)V means that method receives one Int argument and returns Void

53

Swift + Java

Decoding calling from Swift into Java

• Actual calling should be done with Call(ReturnType)MethodA

• All args must be wrapped into jvalue struct

let valueAsJValue = jvalue(i: value) var methodArgs: [jvalue] = [valueAsJValue] jni.CallVoidMethodA(env, jobj, methodID, &methodArgs)

54

What’s good

• Code once, run everywhere (really)

• Speed

• Swift itself

55

Try the future today!Okay, maybe not that exciting…

56

What’s bad not ideal• App size overhead

• No macOS cross-compiling

• Compilation is not easy

• Native stuff (i.e. In-Apps) is not easy as well

• GPU is bad for dirty-rectangles type of apps

• MIPS, x86 and simulators are not supported yet

• No SIMD/NEON support

• No lldb (yet)

57

That’s it!

58

@s1ddok

@s1ddok

siddok@gmail.com