Adaptief uitbesteden en configurerentverbele/PhD_Verbelen.pdf · 2013-06-26 · Adaptief uitbesteden en configureren van complexe mobiele toepassingen Adaptive Offloading and Configuration

Adaptief uitbesteden en configurerenvan complexe mobiele toepassingen

Adaptive Offloading and Configurationof Resource Intensive Mobile Applications

Tim Verbelen

Promotoren: prof. dr. ir. B. Dhoedt, prof. dr. ir. F. De TurckProefschrift ingediend tot het behalen van de graad van Doctor in de Ingenieurswetenschappen: Computerwetenschappen

Vakgroep InformatietechnologieVoorzitter: prof. dr. ir. D. De ZutterFaculteit Ingenieurswetenschappen en ArchitectuurAcademiejaar 2012 - 2013

ISBN 978-90-8578-611-5NUR 986, 988Wettelijk depot: D/2013/10.500/44

Universiteit GentFaculteit Ingenieurswetenschappen en Architectuur

Vakgroep Informatietechnologie

Promotoren: prof. dr. ir. Bart Dhoedtprof. dr. ir. Filip De Turck

Universiteit GentFaculteit Ingenieurswetenschappen en Architectuur

Vakgroep InformatietechnologieGaston Crommenlaan 8 bus 201, B-9050 Gent, België

Tel.: +32-9-331.49.00Fax.: +32-9-331.48.99

Dit werk kwam tot stand in het kader van eenmandaat als Aspirant bij het Fonds voor

Wetenschappelijk Onderzoek Vlaanderen.

Proefschrift tot het behalen van de graad vanDoctor in de Ingenieurswetenschappen:

ComputerwetenschappenAcademiejaar 2012-2013

Dankwoord

“When I get sad, I stop being sad and be awesome instead. True story.”

–Barney Stinson, How I Met Your Mother

4 juli 1776. Op die dag werd de onafhankelijkheidsverklaring van de VerenigdeStaten opgesteld door Thomas Jefferson, en ondertekend door de vertegenwoordi-gers van de Amerikaanse kolonies. Elk jaar herdenkt men in de Verenigde Statennog steeds de onafhankelijkheid, en het einde van de Britse overheersing. Voor mijzal the Fourth of July voortaan een andere betekenis krijgen: het einde van mijndoctoraat. Alvorens jullie binnen te laten in de wondere wereld van mijn docto-raatsonderzoek, neem ik toch graag de tijd om eerst enkele mensen te bedanken.

Vooreerst was dit doctoraat nooit mogelijk geweest zonder financiering. Ikbedank dan ook het Fonds Wetenschappelijk Onderzoek (FWO), dat mij het ver-trouwen gaf en in mij investeerde door het toekennen van een aspirantenmandaat.Mijn onderzoek was ook niet mogelijk geweest zonder het harde werk van prof.Piet Demeester, door wiens toedoen IBCN is uitgegroeid tot een dynamische enboeiende onderzoeksgroep.

De belangrijkste drijfveren achter mijn onderzoek waren ongetwijfeld mijnpromotoren, prof. Bart Dhoedt en prof. Filip De Turck. Bedankt voor de tallozevergaderingen, vele brainstormsessies en ontelbare paper reviews. In het bijzonderwil ik Bart bedanken voor de steun, het vertrouwen en om mij telkens opnieuw temotiveren om toch maar nog een paper te schrijven. Ook een woord van dank aanTim Stevens en Pieter Simoens om telkens met een verfrissende blik mijn paperste bekijken, en steeds opnieuw constructieve feedback te geven.

Net als bij elke job komt bij een doctoraat ook (te) veel administratie kij-ken. Gelukkig kon ik hiervoor altijd rekenen op de hulp van Martine en Daviniavan het secretariaat, en de financiële adminstratie van Joke, Nathalie, Karien enBernadette. Ook het matriaal liet het af en toe eens afweten, maar gelukkig stondhet A-team steeds paraat met Brecht, Bert, Jonathan en Joeri. Bedankt ook aanSandra en Sabrina, voor het proper houden van de werkomgeving, en de immervriendelijke begroeting om de dag te beginnen.

Naast onderzoek werd ik afgelopen vier jaar ook ingezet in de talloze onder-wijsopdrachten die onze onderzoeksgroep verzorgt. Onder het motto “We makenezels tot renpaarden” probeerde ik de studenten iets bij te brengen tijdens Inge-nieursproject I met Bas en Steven, tijdens Softwareontwikkeling 2 in de rol vangood cop, bad cop met Olivier en Steven, tijdens VOP project met Klaas, tijdensODS project met Hendrik en Bert, en als thesisbegeleider met Hendrik en Niels.

ii

Tijdens de voorbije vier jaar had ik ook het geluk te mogen vertoeven in Bu-reau 2.21. Vele leuke herinneringen staan voor altijd in mijn geheugen gegrift,zoals de Counter Strike sessies, Friday drinks, Bureau-Bowlings, pokeravonden enhet fantastische Indische restaurant dat we dankzij Philip hebben leren kennen. Ikwil dan ook van harte iedereen bedanken met wie ik deze fantastische bureau hebmogen delen, voor de dagdagelijkse babbels tussendoor, de leuke middagpauzesen de talloze running jokes. Ik waag mij niet aan een opsomming van alle namen– better safe than sorry – maar enkele mensen wil ik er toch nog extra uitpikken.Als eerste Wim Van de Meerssche; ik denk dat geen enkele cache een hogere hitrate heeft dan Wim wanneer je een computervraag hebt. Ook bedankt aan Niels“den Hollander” Sluijs en Dieter “Marcel” Verslype om anderhalf jaar mijn vastesport-buddies te zijn, waarbij we vooral uitblonken in het “hijsen”. Klaas, dankzijjou heeft het woord kluchtig ook een gezicht gekregen! Tot slot wil ik ook OlivierVan Laere bedanken, die ik voor het eerst heb leren kennen in mijn 3e bachelorjaarals assistent voor het vak Softwareontwikkeling 2 en voor mijn VOP project, dieik twee jaar later opnieuw tegenkwam als mijn thesisbegeleider, en die ik daarnavier jaar als collega op de bureau heb mogen kennen. Mede door zijn toedoen kanik hier nu mijn dankwoord neerschrijven.

Ook over de bureaugrenzen heen was het leuk samenwerken met Bert en Elias.Met DREAMaaS hebben we alleszinds het project binnengehaald met de meestdubieuze naam. Ik bedank ook Philip, Niels B. en Farhan voor het ter beschikkingstellen van de data voor een aantal grafieken in dit proefschrijft en mijn presen-taties.

Naast het werk was er ook tijd voor ontspanning. Bedankt aan al mijn vriendenvan HSC Moeder Malheur, voor de schitterende studententijd, de vele inspirerendeavonden en de legendarische verhalen die we er aan hebben overgehouden. Inhet bijzonder Bad-Man, mijn beste maat, Giedo, mijn beste vriend, Sam-Joris,mijn beste vice (of was het omgekeerd), Vallie, mijn beste peter, Kerel, mijn bestepetekindje, Thomas, de beste brouwer, Kurmit, de beste cantor, Zazoe, de bestepenning, Romeo, de echte speechkampioen, Bender, die mij introduceerde in declub en Pippi, de enige die echt begreep waar mijn doctoraat over ging.

Ook bedankt aan alle anderen waarmee ik menige terras-, restaurant-, salon- ofkeukentafel gedeeld heb: Evelyne, Jacques, Marie-Anne, Laura, Tinnie, Gunther,Pieter en al wie ik nog vergeten ben.

De grootste steun kreeg ik wellicht van mijn familie: bedankt broer, zus, moeen va, voor de nimmer aflatende zorgen en de thuishaven waar ik steeds op terugkan vallen. Tot slot ben ik één persoon ontzettend dankbaar die de laatste tien jaarvan mijn leven verlicht heeft: Heleen, mijn allerliefste. Ik zie je graag!

Gent, 4 juli 2013Tim Verbelen

Table of Contents

Dankwoord i

Samenvatting xxiii

Summary xxvii

1 Introduction 11.1 The mobile (r)evolution . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Research Contributions . . . . . . . . . . . . . . . . . . . . . . . 41.4 Outline of this dissertation . . . . . . . . . . . . . . . . . . . . . 51.5 General applicability . . . . . . . . . . . . . . . . . . . . . . . . 81.6 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.6.1 Publications in international journals(listed in the Science Citation Index) . . . . . . . . . . . . 10

1.6.2 Publications in international conferences . . . . . . . . . 111.6.3 Poster publications in international conferences . . . . . . 121.6.4 Publications in national conferences . . . . . . . . . . . . 12

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2 Leveraging the Cloudlet for Immersive Collaborative Applications 152.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.2 Related work in Cyber Foraging . . . . . . . . . . . . . . . . . . 172.3 Use case: parallel tracking and mapping . . . . . . . . . . . . . . 192.4 A component-based cyber foraging approach . . . . . . . . . . . 202.5 An autonomic cloudlet management platform . . . . . . . . . . . 22

2.5.1 Execution Environment . . . . . . . . . . . . . . . . . . . 222.5.2 Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.5.3 Cloudlet . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.6 Dynamic configuration and deployment . . . . . . . . . . . . . . 232.7 Collaborative immersive applications . . . . . . . . . . . . . . . . 272.8 The future of the Cloudlet . . . . . . . . . . . . . . . . . . . . . . 28References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

iv

3 Dynamic Deployment and Quality Adaptation for Mobile AugmentedReality Applications 313.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333.3 Offloading middleware overview . . . . . . . . . . . . . . . . . . 35

3.3.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . 353.3.2 Offloading framework . . . . . . . . . . . . . . . . . . . 363.3.3 Application programming model . . . . . . . . . . . . . . 37

3.4 Optimal deployment calculation . . . . . . . . . . . . . . . . . . 393.5 Augmented reality use case . . . . . . . . . . . . . . . . . . . . . 433.6 Experimental validation . . . . . . . . . . . . . . . . . . . . . . . 46

3.6.1 Set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473.6.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.6.2.1 Scenario 1: CPU adaptation . . . . . . . . . . . 483.6.2.2 Scenario 2: Server discovery . . . . . . . . . . 503.6.2.3 Scenario 3: Bandwidth adaptation . . . . . . . . 513.6.2.4 Scenario 4: Network failure recovery . . . . . . 53

3.6.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 533.7 Conclusions and future work . . . . . . . . . . . . . . . . . . . . 54References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4 Graph Partitioning Algorithms for Optimizing Software Deploymentin Mobile Cloud Computing 594.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

4.2.1 Graph partitioning . . . . . . . . . . . . . . . . . . . . . 634.2.2 Task allocation on the grid . . . . . . . . . . . . . . . . . 64

4.3 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . 644.4 Algorithms description . . . . . . . . . . . . . . . . . . . . . . . 65

4.4.1 Integer Linear Programming (ILP) . . . . . . . . . . . . . 654.4.2 Multi level graph partitioning (MLKL) . . . . . . . . . . 66

4.4.2.1 Coarsening . . . . . . . . . . . . . . . . . . . . 664.4.2.2 Initial partitioning . . . . . . . . . . . . . . . . 674.4.2.3 Uncoarsening and refinement . . . . . . . . . . 67

4.4.3 Simulated Annealing (SA) . . . . . . . . . . . . . . . . . 704.4.3.1 Initial temperature T1 . . . . . . . . . . . . . . 704.4.3.2 Cooling function . . . . . . . . . . . . . . . . . 704.4.3.3 Epoch length L . . . . . . . . . . . . . . . . . 714.4.3.4 Stopping condition . . . . . . . . . . . . . . . . 71

4.4.4 Hybrid algorithm (KLSA) . . . . . . . . . . . . . . . . . 724.5 Evaluation Results . . . . . . . . . . . . . . . . . . . . . . . . . 72

4.5.1 Evaluation of MLKL, SA and KLSA . . . . . . . . . . . 734.5.2 Comparison with METIS 4.0 . . . . . . . . . . . . . . . . 784.5.3 Offloading scenario . . . . . . . . . . . . . . . . . . . . . 79

4.6 Conclusions and future work . . . . . . . . . . . . . . . . . . . . 81

v

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5 AIOLOS: middleware for improving mobile application performancethrough cyber foraging 875.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 905.3 Decision algorithm for offloading method calls . . . . . . . . . . 92

5.3.1 Optimize execution time . . . . . . . . . . . . . . . . . . 925.3.2 Optimize energy . . . . . . . . . . . . . . . . . . . . . . 93

5.4 Parameter estimation using history based profiles . . . . . . . . . 945.4.1 Estimating Tlocal and R . . . . . . . . . . . . . . . . . . 945.4.2 Estimating α, β and γ . . . . . . . . . . . . . . . . . . . 95

5.5 AIOLOS architecture . . . . . . . . . . . . . . . . . . . . . . . . 955.5.1 Android . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.5.2 OSGi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 965.5.3 AIOLOS architecture . . . . . . . . . . . . . . . . . . . . 96

5.6 AIOLOS and Android integration . . . . . . . . . . . . . . . . . 985.7 Development tools for AIOLOS . . . . . . . . . . . . . . . . . . 1005.8 Evaluation results . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5.8.1 Estimation accuracy . . . . . . . . . . . . . . . . . . . . 1045.8.2 Honza’s Chess . . . . . . . . . . . . . . . . . . . . . . . 1085.8.3 Photo Editor . . . . . . . . . . . . . . . . . . . . . . . . 1095.8.4 AIOLOS overhead . . . . . . . . . . . . . . . . . . . . . 1115.8.5 Development tools . . . . . . . . . . . . . . . . . . . . . 111

5.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

6 Adaptive Deployment and Configuration for Mobile Augmented Real-ity in the Cloudlet 1176.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1186.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1206.3 Use case: Mobile Augmented Reality . . . . . . . . . . . . . . . 1226.4 Cloudlet architecture . . . . . . . . . . . . . . . . . . . . . . . . 1246.5 Cloudlet platform design and implementation . . . . . . . . . . . 126

6.5.1 OSGi . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1276.5.2 Execution Environment . . . . . . . . . . . . . . . . . . . 1276.5.3 Node and Cloudlet Agent . . . . . . . . . . . . . . . . . . 1306.5.4 OSGi and Android integration . . . . . . . . . . . . . . . 131

6.6 Programming model . . . . . . . . . . . . . . . . . . . . . . . . 1326.7 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

6.7.1 The need for dynamic configuration . . . . . . . . . . . . 1346.7.2 The benefits of offloading . . . . . . . . . . . . . . . . . 136

6.8 Conclusions and future work . . . . . . . . . . . . . . . . . . . . 137References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

vi

7 Model-driven Configuration and Deployment Optimization in MobileCloud Computing 1437.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1447.2 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

7.2.1 Cyber foraging . . . . . . . . . . . . . . . . . . . . . . . 1467.2.2 Dynamic QoS adaptation . . . . . . . . . . . . . . . . . . 147

7.3 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1487.4 Application and infrastructure model . . . . . . . . . . . . . . . . 150

7.4.1 Application model . . . . . . . . . . . . . . . . . . . . . 1507.4.2 Infrastructure model . . . . . . . . . . . . . . . . . . . . 1527.4.3 Constraints . . . . . . . . . . . . . . . . . . . . . . . . . 153

7.5 Decision Algorithms . . . . . . . . . . . . . . . . . . . . . . . . 1547.5.1 Optimization goal . . . . . . . . . . . . . . . . . . . . . 1547.5.2 Objective function . . . . . . . . . . . . . . . . . . . . . 1557.5.3 KL-based search heuristic . . . . . . . . . . . . . . . . . 1567.5.4 Simulated annealing (SA) . . . . . . . . . . . . . . . . . 1577.5.5 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 158

7.6 Experimental results . . . . . . . . . . . . . . . . . . . . . . . . 1617.6.1 AR use case scenario . . . . . . . . . . . . . . . . . . . . 161

7.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

8 Conclusions and perspectives 171

List of Figures

1.1 Worldwide amount of shipments of smartphones versus desktopand portable PCs (2005–2017), data taken from IDC [2] . . . . . . 2

1.2 The 2012 shipments market share of mobile operating systems (a)and distribution of Android versions in use in March 2013 (b) illus-trate the fragmented mobile phone market and device heterogeneity. 3

1.3 Comparison of different processors using the Geekbench cross-platform processor benchmark [6]. . . . . . . . . . . . . . . . . . 3

1.4 Overview of the structure and the different chapters of this disser-tation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2.1 Identifying loosely coupled software components in an exampleimmersive application. . . . . . . . . . . . . . . . . . . . . . . . 21

2.2 Overview of the cloudlet platform. Execution environments arecontainers for application components. All communication be-tween components passes through the execution environments en-abling transparent monitoring and offloading. The Node Agentsaggregate monitoring information for each device. The cloudletagent analyzes the monitoring information and decides whetherthe deployment or configuration has to be adapted. . . . . . . . . 22

2.3 As the bandwidth decreases the configuration quality is lowered inorder to keep the achieved framerate between 15 and 20 FPS. Theconfiguration is changed within 10 to 20 seconds. . . . . . . . . . 25

2.4 By sharing components multiple devices can track and expand thesame map and recognize the same objects. . . . . . . . . . . . . . 26

2.5 The scene of the desk in (a) is explored by 3 devices resulting inthe mapped features points and recognized object in (b). By shar-ing the detected feature points of all devices, more feature pointsfrom multiple viewpoints are added to the map, leading to robustertracking and faster map expansion. . . . . . . . . . . . . . . . . . 27

viii

3.1 Overview of the offloading middleware. The Agent fetches themonitor information from the monitor bundles and calculates thebest deployment. The Bundle Installer distributes the componentsto the discovered servers. Our framework is built upon the OSGiframework in Java and uses R-OSGi for remote execution, jSLPfor server discovery and DBUS as an interface to the operatingsystem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.2 The application developer provides an application consisting of 6bundles. These can be used in two configurations that each offer adifferent quality level. Each bundle has an associated resource costwi and interacting bundles have a communication cost eij . Theframework will decide at runtime which configuration to deployand which bundles should be offloaded. . . . . . . . . . . . . . . 40

3.3 The best configuration is the one that matches with the availableresources on the mobile device (M0) and a possibly discoveredserver (M1) and the available bandwidth (L) while maximizingthe perceived quality. This configuration is then deployed on themobile device and the server. . . . . . . . . . . . . . . . . . . . . 40

3.4 Flowchart of the algorithm to calculate the best deployment out ofdifferent possible configurations. . . . . . . . . . . . . . . . . . . 42

3.5 Sequence diagram of the augmented reality shopping assistant. . . 443.6 The left screenshot shows the high quality configuration where the

book is visually recognized. In the right screenshot the barcodehas to be in view to recognize the object. . . . . . . . . . . . . . . 46

3.7 Scenario 1. The upper graph shows the CPU usage on the mobiledevice, the lower graph shows the frames analyzed per second ofthe shopping assistant application. When another application isstarted, a lower quality configuration is deployed. . . . . . . . . . 49

3.8 Scenario 2. The upper graph shows the CPU usage on the mo-bile device and the server, the middle graph shows the frames an-alyzed per second of the shopping assistant application and thelower graph shows the bandwidth usage. When a remote server isdiscovered, a higher quality configuration is deployed and compo-nents are outsourced. . . . . . . . . . . . . . . . . . . . . . . . . 50

3.9 Scenario 3. The upper graph shows the CPU usage on the mo-bile device and the server, the middle graph shows the frames an-alyzed per second of the shopping assistant application and thelower graph shows the bandwidth usage. As the available band-width is lowered, the framework deploys lower quality configura-tions that use less bandwidth. . . . . . . . . . . . . . . . . . . . . 52

3.10 Scenario 4. The upper graph shows the CPU usage on the mobiledevice and the server, the lower graph shows the frames analyzedper second of the shopping assistant application. When there is afailure of the server or the network, the framework recovers fromthe failure and switches back to a degraded version of the application. 53

ix

4.1 Work is offloaded from private infrastructure to a public cloud onpeak moments, reducing underutilization (and the cost) of the pri-vate infrastructure. . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.2 A cloudlet consisting of four devices connected by a wireless net-work sharing their resources. Application components are dis-tributed among all devices in the cloudlet, in order to enhance theuser experience of all users. . . . . . . . . . . . . . . . . . . . . . 62

4.3 This partition cannot be optimized when no moves exceeding thetarget parts maximum boundary are allowed. . . . . . . . . . . . . 69

4.4 In (a) the median of the resulting graph cut size is shown of theMLKL, SA and KLSA algorithms relative to the optimal ILP so-lution. Simulated annealing reaches the optimum in more than50% of the cases, while MLKL performs worse. KLSA is a bitbetter than MLKL but can not achieve the quality of SA. In (b) theexecution time of the different algorithms is shown. Better solu-tion quality comes at a price: KLSA and SA use respectively 2 to10 times more execution time than MLKL. . . . . . . . . . . . . . 74

4.5 In (a) the influence of a cutoff threshold is shown on the solutionquality relative to the MLKL algorithm without cutoff threshold,while in (b) the execution time is plotted. A lower cutoff thresholdlowers execution time at the cost of a lower solution quality. Theimprovement of solution quality by allowing moves with negativegains is lowered as graph size gets larger. . . . . . . . . . . . . . 75

4.6 In (a) the solution quality of SA relative to the MLKL algorithmis plotted, while (b) shows the execution time. For smaller graphsSA results in better partitions, but uses much more execution timethan MLKL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

4.7 In (a) the solution quality of KLSA relative to the MLKL algo-rithm is plotted, while (b) shows the execution time. KLSA resultsin slightly better partitions than MLKL, but this improvement di-minishes as the graph size gets larger. Regarding execution time itperforms better than SA, but it is still slower than MLKL. . . . . . 77

4.8 Solution quality of the MLKL algorithm relative to METIS 4.0 topartition the graph in 2, 4, 8, 16, 32 or 64 balanced parts. Ouralgorithm results in 10 to 40% lower graph cuts. . . . . . . . . . . 79

4.9 By increasing the parameter α the bandwidth on the wireless linkcan be influenced. For fine grained graphs composed of manysmall components shown in (a) the influence is more gradual thanfor coarser graphs shown in (b). . . . . . . . . . . . . . . . . . . 80

5.1 A mobile device’s capabilities are enhanced by offloading to aserver available either nearby or far away in the cloud. . . . . . . 89

5.2 Time needed for local execution Tlocal (on the left) and for remoteexecution Tremote (on the right). . . . . . . . . . . . . . . . . . . 93

x

5.3 Overview of the offloading middleware. The Proxy Manager cre-ates proxies for the registered services (s1 and s2) of the appli-cation bundles. These proxies can switch between local and re-mote method execution and monitor all method calls. The BundleMonitor aggregates this monitor information to build profiles of allmethods called. This information is used by the Offloading Logicto instruct the Deployment Manager to outsource bundles, or toupdate the proxy policies when to offload. . . . . . . . . . . . . . 97

5.4 An OSGi container is embedded within the Android application.Offloadable parts of the Android application are packaged as OSGibundles and deployed in the embedded OSGi container where theAIOLOS framework handles the cyber foraging functionalities.The Android application can lookup services through the OSGiruntime, and can also register callback services with the OSGi run-time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

5.5 Bundles are formed from the class dependency graph. Classes Cand H are marked as offloadable by the developer, and will formapplication bundles with the classes that only depend on them.Classes A, G and J depend on the Android SDK and will be putin the Android application bundle that cannot be offloaded. More-over J will be marked as a callback class from which a callbackservice is formed. Classes B and F are shared between the differ-ent bundles and are packaged as a shared bundle. . . . . . . . . . 102

5.6 Two Android applications running with the AIOLOS framework.On the left Honza’s Chess [30] and on the right a Photo Editorapplication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

5.7 The 0.1 to 0.9 percentiles of the relative error of the estimatedremote execution time using a moving average (MA), a linear leastsquares (LLS) and a RANSAC estimation strategy. The LLS andRANSAC use a history of k = 5, 10 and 15 calls. RANSAC offersthe best results for larger history k. . . . . . . . . . . . . . . . . . 105

5.8 Estimated versus the actual monitored execution time of a blur op-eration on images of different size, using the RANSAC algorithmwith k = 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

5.9 The ability to adapt to a bandwidth change in time using a movingaverage (MA), a linear least squares (LLS) and a RANSAC esti-mation strategy. The LLS and RANSAC use a history of k = 5, 10and 15 calls. The MA approach adapts after one method call, theLLS and RANSAC need respectively k and k/2 method calls be-fore the bandwidth change is taken into account. . . . . . . . . . . 107

xi

5.10 Execution time of the Chess AI algorithm for different chess movesfor local execution on both the Motorola Milestone and the LG Op-timus 2x, as well as remote execution on a remote server deployedin the LAN network and one deployed in the Amazon EC2 Cloud.Error bars are omitted as standard deviations are too small to bevisible. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

5.11 Execution time of a sharpening filter on a 500×500, 1000×1000and 1500×1500 RGB image for local execution on both the Mo-torola Milestone and the LG Optimus 2x, as well as remote exe-cution on a remote server deployed in the LAN network and onedeployed in the Amazon EC2 Cloud. In the case of the 1500x1500image local execution on the Milestone is omitted for the clarityof the figure and was more than 30s. . . . . . . . . . . . . . . . . 110

6.1 Static cloudlets can be provided by a corporation in a corporatecloudlet, or by a service provider in the mobile network. Ad hoccloudlets can be formed in the home network or within a railwaycarriage. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

6.2 The augmented reality application tracks feature points in the videoframes (right) to enable overlaying of 3D objects (left). . . . . . . 123

6.3 The component diagram of the AR application. The VideoSourceand the Renderer (dark gray) are fixed on the mobile device. TheRelocalizer and Tracker (gray) have real-time constraints (≤ 50 ms).124

6.4 The application components are distributed among nodes in twocloudlets. An ad hoc cloudlet consisting of a mobile phone, alaptop and a desktop computer, and a distant elastic cloudlet ina public cloud infrastructure. . . . . . . . . . . . . . . . . . . . . 125

6.5 The Execution Environment proxies all component interfaces inorder to monitor all (local and remote via RemoteServiceAdmin)service method calls. The Configuration Admin and Metatype ser-vice enable the EE to dynamically configure components, whichare defined using the Declarative Services specification. . . . . . . 127

6.6 The Node Agent and Cloudlet Agent bundles run on an OSGi run-time in a separate process. The Cloudlet Agent bundle is onlypresent when the current node is also the master of the cloudlet. . 131

6.7 On the Android platform a separate application hosts the NodeAgent and the Cloudlet Agent bundles, that run in a backgroundprocess using an Android Service. Applications having offload-able and/or configurable components deploy these on an OSGiruntime hosting the EE, which proxies the provided interfaces.Components requiring the Android internals (e.g. drawing to screen)are embedded in the Android Activity and also provide their inter-faces through the EE. . . . . . . . . . . . . . . . . . . . . . . . . 132

xii

6.8 Overview of the steps done by the build system. Annotated sourcecode is preprocessed and application components are generated.These are bundled with the middleware components to create theresulting Android .apk which can be deployed on the Android OS. 134

6.9 In order to meet the constraint of 50 ms to process a frame, theHTC Desire can process at most 200 feature points per frame,while the LG Optimus can handle 450 feature points. . . . . . . . 135

6.10 The time needed to track a frame as a function of the number offeature points. Using a negative feedback loop, the frameworkadapts the parameter configuration at runtime in order to meet themaximum processing time constraint. . . . . . . . . . . . . . . . 136

6.11 The time to refine the map when a new keyframe is added, as afunction of the total number of keyframes in the map. Offloadingthe Mapper component results in a speedup factor of 10. . . . . . 137

7.1 The overall architecture of our adaptive framework. Applicationcomponents are deployed in containers – called Execution En-vironments (EE) – that run on different devices. Adaptation isachieved by monitoring the EEs and take action through (re)configurationor migration of components at runtime. . . . . . . . . . . . . . . 148

7.2 An example of a component-based application. Each componentconsumes and/or provides service interfaces. Components 1 and 4also offer a parameter that can be configured at runtime. . . . . . . 151

7.3 The actual behavior of the application is captured in UML se-quence diagrams, showing the actual method calls, loops and con-ditional flows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

7.4 The two UML sequence diagrams shown in Figure 7.3 are splitup in three sequences. The conditional sequences are split up ina separate sequence for each condition, and loops are modeled byan annotation stating how many times each method is called withinthe sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

7.5 The weight function S(x) used for constructing the objective func-tion. The weight function introduces a large penalty factor whenx > b. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156

7.6 The KL algorithm produces close to optimal results for small prob-lem instances, but solution quality quickly degrades as the problemsize increases. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

7.7 SA finds the optimal solution in more than 85% of the cases forproblem sizes up to 15 components. Performance degrades slowlyas the problem size increases. . . . . . . . . . . . . . . . . . . . . 160

7.8 The KL algorithm is about hundred times faster than SA, but at thecost of solution quality. . . . . . . . . . . . . . . . . . . . . . . . 161

7.9 The augmented reality application tracks feature points in the videoframes (right) to enable the overlay of 3D objects (left). . . . . . . 162

xiii

7.10 The AR application is implemented as a set of loosely coupledsoftware components. The VideoSource and Tracker componenteach expose a configuration parameter affecting the QoS of theapplication. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162

7.11 The augmented reality application consists of three sequences. In(a) the tracking and rendering sequence is shown, which processesthe video frames. The map refinement sequence is shown in (b),and (c) depicts the object recognition sequence. . . . . . . . . . . 163

7.12 Monitored execution times of the tracker (a) and object recognition(b) sequences, for different configurations. . . . . . . . . . . . . . 165

List of Tables

1.1 An overview of the contributions per chapter in this dissertation.The light gray bullets are quickly addressed in the chapter but dis-cussed more thoroughly elsewhere. . . . . . . . . . . . . . . . . . 6

2.1 Overview of cyber foraging systems and their characteristics, basedon [9] . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.1 Different configurations for the AR use case. . . . . . . . . . . . . 453.2 Measurements of the CPU time spent (in ms) in the components

for each application loop for the different configurations on theN900. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

4.1 Simulated annealing parameter values used in our SA refinementalgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.1 Values of the parameters α, β and γ for both the Motorola Mile-stone and the LG Optimus 2x when offloading to a server in theLAN network and to the Amazon EC2 cloud. . . . . . . . . . . . 108

List of Acronyms

0-9

3G 3rd Generation4G 4th Generation

A

AD Attribute DefinitionAPI Application Programming InterfaceAR Augmented Reality

B

BF Brute ForceBOF Bijzonder Onderzoeksfonds

C

CA Cloudlet AgentCOM Component Object ModelCPU Central Processing Unit

E

EE Execution Environment

xviii

F

FM Frequency ModulationFPS Frames Per SecondFWO-V Fonds voor Wetenschappelijk Onderzoek Vlaanderen

G

GHz GigahertzGOA Geconcerteerde OnderzoeksactieGPS Global Positioning SystemGSM Global System for MobileGUI Graphical User Interface

H

HEM Heave Edge MatchingHMD Head Mounted Display

I

IDC International Data CorporationIDE Integrated Development EnvironmentIEEE Institute of Electrical and Electronics EngineersILP Integer Linear ProgrammingINTEC Information TechnologyIPC Inter Proces CommunicationITU International Telecommunication Union

J

JNI Java Native InterfaceJVM Java Virtual MachineJPEG Joint Photographic Experts Group

xix

K

kbps kilobit per secondKL Kernighan-Lin

L

LAN Local Area NetworkLLS Linear Least Squares

M

MA Moving AverageMAPE Monitor-Analyze-Plan-ExecuteMB MegabyteMHz MegahertzMJPEG Motion JPEG

N

NA Node Agent

O

OCD Object Class DefinitionOS Operating SystemOSGi Open Service Gateway initiative

P

PC Personal ComputerPhD Philosophiae DoctorPTAM Parallel Tracking And Mapping

xx

Q

QoE Quality of ExperienceQoS Quality of Service

R

RAM Random Access MemoryRANSAC RANdom SAmple ConsenusRFC Request for CommentsRGB Red Green BlueR-OSGi Remote OSGi

S

SA Simulated AnnealingSDK Software Development KitSIFT Scale Invariant Feature TransformSLA Service Level AgreementSLP Service Location ProtocolSURF Speeded Up Robust Features

T

TC Traffic Control

U

UML Unified Modelling LanguageUSB Universal Serial Bus

V

VLSI Very Large Scale IntegrationVM Virtual Machine

xxi

W

WAN Wide Area NetworkWiFi Wireless FidelityWLAN Wireless Local Area Network

X

XML eXtensible Markup Language

Samenvatting– Summary in Dutch –

De uitvinding van de eerste draagbare mobiele telefoon in 1973 heeft ons levendrastisch veranderd. Mobiele telefoons worden niet meer uitsluitend gebruikt voorhet voeren van telefoongesprekken of het verzenden van tekstberichten, maar zijnuitgegroeid tot “slimme” telefoons, uitgerust met een krachtige processor, verbin-ding met het Internet via WiFi of 4G, verschillende sensoren zoals gyroscopenvoor interactie, een GPS-module voor localisatie, enz. Smartphones overspoelende elektronicamarkt, en sinds 2011 worden er jaarlijks zelfs meer smartphonesverkocht dan laptop en desktop PCs samen. Smartphones bieden ook de moge-lijkheid om diverse applicaties te downloaden en te installeren. Vandaag de dagbestaan er verschillende digitale applicatiewinkels, die een groot aantal toepas-singen aanbieden in een breed scala van categorieën, zoals spelletjes, socialeme-diatoepassingen, webbrowsers, diensten gebaseerd op locatie-informatie, muziek-en videospelers, enz.

Als gevolg van de toenemende populariteit van smartphones en het gemak omtoepassingen in een appmarkt te publiceren, richten meer en meer ontwikkelaarszich tot het mobiele platform. Bij de ontwikkeling van mobiele toepassingen dientmen wel rekening te houden met een aantal problemen. Eerst en vooral zullen mo-biele apparaten altijd beperkt zijn qua capaciteiten in vergelijking met vaste infra-structuur zoals desktop PCs of servers, als gevolg van beperkingen met betrekkingtot gewicht, grootte, levensduur van de batterij en warmteafvoer. Daarom schie-ten mobiele toestellen nog steeds te kort om complexe toepassingen uit te voerendie veel rekenkracht en/of geheugen vereisen. Ten tweede is er een enorm aantalverschillende types toestellen op de markt, die bovendien door de snelle techno-logische vooruitgang allemaal verschillende specificaties hebben. Ontwikkelaarsdienen dus rekening te houden met deze diversiteit aan toestellen, en de toepas-sing te testen en af te stemmen op elk type apparaat. Ten derde moet men ookrekening houden met de mobiliteit van de gebruiker, waardoor de toepassing uit-gevoerd kan worden in een veranderende context. De kwaliteit van de draadlozenetwerkverbinding is bijvoorbeeld afhankelijk van de locatie van de gebruiker, ende beschikbare bandbreedte en netwerkvertraging kunnen sterk variëren in de tijd.

In dit proefschrift worden deze problemen aangepakt en wordt de taak vande mobiele applicatie-ontwikkelaar verlicht door de applicatie adaptief te maken,waardoor deze zichzelf optimaal kan aanpassen aan de context. Om tegemoet tekomen aan de beperkte rekenkracht van mobiele toestellen, kunnen delen van de

xxiv SAMENVATTING

applicatie uitbesteed worden aan vaste infrastructuur in het netwerk. Omdat detoepassing ook moet werken op diverse toestellen, wordt deze automatisch gecon-figureerd op basis van de mogelijkheden van het toestel.

Om deze aanpassingsmechanismen te ondersteunen, werd in dit proefschrifteen middleware platform ontworpen en geïmplementeerd dat de applicatie be-heert. We veronderstellen dat de toepassing is opgebouwd uit een aantal zwakgekoppelde componenten, waardoor de middleware op transparante wijze dezecomponenten kan distribueren en configureren. Naast het distribueren en confi-gureren van applicatiecomponenten, staat de middleware ook in voor het vindenvan beschikbare infrastructuur in het netwerk. Daarnaast houdt het platform ookgegevens bij over de uitvoeringstijd van componenten en hoe deze onderling com-municeren, om zo het gedrag van de applicatie te modelleren en te kunnen voor-spellen voor de nabije toekomst.

Om te beslissen welke componenten uitbesteed moeten worden, werden eenaantal algoritmen ontworpen om de optimale verdeling van de applicatie te vin-den, gegeven een aantal mogelijke plaatsen waar de componenten kunnen wordenuitgevoerd. Aangezien de bandbreedte van het draadloos netwerk vaak een be-perkende factor is, hebben we ons in een eerste fase gericht op het minimaliserenvan de nodige bandbreedte, rekening houdend met de beperkingen in rekenkracht.Initieel werd dit probleem aangepakt met behulp van lineaire programmering,maar omdat de complexiteit van dit probleem exponentieel toeneemt voor een stij-gend aantal componenten, werden een aantal heuristische zoekalgoritmen ontwor-pen. Twee heuristische optimalisatietechnieken zijn beschouwd: een eerste geïn-spireerd op het Kernighan-Lin (KL) graaf-partitioneringsalgoritme, en een tweedeop basis van Simulated Annealing (SA). Experimentele resultaten tonen aan datbeide technieken voor kleine probleeminstanties vaak de theoretisch optimale op-lossing terugvinden, en dat deze beter schalen dan lineaire programmering in ter-men van uitvoeringstijd. In het algemeen geeft SA oplossingen terug die dichterbij het optimum liggen dan de KL gebaseerde technieken, ten koste van langereuitvoeringstijd.

Om ook de configuratie van de applicatie in rekening te brengen, werden ineen tweede fase de voorgestelde algoritmen uitgebreid om zowel de distributie alsde configuratie van de componenten te optimaliseren. In plaats van de benodigdebandbreedte te minimaliseren, wordt nu de beschikbare bandbreedte beschouwdals een beperking van de infrastructuur. Het doel hier is om de Quality of Service(QoS) die wordt waargenomen door de eindgebruiker te maximaliseren, dewelkewordt beïnvloed door de configuratie van de toepassing. Naast de restricties inhe-rent aan de infrastructuur, zoals beperkingen in rekenkracht of gelimiteerde band-breedte van het netwerk, kan de ontwikkelaar ook bepaalde applicatie-specifiekebeperkingen opleggen, bijvoorbeeld door het opleggen van deadlines bij het uit-voeren van bepaalde methoden. Door deze aanpak wordt de applicatie aangepastaan de capaciteiten van het toestel waarop deze wordt uitgevoerd, en kan eventueelde kwaliteit verbeterd worden wanneer uitbesteden mogelijk is.

Een belangrijk aspect van middleware is het programmeermodel, hetgeen be-paalt hoe de functionaliteit wordt aangeboden aan de ontwikkelaar. Om de be-

SUMMARY IN DUTCH xxv

lasting van de applicatie-ontwikkelaar te minimaliseren, werd een programmeer-model ontwikkeld dat gebaseerd is op code-annotaties. Dit model is eenvoudig tegebruiken en laat de ontwikkelaar toe om een applicatiecomponent te definiërendoor slechts één annotatie toe te voegen. Bijkomende annotaties kunnen dan wor-den gebruikt om configureerbare parameters, deadlines, enz. te specificeren. Bijhet compileren van het programma worden deze annotaties verwerkt en wordenautomatisch de verschillende softwarecomponenten gegenereerd en geïntegreerdmet het middleware platform.

Om de middleware en het programmeermodel te valideren werden een aan-tal toepassingen ontwikkeld die in dit proefschrift gebruikt worden in ter evalu-atie. Om de voordelen van het uitbesteden van componenten te illustreren, werdenmet behulp van het programmeermodel een aantal veelvoorkomende smartphone-toepassingen ontwikkeld, zoals een schaakspel en een fotobewerkingsprogramma.Experimenten tonen aan dat een versnelling van 20 tot 90% mogelijk is, uiteraardafhankelijk van de rekenkracht van het mobiel toestel en de server infrastructuur,alsook van de netwerkconnectiviteit. Het volledige potentieel van het middleware-platform wordt pas duidelijk wanneer het gaat om complexe applicaties die instaanvoor het verwerken van media, aangezien deze hoge eisen hebben qua rekenkrachten vaak ook in ware tijd moeten werken. Daarom werd ook een mobiele aug-mented reality (AR) toepassing ontwikkeld met behulp van recente computervi-sietechnieken. Experimenten laten zien hoe de middleware in staat is om de appli-catie optimaal te configureren op verschillende mobiele toestellen, hoe de kwaliteitverbeterd wordt door het uitbesteden rekenintensieve componenten en hoe de ap-plicatie aangepast wordt aan de veranderende context.

Doorheen dit proefschrift worden de verschillende aspecten van de ontwikke-ling van mobiele toepassingen behandeld, en worden oplossingen aangerijkt vooralle betrokken actoren: enerzijds wordt voor de applicatie-ontwikkelaar een een-voudig programmeermodel aangeboden voor de ontwikkeling van adaptieve appli-caties, en anderzijds wordt voor de eindgebruikers de best mogelijke kwaliteit vande applicatie aangeboden. In de toekomst zal door de verdere schaalverkleiningvan de hardware nieuwe randapparatuur ontstaan, zoals slimme horloges, head-mounted displays of slimme textiel, waardoor de problemen aangepakt in dit doc-toraatsonderzoek zich nog duidelijker zullen manifesteren. De verworven inzichtenvan dit proefschrift vormen dan ook een goede basis voor verder onderzoek in dezerichting.

Summary

Since the demonstration of the first handheld mobile phone in 1973, the mobilephone ecosystem has undergone drastic changes. Mobile phones are no longersolely used for making phone calls or sending text messages, but have become“smart” phones, equipped with a powerful processor, Internet connectivity by WiFior 4G, various sensors such as accelerometers for user control, a GPS module forlocalization, etc. Smartphones have overwhelmed the electronic device market,and from 2011 on, smartphone shipments overtook the total number of portableand desktop shipments all together. Smartphones also offer the ability to downloadand install various 3rd party applications. Today, various mobile app markets exist,that offer a myriad of applications in a wide range of categories such as games,social networking and messaging clients, web browsers, location-based services,audio and video players, etc.

Due to the increasing popularity of smartphones, and the ease of publishingapplications in the app markets, more and more developers target their applicationto the mobile platform. However, various challenges have to be taken into accountwhen developing mobile applications. First, due to the limitations in weight, size,battery life and heat dissipation, mobile devices will always be more resource con-strained than fixed infrastructures such as desktop PCs or servers. Therefore, mo-bile devices still fall short to execute complex applications that require heavy CPUprocessing or have high memory requirements. Second, due to the rapid changesin mobile hardware, a wide range of mobile devices exists, all with different ca-pabilities. Mobile application developers have to take into account this deviceheterogeneity, and test and fine-tune their application on each device. Third, dueto the user mobility, the application can be executed in a wide variety of contexts.For example wireless network connectivity is location dependent, and the networkbandwidth and latency will vary heavily over time.

In this dissertation, we aim to tackle these challenges, and alleviate the taskof the mobile application developer, by making the application self-adaptive. Inorder to mitigate the resource limitations of the mobile device, one adaptationmechanism is to offload parts of the application to fixed infrastructure in the net-work. To take into account device heterogeneity, a second adaptation mechanismis to automatically configure the application depending on the device capabilities.Of course, due to the changing context, the deployment and configuration of theapplication has to change over time.

To support these adaptation mechanisms, we have designed and implemented across-platform middleware framework. We assume the application is built up from

xxviii SUMMARY

a set of loosely coupled components, which enables the middleware to transpar-ently distribute and configure separate components at runtime. Besides offload-ing and reconfiguring the application components, the middleware also handlesresources discovery, in order to find available infrastructure in the network. Addi-tionally, application components are monitored at runtime and an application pro-file is maintained, capturing the runtime behavior which is used to predict futureresource usage of the application, and to detect changes in the context.

To decide which components should be offloaded, we have developed a num-ber of algorithms to optimize the deployment of the application components givena collection of available resources. As wireless network bandwidth is often a lim-iting factor, we first focused on an integer linear programming (ILP) solver, thatminimizes the bandwidth used, while taking into account the limited processingresources of the devices. As the complexity of this problem increases exponen-tially with an increasing number of components, we have designed heuristic searchalgorithms, enabling to calculate a good solution at runtime. Two heuristic opti-mization techniques are considered: the first inspired by the Kernighan-Lin (KL)graph partitioning algorithm, and the second based on Simulated Annealing (SA).Experimental results show that both techniques provide near to optimal results forsmall problem instances, and scale better in terms of execution time than ILP. Ingeneral, the SA based algorithms return solutions closer to the optimum than theKL based techniques, at the cost of longer execution time.

In order to take into account application configuration, we extended the pro-posed search heuristics to co-optimize both component deployment and configura-tion. Here, instead of minimizing the required bandwidth, the available bandwidthis considered as an infrastructure constraint. The goal is to maximize the Qualityof Service (QoS) as perceived by the end user, which is influenced by the applica-tion configuration. Next to the limitations on processing capabilities and networkbandwidth, also application specific constraints can be defined by the applicationdeveloper, for example by imposing execution deadlines on particular methods.This approach will adapt the application configuration depending on the mobiledevice capabilities, and by opportunistically offloading the QoS is enhanced whenpossible.

An important aspect of middleware is the programming model, which defineshow the functionality is exposed to the application developer. To minimize theburden on the application developer, our programming model is based on codeannotations. The programming model is easy to use, as the developer can declar-atively specify an offloadable component by only adding one annotation. Moreannotations can be used in order to declare configurable parameters, performanceconstraints, etc. At build time the annotations are processed and automaticallysoftware components are constructed and integrated with the middleware frame-work.

To validate the middleware and the programming model, we have developed anumber of applications that are used in this dissertation for evaluation. To illustratethe benefits of offloading, a component-based implementation of common smart-phone applications such as a chess game and a photo editor application are used,

SUMMARY xxix

showing a speedup of 20 to 90% depending on the processing power of the mo-bile device and the server infrastructure and the network connectivity. However,the potential of our middleware platform only becomes apparent when dealingwith complex media processing applications, that have stringent resource require-ments and often soft real-time constraints. Therefore we also developed a mobileaugmented reality (AR) application implementing state-of-the-art computer visiontechniques. Experiments show how the middleware is able to configure the ap-plication for different mobile devices, to offload CPU intensive components toenhance the QoS, and to adapt to changing context.

Throughout this dissertation, various aspects of software development of mo-bile applications are covered, addressing all actors involved: from the applicationdeveloper that is offered a helpful programming model, to the end users who’sperceived quality is optimized. As the miniaturization of electronic devices con-tinues, new peripherals will arise such as smart watches, head-mounted displays orsmart textiles. Due to the even more stringent restrictions on weight, form factorand heat dissipation of these wearable devices, the challenges regarding applica-tion development addressed in this work will become even more apparent, and theinsights gained in this dissertation form a sound basis for further research in thisdirection.

1Introduction

“If you can dream it, you can do it.”

–Walt Disney (1901 - 1966)

1.1 The mobile (r)evolutionBack in 1973, John F. Mitchell and Martin Cooper, both working at Motorola,demonstrated the first handheld mobile phone. Ten years later, in 1983, Motorolareleased the DynaTAC 8000x, the first handset to be commercially available. Thedevice weighted over 1 kg and offered 30 minutes of talk time and 8 hours ofstandby. Now, thirty years later, ITU reports that the number of worldwide mo-bile phone subscriptions exceeds 6 billion, accounting for more than 85% of theglobal population [1]. In the mean time, the mobile phone ecosystem has under-gone drastic changes. Mobile phones are no longer solely used for making phonecalls or sending text messages, but have become “smart” phones, equipped witha powerful central processing unit (CPU), Internet connectivity by WiFi or 4G,various sensors such as accelerometers for user control, a Global Positioning Sys-tem (GPS) module for localization, etc. As illustrated in Figure 1.1, the popularityof smartphones is still increasing, and the number of shipped smartphones over-took the number of portable and desktop PCs in 2011. By 2017, IDC expects theworldwide amount of smartphone shipments will exceed 1.5 billion [2].

An important driver of smartphone adoption is the development of mobile ap-plication stores, that offer the ability to download and install various 3rd party

2 CHAPTER 1

Figure 1.1: Worldwide amount of shipments of smartphones versus desktop and portablePCs (2005–2017), data taken from IDC [2]

applications. Currently mobile application markets offer a myriad of applicationsin a wide range of categories such as games, social networking and messagingclients, web browsers, location-based services, audio and video players, etc. Gart-ner forecasts over 80 billion mobile application downloads worldwide for 2013,exceeding 300 billion downloads by 2016 [3]. Application developers can publishtheir applications to these stores, being rewarded by a revenue sharing of the sales,and/or by revenues from in-app purchases or advertisements. Therefore, more andmore software developers target mobile devices such as smartphones and tabletsas platform for their applications.

1.2 Problem Statement

Due to the explosive growth of the mobile market many opportunities are given,but also many challenges are posed to mobile application developers. A first chal-lenge is imposed by the highly fragmented mobile phone market, where manyhardware and operating system vendors strive for the market leadership. Look-ing at 2012 smartphone shipments by mobile operating system, as shown in Fig-ure 1.2(a), currently Android and iOS stand out capturing 87.6% of the market [4].However, a lot of devices are still in use running Symbian or Windows Mobile,and many new operating systems are being released next year, such as Firefox OS,Tizen and Ubuntu Phone, which could increase the competition. Also within oneoperating system, one witnesses a fragmentation due to the different versions inuse. For example looking at the distribution of the different Android versions in

INTRODUCTION 3

(a) 2012 mobile OS market share [4] (b) March 2013 Android versions in use [5]

Figure 1.2: The 2012 shipments market share of mobile operating systems (a) and distri-bution of Android versions in use in March 2013 (b) illustrate the fragmentedmobile phone market and device heterogeneity.

use in March 2013 as illustrated on Figure 1.2(b), only 16.5% of the devices runthe most recent Android version. This is due to the fact that many legacy devicesin use do not have the required hardware capabilities to update to the latest version.Thus, in order to reach a large part of the smartphone users, application develop-ers are inclined to target multiple operating systems, and to be compatible withmultiple versions.

A second challenge arises from restrictions on weight, size and heat dissipationwhen dealing with mobile devices, which impose limitations on mobile devicesÂt’

Figure 1.3: Comparison of different processors using the Geekbench cross-platform pro-cessor benchmark [6].

4 CHAPTER 1

computational and memory resources, as well as on battery life. Despite rapidadvances in hardware technologies, due to these restrictions mobile devices willalways lag behind their non-mobile counterparts. This is illustrated by Figure 1.3,showing the Geekbench cross-platform processor benchmark [6] comparing pro-cessors found in smartphones to processors assembled in current laptop, desktopor server machines. Although mobile processor capabilities have increased expo-nentially the past five years, current server processors achieve a factor 10 higherperformance. Therefore, mobile devices still fall short to execute complex ap-plications, i.e. involving state-of-the-art media processing algorithms or big dataanalysis applications.

A third challenge for mobile application developers is to take into account theuser mobility, as the application should work in a wide number of contexts. Animportant factor in this regard is the network. Due to the popularity of variousInternet-based services such as social networks and media, a lot of applicationsrequire Internet connectivity. Although nowadays wireless network coverage isvastly improving in many areas, various network technologies still suffer fromlimited and varying bandwidth and/or high and variable latencies. Therefore, theoperating context of the application can have a huge impact on the Quality ofService (QoS) perceived by the end user, and should ideally be taken into accountby the application developer.

1.3 Research Contributions

The main research contributions of this dissertation address the challenges in-volved with mobile application development. The key goal is to make mobileapplications self-adaptive, optimizing their behavior taking into account variousaspects, such as the capabilities of the mobile devices, resources available in thenetwork, characteristics of the network connection, constraints imposed by the ap-plication developer, preferences of the user, etc.

The following contributions are presented:

Contribution 1 A middleware framework for transparent component offloadingand configuration.In order to enable self-adaptiveness of applications, a first contribution fo-cuses on the design and implementation of a middleware framework thatprovides adaptation mechanisms. In this dissertation two adaptation mech-anisms are considered. A first adaptation mechanism is to offload parts ofthe application to a remote infrastructure in the network. This enables toreduce the load on the mobile device, which can lead to lower energy con-sumption and/or faster execution times. A second adaptation mechanism isto change the configuration of the application, either by switching between

INTRODUCTION 5

different components, or by tuning configurable parameters. Throughoutthis dissertation, we present a cross-platform middleware supporting theseadaptations, as well as other core building blocks such as resource discovery,monitoring, etc.

Contribution 2 Algorithms for adaptive software deployment and configuration.To decide which deployment and configuration is optimal given a certaincontext at runtime, we present heuristic search algorithms for solving thisoptimization problem. Different optimization goals are considered, suchas minimizing bandwidth, execution time or energy usage, or maximizingQoS. The performance of these heuristics is evaluated by comparing theirsolution quality to the theoretic optimal solution given by the integer linearprogramming (ILP) solution or a brute force approach.

Contribution 3 Developer tools for creating component-based adaptive applica-tions.An important aspect of middleware is how the functionality is exposed to theapplication developer, the so called “programming model”. To minimize theburden on the application developer, we present a programming model basedon code annotations, allowing to declaratively specify software componentsand their properties. At build time the annotations are processed and auto-matically software components are created that are usable at runtime withour framework.

Contribution 4 Apply optimization techniques on a state-of-the-art augmentedreality (AR) use case.Since the concepts in this dissertation are not yet commonly used in mo-bile application development, apps that could benefit from the frameworkare scarce, and therefore evaluations are often limited to small and trivialapplications such as a chess game, an image editor program, etc. In orderto validate the effectiveness of our optimization techniques on a demandinguse case, we designed and implemented an augmented reality use case, us-ing state-of-the-art computer vision algorithms, that better reflects the needsand requirements of next-generation mobile applications, and is used forevaluating our framework.

1.4 Outline of this dissertation

This dissertation is composed of a number of publications that were realized withinthe scope of this PhD. The selected publications provide an integral and consistentoverview of the work performed. The different research contributions were de-tailed in Section 1.3 and the complete list of publications that resulted from this

6 CHAPTER 1

work is presented in Section 1.6.Within this section we give an overview of the remainder of this dissertation

and explain how the different chapters are linked together. Fig. 1.4 positions thedifferent contributions that are presented in each chapter (Ch.). Table 1.1 showsthe contributions that were highlighted in Section 1.3 and indicates which weretargeted per chapter.

Table 1.1: An overview of the contributions per chapter in this dissertation. The light graybullets are quickly addressed in the chapter but discussed more thoroughly else-where.

Ch.2 Ch.3 Ch.4 Ch.5 Ch.6 Ch.7Middleware platform fordynamic redeployment • •- runtime configuration • •- monitoring and prediction •

Optimization algorithms for- minimizing bandwidth •- maximizing QoS • •

Programming model • • •

AR use case • • •

We start by showing the need for dynamic deployment and configuration formobile immersive applications in Chapter 2. As use case, a collaborative aug-mented reality application is presented, based on state-of-the art computer visionalgorithms. An overview of our middleware platform and the accompanying pro-gramming model is given, as well as some preliminary evaluation results, forwhich more implementation details are given in Chapter 6, and a more throughoutdiscussion on the decision algorithm in Chapter 7.

Chapter 3 presents our first middleware system, offering basic resource discov-ery, bundle offloading, resource monitoring etc. A decision algorithm is proposed,choosing the best of a set of predefined configurations for which a partitioningexists restricting the CPU load on the device, and consuming less bandwidth thanavailable. To calculate the partitioning minimizing the bandwidth usage for oneconfiguration, an integer linear programming (ILP) solver is used. The system isevaluated using an object recognition application in a number of runtime scenarios,such as variations in available bandwidth or CPU, network failure, etc.

In Chapter 4, we present heuristic algorithms to solve the bandwidth mini-mization problem addressed in Chapter 3, as ILP is known not to scale well tolarger problem instances. Two heuristic search techniques are explored: one based

INTRODUCTION 7

Cro

ss-p

latfo

rm M

idd

lew

are

pro

toty

pe

Co

mpo

ne

nt-

Base

d N

ext-

Gen

Mo

bile

Ap

ps

<<

cre

ate

s>

>

<<

uses>

>

App

licatio

n

De

velo

per

End

User

Co

mp

one

nt M

ana

ge

me

nt

Pla

tfo

rm

Mob

ile D

evic

es

Ed

ge D

evic

es

Clo

ud In

fra

str

uctu

re

Mo

nito

rin

g

& P

red

ictio

nC

on

figu

ratio

nD

ep

loym

en

tD

eci

sio

n

Alg

ori

thm

s

C1

C2

C3

C4

Ch.2

Ch.4

Ch.7

Ch.5

Ch.6

Pro

vid

e

de

velo

per

tools

Optim

ize

Qualit

y o

f

Expe

rie

nce

Ch.7

Ch

.3C

h.5

Ch

.6

Figure 1.4: Overview of the structure and the different chapters of this dissertation

8 CHAPTER 1

on the Kernihan-Lin (KL) graph partitioning algorithm, and a second on simu-lated annealing. Comparison with ILP for small problem instances shows that theheuristics achieve near to optimal results.

While the system presented in Chapter 3 focused on restrictions on CPU loadand bandwidth, in Chapter 5 we introduce AIOLOS, an offloading framework forAndroid which focuses on minimizing execution time. In order to estimate theexecution time upfront, a random sample consensus algorithm is proposed for pre-diction based on monitoring information from previous executions. To enable easyapplication development for the AIOLOS framework, we present a programmingmodel based on code annotations, which minimizes the burden on the applicationdeveloper. The framework and programming model are validated using a Chessgame application and a Photo Editor application, when offloading either to theAmazon EC2 Cloud or to infrastructure in the LAN network.

In Chapter 6 a component execution environment is presented based on AIO-LOS, extended with mechanisms for dynamic (re)configuration of component pa-rameters next to component offloading. This execution environment is fitted ina hierarchical framework for managing all devices in the vicinity, denoted as the“cloudlet”. Instead of focusing on generic metrics such as bandwidth or executiontime, taking configuration into account allows us to focus on higher level Quality ofService (QoS) metrics. The developer is able to specify application specific qualityparameters and performance constraints, in order to optimize the QoS perceived bythe end user. To capture these semantically richer specifications, the programmingmodel from Chapter 6 is extended. The cloudlet middleware platform proposed isvalidated using the augmented reality use case presented in Chapter 2.

Chapter 7 elaborates on algorithms for optimizing the QoS by offloading and/orconfiguring components to apply in the middleware framework presented in Chap-ter 6. A formal model is defined that represents the application components andinfrastructure, and contains the parameters and constraints defined by the appli-cation developer. Two search heuristics are compared based on the techniquesproposed in Chapter 4, and are validated using the middleware framework fromChapter 6.

1.5 General applicability

Throughout this dissertation we introduce a number of prototypes for offloadingparts of applications. In our work, we have chosen for the OSGi [7] componentframework, which is written in Java and thus runs on all systems where a Javaruntime environment is available, such as regular desktop and server systems, aswell as mobile devices running Android or a Linux-based operating system. Thequestion may rise whether the techniques presented in this dissertation are alsoapplicable to other mobile platforms, such as iOS or the upcoming platforms based

INTRODUCTION 9

on web technology such as Tizen or Firefox OS.All of the approaches presented in this dissertation rely on two core concepts.

The first requirement is the notion of a “component”, which contains part of thefunctionality of the application, and both the client and the server devices have tobe able to execute this functionality. In our case a component consists of a .jar filecontaining Java bytecodes. The second requirement is the presence some kind ofremote procedure call (RPC) protocol, that enables one to call the functionality ofa remote component. In the remainder of this work we adopt the R-OSGi protocol[8], but also other protocols could be integrated. A third but optional entity usedin this dissertation is the notion of a “service”. A service defines the interface ofthe component, containing only methods that should be visible from outside thecomponent.

In order to be applicable on other platforms, at least some sort of componentsand a method for RPC should be available. To apply these concepts to iOS, onesolution would be to install a Java runtime environment, and use the same mid-dleware as presented in this dissertation. Although this is technically possible,this is hindered by Apple’s strict application approval policy. A second approachcould be to cross-compile from Android to iOS, as proposed by [9]. However, inour case the cross compiler should take into account the component concept, andtranslate Java components to something equivalent in the iOS world. A final optionwould be to develop a component runtime natively in Objective-C, comparable tothe Apache Celix initiative [10], where a native C implementation of the OSGispecification is developed.

Similarly for platforms based on web technologies, our framework could beused in combination with an HTML 5 application, by replacing the R-OSGi pro-tocol by a HTTP based protocol such as REST or SOAP. Another approach is touse web-based components altogether as proposed by Zhang et al. [11], wherecomponents called “weblets” are used. A third option could be to leverage client-side technologies, i.e. developing a component model in Javascript, and an RPCprotocol using HTML 5 WebSockets.

10 CHAPTER 1

1.6 PublicationsThe research results obtained during this PhD research have been published inscientific journals and presented at a series of international conferences. The fol-lowing list provides an overview of the publications during my PhD research.

1.6.1 Publications in international journals(listed in the Science Citation Index 1 )

[1] T. Verbelen, T. Stevens, P. Simoens, F. De Turck, and B. Dhoedt, Dynamicdeployment and quality adaptation for mobile augmented reality applica-tions. Published in the Journal of Systems and Software (JSS), 84(11):1871–1882, 2011.

[2] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt, AIOLOS: middle-ware for improving mobile application performance through cyber foraging.Published in the Journal of Systems and Software (JSS), 85(11):2629–2639,2012.

[3] T. Verbelen, T. Stevens, F. De Turck, and B. Dhoedt, Graph partitioningalgorithms for optimizing software deployment in mobile cloud computing.Published in the Journal of Future Generation Computer Systems (FGCS),29(2):451–459, 2013.

[4] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt, Leveraging theCloudlet for Immersive Collaborative Applications. Revised manuscriptsubmitted to IEEE Pervasive Computing, June, 2013.

[5] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt, Adaptive Deploymentand Configuration for Mobile Augmented Reality in the Cloudlet. Submittedto the Journal of Network and Computer Applications (JNCA), April, 2013.

[6] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt, Model-driven Con-figuration and Deployment Optimization in Mobile Cloud Computing. Sub-mitted to IEEE Transactions on Software Engineering (TSE), April, 2013.

1The publications listed are recognized as ‘A1 publications’, according to the following definitionused by Ghent University: A1 publications are articles listed in the Science Citation Index, the SocialScience Citation Index or the Arts and Humanities Citation Index of the ISI Web of Science, restrictedto contributions listed as article, review, letter, note or proceedings paper.

INTRODUCTION 11

1.6.2 Publications in international conferences

[1] O. Van Laere, M. Strobbe, K. Michiels, S. Schockaert, S. Dauwe, J. Vanat-tenhoven, C. Van Nimwegen, P. Dhondt, T. Verbelen, B. Dhoedt, F. De Turck,and P. Demeester, Enriching networked applications and services throughusergenerated content. Published in proceedings of the 48th FITCE Congress,Prague, Czech Republic, September 2009.

[2] T. Verbelen, R. Hens, T. Stevens, F. De Turck, and B. Dhoedt, Adaptiveonline deployment for resource constrained mobile smart clients. Publishedin proceedings of the 3rd Conference on Mobile Wireless Middleware, Op-erating Systems, and Applications (Mobilware), Chicago, USA, July 2010.

[3] T. Verbelen, T. Stevens, and F. De Turck, Fast dynamic deployment adap-tation for mobile devices. Published in proceedings of the 4th Interna-tional Workshop on Adaptive and DependAble Mobile Ubiquitous Systems(ADAMUS) in conjunction with the 7th International Conference on Perva-sive Services (ICPS), Berlin, Germany, July 2010.

[4] P. Simoens, T. Verbelen, F. De Turck, and B. Dhoedt, Mobile AR in the out-doors: watch the clouds. Published in proceedings of the Workshop “Howto industrialize Wearable AR?” in conjunction with the 16th InternationalSymposium on Wearable Computing (ISWC), Newcastle, United Kingdom,June 2012.

[5] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt, Cloudlets: bringingthe cloud to the mobile user. Published in proceedings of the 3rd ACMWorkshop on Mobile Cloud Computing and Services (MCS) in conjunctionwith the 10th International Conference on Mobile Systems, Applications,and Services (MobiSys), Low Wood Bay, United Kingdom, June 2012.

[6] Tim Verbelen, Pieter Simoens, Filip De Turck, and Bart Dhoedt, Adaptiveapplication configuration and distribution in mobile cloudlet middleware.Published in proceedings of the 5th Conference on Mobile Wireless Middle-ware, Operating Systems, and Applications (Mobilware), Berlin, Germany,November 2012.This paper received the Best Paper Award.

[7] B. Vankeirsbilck, T. Verbelen, D. Verslype, N. Staelens, F. De Turck, P. De-meester, and B. Dhoedt, Quality of experience driven control of interac-tive media stream parameters. Published in proceedings of the IFIP/IEEEworkshop on QoE centric management (QCMAN) in conjunction with theIFIP/IEEE International Conference on Integrated Network Management(IM), Ghent, Belgium, May 2013.

12 CHAPTER 1

[8] P. Simoens, T. Verbelen, and B. Dhoedt, Vision: Mapping the world in 3Dthrough first-person vision devices with Mercator. Accepted for publicationin proceedings of the 4th ACM Workshop on Mobile Cloud Computing andServices (MCS) in conjunction with the 11th International Conference onMobile Systems, Applications, and Services (MobiSys), Tapei, Taiwan, June2013.

[9] T. Verbelen, P. Simoens, and B. Dhoedt, Towards a Component-Based Plat-form for Industrial AR. Accepted for pubication in proceedings of the 2ndworkshopon Wearable AR Systems for Industrial Applications in conjunc-tion with the 17th International Symposium on Wearable Computing (ISWC),Zurich, Switzerland, September 2013.

1.6.3 Poster publications in international conferences

[1] T. Verbelen, P. Simoens, F. De Turck, B. Dhoedt, AIOLOS: mobile mid-dleware for adaptive offloading. Published in proceedings of the ACM/I-FIP/USENIX 12th International Middleware Conference (Middleware’11),Lisbon, Portugal, December 2011.

[2] T. Verbelen, P. Simoens, F. De Turck, B. Dhoedt, A Component-Based Ap-proach towards Mobile Distributed and Collaborative PTAM. Published inproceedings of the International Symposium on Mixed and Augmented Re-ality (ISMAR), Atlanta, USA, November 2012.

1.6.4 Publications in national conferences

[1] T. Verbelen, Augmenting mobile phone capabilities by adaptive offloading.Published in the 12th FEA PhD symposium, Ghent, Belgium, December2011.

INTRODUCTION 13

References[1] International Telecommunication Union (ITU). Measuring the Informa-

tion Society 2012. 2012. Available from: http://www.itu.int/ITU-D/ict/publications/idi/material/2012/MIS2012_without_Annex_4.pdf.

[2] International Data Corporation (IDC). Worldwide Smartphone 2013–2017Forecast and Analysis, March 2013.

[3] Gartner Group. Press Release: Gartner Says Free Apps Will Account forNearly 90 Percent of Total Mobile App Store Downloads in 2012, September2012. Available from: http://www.gartner.com/newsroom/id/2153215.

[4] International Data Corporation (IDC). Android and iOS Combine for 91.1%of the Worldwide Smartphone OS Market in 4Q12 and 87.6% for the Year,Februari 2013. Available from: http://www.idc.com/getdoc.jsp?containerId=prUS23946013#.UVhP2xKJA7x.

[5] http://developer.android.com. Platform versions distribution, March 2013.Available from: http://developer.android.com/about/dashboards/index.html.

[6] Primate Labs. Geekbench, Cross-Platform Processor Benchmark, March2013. Available from: http://www.primatelabs.com/geekbench/.

[7] The OSGi Alliance. OSGi Service Platform, Core Specification, Release 4,Version 4.2. aQute, September 2009.

[8] J. S. Rellermeyer, G. Alonso, and T. Roscoe. R-OSGi: distributed applica-tions through software modularization. In Proceedings of the InternationalConference on Middleware, Middleware ’07, pages 1–20, 2007.

[9] A. Puder and O. Antebi. Cross-Compiling Android Applications to iOS andWindows Phone 7. Mobile Networks and Applications, pages 1–19, 2012.

[10] The Apache Software Foundation. Apache Celix, 2013. Available from:http://incubator.apache.org/celix/.

[11] X. Zhang, S. Jeong, A. Kunjithapatham, and S. Gibbs. Towards an Elas-tic Application Model for Augmenting Computing Capabilities of MobilePlatforms. In Proceedings of the 3rd Conference on Mobile Wireless Mid-dleware, Operating Systems, and Applications, volume 48, pages 161–174,2010.

http://www.itu.int/ITU-D/ict/publications/idi/material/2012/MIS2012_without_Annex_4.pdf

http://www.itu.int/ITU-D/ict/publications/idi/material/2012/MIS2012_without_Annex_4.pdf

http://www.gartner.com/newsroom/id/2153215

http://www.idc.com/getdoc.jsp?containerId=prUS23946013#.UVhP2xKJA7x

http://www.idc.com/getdoc.jsp?containerId=prUS23946013#.UVhP2xKJA7x

http://developer.android.com/about/dashboards/index.html

http://www.primatelabs.com/geekbench/

http://incubator.apache.org/celix/

2Leveraging the Cloudlet for Immersive

Collaborative Applications

“The most powerful natural species are those that adapt to environmental change.”

–Charles Darwin (1809 - 1882)

Augmented reality (AR) is often considered as one of the key applications thatwill drive mobile growth in the future. With the emergence of new head mounteddisplay (HMD) technologies such as Google Glass, mobile AR applications aregaining a lot of attention. However, a lot of challenges still remain when develop-ing demanding AR applications for resource-constrained mobile devices. In thischapter, we focus on a mobile augmented reality use case, and show the need fordynamic adaptation at runtime. We present a generic framework for discovering,managing and utilizing resources in the near vicinity of the mobile user – denotedas the “Cloudlet” – which is discussed more thoroughly in Chapter 6 of this dis-sertation.

? ? ?

T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt.Revised manuscript submitted to IEEE Pervasive Computing, June 2013.

16 CHAPTER 2

Abstract Cyber foraging has already shown itself to be succesful for enablingresource-intensive applications on mobile devices. Offloading applications or partsthereof to nearby infrastructure – also denoted as cloudlets – can lower executiontime and/or save energy. However, in view of immersive applications, two chal-lenges still remain. First, due to the real-time constraints of immersive applica-tions, optimizing generic metrics such as execution time or energy is not suffi-cient. Therefore we propose a component-based cyber foraging framework thatoptimizes application specific metrics, not only by offloading, but also by config-uring application components at runtime. Second, as immersive applications tendto process location aware data, much processing is replicated when multiple usersare in the same environment. Therefore, our framework also enables collaborativescenarios by sharing components between multiple devices.

2.1 Introduction

Mobile devices are pervading people’s daily lives. Many use a smartphone toaccess their e-mails on the road, read an e-book on their tablet, or use a laptopto work on the train. Due to recent advances in hardware technology, currentmobile devices are not only gaining more memory and processing power, they arealso packed with a large number of sensors. In combination with developments innear-to-eye display technologies such as Google Glasses, these advances will leadto the emergence of mobile immersive applications such as augmented reality [1].By combining virtual 3D objects with the real world, such applications submergethe user in a new, mixed world to interact with. These applications utilize complexalgorithms for camera tracking, object recognition etc. that require considerablecomputational resources.

Despite the increasing hardware capabilities, mobile devices will always beresource poor compared to their desktop or server counterparts. Due to additionallimitations concerning weight, size, battery autonomy and heat dissipation, mo-bile devices simply cannot compete with fixed hardware. These resource limi-tations imply considerable obstacles for realizing mobile immersive applications,that typically require heavy and real-time processing. As a developer, one also hasto cope with a high level of device heterogeneity, meaning that one has either todevelop for the lowest common denominator, either to tailor the application foreach device by providing a specific configuration.

In order to be able to run such resource-intensive applications on a mobile de-vice, a solution is to offload the application or parts thereof to resources in thenetwork, also known as cyber foraging [2]. With the emergence of cloud comput-ing, processing power is nowadays available in the network as a commodity [3].However, due to the high and variable latency to distant datacenters, the cloud isnot always suitable for code offload, especially not for real-time immersive appli-

LEVERAGING THE CLOUDLET FOR IMMERSIVE COLLABORATIVE APPLICATIONS 17

cations. Therefore, Satyanarayanan et al. introduced “cloudlets”: infrastructuredeployed in the vicinity of the mobile user [4].

Current cyber foraging systems face two major challenges for supporting im-mersive applications. First, most cyber foraging systems focus on optimizinggeneric metrics such as energy usage [5] [6] or execution time [7] [8]. Immersiveapplications however, often need to execute complex image processing algorithmsin a timely manner, and thus it makes more sense to take into account applica-tion specific metrics such as the framerate, image resolution or the robustness ofcamera tracking. From a developer’s perspective, the goal is then to achieve an ac-ceptable framerate at the highest possible quality for each device, where “quality”is related to the image resolution and the tracking configuration. To address thisissue, we propose a cyber foraging framework that not only decides on offloading,but also takes into account application specific configuration parameters in order toenhance application quality. We adopt a component-based approach, where appli-cation developers define software components with their dependencies, configura-tion parameters (i.e. image resolution, tracking parameters) and constraints (i.e.minimal required framerate). These components are then distributed and config-ured at runtime, in order to optimize the user experience and meet all performanceconstraints.

A second challenge consists of the fact that existing cyber foraging solutionsonly decide on offloading from one mobile device to one or more servers. How-ever, when multiple devices are connecting to one or more servers, a global optimalsolution is preferred, instead of all devices competing for the available network orcompute resources. Moreover, in the case of immersive applications, devices inthe same neighbourhood will often process the same or similar data. For exampleregarding object recognition, co-located devices will recognize the same objects,and could benefit from one another. By sharing offloaded components betweenmultiple devices, collaborative immersive application scenarios become possible.

2.2 Related work in Cyber Foraging

Cyber foraging has been a research topic for over a decade [9]. An overview of thedifferent approaches used is shown in Table 2.1. Early systems such as Spectra [10]offer an API to let programmers identify methods that should possibly be executedremotely. At runtime the system selects the best option using a greedy heuristicoptimizing an utility function taking into account execution time, energy usage andfidelity. Chroma [11] follows a similar approach, but uses a tactics-based decisionengine optimizing a user-specific utility function.

Later systems often use a bytecode format running in an application VM inorder to enable more transparent code offloading. MAUI [5] offloads methods inthe Microsoft .Net framework aiming to optimize the energy usage. An ILP solver

18 CHAPTER 2

Table 2.1: Overview of cyber foraging systems and their characteristics, based on [9]

Obj

ectiv

eG

ranu

lari

tyD

ecis

ion

Alg

orith

mPr

ogra

mm

ing

Mod

elSp

ectr

aE

xecu

tion

time,

Met

hod

Gre

edy

heur

istic

Use

Spec

tra

API

[10]

ener

gyan

dfid

elity

Chr

oma

Exe

cutio

ntim

eM

etho

dTa

ctic

s-ba

sed

Defi

neta

ctic

s

[11]

and

fidel

ityse

lect

ion

MA

UI

Ene

rgy

Met

hod

Inte

gerL

inea

rC

ode

anno

tatio

ns

[5]

Prog

ram

min

g

Scav

enge

rE

xecu

tion

time

Met

hod

His

tory

-bas

edC

ode

anno

tatio

ns

[7]

profi

le

AIO

LO

SE

xecu

tion

time

OSG

icom

pone

ntH

isto

ry-b

ased

Use

OSG

ior

[8]

profi

leco

dean

nota

tions

Ode

ssa

Mak

espa

nan

dSp

rout

com

pone

ntIn

crem

enta

lgre

edy

Use

Spro

ut

[12]

thro

ughp

utal

gori

thm

fram

ewor

k

Clo

neC

loud

Exe

cutio

ntim

eT

hrea

dIn

tege

rLin

ear

Non

e

[6]

oren

ergy

Prog

ram

min

g

CO

ME

TE

xecu

tion

time

Thr

ead

Thr

esho

ld-b

ased

Non

e

[13]

and

ener

gysc

hedu

ler


is used to decide whether or not offload a method call. Scavenger [7] offloadsPython methods, minimizing the execution time depending on the method callarguments using history based profiles. Both systems require code annotationsfrom the application developer to identify offloadable methods.

AIOLOS [8] is a cyber foraging framework on Android that takes a component-based approach, replicating OSGi components on the remote server. At runtimemethod calls are forwarded either to the local or remote component instance usinga profile built from monitoring information similar to Scavenger. Programmers candevelop their applications as a number of OSGi components, or use code annota-tions to automate this process. Odessa [12] uses the Sprout component framework,employing an incremental greedy strategy trying to exploit parallelism to optimizethe throughput and makespan.

CloneCloud [6] operates at a lower level, migrating at thread granularity inthe Dalvik VM. The CloneCloud partitioner automatically identifies offloadableparts of the application in an offline stage, using static and dynamic code analy-sis and without the need for programmer’s annotations. At runtime the partitionminimizing the execution time is selected using an ILP solver. COMET [13] alsooffloads threads in the Dalvik VM, but instead of rewriting the software offline, anonline approach is presented using distributed shared memory (DSM). This allowsto migrate threads at runtime using a threshold based scheduler, at the cost of morecommunication overhead for keeping the DSM synchronized.

Our framework uses an application-specific utility function similarly as [11] inorder to optimize the application quality. We adopt a component-based approachas in [8], which gives a good granularity to offload, while keeping the monitoringoverhead low. In contrast to existing systems, our framework not only adapts thedeployment of parts of the application but also autonomically adapts configurationparameters of the components in order to adhere to constraints given by the devel-oper and the device context. A heuristic search algorithm allows fast optimizationof both the deployment and configuration for all connected devices in the vicin-ity. Second, the framework takes into account collaborative components, that canbe shared among multiple users. This way, users can benefit from computationsalready done by others, thus reducing the overall computational load.

2.3 Use case: parallel tracking and mapping

With the breakthrough of head-mounted displays, a whole new type of immer-sive applications becomes possible. In a museum, in addition to listening to anexplanation through headphones, visitors can be guided using visual annotations.Scenes can come to life right before their eyes immersing them in ancient history.An immersive shopping application can guide customers through the mall, anno-tating all products on their shopping list. Children can play with virtual toy cars or

20 CHAPTER 2

spaceships racing on a table top, colliding with real obstacles.The above mentioned applications require similar functionality:

• The position of the glasses with respect to the world has to be known, inorder to correctly position an overlay.

• There is a need for a correct model of the world, to let the overlay “blendin” without becoming too intrusive.

• Many applications require reliable object recognition, to identify objects thatneed to be annotated.

When multiple users run such applications in the same environment, they canclearly benefit from collaboration as they will require a model of the same en-vironment and likely recognize the same objects.

Current state of the art algorithms providing these features rely on visual infor-mation. The Parallel Tracking and Mapping (PTAM) algorithm by Klein et al. [14]tracks the position of a camera with respect to the world based on detected visualfeatures. In parallel with the tracking, a model of the world’s visual features isconstructed in a mapping phase. Castle et al. [15] combined this approach withvisual object recognition, allowing to recognize objects and localize them in themap. However due to the complexity of these algorithms, current high end mobiledevices still fall short to execute this in a timely manner, especially when scalingto large environments and many objects to be recognized.

2.4 A component-based cyber foraging approachIn order to run resource-intensive immersive applications on a mobile device, a firstsolution would be to run the application on a remote server and use the device as athin client. This however requires the device to continuously send camera framesto the server, and receive rendered frames as result, which would require consid-erable bandwidth, and would not scale to multiple devices in the same wirelessnetwork. Therefore we adopt a component-based approach, splitting the applica-tion up into a number of loosely coupled software components that each can beoffloaded and/or configured in order to optimize the user experience.

We identified different components for our use case augmented reality appli-cation as shown in Figure 2.1. In addition to a component for fetching videoframes from the camera hardware (VideoSource) and for rendering an overlay onthe screen (Renderer), there are three components for each main functionality. TheTracker component processes the camera frames and estimates the camera positionwith respect to the world based on a number of visual feature points. The morefeature points used for tracking, the more stable the tracking becomes and the morerobust to sudden camera movements. In parallel, the Mapper creates a model of


VideoSource Renderer

Tracker Mapper

Object

Recognizer

resolution

#features

image

format

Figure 2.1: Identifying loosely coupled software components in an example immersive ap-plication.

the world by identifying new feature points and estimating their position in theworld, which can then be used for tracking. The ObjectRecognizer tries to recog-nize known objects in the world and notifies the Renderer of their 3D position inthe world when found.

When executing such an augmented reality application, optimizing genericmetrics such as energy or execution time is not a silver bullet. More importantis to optimize some application specific metrics. For example, to achieve smoothcamera tracking, the Tracker should be able to process 15 to 20 frames per second.On the other hand, a higher resolution of the camera frames or a higher numberof tracked feature points will also enhance the user experience. Therefore, ourframework enables developers to specify application performance constraints, aswell as configuration parameters that influence the end user quality. To meet theimposed constraints the framework can not only offload application components,but can also configure their quality parameters.

In our use case we defined three configurable parameters. Increasing the cam-era resolution or the number of features used to track the camera position increasethe quality, at the cost of more required processing time. The images can also befetched either as raw image data or a compressed JPEG format. A compressed im-age format requires extra processing time for encoding and decoding the frames,but can lower the required bandwidth when offloading.

By adopting a component-based approach, collaborative scenarios can easilybe constructed by sharing application components. In the parallel tracking andmapping use case for example, the Mapper component can be shared betweenmultiple mobile devices in the same physical environment. This way, all devicesreceive the same world model to track the camera position. Because the model isupdated and refined using camera images from multiple devices, the model willalso be much more accurate than one device could create on its own.

22 CHAPTER 2

Video

SourceRenderer

Tracker Video

SourceRenderer

MapperObject

Recognizer

Tracker

Node

Agent

Node

Agent

Node

Agent

Cloudlet Agent

Execution Environment Execution Environment

Execution Environment

Monitor

Analyze Plan

Redeploy

Reconfigure

Figure 2.2: Overview of the cloudlet platform. Execution environments are containersfor application components. All communication between components passesthrough the execution environments enabling transparent monitoring and of-floading. The Node Agents aggregate monitoring information for each device.The cloudlet agent analyzes the monitoring information and decides whetherthe deployment or configuration has to be adapted.

2.5 An autonomic cloudlet management platform

In order to benefit from the effort of developing applications as a set of looselycoupled components, a platform is required that is able to manage and configurecomponents at runtime, as well as to distribute them among available resources inthe near vicinity, called the “cloudlet”. We propose a hierarchical architecture asdepicted on Figure 2.2 that operates on three management levels: the executionenvironment, the node and the cloudlet.

2.5.1 Execution Environment

An execution environment is to be thought of as a container into which componentscan be deployed. When a component is deployed, the execution environment man-ages the component life cycle and resolves the dependencies to other components.Each component can expose configuration parameters that can be adapted by theexecution environment at runtime. The execution environment also enables mon-itoring of method calls between components, capturing the size of the arguments,return value and the execution time. This monitoring information is then sent tothe node level, where the information is aggregated and used as input for decisiontaking on the cloudlet level.


2.5.2 Node

Each device runs a node agent, that manages the device as a whole. The nodeagent aggregates monitoring information from all the execution environments run-ning on the device, and sends these to the cloudlet agent. In addition to the in-formation about the execution environments, the node agent also provides devicespecific information, such as the device capabilities (i.e. processor speed, numberof processor cores, etc.).

2.5.3 Cloudlet

Among all devices in the network, one is chosen to host the cloudlet agent. Thecloudlet agent receives monitoring information about all the nodes, and has aglobal overview of all execution environments, their components and how thesecomponents interact. This information is then analyzed and used as input for aglobal optimization planning algorithm, that calculates both the deployment andthe configuration for all components [16]. This solution is then enforced and newmonitoring information is used again as feedback. In this way we realize an auto-nomic feedback loop that on the one hand is able to adapt to changing context, buton the other hand strives to a stable deployment, as component migration is alsocostly.

The device that runs the cloudlet agent can be predefined (i.e. a fixed serveravailable in the network), or can be determined at runtime as part of the discoveryprocess. When first initialized, each device runs both a node and cloudlet agent.When another device running a cloudlet agent is discovered, the most suitable ofthe two devices (i.e. the one with the most CPU power) is chosen as the host forthe cloudlet agent, and the other one joins as a node. This way an ad hoc cloudletcan be constructed, where the strongest device runs the cloudlet agent performingglobal optimization for all others.

The cloudlet agent also identifies collaborative components, that can be sharedamong multiple users. By offloading collaborative components to the most re-sourceful node available in the network and redirecting calls of all users to thisnode, users not only save computational resources, but also gain information fromthe input of others. As depicted on Figure 2.2, in our use case example the Map-per and ObjectRecognizer components are shared between two devices. This way,both users receive updates to the map provided by one another, and objects haveto be recognized only once in the scene in order to annotate them on each device.

2.6 Dynamic configuration and deploymentTo illustrate the effectiveness of dynamic configuration and distribution of applica-tion components, we have built a prototype framework. Our implementation builds

24 CHAPTER 2

on OSGi, a standard for creating modular applications in Java, and which we alsoused in previous cyber foraging implementations [8]. In order to allow easy appli-cation development targeting our platform, we use a programming model based onannotations. By adding annotations to the source code, the developer can definesoftware components, their dependencies and their configuration parameters thatare to be optimized, or define application specific constraints. Listing 2.1 showsthe annotated source code of the Tracker class.

The Tracker component is defined, with a dependency to the Mapper compo-nent, and the number of features to track as a configurable parameter. To imposea minimum number of frames processed per second, the Tracker has to process aframe within 60 ms. These annotations are processed at build time, from whichOSGi components are generated [8], as well as a number of XML descriptorsdefining the configurable properties and constraints which are used by the frame-work at runtime.

Our cloudlet agent implements an autonomic feedback loop, that periodicallygathers all monitoring information from all nodes and execution environments.From this information a new deployment and configuration is calculated. In orderto predict the change in behavior when changing configuration parameters, thecloudlet agent is bootstrapped with monitoring information gathered in an offlineprofiling stage.

Suppose N migratable components, M available devices and K configurationparameters with each ki possible parameter values, the total number of possible

package ptam . t r a c k e r ;

@Componentp u b l i c c l a s s T r a c k e r implements T r a c k e r S e r v i c e , V i d e o L i s t e n e r {

@Proper ty ( v a l u e s =200 ,500 ,1000 ,1500 ,2000)p r i v a t e i n t f e a t u r e P o i n t s = 1000 ;

@Reference ( p o l i c y =dynamic )p r i v a t e MapperSe rv ice mapper ;

@ C o n s t r a i n t ( maxTime =60)p u b l i c vo id p r o c e s s F r a m e ( byte [ ] d a t a ) {

/ / p r o c e s s f rame. . .

}. . .

}

Listing 2.1: Example annotated Java source file defining the Tracker component, aconfigurable property (i.e. the number of tracked feature points) and a timingconstraint (i.e. processing a frame should take less than 60 ms).


solutions becomes MN × k1 × ... × kK . As the solution space rapidly growswith the number of devices, components and parameters, we propose a greedysearch heuristic, which calculates close-to-optimal solutions in general cases, andhas been shown to yield the optimal solution in our specific use case [16].

As our framework is implemented in Java, it runs on both Android as on regulardesktop or server machines. The components of the use case application are builtas OSGi modules in Java, and use native C libraries for the complex computervision routines. The native libraries are included compiled for both ARM as x86architectures in order to have cross-platform components.

The developer provides three parameters to configure: the input size of thecamera frames (640x480 or 320x240), the format of the camera frames (raw gray-scale or JPEG) and the number of feature points to track in the frame (2000, 1500,1000, 500 or 200). By limiting the time required to process a frame for tracking,a constraint is imposed to assure a tracker framerate between 15 and 20 framesper second. Maximizing the quality is reflected as maximizing both the cameraresolution and the number of tracked feature points.

We evaluate the framework on a Samsung Galaxy S2 Android device, equippedwith a dual core 1.2 GHz processor and a laptop with an Intel Core 2 Duo clockedat 2.26GHz. The device and laptop are connected using a virtual network overUSB. This enables us to accurately set the available bandwidth on the link, inorder to show how different configurations and deployments are chosen in differentscenarios.

Every 5 seconds the monitoring information is sent from the execution envi-ronment to the cloudlet agent, which decides whether an action is needed. Fromthis monitoring information we calculate the number of processed frames per sec-

Figure 2.3: As the bandwidth decreases the configuration quality is lowered in order to keepthe achieved framerate between 15 and 20 FPS. The configuration is changedwithin 10 to 20 seconds.

26 CHAPTER 2

ond, which is plotted in Figure 2.3. The experiment starts with a bandwidth of100 Mbps, which allows the device to send the raw camera frames with a resolu-tion of 640x480 over the network, and offload the Tracker, Mapping and ObjectRecognition to the laptop. After two minutes the bandwidth is lowered to 20 Mbps,resulting in a drop to 5 FPS. The framework thus adapts the configuration of theVideoSource to change to the MJPEG format. Now extra CPU cycles are usedfor compressing and decompressing the video frames to JPEG, which lowers therequired bandwidth. The tracking is still executed on the laptop, as this allowsto keep the number of tracked feature points (and thus the quality) high. Whenthe bandwidth is even further reduced to 5 Mbps, the framerate drops again to 5FPS, and the Tracker component is moved back to the smartphone. Because of thelimited processing power on the smartphone, only 500 points can be tracked now,lowering the tracking quality.

Figure 2.3 shows that it takes between 10 (first adaptation) and 20 (secondadaptation) seconds to fully recover the framerate. This is due to the fact that themonitoring information is only sent each 5 seconds, and thus only after 5 secondsthe deployment and configuration are re-evaluated and the adaptation is performed.During the second adaptation, the longer migration time is due to the Tracker com-ponent that has to be migrated from the server to the client, and also its state hasto be sent over the network, while at the time of migration the network bandwidthis scarce. To further reduce the adaptation time one could for example notify thecloudlet agent much earlier when a performance drop is experienced, instead ofwaiting during the full 5 second period. To reduce the migration time one couldperiodically synchronize the state of offloaded components with the clients, suchthat when the component is migrated back only an incremental state change has tobe transferred.

Figure 2.4: By sharing components multiple devices can track and expand the same mapand recognize the same objects.


2.7 Collaborative immersive applications

As multiple users in the same environment typically look at the same scene, trackthe same environment and need to recognize the same objects, it makes senseto share computation results with each other. By adopting the component-basedapplication offloading model, such collaborations can be easily implemented bysharing components between different devices. When a component is marked asshared by the developer, the framework can transparently redirect calls from dif-ferent devices to the same instance of this component.

An example application for our use case is shown in Figure 2.4, where twodevices share the same Mapper and Object Recognizer which are offloaded to thelaptop. This way the map of the environment is generated twice as fast (as bothdevices send updates to the Mapper), and processing is only done once (instead offor each device).

Using multiple devices to add feature points to the map also leads to a more

(a)

(b)

Figure 2.5: The scene of the desk in (a) is explored by 3 devices resulting in the mappedfeatures points and recognized object in (b). By sharing the detected featurepoints of all devices, more feature points from multiple viewpoints are added tothe map, leading to robuster tracking and faster map expansion.

28 CHAPTER 2

complete map of the environment. Each device now has a complete overview ofthe environment, whereas the map is limited to the device’s own viewpoint in thenon-collaborative case. In Figure 2.5 a map is shown from a desk environmentgenerated with 3 devices. As each device looks to the desk from other viewpoints,they all map different (overlapping) parts of the desk.

2.8 The future of the CloudletDue to the real-time constraints and the resource-intensive tasks of immersive ap-plications, these represent very important scenarios for leveraging the cloudlet.However, a thin client approach on cyber foraging is not sufficient due to the hugebandwidth requirements, and offloading at a more fine grained level is necessary.We have proposed a component-based platform for cyber foraging, not only de-ciding on component deployment, but also taking into account application spe-cific metrics and constraints provided by the developer. We have shown that ourplatform is able to optimize application quality and to adapt to changing networkconditions using an implemented immersive application. Due to the component-based approach, our platform is able to share components between multiple de-vices, which opens doors to a whole new type of collaborative immersive applica-tions.

AcknowledgmentTim Verbelen is funded by PhD grant of the Fund for Scientific Research, Flanders(FWO-V).Part of this work was funded by the Ghent University GOA-grant "Autonomicnetworked multimedia systems".


References

[1] T. Langlotz, D. Wagner, A. Mulloni, and D. Schmalstieg. Online Creation ofPanoramic Augmented Reality Annotations on Mobile Phones. IEEE Perva-sive Computing, 11(2):56–63, Februari 2012.

[2] R. Balan, J. Flinn, M. Satyanarayanan, S. Sinnamohideen, and H.-I. Yang.The case for cyber foraging. In Proceedings of the 10th workshop on ACMSIGOPS European workshop, EW ’10, pages 87–92, 2002.

[3] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic. Cloud comput-ing and emerging IT platforms: Vision, hype, and reality for delivering com-puting as the 5th utility. Future Generation Computer Systems, 25(6):599–616, June 2009.

[4] M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies. The Case for VM-Based Cloudlets in Mobile Computing. IEEE Pervasive Computing, 8(4):14–23, October-December 2009.

[5] E. Cuervo, A. Balasubramanian, D.-k. Cho, A. Wolman, S. Saroiu, R. Chan-dra, and P. Bahl. MAUI: making smartphones last longer with code offload.In Proceedings of the 8th international conference on Mobile systems, appli-cations, and services, MobiSys ’10, pages 49–62, 2010.

[6] B.-G. Chun, S. Ihm, P. Maniatis, M. Naik, and A. Patti. CloneCloud: elas-tic execution between mobile device and cloud. In Proceedings of the sixthconference on Computer systems, EuroSys ’11, pages 301–314, 2011.

[7] M. Kristensen. Scavenger: Transparent development of efficient cyber for-aging applications. In Proceedings of the IEEE International Conferenceon Pervasive Computing and Communications, Percom ’10, pages 217–226,April 2010.

[8] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt. AIOLOS : middle-ware for improving mobile application performance through cyber foraging.Journal of Systems and Software, 85(11):2629–2639, 2012.

[9] J. Flinn. Cyber Foraging: Bridging Mobile and Cloud Computing. SynthesisLectures on Mobile and Pervasive Computing. Morgan & Claypool Publish-ers, 2012.

[10] J. Flinn, S. Park, and M. Satyanarayanan. Balancing Performance, Energy,and Quality in Pervasive Computing. In Proceedings of the 22nd Interna-tional Conference on Distributed Computing Systems, ICDCS ’02, pages217–226, 2002.

30 CHAPTER 2

[11] R. K. Balan, M. Satyanarayanan, S. Y. Park, and T. Okoshi. Tactics-basedremote execution for mobile computing. In Proceedings of the 1st interna-tional conference on Mobile systems, applications and services, MobiSys’03, pages 273–286, 2003.

[12] M.-R. Ra, A. Sheth, L. Mummert, P. Pillai, D. Wetherall, and R. Govindan.Odessa: enabling interactive perception applications on mobile devices. InProceedings of the 9th international conference on Mobile systems, applica-tions, and services, MobiSys ’11, pages 43–56, 2011.

[13] M. S. Gordon, D. A. Jamshidi, S. Mahlke, Z. M. Mao, and X. Chen. COMET:code offload by migrating execution transparently. In Proceedings of the10th USENIX conference on Operating Systems Design and Implementation,OSDI’12, pages 93–106. USENIX Association, 2012.

[14] G. Klein and D. Murray. Parallel Tracking and Mapping for Small ARWorkspaces. In Proceedings of the 6th International Symposium on Mixedand Augmented Reality, ISMAR ’07, pages 1–10, 2007.

[15] R. Castle and D. Murray. Keyframe-based recognition and localization dur-ing video-rate parallel tracking and mapping. Image and Vision Computing,29(8):524–532, 2011.

[16] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt. Adaptive applicationconfiguration and distribution in mobile cloudlet middleware. In Proceedingsof the 5th Conference on Mobile Wireless Middleware, Operating Systems,and Applications, pages 178–192, 2012.

3Dynamic Deployment and QualityAdaptation for Mobile Augmented

Reality Applications

“Plan to throw one away; you will, anyhow.”

–Frederick Brooks (1931 - ), The Mythical Man-Month

In this chapter, we present our first offloading framework for mobile aug-mented reality applications. When computational resources are available in thenetwork, application components are offloaded from the mobile device. The devel-oper can also define multiple configurations offering a different quality, which areused to gracefully degrade the application when resources are scarce. At runtime,for each possible configuration the optimal deployment consuming least band-width is calculated as a solution to an Integer Linear Programming (ILP) prob-lem. Results show that our framework is able to adapt the application to changingCPU and bandwidth conditions, to server discovery and to network failures. Onthe downside the use of an ILP solver makes the algorithm used not scalable forlarge applications, and a lot of effort is put on the application developer that has tocarefully define the configurations at hand. Therefore, scalable heuristics are pre-sented in Chapter 4, while the programming model is addressed in Chapter 5 & 6.

? ? ?

T. Verbelen, T. Stevens, P. Simoens, F. De Turck, and B. Dhoedt.

Journal of Systems and Software, 84(11):1871–1882, 2011.

32 CHAPTER 3

Abstract With the increasing popularity of smartphones and netbooks, more andmore applications are developed for the mobile platform. Notwithstanding therecent advances in mobile hardware, most mobile devices still lack sufficient re-sources (e.g. CPU power, memory) to execute complex multimedia applicationssuch as augmented reality. Application developers also have difficulties to copewith the changing device context (e.g. network connectivity, remaining batterylife) and the many different hardware platforms and operating systems to run appli-cations on. Therefore, we introduce the concept where the developer can providedifferent configurations of an application, each having different resource require-ments and a different quality offered to the end user. The middleware frameworkpresented in this paper will select and deploy the configuration offering the bestquality possible for the current connectivity and available resources. As thesechange over time, the framework will dynamically adapt the configuration anddeployment at runtime, enhancing the quality by offloading parts of the applica-tion when a remote server is discovered, or gracefully degrading the quality whenthe network connection is lost. Based on experimental results on the augmentedreality use case the performance and effectiveness of our middleware has beencharacterized in different scenarios.

3.1 Introduction

According to a Gartner press release from 12 August 2010 [1], smartphone salesin the second quarter of 2010 totaled 61.6 million units, a 50.5 percent increasefrom the same period in 2009. These numbers illustrate that a significant andgrowing segment of the world population wants to use their handheld device ina more versatile way than the traditional mobile phone. Due to recent advancesin mobile hardware, such as increased battery capacity, CPU power, display res-olution, and improved network connectivity, new and more advanced services arebeing offered for use on handheld devices. Today, a myriad of applications can bedownloaded and installed on smartphones, in a wide range of categories such asweb browsers, games, social and messaging clients, location-based services, audioand video players, just to name a few. As an application developer, it is difficultto design and develop such applications specifically for the mobile market, as dueto the still limited available resources (e.g. CPU power, memory) on the mobileplatform, mobile applications are usually less advanced in comparison with theirdesktop counterparts and hence cannot match the desktop experience for similarapplications. It is also a burden to deal with the variety of mobile devices, differ-ent mobile platforms and the varying mobile context (e.g. connectivity, location,etc.) [2], which rises the need for run-time adaptation and recomposition as de-picted in [3].

Especially for real-time demanding multimedia applications such as augmented

DYNAMIC DEPLOYMENT AND QUALITY ADAPTATION FOR MOBILE AR 33

reality (AR), the available CPU power and memory falls short to execute complexcomputer vision algorithms developed for the desktop. In order to run augmentedreality applications on the mobile device one approach is to reduce the complexityof state-of-the-art computer vision algorithms, that lower the required CPU powerbut also reduce the quality compared to their desktop counterparts [4], [5]. Anotherapproach is to split the application in a client-side and a server-side task as inves-tigated in [6]. Object recognition is performed on a remote server, while objecttracking is still done at the mobile device. The biggest drawback of outsourcingtasks to a server is that the possibly large delay between the client and server candeteriorate the usability of the application [7].

In this paper we present a hybrid approach to tackle the challenges of mo-bile multimedia applications such as augmented reality. We present a middlewareframework that will adapt the application depending on the mobile device and itsavailable resources. In order to be able to adapt the application we assume that thedeveloper provides different configurations of the application that have differentresource requirements and thus offer a different quality level to the end user. Forexample in the object recognition scenario an object could be recognized usingcomplex computer vision algorithms, but also by scanning the image for a prod-uct barcode. When a low latency remote server is discovered the framework willoutsource parts of the application and run complex object recognition algorithmsremotely. When the connection is deteriorated, the framework will try to decreasethe network usage, most often resulting in more mobile processing and if necessarydegrade the quality of the application by switching to barcode detection to be ableto meet the real-time constraints. The end user will experience a higher qualitywhen the complex object recognition is working, but when not enough resourcesare available he can still use the application using the barcode scanner.

The reminder of this paper is structured as follows. In the next section wediscuss related work on code outsourcing and cyber foraging. Section 3.3 describesthe architecture of our offloading framework and Section 3.4 presents our methodto calculate the optimal deployment of the application. In Section 3.5 a detaileduse case is described which is used to evaluate our framework in Section 3.6 andfinally Section 3.7 concludes this paper.

3.2 Related work

In this section an overview is given of related work done in automatic partitioningand remote execution of software. Early software partitioners such as JavaParty[8], Doorastha [9] and AdJava [10] use an additional preprocessing step beforecompiling to insert remote invocation code and expect the programmer to insertspecial keywords indicating parts of the software to be run remotely. However,these methods fully rely on the application programmer and the source code is

34 CHAPTER 3

needed to partition the program.This problem is addressed by using binary rewriting to introduce remote ex-

ecution of parts of the software. This technique is used by Addistant [11] andJ-Orchestra [12] that impact the Java bytecodes to create a distributed application.Similarly, Coign [13] distributes Microsoft COM objects and MAUI [14] rewritesprograms written for the Microsoft .NET Common Language Runtime. In orderto decide which parts to offload, an offline profiling phase is used to calculate theoptimal partitioning.

With the emergence of pervasive computing, the idea of cyber foraging wasintroduced by Satyanarayanan [15]. The idea is to use available resources in theneighborhood of a mobile device to outsource parts of the application and utilizethe available resources to increase the performance on the device. The earliestcyber foraging systems such as Spectra [16] and Chroma [17] introduce a tacticsbased scheduler to decide which methods to outsource. However, both systemsrely on pre-installed remote procedure calls, implying that all remoting has to beprogrammed by the application developer.

A better approach is to make use of mobile code, where the system is ableto outsource parts of the software at runtime, with minimal interference from thedeveloper. Gu et al. [18] present an adaptive offloading framework where Javaclasses can be placed remotely. A fuzzy control model is used to offload classesat runtime and an adapted MINCUT heuristic is used to minimize interactionsbetween partitions. Ou et al. also use class outsourcing, proposing a (k+1) parti-tioning algorithm to outsource k parts on k remote servers [19]. Han et al. presenta flow-based algorithm to partition software which is evaluated by simulation [20].

The Scavenger cyber foraging [21] system uses a scheduler using adaptivehistory-based profiling in order to outsource Python methods, where candidatemethods for outsourcing have to be annotated by the application developer. Zhanget al. [22] propose a mobile code framework where platform independent softwarecomponents – called weblets – can be outsourced to the cloud based on a Bayesianlearning scheduler.

Lastly, recent advances in cloud computing have led to the use of virtual ma-chines in the cloud as surrogates [23]. The biggest problem in using the cloudfor cyber foraging is presented by the typically high latency of the WAN linkcompared to the WLAN connection. To cope with this bottleneck Su et al. pro-posed Slingshot [24], where the VMs are co-located with the wireless access point.Satyanarayanan et al. [25] also placed the surrogates, called cloudlets, in the phys-ical proximity of the mobile user. Chun et al. [26] uses virtualization technology tocreate a clone of the mobile device platform to be able to migrate parts of mobileapplications without modification.

In this paper we describe a framework for adaptive deployment that adoptsthe cyber foraging concept to offload parts of the software to nearby discovered


servers. There are two main research contributions in this paper. First we focuson real-time multimedia applications such as augmented reality, where multiplecomponents are candidate for outsourcing, instead of focusing on few well definedtasks as candidates for outsourcing such as image filtering [21], [22] or speechrecognition [23]. We take the same assumption as Satyanarayanan et al. [25],which implies that the remote processing power is in the vicinity of the user, sincehigh latencies on a WAN link make it useless to offload for real-time applications.Second, we present an algorithm for optimizing the quality as perceived by the enduser, rather than focusing solely on minimizing memory [18], CPU [19] or energyrequirements [20]. An application developer can provide high and low qualityversions of application components, in order to target a wide range of mobile plat-forms and to cope with the changing context. The proposed middleware is able todynamically adapt the application when the device context changes, i.e. offloadparts to a remote server when one is discovered, or gracefully degrade to a lowerquality version of the application when the network connection is lost, taking intoaccount the real-time constraints.

3.3 Offloading middleware overview

3.3.1 Requirements

To build such an offloading framework, different requirements have to be filled in.The application has to be able to adapt to the ever changing available resources,which must happen at run-time since it is impossible to predict all contexts theapplication has to operate in. This means the framework has to be able to sense thecontext to gather information about all available resources, which is done by mon-itoring all known resources on the one hand, and by discovering possible remoteresources on the other hand.

In order to offload parts of the software to a remote server a mobile code ap-proach is adopted. This means units of deployment have to be defined that canmigrate from one machine to another, a remote procedure call technique has tobe used to call these offloaded components, while keeping this as transparent aspossible to the application programmer.

Last but not least an algorithm is needed to decide when to redeploy and whichdeployment is best given the current device context. Given the fact that one ap-plication can consist of different versions of different quality levels, the goal is todeploy the best quality version that is feasible given the available resources. Thedetails of this algorithm are described in Section 3.4.

36 CHAPTER 3

Operating System

OSGi Framework

R-O

SG

i

jSLP

D-B

US-j

ava

Resourc

e M

onit

or

Bundle

Monit

or

Bundle

Insta

ller

Agent

Figure 3.1: Overview of the offloading middleware. The Agent fetches the monitor infor-mation from the monitor bundles and calculates the best deployment. The Bun-dle Installer distributes the components to the discovered servers. Our frame-work is built upon the OSGi framework in Java and uses R-OSGi for remoteexecution, jSLP for server discovery and DBUS as an interface to the operatingsystem.

3.3.2 Offloading framework

This section presents the different components of our offloading framework as il-lustrated on Figure 3.1. Our middleware is built upon OSGi [27], a module systemand service platform that enables runtime deployment of application components– called bundles – implemented in Java. The portability of Java enables our frame-work to migrate and execute code on different hardware platform and architectures.

The OSGi bundle concept gives us a unit of deployment, which can be mi-grated to other machines, while being transparent for the application developer:by developing the application as OSGi bundles the framework will be able to par-tition it and outsource parts of it without the need of special code or annotations.

On top of the OSGi framework, the following three components provide dif-ferent functionalities:

R-OSGi extends the functionality of OSGi for distributed systems [28] and pro-vides remote invocation of methods exposed by the different OSGi bundles.By creating a local proxy of a bundle running on a remote server applicationfunctionality can be outsourced transparently for the application.

jSLP is a pure Java implementation of the Service Location Protocol (SLP) asspecified in RFC 2608 [29]. It has a small footprint which makes it feasible


for embedded devices and is used for the discovery of remote servers in ourframework.

Java D-Bus offers access to the D-Bus message bus system, which is used forinter-process communication between different desktop applications. D-Busis commonly used on Linux-based systems and enables us to integrate appli-cations that use our middleware in the operating system, for example start-upthrough shortcuts.

The last four bundles make up our offloading middleware:

Resource Monitor collects system wide monitoring information such as the usedand available CPU time and bandwidth.

Bundle Monitor fetches monitor information on a more fine-grained level. Thisbundle monitors the CPU time used for each bundle and captures the com-munication between different bundles.

Bundle Installer is responsible for the installation and migration of applicationbundles. When a bundle is migrated the Bundle Installer will send the codeto the destined machine, start it up there and set up all proxies to enableremote execution through R-OSGi.

Agent is the core of our offloading framework. The Agent keeps track of theremote servers discovered by jSLP and periodically checks the monitoringinformation collected by the Resource Monitor and the Bundle Monitor todecide if a redeployment is needed. Using the current available bandwidthand CPU resources the best deployment is calculated – on a mobile devicethis calculation is usually outsourced to the server side – and the BundleInstaller is instructed to migrate the necessary bundles. The algorithm fordetermining the best deployment is presented in Section 3.4.

3.3.3 Application programming model

Because the proposed adaptation framework is built upon OSGi, application devel-opers must adhere to the OSGi programming model. OSGi uses a service orientedprogramming model, where bundles can expose a service by registering a serviceimplementation with the OSGi service registry. When a bundle wants to call an-other service, it queries the service registry.

As the adaptation system uses the bundle as unit of deployment, the applicationdeveloper is responsible of composing his application in multiple OSGi bundles.This can be done with the OSGi APIs or using one of the many specificationssuch as Declarative Services or Blueprint Container that facilitate OSGi program-ming model using declarative approaches and dependency injection [30]. By split-ting up the application functionality in different bundles, the developer has control

38 CHAPTER 3

over the adaptability of the application: a fine grained composition of many smallcomponents offers the adaptation framework more choices to optimally deploy theapplication. As long as all components are valid OSGi bundles the framework willbe able to transparently offload these to remote resources.

In order to further improve the adaptability, the developer can provide differentcomponent implementations, providing the same functionality at different qualitylevels. Depending on the available resources and the mobile device capabilitiesthe best possible quality will be chosen. To make the system aware of the dif-ferent quality levels, the developer has to provide an application description thatstates which different configurations can be deployed. An example is given in List-ing 3.1. The application named “ExampleApp” consists of 3 components and has2 configurations, because of the two implementations of component2.

<application name="ExampleApp"><constraint deadline="100" /><configuration name="config1">

<bundle name="component1"location="bundles/component1.jar"time="10">

<uses bundle="component2_v1" data="5000" /><uses bundle="component3" data="500" />

</bundle><bundle name="component2_v1"

location="bundles/component2_v1.jar"time="150"/>

<bundle name="component3"location="bundles/component3.jar"time="40"/>

</configuration><configuration name="config2">

<bundle name="component1"location="bundles/component1.jar"time="10">

<uses bundle="component2_v2" data="1000" /><uses bundle="component3" data="500" />

</bundle><bundle name="component2_v2"

location="bundles/component2_v2.jar"time="30"/>

<bundle name="component3"location="bundles/component3.jar"time="40"/>

</configuration></application>

Listing 3.1: Example of an application description consisting of 2 configurations with analternative implementation of component2.


Next to the structure of the application, for each bundle also the estimatedaverage CPU time spent (measured on the mobile device without other load) andamount of data communicated with other bundles is stated. These values can beprovided by the application developer, or be generated by the Bundle Monitor inan offline profiling phase or at runtime. The developer can also state a constraintsuch as a deadline in which all operations should finish. In this example it isclear that config2 can be executed on the mobile device, whereas config1 will needremote resources to be able to meet the deadline. A more detailed example ofhow the deadline and resource information is used in the framework is given inSection 3.6.

Currently, mobile application developers have to tailor their application fora wide range of mobile platforms. Using our OSGi-based middleware platform,this is no longer necessary. Instead, the application developer can concentrate hisefforts on the development of component implementations for different qualitylevels. The middleware will dynamically choose the best deployment, taking intoaccount the capabilities of the device at hand and the currently available remoteresources. It is important to note that even if the application developer decides toprovide only one configuration, the system will still be able to offload componentsdependent on the discovered remote resources.

3.4 Optimal deployment calculation

We assume that the application developer provides all software bundles that makeup an application. An application can consist of several configurations where dif-ferent configurations can implement the application on a different quality level.For example in a location based application, there can be a bundle to estimate thelocation by using the GPS sensor or by using GSM localization using the roam-ing signal to the antenna tower. This can result in two different configurationswhere the framework will choose the most appropriate configuration offering thebest quality possible in the current device context. For each configuration also dif-ferent deployments are possible, as bundles can be migrated to remote servers ifavailable. An example application with two configurations is shown on Figure 3.2.

More formally an application can be represented as a set of possible graphsG = (V,E), each representing a single configuration, where V is a set of N ver-tices and E is a set of edges connecting these vertices. The vertices represent thesoftware bundles in a configuration and the edges represent communication be-tween those bundles. We associate a cost wi with each vertex vi indicating theamount of resources (i.e. CPU power) this component needs. C = (cij) is the ad-jacency matrix of G, i.e. if there exists an edge between vi and vj then cij equalsthe weight of this edge, otherwise cij = 0. The edge weights represent the cost

40 CHAPTER 3

w1

w2

w4

w3w1

w6

w5

e14

e12 e23 e15

e16

configuration 1 configuration 2

application

Figure 3.2: The application developer provides an application consisting of 6 bundles.These can be used in two configurations that each offer a different quality level.Each bundle has an associated resource cost wi and interacting bundles have acommunication cost eij . The framework will decide at runtime which configu-ration to deploy and which bundles should be offloaded.

of communication (i.e. bandwidth) between different software components. Theweights of the vertices and edges can be found by profiling the different config-urations of the application. Each software bundle can be deployed on the mobiledevice or on a discovered server with respectivelyM0 andM1 maximum availableresources. The wireless link has a maximum link capacity denoted by L.

The goal now is to find the configuration where all bundles can be deployed oneither the mobile device or the discovered server, taking into account their maxi-mum available resources and the wireless link capacity, and which offers the bestuser experience, or thus maximizing the quality of the application, as shown inFigure 3.3.

We assume here that the quality of software bundle vi increases with its weightwi meaning that the software bundle does more useful work, as it is useless to sac-rifice more CPU power for lower quality. The best configuration therefore maxi-

M0 M1

Figure 3.3: The best configuration is the one that matches with the available resources onthe mobile device (M0) and a possibly discovered server (M1) and the availablebandwidth (L) while maximizing the perceived quality. This configuration isthen deployed on the mobile device and the server.


mizes the total node weights, i.e.

maxconfigurations

∑i

wi (3.1)

However, due to the constrained available resources and limited bandwidth,it is not always possible to deploy this configuration. Therefore we calculate apossible deployment for each configuration by solving the following ILP problemfor one specific configuration and its associated graph G. Let Xim be the decisionvariable that is equal to 1 if software bundle vi is assigned to machine m ∈ {0, 1}and 0 otherwise.

In order to find a valid deployment, the following constraints are imposed. Firstof all the sum of the weights of all bundles deployed on the device or the servercannot exceed the maximum allowed weight.

∀m :∑i

Xim × wi ≤Mm (3.2)

Next, in order to have a fully functional application, each bundle of the config-uration has to be deployed on either the device or the server.

∀i :∑m

Xim = 1 (3.3)

Lastly the bandwidth needed should be less than the available bandwidth L.Therefore the bandwidth usage for the configuration is minimized as the objectivefunction of our ILP problem and afterwards is checked if the resulting bandwidthis less than L.

Let hij take the value 1 when vi and vj are deployed on a different machine, 0otherwise. Then the objective function becomes:

min1

2

∑i

∑j

hij × cij (3.4)

Variables hij can be expressed as a function of decision variables Xim as fol-lows:

∀i, j : hij = 1−∑m

Xim ×Xjm, (3.5)

Indeed, if vi and vj are deployed on the same machine M , all Xim and Xjm

will be zero except for m = M where they are both 1, making the resulting sumequal to 1. When they are deployed on different machines, for each m at least oneof the two variables Xim or Xjm will equal 0. Replacing 3.5 in equation 3.4 givesa formal description that can be solved with a solver that can handle quadraticobjective functions (e.g. CPLEX [31]). It can also be converted to a true IntegerLinear Programming (ILP) problem by stating that:

∀i, j : hij = 1−∑m

hijm, (3.6)

42 CHAPTER 3

Equation 3.6 then introduces extra decision variables hijm subjected to thefollowing constraints:

∀i, j,m :

hijm ≤ Xim

hijm ≤ Xjm

hijm ≥ Xim +Xjm − 1(3.7)

The final algorithm to calculate the best deployment is illustrated on Figure 3.4.The configurations are processed in order of decreasing total weight. When no re-mote server is discovered, the configuration is chosen with maximum total nodeweight that fits on the client device or thus where

∑i wi ≤ M0. If no such so-

lution exists the configuration is chosen with minimal total node weight. When aremote server is available, the ILP problem is solved and if a solution exists withbandwidth smaller or equal to the available bandwidth L this result is returned.

Start

Select G(V,E) with

max w

not yet visitedi

P

L>0

Solve ILP problem

Solution has minimum

bandwidth b

b<L

Return G(V,E) as solution

End

NO

YES

YES

NO NO

YES

Figure 3.4: Flowchart of the algorithm to calculate the best deployment out of differentpossible configurations.


Note that the ILP problem – which is computationally expensive – only hasto be solved when a remote server is available. This calculation can thus be out-sourced and executed at the server side which can solve this problem in less thana second for graphs of 10 to 20 vertices. For larger graphs a set of deploymentscould also be calculated upfront for a wide range of parameters.

3.5 Augmented reality use case

As use case for our offloading framework, we focus on augmented reality appli-cations for two main reasons. Firstly, it is much more challenging to identify thebest deployment and identify good candidates for outsourcing in an augmentedreality application due to its real-time character, as opposed to non real-time appli-cations such as a photo gallery application where the execution of an image filteris almost always a good task to outsource. Secondly the complexity of augmentedreality and the various computer vision algorithms used make it a mere necessityto outsource parts of the application in order to be able to run the most advancedtechniques on the mobile device.

As a sample scenario, we present an augmented reality shopping assistant thatoffers the user extra information on the product(s) in view in the form of an overlay.The most resource intensive part of this application is the detection and recognitionof the products. In the ideal case the user will point his smartphone at the producton the shelve, where the application uses complex object recognition algorithms toidentify the product. In order to also let the application work on lower end devicesor when no remote resources are available, another way to identify a product isby scanning its barcode. However, in this case the user has to take the productform the shelve, look for this barcode and scan it, which results in a lesser userexperience.

The general architecture and different components of this application are illus-trated in the sequence diagram on Figure 3.5. The Video bundle has a separatethread to continuously capture frames and show them on the screen. Anotherthread is run by the Analyzer bundle that will fetch the last captured frame andanalyze it for possible objects. The Analyzer passes the frame to the Detectorthat can identify possible interesting feature points or regions. Next the Recogniz-er/Decoder bundle will try to decode a barcode from the detected region or matchit to a known set of feature points associated to a certain object. When an objectis found its ID is passed to the ContentProvider bundle that fetches the associatedcontent that is then displayed on top of the video.

Of each component type one or more implementations are developed.

Video – Because the Video bundle is the same for all configurations, only oneimplementation is provided. This bundle also has to access the device cam-

44 CHAPTER 3

getFrame

detect

recognizeObject

lookupContent

overlayContent

getFrame

Figure 3.5: Sequence diagram of the augmented reality shopping assistant.

era and can not be outsourced. The video frames are captured in a nativethread by use of the Java Native Interface (JNI) and rendered on screen withOpenGL ES. All other components are pure Java implementations in orderto be able to migrate.

Content Provider – The Content Provider bundle simply looks up product infor-mation by product ID. Currently only one provider is implemented.

Analyzer – The Analyzer bundle contains the main application loop and orches-trates between the different bundles. There are two versions of this bundledifferentiated by the way they fetch the captured frames: one fetches theimage at full 640x480 resolution, while the other one subsamples the imageto a 320x240 resolution.

Detector – There are two types of detector bundles depending on the method ofobject identification. The first type is a Barcode Detector that returns a re-gion that possibly contains a barcode. We implemented two versions : onedetector analyzes the full image for image regions with many vertical edgesusing Sobel filtering that possibly represent barcodes, a more simple imple-mentation only looks at the center image patch. The second type detectorwill detect image features for object recognition. Again we implementedtwo versions: one generates SURF feature descriptors as described in [32],which results in scale invariant and rotation invariant feature descriptors,and a faster implementation combines FAST corner detection with SURF-like feature descriptors inspired by [5], which gives up the scale invariance


for more speed.

Recognizer/Decoder – Depending on the type of detector bundle a suitable im-plementation of the Recognizer/Decoder bundle is deployed. The FeatureRecognizer matches the features from the image to a set of known featuresof objects to recognize. We implemented a simple matching algorithm basedon cross-correlation matching that lets us differentiate a few objects, but forlarge product databases a more complex algorithm should be used like theone in [6]. A Barcode Decoder bundle will try to decode a possible barcodein the region detected by one of the Barcode Detectors, using the ZXingbarcode image processing library [33].

By combining the different implementations of the Analyzer, Detector andRecognizer/Decoder bundles the following five configurations can be distilled asshown in Table 3.1.

1 The first one combines the Center Barcode Detector and Barcode Decoderwith the 320x240 Analyzer. This configuration will need the least CPUpower, but will also offer the least quality to the end user: he/she will alwayshave to make sure the barcode of the product is in the center of the view toget the information displayed.

2 A second configuration uses the Sobel Barcode Detector that will scan thewhole image together with the 320x240 Analyzer and Barcode Decoder.Now the information will be displayed when the barcode is in view, but itdoes not necessarily need to be in the center.

Table 3.1: Different configurations for the AR use case.

conf. 1 conf. 2 conf. 3 conf. 4 conf. 5Video x x x x xContentProvider x x x x xAnalyzer- Full resolution x x- Half resolution x x xDetector- Center Barcode Detector x- Sobel Barcode Detector x- FAST Feature Detector x- SURF Feature Detector x xRecognizer/Decoder- Barcode Decoder x x- Feature Recognizer x x x

46 CHAPTER 3

3 The third configuration will combine the 640x480 Analyzer with the FASTFeature Detector and the Feature Recognizer. Because the FAST featuredetection the detection phase can be executed on the device and only the de-tected feature data has to be transmitted to the server which can be beneficialwhen not much bandwidth is available. However, due to the lack of scaleinvariance the object can only be detected when it has the same size in theimage as in the reference image.

4 The fourth configuration uses the 320x240 Analyzer with the SURF FeatureDetector and the Feature Recognizer. Because the SURF feature detectionis rather slow, it will probably be outsourced and thus the whole 320x240frame will have to be sent to the server.

5 The final configuration uses the 640x480 Analyzer with the SURF FeatureDetector and the Feature Recognizer. By using the full resolution imageobjects can be identified from a bigger distance, but at the cost of moreprocessing power and most likely more bandwidth when the detector com-ponent is outsourced.

Figure 3.6 shows two screenshots of different configurations of the application.In the left screenshot configuration 5 is deployed and the book is recognized by itsvisual features. In the right screenshot the barcode is used to identify the object.

(a) Config 5 (b) Config 2

Figure 3.6: The left screenshot shows the high quality configuration where the book is vi-sually recognized. In the right screenshot the barcode has to be in view torecognize the object.

3.6 Experimental validation

In this section we validate our offloading framework using the use case presentedin Section 3.5 and a number of relevant scenarios. The first scenario is adaptation


on the device locally, when the available CPU changes (e.g. when another applica-tion starts up in the background). A second scenario shows how our framework isable to discover a remote server on the network and outsources some componentsto this discovered server. The third scenarios introduces changes in the availablebandwidth on the network and illustrates how the application is adapted accord-ingly. Lastly we show how the framework detects network failures and tries torecover from this situation.

3.6.1 Set-up

We first describe the hardware and software set-up used for our experiments. Asmobile device we used a Nokia N900 smartphone equipped with a 600 MHz ARMCortex A8 processor and 256 MB RAM. This device runs Maemo 5 Linux onwhich we run a Sun Java SE for Embedded 6 JVM [34] and a Felix OSGi instance[35], adapted to be able to monitor the bundles as described in [36]. It has acamera capable of video recording at a resolution of 640x480. The server machineis equipped with an Intel Core 2 Duo P8400 CPU clocked at 2.26GHz and runsUbuntu Linux.

The mobile device is connected to the server with a USB cable and they com-municate using Ethernet over USB. The bandwidth on the link can be controlledusing the Linux traffic control (tc) tool. The introduced bandwidth value is alsofed into our monitoring bundle since algorithms for accurate bandwidth estimationusing probe traffic is out of scope of this research [37].

Finally we have to set up the different parameters of our offloading algorithmin order to correctly deploy our augmented reality application. As shown in the se-quence diagram on Figure 3.5, the whole application consists of a repetitive loopof fetching a frame, analyzing it, detecting an object and augmenting the view.This repetitive structure is typical in multimedia applications that involve videoprocessing. Using the Bundle Monitor we collect profile data for each configu-ration, where we collect for one loop the time (in ms) spent in each component(vertex weight wi) and the data communicated (in bits) at each method call (edgeweight cij). The experimental values are shown in Table 3.2.

The remaining parameters to set are the maximum client and server weightsM0 and M1 and the available bandwidth L. In order to calculate a meaningfuldeployment one has to impose a deadline on how long one single loop can take.For example for our augmented reality application we want to process a frame inat most 500 ms, which is an intuitively defined threshold that means one shouldrecognize an object within half a second. The available client weight is then set tothe deadline (500 ms) multiplied by the available percentage of CPU power. Forexample, when there is 90% CPU available, the application has to execute the loopwithin the 450 ms processor time available.

48 CHAPTER 3

Table 3.2: Measurements of the CPU time spent (in ms) in the components for each appli-cation loop for the different configurations on the N900.

conf. 1 conf. 2 conf. 3 conf. 4 conf. 5Video 40 20 40 20 40ContentProvider 40 40 40 40 40Analyzer- Full resolution 10 10 10- Half resolution 10 10Detector- Center Barcode Detector 50- Sobel Barcode Detector 150- FAST Feature Detector 220- SURF Feature Detector 2390 8990Recognizer/Decoder- Barcode Decoder 20 20- Feature Recognizer 500 820 2810

Of course, since the CPU of the server is more powerful than the CPU ofthe mobile device, the instructions executed in 1 ms on the client device will beexecuted much faster on the server. Therefore to determine the server weight M1

one has to multiply by a scale factor, which can be found by comparing executiontimes of the same bundle on the client device and the server device (e.g. in ourset-up the server executes about 30 times faster than the mobile device). Thussuppose there is 50% CPU time available at the server, M1 is given by 500 ms×50%× 30 = 7500 ms. Lastly the available bandwidth which is expressed as bitsper second has to be rescaled taking into account the deadline: in our case it has tobe rescaled to bits per half a second.

3.6.2 Results3.6.2.1 Scenario 1: CPU adaptation

In the first scenario the user is running the shopping assistant application withoutconnection to any remote resources. However, during shopping, the user wantsto listen to some background music and starts up the FM radio player. Becausethe available CPU has to be shared between both applications, the framework willadapt the application to reduce the CPU usage.

Figure 3.7 shows what happens in this scenario: the upper graph presents theCPU usage of the mobile device running the shopping assistant application, thelower graph shows the frames analyzed per second by the shopping assistant ap-plication. At first configuration 2 is deployed: frames are analyzed for detectingbarcodes using sobel filtering and when a barcode is decoded the product informa-


tion is shown on the screen. This happens at a rate of 3 to 4 analyzed frames persecond using all CPU power.

At t = 30s the FM Radio player application is started and less CPU power isavailable for our application, which lead to less frames analyzed per second. TheAgent performs a check at t = 55s and detects less CPU power is available bycalling the Resource Monitor. After recalculation the Agent instructs the BundleInstaller to change to configuration 1, where only the center of the frame is ana-lyzed. After redeploying the application again reaches 5 to 6 analyzed frames persecond, but at the cost of quality to the user: he will have to capture the barcode inthe center of his view before the object is identified.

Figure 3.7: Scenario 1. The upper graph shows the CPU usage on the mobile device, thelower graph shows the frames analyzed per second of the shopping assistantapplication. When another application is started, a lower quality configurationis deployed.

50 CHAPTER 3

3.6.2.2 Scenario 2: Server discovery

Now the user enters a supermarket that wants to enable the customers to use moreadvanced mobile applications and has a set up a local WLAN network with aserver where components can be outsourced to. On entering the supermarket theuser’s mobile device will discover the server and reconfigure the application. Theuser has good connectivity with the WLAN network and the available bandwidthis 10 Mbit/s.

At first configuration 2 is deployed because no remote server is found and noother applications are running on the device. When the jSLP bundle discovers aserver, the Agent is notified and R-OSGi is configured to create the needed end-points for remote calls. The Agent then gathers all resource monitor informationof both the client and the discovered server and a new deployment is calculated.

Figure 3.8: Scenario 2. The upper graph shows the CPU usage on the mobile device and theserver, the middle graph shows the frames analyzed per second of the shoppingassistant application and the lower graph shows the bandwidth usage. Whena remote server is discovered, a higher quality configuration is deployed andcomponents are outsourced.


Since there is high bandwidth available the Bundle Installer is instructed to startconfiguration 5 and outsource all components except the Video bundle. This meansthat all full resolution images are sent to the server for analysis. Figure 3.8 showsthis process: at t = 36s the remote server is discovered and the client is configuredto be able to make remote calls which takes about 10 seconds. Then the compo-nents consisting of configuration 5 are started and outsourced, which is finished att = 70s. One notices that during configuration of the server and the redeploymentthe number of frames analyzed per second drops, since quite some CPU goes tothe redeployment and the responsiveness of the application will diminish. Afterredeployment, the user will have a higher quality version of the application run-ning, and objects will be identified just by looking at them, rather than searchingfor a barcode and scanning it. The bandwidth usage is then around 7 MBit/s whichis the bandwidth needed to send 3 frames per second (1 frame = 640 × 480 × 8

bits).

3.6.2.3 Scenario 3: Bandwidth adaptation

In the third scenario the available bandwidth changes and the framework will adaptthe application accordingly. At the start there is an available bandwidth of 10Mbit/s, but it is first lowered to 3 Mbit/s, and later even to 750 Kbit/s. The frame-work will deploy respectively configuration 4 (sending lower resolution frames)and configuration 3 (keep feature detection local and only send detected featuresto the server) to lower the bandwidth needed by the application as shown on Fig-ure 3.9.

Initially a high bandwidth network is available and configuration 5 that pro-vides the highest quality is deployed. At t = 50s the available bandwidth islowered from 10 to 3 Mbit/s. Due to the lower bandwidth, the number of analyzedframes drops from 1-2 to 0-1 and the framework decides to redeploy. This rede-ployment takes little time since only one bundle has to be changed and both on thedevice and the server there is CPU available to start and outsource this bundle.

At t = 135s the available bandwidth is further lowered to 750 Kbit/s. Againthe analyzed frames per second are lowered and the framework decides again toredeploy. Now more bundles have to be started and little bandwidth is availableto outsource bundles, leading to a redeployment time of 25 seconds. Again duringredeployment the number of analyzed frames per seconds diminishes.

52 CHAPTER 3

Figure 3.9: Scenario 3. The upper graph shows the CPU usage on the mobile device and theserver, the middle graph shows the frames analyzed per second of the shoppingassistant application and the lower graph shows the bandwidth usage. As theavailable bandwidth is lowered, the framework deploys lower quality configu-rations that use less bandwidth.


3.6.2.4 Scenario 4: Network failure recovery

When the connection to the server is lost due to a network failure, the frameworkwill detect the remote server is no longer available and recover from this failure bystarting configuration 2, which can run locally on the device.

Figure 3.10 shows this scenario. Initially configuration 5 is deployed using thediscovered server on a 10 Mbit/s network and at t = 50s the server is stoppedcausing no more frames to be analyzed. At t = 75s the framework detects thefailure and switches back to the local configuration 2.

Figure 3.10: Scenario 4. The upper graph shows the CPU usage on the mobile device andthe server, the lower graph shows the frames analyzed per second of the shop-ping assistant application. When there is a failure of the server or the network,the framework recovers from the failure and switches back to a degraded ver-sion of the application.

3.6.3 Discussion

In four scenarios, we have demonstrated the validity of our framework, the suit-ability to develop applications in different quality configurations and how ourframework switches between those configurations. The only issue that can be

54 CHAPTER 3

experienced by the user is a drop in performance during the redeployment phase,especially in the case when there is not enough CPU power left on the device (Re-deployment in Scenario 2) to start the bundles, or when there is little bandwidthavailable to outsource the bundles (Redeployment 2 in Scenario 3). However, inthis experiment all bundles are started ’cold’, meaning they are only started whenthey are needed. The redeployment time could be further lowered by caching se-lected bundles of all configurations at both the device and the server, ready to beused when a redeployment occurs.

Because the bandwidth limit is imposed using the Linux Traffic Control (tc)tool, the traffic is actually shaped when the available bandwidth limit is exceeded.In a real network setting, this is not the case and trying to send more packets in thenetwork would lead to packet loss, which will deteriorate the application qualityeven more and make it more important to redeploy in order to reduce bandwidthusage.

3.7 Conclusions and future workIn this paper we presented a mobile middleware for adaptive deployment of mul-timedia applications, specifically augmented reality. Using the use case of an aug-mented reality shopping assistant, we introduce the concept of multiple configu-rations of an application offering different qualities. We presented an algorithmfor calculating the deployment offering the best quality possible given the currentconnectivity and available resources. Our framework is built upon the OSGi frame-work and is able to switch configurations and outsource components at runtime.When a remote server is discovered a higher quality configuration is deployed asmore load can be outsourced. When the connection is lost, the application grace-fully degrades to a lower quality configuration. We presented experimental resultsof different scenarios with an implementation of our augmented reality use casethat illustrate the performance and effectiveness of our approach.

Important points for future work are to lower the redeployment time by study-ing caching strategies for bundles and to evaluate our middleware in other scenar-ios.

AcknowledgmentTim Verbelen is funded by PhD grant of the Fund for Scientific Research, Flanders(FWO-V).


References

[1] Gartner Group. Press release : Gartner Says Worldwide Mobile Device SalesGrew 13.8 Percent in Second Quarter of 2010, But Competition Drove PricesDown, August 2010. Available from: http://www.gartner.com/it/page.jsp?id=1421013.

[2] A. Fortier, G. Rossi, S. E. Gordillo, and C. Challiol. Dealing with vari-ability in context-aware mobile software. Journal of Systems and Software,83(6):915–936, 2010.

[3] N. Gui, V. D. Florio, H. Sun, and C. Blondia. Toward architecture-basedcontext-aware deployment and adaptation. Journal of Systems and Software,84(2):185–197, 2011.

[4] G. Klein and D. Murray. Parallel Tracking and Mapping on a camera phone.In Proceedings of the 8th IEEE International Symposium on Mixed and Aug-mented Reality, ISMAR ’09, pages 83–86, 2009.

[5] D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond, and D. Schmalstieg.Real-Time Detection and Tracking for Augmented Reality on Mobile Phones.IEEE Transactions on Visualization and Computer Graphics, 16:355–368,2010.

[6] S. Gammeter, A. Gassmann, L. Bossard, T. Quack, and L. Van Gool. Server-side object recognition and client-side object tracking for mobile augmentedreality. In IEEE Computer Society Conference on Computer Vision and Pat-tern Recognition Workshops, CVPRW ’10, pages 1–8, 2010.

[7] J. J. Hull, X. Liu, B. Erol, J. Graham, and J. Moraleda. Mobile image recog-nition: architectures and tradeoffs. In Proceedings of the Eleventh Workshopon Mobile Computing Systems & Applications, HotMobile ’10, pages 84–88, 2010.

[8] M. Philippsen and M. Zenger. JavaParty – transparent remote objects inJava. Concurrency: Practice and Experience, 9(11):1225–1242, December1997.

[9] M. Dahm. Doorastha – a step towards distribution transparency. In JIT,2000.

[10] M. M. Fuad and M. J. Oudshoorn. AdJava: automatic distribution of Javaapplications. In Proceedings of the 25th Australasian conference on Com-puter science, ACSC ’02, pages 65–75, 2002.

http://www.gartner.com/it/page.jsp?id=1421013


56 CHAPTER 3

[11] M. Tatsubori, T. Sasaki, S. Chiba, and K. Itano. A Bytecode Translator forDistributed Execution of "Legacy" Java Software. In Object-Oriented Pro-gramming, pages 236–255, 2001.

[12] E. Tilevich and Y. Smaragdakis. J-Orchestra: Enhancing Java programswith distribution capabilities. ACM Transactions on Software Engineeringand Methodology, 19(1):1–40, 2009.

[13] G. C. Hunt and M. L. Scott. The Coign automatic distributed partitioningsystem. In Proceedings of the 3rd symposium on Operating systems designand implementation, OSDI ’99, pages 187–200, 1999.

[14] E. Cuervo, A. Balasubramanian, D.-k. Cho, A. Wolman, S. Saroiu, R. Chan-dra, and P. Bahl. MAUI: making smartphones last longer with code offload.In Proceedings of the 8th international conference on Mobile Systems, Ap-plications, and Services, MobiSys ’10, pages 49–62, 2010.

[15] M. Satyanarayanan. Pervasive Computing: Vision and Challenges. IEEEPersonal Communications, 8:10–17, 2001.



[18] X. Gu, A. Messer, I. Greenberg, D. Milojicic, and K. Nahrstedt. AdaptiveOffloading for Pervasive Computing. IEEE Pervasive Computing, 3(3):66–73, 2004.

[19] S. Ou, K. Yang, and J. Zhang. An effective offloading middleware for perva-sive services on mobile devices. Pervasive and Mobile Computing, 3(4):362–385, 2007.

[20] S. Han, S. Zhang, J. Cao, Y. Wen, and Y. Zhang. A resource aware soft-ware partitioning algorithm based on mobility constraints in pervasive gridenvironments. Future Generation Computer Systems, 24(6):512–529, 2008.

[21] M. Kristensen and N. Bouvin. Scheduling and development support inthe Scavenger cyber foraging system. Pervasive and Mobile Computing,6(6):677–692, 2010.



[23] S. Goyal and J. Carter. A Lightweight Secure Cyber Foraging Infrastructurefor Resource-Constrained Devices. In Proceedings of the Sixth IEEE Work-shop on Mobile Computing Systems and Applications, WMCSA ’04, pages186–195, 2004.

[24] Y.-Y. Su and J. Flinn. Slingshot: deploying stateful services in wirelesshotspots. In Proceedings of the 3rd international conference on Mobile sys-tems, applications, and services, MobiSys ’05, pages 79–92, 2005.

[25] M. Satyanarayanan, P. Bahl, R. Caceres, and N. Davies. The Case for VM-Based Cloudlets in Mobile Computing. IEEE Pervasive Computing, 8(4):14–23, 2009.

[26] B.-G. Chun, S. Ihm, P. Maniatis, and M. Naik. CloneCloud: Boosting Mo-bile Device Applications Through Cloud Clone Execution. Technical ReportarXiv:1009.3088, 2010.



[29] E. Guttman, C. Perkins, J. Veizades, and M. Day. Service Location Protocol,Version 2, 1999.

[30] The OSGi Alliance. OSGi Service Platform, Service Compendium, Release4, Version 4.2. aQute, September 2009.

[31] IBM ILOG CPLEX. Available from: http://www.ilog.com/products/cplex/.

[32] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool. Speeded-Up Robust Features(SURF). Computer Vision and Image Understanding, 110:346–359, June2008.

[33] ZXing. Available from: http://code.google.com/p/zxing.

[34] Sun Java SE for Embedded 6. Available from: http://java.sun.com/javase/embedded/index.jsp.

http://www.ilog.com/products/cplex/

http://code.google.com/p/zxing

http://java.sun.com/javase/embedded/index.jsp

http://java.sun.com/javase/embedded/index.jsp

58 CHAPTER 3

[35] Apache Felix. Available from: http://felix.apache.org/site/index.html.

[36] T. Verbelen, R. Hens, T. Stevens, F. Turck, and B. Dhoedt. Adaptive OnlineDeployment for Resource Constrained Mobile Smart Clients. In Proceedingsof the 3rd Conference on Mobile Wireless Middleware, Operating Systems,and Applications, volume 48, pages 115–128, 2010.

[37] C. D. Guerrero and M. A. Labrador. On the applicability of availablebandwidth estimation techniques and tools. Computer Communications,33(1):11–22, 2010.

http://felix.apache.org/site/index.html

4Graph Partitioning Algorithms for

Optimizing Software Deployment inMobile Cloud Computing

“This theory goes as follows and begins now. All brontosauruses are thin at oneend, much much thicker in the middle, and the thin again at the far end.”

–Miss Anne Elk, Monty Python’s Flying Circus

In the previous chapter we introduced an ILP formulation to find the optimalcomponent deployment in order to minimize the required bandwidth. Because ILPis known not to scale well for bigger problem instances, in this chapter we delveinto graph theory and propose a number of graph partitioning heuristics that tradesolution optimality for calculation time. Next to the mobile offloading scenario,these heuristics also prove useful for large software deployment problems, i.e. inthe context of cloud computing.

? ? ?

T. Verbelen, T. Stevens, F. De Turck, and B. Dhoedt.

Future Generation Computer Systems, 29(2):451–459, 2012.

60 CHAPTER 4

Abstract As cloud computing is gaining popularity, an important question is howto optimally deploy software applications on the offered infrastructure in the cloud.Especially in the context of mobile computing where software components couldbe offloaded from the mobile device to the cloud, it is important to optimize thedeployment, by minimizing the network usage. Therefore we have designed andevaluated graph partitioning algorithms that allocate software components to ma-chines in the cloud while minimizing the required bandwidth. Contrary to the tra-ditional graph partitioning problem our algorithms are not restricted to balancedpartitions and take into account infrastructure heterogeneity. To benchmark ouralgorithms we evaluated their performance and found they produce 10 to 40%smaller graph cut sizes than METIS 4.0 for typical mobile computing scenarios.

4.1 Introduction

Nowadays, the emergence of cloud computing is leading to a new paradigm ofutility computing [1], where computing power is offered on an on-demand basis.Users are able to access applications, storage and processing over the Internet, viaservices offered by cloud providers on a pay-as-you-use scheme. The advantagesfor the end users are reduced cost, higher scalability and improved performance incomparison to maintaining own private computer systems, dimensioned for peakload conditions. Moreover the elasticity of the cloud reduces the risks of overpro-visioning (underutilization) or underprovisioning (saturation) [2].

The usage of the cloud is not only beneficial for web based applications, butcan also be used for other applications composed of many service components fol-lowing the service-oriented programming paradigm. Some of these service com-ponents may have high needs regarding CPU power or memory consumption, andshould therefore be executed on dedicated server machines in the cloud rather thanon a regular desktop PC or mobile terminal, for example recognition componentsin an object recognition or speech to text application.

The adoption of the cloud paradigm poses the problem where to deploy soft-ware components, given the many options in terms of available hardware nodesin even moderate scale data centers. This deployment optimization is importantto both the cloud user and the cloud provider in order to reduce costs. All com-ponents need to be deployed on a machine with sufficient CPU power while thecommunication overhead between different machines is preferably minimized asthis introduces extra latency and network load. This problem can be modeled asa graph partitioning problem where a weighted graph of software components hasto be partitioned in a number of parts representing the available machines. More-over the optimal deployment can change over time, and thus in order to realize anoptimal deployment a fast algorithm is desired.

An important scenario in this respect is cloud overspilling. In this scenario,

GRAPH PARTITIONING ALGORITHMS FOR OPTIMIZING SOFTWARE DEPLOYMENT 61

Internet

Public CloudPrivate Cloud

overspill to public cloud

Figure 4.1: Work is offloaded from private infrastructure to a public cloud on peak mo-ments, reducing underutilization (and the cost) of the private infrastructure.

a company offloads work from its own private infrastructure to a public cloud in-frastructure on peak moments as shown in Figure 4.1. This enables the companyto dimension its infrastructure for the average workload instead of the peak work-load, reducing the underutilization and the cost of the private infrastructure. Thissituation is typical in digital document processing where one faces strict month-end or year-end deadlines, and thousands of batches of documents are to be pro-cessed. Typically, document processing systems support workflows consisting ofa few tens of components (including content ingest, reformatting, layouting, merg-ing, versioning, logging, output generation and printing components). As many ofthese components come in different versions, and potentially need to be instanti-ated for each customer separately, the number of components in such a scenarioquickly amounts to a few hundreds.

A second use case is situated in the area of integrated simulation tools forengineering purposes [3]. These integrated tools typically involve multi-physicssimulation (e.g. structural analysis, acoustic simulation and engine dynamics sim-ulation in the case of designing a new automobile engine), and the number of sim-ulation tools involved can easily amount to 10-20 for small engineering projectsto a few hundreds individually deployable components for a realistic engineeringproject. In such engineering endeavours, often parameter sweeps are executed tooptimize the design (or to assess the sensitivity of the resulting design performancewith respect to these parameters), necessitating multiple instances of these simula-tion components running concurrently, in order to arrive at realistic design times.Again, we end up with component graphs containing a few hundreds to thousandscomponents.

This overspilling problem also arises in the context of mobile computing wherethe cloud can be used to enhance the capabilities of a mobile device. Due tothe restricted CPU power of mobile devices, the idea is to offload parts of theapplication at runtime to a cloud infrastructure [4]. The question then is whichparts to offload and to deploy on which machines in the cloud – and how many

62 CHAPTER 4

– in order to spread the load while keeping the needed bandwidth low. In thiscase, the complexity of the partitioning problem depends on the granularity of theoffloading, as offloading can be done on component [4], Java class [5] or method[6] level, resulting in graphs of tens, hundreds or thousands components.

A special case of deployment optimization occurs when multiple mobile de-vices connected via a wireless network share their resources in order to enhancethe user experience of all users, in a so called “cloudlet” [7] as shown in Figure 4.2.This is the case when no Internet uplink is available for offloading to the cloud, orwhen cloud offloading is not beneficial due to a high WAN latency. Because thebandwidth is a scarce resource and shared between all devices in the wireless LAN,a global optimization is needed taking into account all application components ofall devices.

As use case, we mention a mobile augmented reality application. When castingsuch an application into a component framework, the number of independently de-ployable components amounts 5 to 10 [7], with different components for trackingcamera movements, building a 3D map of the environment, recognizing objects,detecting collisions between objects, rendering a 3D overlay, etc. Other appli-cations, such as 3D games, are reported to consist of 10 to 20 components [8].Assuming a few tens of users connected to the same cloudlet and hence sharingcomputing and network resources, the number of components easily exceeds 100.As these users are connected through heterogeneous devices (a mix of low-endand high-end devices), an optimal deployment guaranteeing a minimal quality ofexperience for all users should be aimed for.

In this paper we present algorithms to partition a software application, com-posed of a number of components, on a number of interconnected machines in

Wireless LAN

network

Figure 4.2: A cloudlet consisting of four devices connected by a wireless network sharingtheir resources. Application components are distributed among all devices inthe cloudlet, in order to enhance the user experience of all users.


the cloud with different capacities while minimizing the communication cost be-tween the components. In Section 4.2 related work regarding graph partitioningand task allocation on the grid is discussed. Section 4.3 more formally describesour problem and in Section 4.4 algorithms are proposed that solve the problem. InSection 4.5 the different algorithms are evaluated and compared regarding solutionquality and execution time, and the influence of different parameters is discussed.In view of the use cases mentioned in this introduction, we focus on graphs con-taining hundreds to thousands components for this evaluation. We also compareour solutions to METIS 4.0 for partitioning graphs in k balanced partitions andshow the applicability of our algorithms in the mobile offloading scenario. FinallySection 4.6 concludes this paper.

4.2 Related work

4.2.1 Graph partitioning

Graph partitioning is a fundamental problem in many domains of computer sciencesuch as VLSI design [9], parallel processing [10] and load balancing [11]. Thegraph partitioning problem tackles the problem of dividing a graph in k equal setswhile minimizing the edges between the sets. When k = 2 this is also referred toas the min-cut bi-partitioning problem. Finding a good solution for this problem isknown to be NP-Hard [12]. In the following we give a brief overview of the state-of-the-art and recent advances in graph partitioning. For a more detailed survey ofgraph partitioning techniques we refer to [13] and [14].

A first class of algorithms are the so called move-based approaches, which tryto iteratively improve the partition by vertex moves or swaps between the partssuch as the Kernighan-Lin (KL) algorithm [15]. By choosing moves that intro-duce a cost reduction of the graph cut this algorithm converges to a local optimum.Fiduccia and Mattheyses introduced a number of optimizations to the KL algo-rithm which led to a linear time algorithm for graph partitioning [16]. These move-based algorithms can be combined with stochastic methods such as simulated an-nealing [17], particle swarm optimization [18] or ant colony optimization [19] inorder to escape from local optima. The biggest disadvantage of iterative improve-ment methods is that their performance deteriorates as the graphs get larger.

In order to partition large graphs the multilevel approach became widely adopt-ed [20], [21], [17], [18], [19], [22]. The main idea is to iteratively coarsen theinitial graph by merging vertices according to a matching until a small graph witha similar structure remains. This graph can then be partitioned with a spectralmethod [23], [20] or a greedy graph growing algorithm [24]. Next the graph isagain iteratively uncoarsened and a local improvement heuristic such as the KLalgorithm is applied at each level. The multilevel scheme is also used in state-

64 CHAPTER 4

of-the-art graph partitioning libraries such as METIS [24], SCOTCH [25] andJOSTLE [26].

Recent work in graph partitioning explores methods based on diffusion [11] ormaximum flow [27]. Also the combination of known techniques can result in newheuristics. Chardaire et al. use a PROBE (Population Reinforced OptimizationBased Exploration) heuristic, combining greedy algorithms, genetic algorithmsand KL refinement [28]. Loureiro et al. propose a greedy graph growing heuristiccombined with a local refinement algorithm [29]. Martin uses a genetic algorithmimproved with spectral methods and KL refinement [30].

All these methods partition the graph in a predefined number of parts of equalsizes. In the context of cloud computing, not all machines have equal capacityand also the number of machines that has to be used is not predefined, thus thesealgorithms can not be readily used. Therefore, our algorithms take a number ofpossible machines with different capacities as input with the only constraints thatno machine can exceed its maximum capacity and every component has to bedeployed on exactly one machine. However, we can still use the ideas from theoriginal graph partitioning problem to calculate a good deployment.

4.2.2 Task allocation on the grid

In the context of grid computing the graph partitioning problem is used for theallocation of parallel tasks on heterogeneous infrastructure. Many proposed algo-rithms such as MiniMax [31], VHEM [32], QM [33] PaGrid [34], and MinEX [35]use the multi level paradigm in combination with a refinement algorithm, whileothers like PART [36] use simulated annealing.

These algorithms all focus on parallel applications based on mesh models, suchas fluid dynamics, where a big task is split up in smaller parallel tasks that are eachexecuted on one of the processors in the grid. The goal is then to execute all thesetasks as fast as possible, thus minimizing the execution time of the slowest node.In our problem the execution time metric makes less sense, since we focus onapplications on the cloud, where components generate load as long as the end userruns the application. Also the structure of the task graph for the grid is based onthe input mesh, while in our case the graph of cloud applications is based on thesoftware components and their interactions, which is structured differently and ofa more modest size.

4.3 Problem statement

Let G = (V,E) be an undirected graph where V = {v1, v2, ..., vN} is a set ofN vertices and E is a set of edges connecting the vertices. The vertices repre-sent units of deployment in a distributed software system and the edges represent


communication overhead between those units. Each vertex vi is assigned a costwi that indicates the amount of resources (i.e. CPU power) this component needs.C = (cij) is the adjacency matrix of G, i.e. if there exists an edge between vi andvj then cij equals the weight of this edge, otherwise cij = 0. The edge weightsrepresent the cost of communication (i.e. bandwidth) between different softwarecomponents.

The infrastructure is also modeled as an undirected graph S = (K,L), whereK is a set of available machines and L the links between the different machines.Each machine has a maximum capacity of Mm and each link is assigned a cost touse this link. From this graph a matrix B = (bmn) can be deduced where elementbmn represents the cost to exchange data between machine m and machine n.

The goal now is to assign each vertex vi to one of the machines so that thetotal data exchanged between the machines weighted by B is minimized. Moreformally, let the decision variables Xim represent the graph cut. The value ofXim is equal to 1 if component i is deployed on machine m, and 0 otherwise.We want to minimize the sum of the edge weights of the edges between nodesdeployed on different machines, thus we introduce variable hij which is equal to 1if components i and j are deployed on different machines, 0 otherwise. Then theobjective function to minimize becomes the weight of the graph cut (GC):

GC =∑i,j

hij × cij × bP (i)P (j) (4.1)

With the function P (j) returning the machine where vertex j is deployed on.Variables hij can be expressed in terms of Xim as follows:

hij = 1−∑m

Xim ×Xjm (4.2)

Two more constraints are needed to fully describe our problem.

∀k :∑i

wi ×Xim ≤Mm (4.3)

∀i :∑m

Xim = 1 (4.4)

Equation 4.3 states that the total amount of resources used by nodes on a machinecannot exceed the maximum capacity of that machine. Equation 4.4 makes surethat every node is deployed on exactly one machine.

4.4 Algorithms description

4.4.1 Integer Linear Programming (ILP)

The problem defined in Section 4.3 can be seen as an Integer Linear Programming(ILP) problem, thus an ILP solver (IBM ILOG CPLEX [37]) can be used to de-

66 CHAPTER 4

termine the optimal solution for this problem. The amount of time and resourcesneeded to solve this problem grows exponentially with the size of the graph, soheuristics are needed to find a good solution faster. We can still use the ILP so-lution for smaller graphs to benchmark our algorithms that exhibit better scalingbehavior, possibly at the expense of optimality.

4.4.2 Multi level graph partitioning (MLKL)

In a first heuristic we use a multilevel refinement strategy as first proposed byHendrickson and Leland [20]. The idea is to coarsen down the graph by mergingconnected vertices until a small graph is obtained. Then this graph is partitionedand uncoarsened again, while optimizing the partition in each uncoarsening step.Thus, the partitioning consists of three phases:

Coarsening The graph G is coarsened in a sequence of smaller graphsG1, G2, ..., Gm such that |V1| > |V2| > ... > |Vm|

Initial partitioning A partition P is computed on the coarsest level of the graph.

Uncoarsening and refinement Partition P of graph Gm is uncoarsened backthrough all the intermediate graphs. Each uncoarsening step a refinementalgorithm is executed in order to find a better partition.

4.4.2.1 Coarsening

During coarsening, from graph Gi a graph with fewer vertices Gi+1 is createdby collapsing edges and combining the vertices connected by those edges. Whenan edge is collapsed the two vertices connected by the edge are reduced into onewhose weight is the sum of the weights of both vertices. When both vertices havean edge to a third node, these two edges are collapsed in one edge with a summededge weight. Every iteration of coarsening, a matching – a set of edges withoutcommon vertices – is created and the matched vertices are combined. In order tofind a small edge cut, it is beneficial to collapse heavy weighted edges, since theseare unlikely to be in the best cut.

Our coarsening algorithm uses the heavy-edge matching (HEM) algorithmwhich is a widely used coarsening scheme proposed by [24]. The vertices of thegraph are visited in a random order and each vertex u is matched with an un-matched neighbor v such that the weight of the edge (u, v) is the maximum overall valid incident edges of u. In order to be able to map the vertices of the coars-est graph on the different machines we added one more constraint to only matchtwo vertices when the sum of their vertex weights is smaller than the size of thesmallest part.


4.4.2.2 Initial partitioning

After coarsening the problem consists of choosing a feasible deployment of thesoftware components on the infrastructure at hand. We assume that there areenough machines and resources available to find such a deployment. This prob-lem reduces to a simple bin packing problem that can be solved with a first-fitalgorithm.

The vertices are ordered by descending vertex weight and the machines areordered by descending maximum capacity. For each vertex the list of machines isiterated and it is assigned to the first one that has enough capacity left and capacityof that machine is diminished with the vertex weight of that vertex.

Algorithm 4.1 Calculate initial partition

vertices : OrderedListmachines : OrderedListfor all vertex vi ∈ vertices do

for all machine m ∈ machines doif wi ≤Mm thenP (i)← mMm ←Mm − wi

breakend if

end forend for

4.4.2.3 Uncoarsening and refinement

In the final step the graph is gradually uncoarsened again and a KL-like algorithmis applied to improve the initial partition found in the previous step. The proposedalgorithm is based upon the refinement algorithm of Hendrickson et al. [20]. Thefundamental idea of the algorithm is the concept of the gain associated with mov-ing a vertex to a different part. The gain reflects the net change in cut size thatwould result from switching a vertex from one machine to another. More formally,the gain introduced by moving vertex i from setm to set n, the corresponding gaingmni can be expressed as

gmni =∑

j∈V &P (j)=n

cij × bmn −∑

j∈V &P (j)=m

cij × bmn

+∑

j∈V &P (j)=p,p 6=m,p 6=n

cij × (bmp − bnp) (4.5)

where again the function P (j) returns the current part of vertex j. The base re-finement algorithm is presented in pseudocode as Algorithm 4.2, and is composed

68 CHAPTER 4

of two loops. Each pass of the outer loop will try moving vertices around to find abetter cut. This outer loop ends after a pass that did not improve the best partitionso far, implying that the algorithm has reached a local optimum. To avoid that thealgorithm gets stuck in an infinite loop a vertex can only be moved once during asingle pass of the outer loop. The inner loop will iteratively select a vertex to movei.e. the vertex having the largest gain, subject to some additional rules. Notice thatthis also could be a move with a negative gain. This gives the algorithm the pos-sibility to escape to some extent from getting stuck in a local optimum. Whena vertex is moved the gains of all its neighbors should be updated. This processcompletes when no more suitable move is found.

Algorithm 4.2 KL Refinement

Bestpartition← CurrentpartitionCompute initial gainsCompute ∀p : Free(p) the amount of free space on part prepeatMovedList← EmptyListrepeatMoveFound← Falserepeat

Select vertex move, not in MovedList with highest gain gmni

if ∃p : Free(p) < 0 thenif Free(m) < 0 and Free(n)− wi > Free(m) thenMoveFound← True

end ifelseMoveFound← True

end ifuntil MoveFoundif MoveFound then

Perform move and add move to MovedListUpdate gains of neighbors of moved vertexif Currentpartition < Bestpartition thenBestpartition← Currentpartitionif No possible moves with positive gain then

Breakend if

end ifend if

until No more moves possibleuntil No better partition foundreturn BestPartition


60

4060

m0 = 100 m1=200

40

100200

1000

100

1000

n0

n1

n2

n3

n4

Figure 4.3: This partition cannot be optimized when no moves exceeding the target partsmaximum boundary are allowed.

When a better partition is found, the inner loop will only accept moves withpositive gain and then start another pass. The rationale for this is that when thealgorithm finds a better partition, it will try to descend as deep as possible to getthe local optimum. Then, a new iteration is started, and the moves that were inMovedList in the previous iteration can again be selected to come closer to theoptimum.

Instead of creating balanced partitions, we need to make sure that no machinegets overloaded. A naive rule in this respect would be to forbid vertex moves thatwould violate the capacity condition of the target. However, this rule could causeto move to local optima, from which there is no escape (see Fig. 4.3).

Therefore we do allow a move to a part that exceeds the maximum part weight,provided that there is no other part with excessive weight. If there is a part s withtoo much weight, then we choose the vertex with the highest gain that moves awayfrom m and reduces the amount of overload. This way situations as in Figure 4.3will also converge to an optimum (m0 = {n0, n2},m1 = {n1, n3, n4}).

When two possible moves have equal gain, we prefer moves that do not exceedthe target parts boundary. If that does not differentiate the possible moves, weprefer the ones to a part that is not empty. When still no differentiation is madebetween two moves, ties are broken randomly.

Instead of continuing until all vertices are moved, one can stop the algorithmearlier to save time, for example when the graph cut of the current partition devi-ates more than a certain cutoff threshold of the best solution found at that moment.Proper choices for this threshold can reduce the execution time while not sacri-ficing much solution quality. If a threshold value of 0 is chosen, the algorithmwill not accept any moves with negative gain, and the algorithm will behave like asteepest descent algorithm ending in the nearest local minimum.

70 CHAPTER 4

4.4.3 Simulated Annealing (SA)

A second approach followed for solving the k-partitioning problem, is based onsimulated annealing (SA), a combinatorial optimization technique introduced byKirkpatrick [38] and independently by Cerny [39], inspired by the cooling processof metal. To use the SA technique in the context of the k-partitioning problem, weuse it as a refinement technique after bin packing, inspired by Johnson et al. whoshowed the effectiveness of SA for the graph bi-partitioning problem [40].

The SA algorithm moves from one solution to a neighbor solution by mov-ing a vertex from one part to another. A move will be accepted with probabilityexp(g/T ), in which g is the gain of a move, as introduced in Section 4.4.2, and Tthe temperature parameter that is lowered gradually over time. The algorithm inpseudocode is shown as Algorithm 4.3. In order to find the optimum in situationsas described in Figure 4.3, vertex i is moved with positive gain to a part that hasnot enough capacity left with probability exp(Free(p)−wi

T ) where Free(p) is theamount of free space on part p. By making this also dependent on the tempera-ture this assures that towards the end of the algorithm it will converge to a validsolution without causing over-occupied parts.

The performance of SA is very dependent on the choice of the different an-nealing parameters: the initial temperature T1, the cooling schedule, the epochlength L, and the stopping condition [40]. Park et al. later proposed a better set ofannealing parameters that led to better results [41], on which we have based ourparameter values.

4.4.3.1 Initial temperature T1

Kirkpatrick et al. [38] propose a value of T1 high enough to make the initial prob-ability of accepting transitions to be close to 1. However, a too high initial tem-perature may lead to an unnecessary long computation time as pointed out in [40]and [41], where it is suggested that the initial temperature is chosen such that thefraction of accepted uphill transitions at the initial temperature is equal to a pa-rameter P1, called the initial acceptance probability. The initial temperature isthen calculated as T1 = ∆

P1, where ∆ is calculated with the initial negative gains

only.

4.4.3.2 Cooling function

The cooling function will gradually decrease the temperature until the algorithmreaches its stopping condition. We adopt a simple geometric cooling function,in which the temperature at the kth epoch is given by Tk = α × Tk−1 whereα(0 < α < 1) is a parameter called the cooling ratio. Johnson et al. [40] showedthat other cooling functions such as linear or logarithmic functions do not affectthe performance.


Algorithm 4.3 Simulated Annealing

Bestpartition← CurrentpartitionTemperature← StartTemperatureT1

repeatcounter ← 0repeatcounter + +Calculate gain g to move vertex i to part pif g ≥ 0 then

if Free(p) ≥ wi thenPerform move

elsePerform move with probability e

Free(p)−wiTemperature

end ifelse

if Free(p) ≥ wi thenPerform move with probability e

gTemperature

end ifend ifif Currentpartition < Bestpartition thenBestpartition← Currentpartition

end ifuntil counter ≥ L (epoch length)Temperature← α.Temperature

until stopconditionreturn BestPartition

4.4.3.3 Epoch length L

The epoch length L is the number of moves tried at each temperature level. John-son et al. state that the epoch length should be chosen proportional with the neigh-borhood size. In our case each vertex can be moved to one of the other machines,thus the epoch length becomes L = s×N×(K−1) withN the number of verticesand K the number of machines used. Experiments showed 50 to be a good valuefor s.

4.4.3.4 Stopping condition

The stopping condition decides when the algorithm reaches the frozen state andfurther search for a solution should be stopped. Trivial stopping conditions termi-nate the algorithm when the epoch count reaches a predefined maximum, or whenthe temperature drops below a pre-selected final temperature. We use Johnson’sstopping rule [40] which increments a counter by one at the end of an epoch and

72 CHAPTER 4

the fraction of accepted moves is less than a predefined limit Fmin and the counteris reset to 0 when a better solution is found. The SA is terminated when the counterreaches a predetermined limit I .

The actual values for the different parameters we use are shown in Table 4.1,which are based on the ones stated in [40] and [41], and which lead to good resultsfor our problem.

Table 4.1: Simulated annealing parameter values used in our SA refinement algorithm.

Parameter ValueP1 0.627α 0.908s 50Fmin 0.02I 5

Because of the randomness in Simulated Annealing, different runs can lead todifferent solutions. The quality of the solution can be improved by executing theSA algorithm multiple times and taking the best solution. This comes of course atthe cost of more computation time.

4.4.4 Hybrid algorithm (KLSA)

Because of the random factor of SA, it will typically explore a larger solution spacethan KL based refinement, and hence arrive at better solutions at the cost of morecomputation time. To combine the best of both approaches, a hybrid approach ispursued in this section. The hybrid algorithm very much resemble the multi-levelalgorithm: only at the coarsest level the partition is first refined with a few runs ofSA. At the coarsest level a better solution can be found with SA while the extracomputation cost remains relatively low. In further uncoarsening iterations KLrefinement is used again, since this is faster. Moreover, due to the characteristicsof coarsening and uncoarsening, the globally optimal solution is expected to besituated in the neighborhood of the coarsest graph, and thus the local optimumwhere KL gets stuck in the end is more likely also the global optimum. Again, theSA refinement at the coarsest level can be performed multiple times in order to geta better solution.

4.5 Evaluation ResultsTo evaluate our algorithms we generated test graphs with different node sizes.Graphs were generated with the Eppstein power law generator [42] and the weights


of the nodes and edges are assigned according an exponential distribution with pa-rameter λ equal to 0.1 and 0.005 respectively. Although the actual graph structureof an application will depend on the software design, we chose these graph config-urations as most design principles aim to reduce the dependencies between differ-ent software components and thus resulting in graphs with few nodes with manyneighbors. The resulting node weights are restricted between 1 and 100, whichcould intuitively represent the percentage CPU time needed for this component ona single processor core.

These nodes have to be deployed on machines with randomly generated capac-ities of 100, 200, ..., 800 which represents machines equipped single, dual, ..., eightcore processors. The number of machines is chosen such that their total capacity isabout 150% of the total node weight of the graph. For each graph size we generate100 graphs and machine weights and calculate the best deployment with our dif-ferent algorithms implemented in Java, which are compared in solution graph cutsize and in processing time. All calculations were performed on Intel Xeon L5420(2.5 GHz) quad core processors.

First we compare our algorithms with the theoretically optimal ILP solutionfor small graph sizes to show the effectiveness of our approach. For bigger graphswe compare our three heuristics on solution quality and execution time and lookat the influence of the cutoff threshold for KL refinement and the number of ex-ecutions for SA. To benchmark our algorithms to the state-of-the-art we comparethe solution quality of our algorithms with METIS 4.0. Finally we also show theeffectiveness of our algorithms in the offloading scenario where a mobile devicedistributes its load to the cloud.

4.5.1 Evaluation of MLKL, SA and KLSA

To evaluate how good deployments are found for cloud applications, we assumethat the communication links between the different servers in the datacenter areequally weighted, and thus ∀m,n : bmn = 1. Since the graphs represent interact-ing software components, we also assume that the graph size of interest is a fewhundred nodes. For small graphs we can compare solutions to the optimal ILPsolution as shown in Figure 4.4.

Both the SA and KLSA algorithm use the best solution of 4 parallel executionsof simulated annealing. For small graphs simulated annealing performs best as itresults in the optimal solution in more than 50% of the cases. The hybrid solutiontends to give slightly better results than MLKL. However better solution qualitycomes at the cost of execution time: KLSA and SA use respectively 2 to 10 timesthe execution time of MLKL.

For larger graphs the ILP solver takes too long to find a solution so we comparethe algorithms relative to the MLKL algorithm. We will also discuss the influence

74 CHAPTER 4

0

0.2

0.4

0.6

0.8

1

1.2

1.4

10 20 30 40 50 60 70 80 90 100

Gra

ph

cu

t siz

e r

ela

tive

to

IL

P s

olu

tio

n

#nodes

MLKLKLSA 4

SA 4

(a)

0

100

200

300

400

500

600

700

800

900

1000

10 20 30 40 50 60 70 80 90 100

Exe

cu

tio

n t

ime

(m

s)

#nodes

MLKLKLSA 4

SA 4

(b)

Figure 4.4: In (a) the median of the resulting graph cut size is shown of the MLKL, SAand KLSA algorithms relative to the optimal ILP solution. Simulated annealingreaches the optimum in more than 50% of the cases, while MLKL performsworse. KLSA is a bit better than MLKL but can not achieve the quality of SA.In (b) the execution time of the different algorithms is shown. Better solutionquality comes at a price: KLSA and SA use respectively 2 to 10 times moreexecution time than MLKL.


0

0.2

0.4

0.6

0.8

1

1.2

1.4

100 200 300 400 500 600 700 800 900 1000

Gra

ph

cu

t siz

e r

ela

tive

to

ML

KL

#nodes

MLKL 0MLKL 100MLKL 500

MLKL 1500

(a)

0

200

400

600

800

1000

1200

1400

1600

1800

100 200 300 400 500 600 700 800 900 1000

Exe

cu

tio

n t

ime

(m

s)

#nodes


MLKL 1500MLKL

(b)

Figure 4.5: In (a) the influence of a cutoff threshold is shown on the solution quality relativeto the MLKL algorithm without cutoff threshold, while in (b) the execution timeis plotted. A lower cutoff threshold lowers execution time at the cost of a lowersolution quality. The improvement of solution quality by allowing moves withnegative gains is lowered as graph size gets larger.

76 CHAPTER 4

0

0.2

0.4

0.6

0.8

1

1.2

100 200 300 400 500 600 700 800 900 1000

Gra

ph

cu

t siz

e r

ela

tive

to

ML

KL

#nodes

SA1SA2SA4SA8

(a)

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

100 200 300 400 500 600 700 800 900 1000

Exe

cu

tio

n t

ime

(m

s)

#nodes

SA 1SA 2SA 4SA 8

(b)

Figure 4.6: In (a) the solution quality of SA relative to the MLKL algorithm is plotted, while(b) shows the execution time. For smaller graphs SA results in better partitions,but uses much more execution time than MLKL.


0

0.2

0.4

0.6

0.8

1

1.2

100 200 300 400 500 600 700 800 900 1000

Gra

ph

cu

t siz

e r

ela

tive

to

ML

KL

#nodes

KLSA 1KLSA 4KLSA 8

(a)

0

5000

10000

15000

20000

25000

30000

35000

40000

100 200 300 400 500 600 700 800 900 1000

Exe

cu

tio

n t

ime

(m

s)

#nodes

KLSA 1KLSA 4KLSA 8

(b)

Figure 4.7: In (a) the solution quality of KLSA relative to the MLKL algorithm is plotted,while (b) shows the execution time. KLSA results in slightly better partitionsthan MLKL, but this improvement diminishes as the graph size gets larger. Re-garding execution time it performs better than SA, but it is still slower thanMLKL.

78 CHAPTER 4

of different parameters: the cutoff threshold for the KL refinement and the numberof parallel executions of the SA refinement.

Figure 4.5 on the left shows the solution quality of the MLKL algorithm forvalues 0, 100, 500 and 1500 for the cutoff threshold. KL refinement is stoppedwhen the graph cut of the current partition deviates more than a certain cutoffthreshold of the best solution found at that moment. A cutoff threshold of 0 meansthat no moves with negative gains are allowed and then the refinement behaves asa steepest descent algorithm. The higher the threshold, the more it approaches theoriginal solution without threshold. As the graph size gets larger, less improve-ment is reached by allowing moves with negative gains. On the right the averageexecution time of the different algorithms is plotted. Using a cutoff threshold canlower the execution time at the cost of solution quality. Choosing a threshold highenough (1500 in our case) leads to a small profit in execution time without sacri-ficing much solution quality.

Figure 4.6 shows the solution quality of the SA algorithm compared to theMLKL algorithm (a) and the execution time (b) for different number of executionruns. For smaller graphs SA finds better solutions than MLKL, but as the graphsize gets larger MLKL becomes better, as due to the increase in search space SAwill less likely find the random move that leads to a better solution. More execu-tion runs of the SA algorithm leads to better solution quality at the cost of moreexecution time. Since the experiments were conducted on quad core machinesthere is a much bigger increase between 4 and 8 than between 1 and 4. As alreadyseen for small graphs in Figure 4.4, the execution time of SA is much higher thanof MLKL.

The hybrid approach (KLSA) shown on Figure 4.7 does a few percents betterthan MLKL, but this improvement becomes smaller as the graph size gets larger.Again this improvement comes at the cost of execution time. Since the expensiveSA refinement is now only executed at the coarsest level, its execution time islower than SA but still much higher than MLKL.

4.5.2 Comparison with METIS 4.0

A special case of our problem statement, where ∀m,n : bmn = 1 and ∀m :

Mm = (∑

i wi)(1 + β)/K represents the traditional graph partitioning problemthat partitions the graph in K balanced partitions with imbalance factor β. For thisproblem we can compare our algorithm to well known graph partitioning softwaresuch as METIS [24].

Figure 4.8 shows the solution quality of our MLKL algorithm relative to thesolution of kmetis to partition our graphs in 2, 4, 8, 16, 32 and 64 balanced parts.Although both algorithms are based on the multilevel partitioning paradigm andKL like refinement, our algorithm results in 10 to 40% lower cut sizes. This is due


0

0.2

0.4

0.6

0.8

1

1.2

100 200 300 400 500 600 700 800 900 1000

Gra

ph

cu

t siz

e r

ela

tive

to

Me

tis

#nodes

MLKL 2MLKL 4MLKL 8


Figure 4.8: Solution quality of the MLKL algorithm relative to METIS 4.0 to partition thegraph in 2, 4, 8, 16, 32 or 64 balanced parts. Our algorithm results in 10 to 40%lower graph cuts.

to the fact that our algorithm allows more imbalance during the refinement, andthe faster cutoff during refinement of METIS, as it is designed to scale to larger(> 10000 vertices) graphs.

4.5.3 Offloading scenario

Until now we always assumed that all application components are deployed in thecloud where all servers are interconnected by the same links and thus ∀m,n :

bm,n = 1. However, in the context of mobile cloud computing where the cloudis used to offload parts of the application from the mobile device to the cloud,an important limiting factor will be the wireless link connecting the device to theInternet. Depending on the technology and the signal strength the available band-width between the mobile device and the cloud – and thus the optimal deployment– will vary. In order to calculate an optimal deployment in this case, the bmn pa-rameters can be used to attach more importance to the wireless link and try to limitthe amount of data exchanged via that link.

To calculate the optimal deployment in the offloading scenario, we added oneextra machinemdevice with a small capacity of 25 to our randomly generated list ofmachines and locked one of the nodes to this machine. This represents for examplea node that takes care of the input or the GUI and must be deployed on the mobile

80 CHAPTER 4

0

500

1000

1500

2000

2500

1.0 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.1

Wire

less lin

k w

eig

ht

α

Graph 1Graph 2Graph 3

(a)

200

300

400

500

600

700

800

900

1.0 1.01 1.02 1.03 1.04 1.05 1.06 1.07 1.08 1.09 1.1

Wire

less lin

k w

eig

ht

α

Graph 1Graph 2Graph 3

(b)

Figure 4.9: By increasing the parameter α the bandwidth on the wireless link can be influ-enced. For fine grained graphs composed of many small components shown in(a) the influence is more gradual than for coarser graphs shown in (b).


device. The connectivity weights between the machines are set to 1 between theserver machines, but to α ≥ 1 for each link between a server and mdevice. Whenincreasing α the algorithm will calculate a new deployment and will try to lowerthe bandwidth on the wireless link.

Of course the granularity with which one can influence the bandwidth on thewireless link will depend on the design of the software and the granularity of thecomponents. If the software is composed of many small components one willbe able to gradually tune the bandwidth requirements, but when the software iscomposed of few big components less possibilities are available. Figure 4.9 showsthe weight on the wireless link in function of α for six graphs. On the left resultsare shown for three graphs composed of 500 small (λ=0.5) components, while onthe right the results for three coarser graphs of 200 bigger (λ=0.1) components areplotted. The more fine grained graphs on the left can be tuned more gradually thanthe coarser graphs on the right.

4.6 Conclusions and future workIn this paper we presented algorithms for partitioning software graphs for deploy-ment on the cloud. The best partition is calculated taking into account the infras-tructure heterogeneity. In contrast to deployment optimization for computationalgrids, we do not minimize execution time of a set of mesh structured tasks, but wefocus on minimizing bandwidth between software components. A multilevel KLbased algorithm is presented as a fast partitioner which allows real-time deploy-ment calculations. Simulated annealing improves the solution quality for smallgraph sizes, at the cost of computation capacity. A hybrid algorithm producesslightly better partitions than KL in an intermediate execution time. We comparedour KL based algorithm to METIS for a wide range of graphs and found 10 to40% better partitions. We also showed how our algorithm can be used to adapt thedeployment to take into account wireless link quality in a mobile cloud computinguse case.

Future work consists of integrating these algorithms in an actual software de-ployment framework to automatically distribute software components on a cloudinfrastructure. The algorithms can also be adapted to optimize alternative objec-tives, for example energy consumption.

82 CHAPTER 4

References

[1] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic. Cloud comput-ing and emerging IT platforms: Vision, hype, and reality for delivering com-puting as the 5th utility. Future Generation Computer Systems, 25(6):599–616, 2009.

[2] M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. H. Katz, A. Konwinski,G. Lee, D. A. Patterson, A. Rabkin, and M. Zaharia. Above the Clouds: ABerkeley View of Cloud Computing. Technical report, 2009.

[3] Noesis Solutions. Optimus. Available from: http://www.noesissolutions.com/Noesis/.

[4] T. Verbelen, T. Stevens, P. Simoens, F. De Turck, and B. Dhoedt. Dynamicdeployment and quality adaptation for mobile augmented reality applica-tions. Journal Systems Software, 84:1871–1882, November 2011.


[6] M. Kristensen. Scavenger: Transparent development of efficient cyber forag-ing applications. In Pervasive Computing and Communications (PerCom),2010 IEEE International Conference on, pages 217 –226, 2010.

[7] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt. Cloudlets: Bringingthe cloud to the mobile user. In Proceedings of the 3rd ACM Workshop onMobile Cloud Computing & Services, MCS ’12, pages 29–36, 2012.


[9] C. Alpert. Recent directions in netlist partitioning: a survey. Integration, theVLSI Journal, 19(1-2):1–81, August 1995.

[10] B. Hendrickson. Graph partitioning models for parallel computing. ParallelComputing, 26(12):1519–1534, November 2000.

[11] H. Meyerhenke, B. Monien, and S. Schamberger. Graph partitioning anddisturbed diffusion. Parallel Computing, 35(10-11):544–569, 2009.

[12] T. N. Bui and C. Jones. Finding good approximate vertex and edge partitionsis NP-hard. Information Processing Letters, 42(3):153–159, 1992.

http://www.noesissolutions.com/Noesis/

http://www.noesissolutions.com/Noesis/


[13] P.-O. Fjallstrom. Algorithms for graph partitioning: A survey. Computer andInformation Science, 3(10), 1998.

[14] K. Schloegel, G. Karypis, and V. Kumar. The Sourcebook of Parallel Com-puting, pages 491–541. Morgan Kaufmann, 2003.

[15] B. Kernighan and S. Lin. An efficient heuristic procedure for partitioninggraphs. Bell System Technical Journal, 49(2):291–307, 1970.

[16] C. Fiduccia and R. Mattheyses. A linear-time heuristic for improving networkpartitions. In Papers on Twenty-five years of electronic design automation,page 247, 1988.

[17] L. Sun and M. Leng. An Effective Multi-level Algorithm Based on SimulatedAnnealing for Bisecting Graph. Lecture Notes in Computer Science, 4679:1,2007.

[18] L. Sun, M. Leng, and S. Yu. A New Multi-level Algorithm Based on Par-ticle Swarm Optimization for Bisecting Graph. Lecture Notes in ComputerScience, 4632:69, 2007.

[19] M. Leng and S. Yu. An effective multi-level algorithm based on ant colonyoptimization for bisecting graph. Lecture Notes in Computer Science,4426:138, 2007.

[20] B. Hendrickson and R. Leland. A multilevel algorithm for partitioninggraphs. In Proceedings of the 1995 ACM/IEEE conference on Supercom-puting, Supercomputing ’95, 1995.

[21] Y. Saab. An effective multilevel algorithm for bisecting graphs and hyper-graphs. IEEE Transactions on Computers, 53(6):641–652, 2004.

[22] C. Aykanat, B. Cambazoglu, and B. Ucar. Multi-level direct K-way hyper-graph partitioning with multiple constraints and fixed vertices. Journal ofParallel and Distributed Computing, 68(5):609–625, October 2007.

[23] A. Pothen, H. D. Simon, and K.-P. Liou. Partitioning Sparse Matrices withEigenvectors of Graphs. SIAM Journal on Matrix Analysis and Applications,11(3):430–452, 1990.

[24] G. Karypis and V. Kumar. A fast and high quality multilevel scheme for parti-tioning irregular graphs. SIAM Journal on Scientific Computing, 20(1):359–392, 1999.

[25] F. Pellegrini. Scotch and libScotch 5.1 User’s Guide. LaBRI, Université Bor-deaux, August 2008. Available from: http://www.labri.fr/~pelegrin/scotch/.

http://www.labri.fr/~pelegrin/scotch/

84 CHAPTER 4

[26] C. Walshaw and M. Cross. JOSTLE: Parallel Multilevel Graph-PartitioningSoftwareâASAn Overview. Mesh Partitioning Techniques and Domain De-composition Techniques, pages 27–58, 2007.

[27] R. Khandekar, S. Rao, and U. Vazirani. Graph partitioning using singlecommodity flows. Journal of the ACM, 56(4):1–15, 2009.

[28] P. Chardaire, M. Barake, and G. P. McKeown. A PROBE-Based Heuristic forGraph Partitioning. IEEE Transactions on Computers, 56(12):1707–1720,December 2007.

[29] R. Loureiro and A. Amaral. An efficient approach for large scale graphpartitioning. Journal of combinatorial optimization, 13(4):289âAS320, 2007.

[30] J. G. Martin. Spectral techniques for graph bisection in genetic algorithms.Proceedings of the 8th annual conference on Genetic and evolutionary com-putation, page 1249, 2006.

[31] S. Kumar, S. Das, and R. Biswas. Graph Partitioning for Parallel Applica-tions in Heterogeneous Grid Environments. In Proceedings of the 16th In-ternational Parallel and Distributed Processing Symposium, volume 00, page167, 2002.

[32] B. Arafeh, K. Day, and A. Touzene. A paradigm for allocating parallelapplication tasks to heterogeneous computing resources on the grid. In Pro-ceedings of the International Conference on Parallel Computing, ParCo ’05,2005.

[33] P. Phinjaroenphan, S. Bevinakoppa, and P. A heuristic algorithm for mappingparallel applications on computational grids. Advances in Grid, 3470:1086–1096, 2005.

[34] S. Huang, E. Aubanel, and V. C. Bhavsar. PaGrid: A Mesh Partitioner forComputational Grids. Journal of Grid Computing, 4(1):71–88, February2006.

[35] D. J. Harvey, S. K. Das, and R. Biswas. Design and Performance of a Het-erogeneous Grid Partitioner. Algorithmica, 45(3):509–530, May 2006.

[36] J. Chen and V. Taylor. Mesh Partitioning for Efficient Use of DistributedSystems. IEEE Transactions on Parallel and Distributed Systems, 13(1):67–78, 2002.

[37] IBM ILOG CPLEX. Available from: http://www.ilog.com/products/cplex/.

[38] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulatedannealing. Science, 220(4598):671–80, May 1983.

http://www.ilog.com/products/cplex/


[39] V. Cerny. Thermodynamical approach to the traveling salesman problem:An efficient simulation algorithm. Journal of Optimization Theory and Ap-plications, 45(1):41–51, January 1985.

[40] D. S. Johnson, C. R. Aragon, L. a. McGeoch, and C. Schevon. Optimiza-tion by Simulated Annealing: An Experimental Evaluation; Part I, GraphPartitioning. Operations Research, 37(6):865–892, November 1989.

[41] M.-W. Park and Y.-D. Kim. A systematic procedure for setting parame-ters in simulated annealing algorithms. Computers & Operations Research,25(3):207–217, March 1998.

[42] D. Eppstein and J. Wang. A Steady State Model for Graph Power Laws. InProceedings of the 2nd International workshop on web dynamics, 2002.

5AIOLOS: middleware for improving

mobile application performancethrough cyber foraging

“The measure of intelligence is the ability to change.”

–Albert Einstein (1879 - 1955)

In this chapter we present AIOLOS, a middleware platform for cyber forag-ing. The name AIOLOS refers to ancient Greek mythology, where Aιoλoς is thekeeper of the winds, that changed the winds in favor of Odysseus on his way backto Ithaca. Likewise, AIOLOS changes the deployment of software components atruntime, in favor of a faster execution time. Whereas in Chapter 3 the focus wason minimizing the network bandwidth, in this chapter the execution time of eachmethod call is minimized. For each method call, the execution time is predictedbased on the size of the arguments, and a history-based profile built from mon-itoring information. The AIOLOS platform is also integrated with the popularAndroid mobile operating system, and an annotation based programming model ispresented in order to facilitate application development.

? ? ?

T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt.

Journal of Systems and Software, 85(11):2629–2639, 2012.

88 CHAPTER 5

Abstract As the popularity of smartphones and tablets increases, the mobile plat-form is becoming a very important target for application developers. Despite re-cent advances in mobile hardware, most mobile devices fail to execute complexmultimedia applications (such as image processing) with an acceptable level ofuser experience. Cyber foraging is a well-known computing technique to enhancethe capabilities of mobile devices, where the mobile device offloads parts of theapplication to a nearby discovered server in the network.

Although first introduced in 2001, cyber foraging is still not widely adopted incurrent smartphone platforms or applications. In this respect, two major challengesare to be tackled. First, a suitable adaptive decision engine is needed to determinethe optimal offloading decision, that takes into account the potentially high andvariable latency between the device and the server. Second, an integrated cyberforaging platform with sufficient support for application developers is not publiclyavailable on popular mobile platforms such as Android.

In this paper, we present AIOLOS, a mobile middleware framework for cyberforaging on the Android platform. AIOLOS uses an estimation model that takesinto account server resources and network state to decide at runtime whether or nota method call should be offloaded. We also introduce developer tools to integratethe AIOLOS framework in the Android platform, enabling easy development ofcyber foraging enabled applications. A prototype implementation is presented andevaluated in detail by means of both a chess application and a newly developedphoto editor application.

5.1 Introduction

In the past decade, one has witnessed an increasing popularity of smartphones.According to Gartner this trend will continue, estimating an increase of 57.7% onsmartphone sales in 2011 [1]. Due to advances in mobile hardware (e.g. displayresolution and CPU power) and improved mobile network connectivity, new andmore advanced services can be offered on mobile devices. The introduction ofapplication markets on various mobile platforms has eased the publishing of appli-cations, resulting in a myriad of mobile applications such as social and messagingclients, location-based services, games and many more. As applications and ser-vices become increasingly important, one can also witness a shift from mobiledevices to ecosystems, such as Google’s Android [2], with people no longer onlybuying a device, but an integrated combination of a device, a platform, applicationsand services.

Although the capabilities of mobile devices are increasing, they still cannotmatch their desktop counterparts, especially for complex multimedia applicationssuch as image and video processing, object or face recognition and augmented re-ality applications. This encourages the use of cyber foraging to augment the capa-

AIOLOS: MIDDLEWARE FOR IMPROVING MOBILE APPLICATION PERFORMANCE 89

Figure 5.1: A mobile device’s capabilities are enhanced by offloading to a server availableeither nearby or far away in the cloud.

bilities of the mobile device by outsourcing CPU intensive parts of the applicationto remote servers [3]. These remote servers can be personal machines residingin the LAN network, as well as virtual machines in the cloud, where computingpower is offered on demand [4], as shown in Figure 5.1.

Although considerate research effort has been spent on designing and develop-ing cyber foraging systems [5] [6] [7] [8], this paradigm is still not widely adoptedin current smartphone application development. Two issues still need to be ad-dressed. First, an adaptive offloading strategy is needed that copes with the limitedbandwidth in wireless networks as well as with high and variable network laten-cies in a WAN environment [9]. Second, none of these cyber foraging systemsare integrated in a mobile ecosystem, with sufficient developer support to ease thedevelopment of cyber foraging enabled applications. Only Cuckoo [10] does aneffort offloading Android services and provides tools to assist in the build process,but the developer still has to define interfaces for computationally intensive partsand write both a local and a remote service implementation.

To address those two issues, we present AIOLOS: a cyber foraging frameworkfor Android. AIOLOS is built on OSGi [11], a module system and service plat-form in Java, which enables transparent monitoring and mobile code. For eachmethod call the framework estimates the execution time for both local and remoteexecution based on the argument size, and then chooses the appropriate executionlocation. Afterwards, the estimation parameters are updated, taking into accountthe new profiling data and changing network parameters. In this way the systemwill optimize the application execution, taking into account the device capabilitiesand the network connectivity, without putting an additional burden on the applica-tion developer.

The second issue is addressed by integrating AIOLOS in the Android platform

90 CHAPTER 5

and providing support for the developer creating cyber foraging enabled applica-tions. We present a plugin for the Eclipse IDE, the most popular Android devel-opment environment, that allows the developer to annotate possible offloadableclasses. At build time the plugin generates OSGi bundles that publish the anno-tated classes as OSGi services, and an OSGi runtime with the AIOLOS frameworkis embedded in the application.

To evaluate the AIOLOS framework we used the Eclipse plugin to enable cyberforaging for an existing chess application on Android. We also developed a photoediting application to show the effectiveness of the estimation model.

The remainder of this paper is structured as follows. Section 5.2 gives anoverview of related work on code migration and cyber foraging. Section 5.3 de-scribes the algorithm for deciding when to offload and Section 5.4 shows how weestimate performance from a history based profile. Section 5.5 describes the archi-tecture of our offloading middleware. In Section 5.6, we discuss implementationdetails of how AIOLOS is integrated in Android, and in Section 5.7 we presentthe programming model and developer tools to ease application development. InSection 5.8 we evaluate the framework using relevant use case scenarios. We con-clude this paper in Section 5.9 and discuss the limitations and open problems ofour approach.

5.2 Related work

The idea of using remote servers, also called surrogates, to augment the process-ing capabilities of mobile devices was first proposed by Satyanarayanan [3] ascyber foraging. Early cyber foraging systems such as Spectra [5] and Chroma [12]use a tactics based scheduler to pick methods to outsource taking into accounthistory-based resource predictions [13]. The most important drawback of thesefirst systems is that they rely on pre-installed remote procedure calls, implyingthat all remoting has to be programmed by the application developer. To facilitateapplication development the Vivendi tactics description language and stub gener-ator were proposed [14], where the application developer declares a tactics filedeclaring remote operations, their parameters and fidelities from which stubs aregenerated. With AIOLOS, the developer marks offloadable methods using sourcecode annotations, which avoids the need for a complex language and syntax.

Later systems make use of mobile code, where the system partitions the soft-ware at runtime and migrates code to the server, with minimal interference fromthe developer. Several systems exist, offloading either at a class, method or soft-ware component level.

Gu et al. [15] present an adaptive offloading framework for offloading Javaclasses, using a fuzzy control model. Ou et al. [16] also use class outsourcing, incombination with a (k+1) partitioning algorithm. Because applications typically


consist of hundreds to thousands of classes, extensive monitoring and computationis needed to identify classes to offload, making this approach less suited to adoptat runtime.

MAUI [7] uses extensive profiling to decide to offload method calls on the Mi-crosoft .Net runtime environment. The goal is to optimize the energy usage, and anInteger Linear Programming (ILP) approach is used to optimize the deployment.The goal of our approach is to optimize the execution time, and ILP is not used asit is too time consuming to solve for each method call.

The Scavenger cyber foraging system [6] outsources Python methods with ascheduler using history-based profiling. Similar to AIOLOS, Scavenger requiresthe application developer to annotate methods that are candidates for remote exe-cution. Based on these annotations, the system takes into account the size or thevalue of the input parameters of the method call to estimate the execution time atruntime (e.g. an image processing filter will take longer on a larger image). Scav-enger however does not support callbacks and assumes a constant network latencywhich is statically defined per device, whereas in AIOLOS, the network latency isestimated from previous remote method calls.

Zhang et al. [17] propose a mobile code framework where platform indepen-dent software components – called weblets – can be outsourced to the cloud basedon a Bayesian learning scheduler. Giurgiu et al. [18] and Verbelen et al. [19] useOSGi components to build a graph model of the software and use graph cuttingalgorithms to distribute these components. Han et al. [20] present a flow-based al-gorithm to partition software components, which is evaluated by simulation. Thedrawback of these component based approaches is that a software component willnot continuously generate the same load, potentially leading to instable partition-ings.

Kemp et al. [10] present Cuckoo, an offloading framework for Android that isable to offload Android services. An extra build step is introduced to make an ex-isting Android application capable of offloading its service components. However,it is still up to the developer to choose what to execute remotely. The approachalso does not support callbacks from the offloaded code.

As surrogates for remote execution, Goyal et al. [21] propose the usage of vir-tual machine technology. Because the deployment of virtual machines in a cloudcan lead to high WAN latencies, Su et al. present Slingshot [22], where the VMsare co-located with the wireless access point. Satyanarayanan et al. [9] also placethe surrogates, called “cloudlets”, in the physical proximity of the mobile user.Chun et al. [23] use virtualization to create a clone of the mobile device platformto be able to migrate parts of mobile applications without modification.

In this paper we describe AIOLOS, a cyber foraging framework that combinesthe approaches at both component and method level. On the one hand softwarecomponents are used as unit of deployment to offload, which keeps the overhead

92 CHAPTER 5

to monitor at runtime low as applications only consist of tens of components. Onthe other hand, offloading decisions are made for each method offered by the com-ponents and the framework will select methods to offload. The execution time ofmethod calls is estimated based on the size of their arguments, also taking call-backs into account. The network characteristics are estimated from the durationof previously monitored remote method calls, using a random sample consensusalgorithm.

To facilitate the development of cyber foraging enabled applications, we pro-vide developer tools that integrate the AIOLOS framework in Android applica-tions. The developer only has to annotate offloadable classes, and an extra buildstep will refactor the source code and incorporate AIOLOS in the application,which makes it easy to use AIOLOS in new as well as in existing Android appli-cations.

5.3 Decision algorithm for offloading method calls

When a method of an annotated class is called, a decision has to be made whetherto execute the method call locally or on a discovered remote surrogate. The twomost important objectives to optimize are execution time [6] and energy [7]. Wepresent two decision models that can be used in AIOLOS to optimize each of theseobjectives.

5.3.1 Optimize execution time

When optimizing the execution time, the optimal decision is to call the remoteinstance when the remote execution time is smaller than the local execution time,or as depicted in Figure 5.2 if Tremote < Tlocal.

In the following discussion we assume that callback methods do not need muchCPU resources nor send much data compared to the other service methods. Thisassumption holds if callback functions are just handling small intermediate updatesof the GUI. Figure 5.2 shows an example offloading scenario, where A performs aremote call of B. If C is called by B, it is preferable to also execute this componenton the server, because the cost of the callback from C to A is assumed small. If thecallback method takes large input arguments, it could be preferable to also keep Clocal. However, this solution can only be found using a graph cutting algorithm,which causes significant overhead to calculate before every service method call [8].

To estimate the local execution time Tlocal, a history based profile is main-tained for each service method, as described in Section 5.8. To estimate the re-mote execution time Tremote more information is needed. First, we introduce thespeedup factor α of the processing time of the remote method calls, as the server’s


A

client

B

client

local execution remote execution

C

client

A

client

B

server

C

server

1

23

45

67

8

910

11

12

34

56

78

910

11

Tlo

cal

Tre

mote

Figure 5.2: Time needed for local execution Tlocal (on the left) and for remote executionTremote (on the right).

CPU is likely to be faster than the processor of the mobile device (2, 4, 8 and 10in Figure 5.2). Two parameters β and γ represent the network bandwidth and la-tency, and are used to estimate the network round trip time as a linear function ofthe bytes sent for the remote call and callbacks (1,11 and 5,7 in Figure 5.2). Theprocessing time of the callback (6 in Figure 5.2) remains the same, as a callback isalways executed locally. Let us refer to Tremote as the estimation of Tremote. Theformula to compute Tremote then becomes:

Tremote =1

α×∑

i∈RM

(TCPU,locali) +1

β× (A+R) + γ

+∑

j∈CM

(TCPU,localj +1

β× (Aj +Rj) + γ)

With RM the collection of remotely executed methods, the estimated localprocessing times TCPU,locali are summed and divided by the speedup factor α.The network time of the remote method call is estimated using β, γ and the argu-ment size A and estimated return size R. Finally, CM defines the collection ofcallbacks, for which the local processing time is added, as well as an additionalestimated round trip time. Parameters α, β and γ are determined using historyinformation of local and remote executions, as described in Section 5.4.

5.3.2 Optimize energy

When optimizing the energy usage, the method should be offloaded if energywould be saved by remote execution. As stated by [24], a simple decision modelis to offload if the energy consumed by sending and receiving bytes to and fromthe server is smaller than the energy saved by offloading the computation, thus to

94 CHAPTER 5

offload when

Esaved = ECPU ×∑

i∈RM

(TCPU,locali)− ETR ×A− ERCV ×R

−∑

j∈CM

(ERCV ×Aj + ETR ×Rj)

> 0

In this case, the decision will depend on the energy model of the mobile device,whereECPU denotes the energy consumed per time unit by the CPU, andETR andERCV respectively denote the energy cost for transmitting and receiving a byte.However, it is difficult to estimate these parameters, which are hard to measureusing only software tools, and depend on the hardware and the hardware state [25].Therefore, in the remainder of this paper we focus on optimizing execution time.

5.4 Parameter estimation using history based pro-files

5.4.1 Estimating Tlocal and R

To estimate the remote execution time with the formula presented in Section 5.3,first the local execution time Tlocal and the return size R of the method need to beestimated. This is done using a history based profile, which can be built at runtime,or can be provided from previous sessions. In [13] it is shown that depending onthe application, the load can be accurately predicted from history logs, especiallywhen application specific knowledge about the input parameters is available. Inthis work, we assume to have no a priori knowledge on the application, and theonly parameter we can use to predict the execution time of a method is the size ofthe input parameters.

For each method call that is executed locally, the framework records the sizeof the arguments, the size of the returned result and the local execution time. Thenext time the method is called, the local execution time and the size of the resultare estimated by looking up the closest data points (with respect to argument size)in the history based profile, and by linearly interpolating between these values.To limit the size of the profile and to speed up data lookup in the profile, closedata points are clustered in buckets, for which only an average value is kept, asdescribed in [6].

When the argument size of the method does not relate to the execution time,we will end up with a profile containing average execution times for each argumentsize, which will converge to the same value for each argument size: the averageexecution time of the method.


5.4.2 Estimating α, β and γ

Parameters α, β and γ can be determined using history information of local andremote executions. The speedup factor α is the ratio of the processing time of localexecutions and the processing time of the method call on the server. Parameters βand γ can be estimated as the measured bandwidth and latency of the network. Inprevious work [13] [7] the network characteristics were estimated using a movingaverage of the network transfer time. The main disadvantage is that a single outliermeasurement, although averaged, can quickly reduce the quality of the predictednetwork transfer time.

In order to reduce the effect of outlier measurements, we estimate the param-eters α, β and γ from k previous remote method calls using a random sampleconsensus (RANSAC) algorithm [26], which is commonly used in computer vi-sion, but is also useful in other estimation problems. The algorithm will pick asubset of the input data as hypothetical inliers, and fit a model to those. All otherdata is then fitted to this model, and if data points fit the model well, they are addedto the inlier subset. Finally, the model is re-estimated using all of the found inlierdata points, and the error of the model to the inliers is calculated. This process isrepeated a number of iterations and the model with the smallest error is returned.

The input data to estimate α is the ratio of the (estimated) local processingtime and the (measured) remote processing time of the last k remote method calls.In each iteration, the algorithm randomly selects one ratio value and the absolutedifference with the other values is calculated. The other values for which thisdifference is beneath a predefined threshold are marked as inliers, and the resultingα is the average of this inlier set.

To estimate β and γ, for each of the last k offloaded method calls the timeneeded for sending (A + R) bytes is calculated as the difference of the remotetime measured at the mobile device and the execution time measured at the server.Iteratively two points are randomly picked and a line is fitted through these points.All other points are classified as inliers if the error to the line is beneath a certainthreshold (i.e. 10%). Finally, a linear regression is calculated over the set of inliers.This enables robust estimation of the model parameters without errors caused byoutliers.

5.5 AIOLOS architecture

First, relevant background information on the Android platform and its compo-nents is provided, and the use of OSGi is motivated. Next, the architecture of theAIOLOS cyber foraging framework is presented.

96 CHAPTER 5

5.5.1 Android

Android is an open source platform developed by the Open Handset Alliance tar-geted at smartphones, tablets and other devices with limited resources. Androidapplications are written in Java and compiled to Dalvik bytecodes, which run onthe Dalvik Virtual Machine. Android applications are composed of different com-ponents: Activities, Services, Content Providers and Broadcast Receivers.

An Activity provides the basic interaction logic with the user, containing a userinterface and offering some basic computing capabilities. An Android Service is acomponent that runs in the background, mainly used for long-running backgroundprocesses, e.g. playing music or fetching data without blocking the user interface.Other components can bind to a Service and communicate with the Service throughinter process communication (IPC). Content Providers are used for managing ashared set of application data. Finally, Broadcast Receivers are small componentsthat respond to system-wide broadcast announcements e.g. an announcement thatthe battery is low.

5.5.2 OSGi

OSGi [11] is a service oriented module management system in Java allowing to dy-namically load and unload software modules – called bundles – at runtime. OSGibundles can expose a service interface by registering an implementation of thisinterface with the OSGi service registry. When a bundle wants to call another ser-vice, it queries the service registry for available implementations. The portabilityof Java enables the execution of the same code on different platforms and architec-tures, facilitating remote execution and code migration. Because OSGi was firstdesigned for embedded devices, it is also lightweight.

The biggest hindrance for the straightforward use of OSGi in this context, isthe fact that the developer has to build his application as a collection of OSGibundles, which is not trivial, especially not on the Android platform. To cope withthis issue, we present extra tools for the Eclipse IDE in Section 5.7 that convert astandard Android application into an OSGi enabled application, with a minimumof effort for the application developer.

5.5.3 AIOLOS architecture

The architecture of the AIOLOS framework is shown in Figure 5.3. Both the clientand the server run an OSGi container with AIOLOS bundles and application bun-dles. R-OSGi [27] is used for remote method invocations across OSGi platforms.

Each application bundle’s service interface contains a number of service meth-ods, which will be monitored and profiled at runtime by the framework. Using this


Bundle 3

Bundle 1

Bundle 2

Pro

xy 2

Pro

xy 1

Pro

xy 2

Pro

xy 1

ProxyManager ProxyManager

Deployment

ManagerDeployment

Manager

Offloading

Logic

Bundle

Monitor

Bundle

Monitor

R-O

SG

i

R-O

SG

i

ClientServer

Bundle 1

Bundle 2

s1

s2

s1

s2

s1s1

s2 s2

Discovery Discovery

Figure 5.3: Overview of the offloading middleware. The Proxy Manager creates proxiesfor the registered services (s1 and s2) of the application bundles. These proxiescan switch between local and remote method execution and monitor all methodcalls. The Bundle Monitor aggregates this monitor information to build profilesof all methods called. This information is used by the Offloading Logic toinstruct the Deployment Manager to outsource bundles, or to update the proxypolicies when to offload.

method profile AIOLOS decides for each method call whether it will be executedlocally or remotely.

The middleware is composed of the following OSGi bundles:

1 The Proxy Manager is responsible for creating and managing proxies foreach application bundle’s services. By proxying each registered service, themiddleware can on the one hand decide to forward a service call to a remoteservice implementation rather than to the local one, and on the other handgather monitoring information of all service calls. In order to proxy services,the Proxy Manager uses Service Hooks, introduced in version 4.2 of theOSGi specification [11]. When an application bundle registers a servicewith the OSGi framework, the Proxy Manager gets a callback through theListener Hook, and generates a proxy for this service which will also registerthe service with the OSGi service registry. When a bundle looks up theservice in the OSGi service registry, the Proxy Manager can hook in usingthe Find Hook and return the service registered by the proxy, hiding theoriginal service implementation. In this way, all monitoring and remoteexecution happens transparently to all application bundles. This also allowsto handle errors transparently, for example when the network connection islost during remote execution, a time-out is detected by the proxy and themethod can be executed locally.

98 CHAPTER 5

2 The Bundle Monitor gathers monitor information of all application bun-dles. When a service method is called, the Bundle Monitor gets notified bythe involved proxy and captures the size of the input arguments, the size ofthe return value and the execution time of the method call. Using this infor-mation, an execution profile for each method is constructed, which is usedto identify candidate methods for remote execution by the Offloading Logicbundle.

3 The Offloading Logic implements the algorithm to decide whether to of-fload a method or not. This component periodically checks whether theestimated execution times match the actual monitored values provided bythe Bundle Monitor, and if necessary parameters α, β and γ are recalcu-lated as described in Section 5.4. The Offloading Logic will also instruct theDeployment Manager to copy bundles to the server when methods are iden-tified that would benefit from offloading, in case the associated bundles arenot available at the server yet. Because of the component based architectureof AIOLOS, this bundle can be easily replaced by a bundle implementinganother decision strategy, e.g. for optimizing energy.

4 The Deployment Manager can migrate bundles to remote servers by send-ing the bytecodes to the server and installing the bundle and its dependenciesthere on the OSGi runtime.

5.6 AIOLOS and Android integration

To integrate AIOLOS and Android, an OSGi container has to be deployed on theAndroid platform. One solution, proposed and implemented by ProSyst [28], is tocreate an Android OSGi container application, which can be executed as a regularAndroid app. Developers code their application as a number of OSGi bundles,which can be deployed and started in the OSGi container. However, there area number of drawbacks to this approach. First, a clear separation remains be-tween Android applications and OSGi applications, as the OSGi container has tobe started first before one can access the OSGi applications. Second, this forcesthe developer to create OSGi bundles instead of a regular Android application, andto rely on an SDK provided by ProSyst to access Android internals.

Therefore, we choose to embed an OSGi container within the Android appli-cation, and only package offloadable components as OSGi bundles as depicted inFigure 5.4. In order to let the Android application communicate with the bundles,the bundle’s services can be looked up in the OSGi container. The Android ap-plication can also register callback services with the OSGi runtime, so the OSGibundles are able to make callbacks. In order that the offloadable components can


App

Bundle 1

OSGi Runtime

Android

App

App

Bundle 2

AIOLOS

Application .apk

Figure 5.4: An OSGi container is embedded within the Android application. Offloadableparts of the Android application are packaged as OSGi bundles and deployedin the embedded OSGi container where the AIOLOS framework handles thecyber foraging functionalities. The Android application can lookup servicesthrough the OSGi runtime, and can also register callback services with the OSGiruntime.

be executed both on the Android device and on a remote server, the code is com-piled in both the Dalvik bytecode format and the regular Java bytecode format.

The resulting application acts as a regular Android application, and no addi-tional SDK is needed. However, a considerate burden is put on the applicationdeveloper when he wants to develop an AIOLOS enabled application. The follow-ing steps are required to convert a regular Android application to an offloadableapplication:

1 Identify parts of the application suitable to offload.2 Create a separate OSGi project for each offloadable part.3 Identify and create service interfaces that will act as access points to the

bundle.4 Identify possible callback interfaces that allow the bundle to perform certain

callbacks to the rest of the application.5 Write the necessary OSGi code that manages the lifecycle of the bundle.

This includes correct initialization of the bundle, registration of the bundle’sservices, searching for dependent services, etc.

6 Build and package each OSGi bundle, including a correct bundle manifestdescribing the OSGi bundle and its dependencies.

7 Include all the resulting OSGi bundles as resources in the Android project.8 Embed an OSGi runtime in the Android application, that manages all the

generated OSGi bundles, together with the AIOLOS framework bundles.

100 CHAPTER 5

This task can be tedious and error prone, and requires a lot of knowledge aboutthe OSGi specification and APIs. To simplify this process, we developed a pluginfor the Eclipse IDE that automates these steps as much as possible and reducesthe task of the application developer to step 1: identifying offloadable parts of theapplication using annotations which is discussed in Section 5.7.

5.7 Development tools for AIOLOS

In order to identify offloadable parts of the application, the developer can use JavaAnnotations to indicate classes that should act as services for offloadable com-ponents. There are a few rules the class must adhere to in order to enable cyberforaging.

1 The class needs a no-argument constructor to be present, either explicitly orimplicitly, to enable the framework to create class instances.

2 All arguments and return values used in the public class methods should beserializable, to be able to send these values over the network.

3 The class should not have dependencies to the Android APIs, as these areplatform specific and are likely not present on the surrogate.

4 At this moment only stateless services are supported for offloading, so allstate needed should be passed as a method argument.

When one of these rules is violated, the build tool will show an error or warningmessage indicating the problem, if possible hinting the developer how to fix theissue (e.g. by indicating which argument type should be made serializable).

An example offloadable class is given in Listing 5.1. The Worker class hasone stateless public method doWork. In order to work the Return class shouldbe serializable.

@Off loadab lep u b l i c c l a s s Worker {

p u b l i c Re tu rn doWork ( ) {/ / do s o m e t h i n g

}}

Listing 5.1: Code snippet of a class annotated @Offloadable.

When the builder finds an annotated class that adheres to the rules, a service in-terface is extracted from this class. This service interface is then registered with theOSGi runtime at startup, and an implementation object is bound to this interface.The extracted service interface and its implementation are given in Listing 5.2.


p u b l i c i n t e r f a c e Worke rSe rv i ce {p u b l i c Re tu rn doWork ( ) ;

}

p u b l i c c l a s s Worker implements Worke rSe rv i ce {p u b l i c Re tu rn doWork ( ) {

/ / do s o m e t h i n g}

}

Listing 5.2: After refactoring a service interface is extracted from the offloadable class.

Next, all method calls to the offloadable class are automatically refactored.Instead of invoking the method on a known reference to this object, first a referencewill be looked up in the OSGi registry. Suppose for example that the applicationcontains a button that executes the onWork method when the button is clicked, asample onClick method is given in Listing 5.3.

/ / method c a l l e d when c l i c k e d on b u t t o np u b l i c vo id o n C l i c k ( ) {

Worker w = new Worker ( ) ;Re tu rn r = w. doWork ( ) ;/ / p r i n t r e s u l t on s c r e e n

}

Listing 5.3: Code snippet of a button onClick method calling the offloadable class.

Listing 5.4 shows the refactored onClick method. A reference to an imple-mentation of the WorkerService interface is looked up in the OSGi registry.The OSGi runtime is wrapped in a singleton class so that the object is accessiblefrom anywhere in the application.

p u b l i c vo id o n C l i c k ( ) {Worke rSe rv i ce w = n u l l ;w = OSGiRuntime . g e t I n s t a n c e ( ) . l o o k u p S e r v i c e ( Worke rSe rv i ce . c l a s s .

getName ( ) ) ;Re tu rn r = w. doWork ( ) ;/ / p r i n t r e s u l t on s c r e e n

}

Listing 5.4: Refactored method call to the offloadable class. The reference is looked upwith the OSGi Runtime.

Because the offloadable class has to be executable on a surrogate, it cannotcontain dependencies to classes of the Android APIs, as these are specific to theAndroid OS, and hence indeed not executable at the surrogate. This type of calls

102 CHAPTER 5

are often system calls to the mobile device hardware. However, dependencies ofthe offloadable class can still invoke system calls. If this is the case, a callbackservice interface is extracted and registered with the OSGi runtime. As the systemcalls can only be executed on the device itself, this callback service cannot beoutsourced.

Next, for each offloadable class an OSGi bundle is formed based on the classdependency graph. Classes depending on a single offloadable class are added tothe OSGi bundle of the offloadable class. Classes that are shared between multiplebundles are put in a separate OSGi bundle. An example is shown in Figure 5.5,where a dependency graph of an application’s classes is given. Classes A, G andJ are dependent on the Android SDK, and therefore are put in the Android appli-cation part that cannot be outsourced. Classes C and H are marked as offloadable,thus two application bundles will be formed. Because classes D and E are onlydependent on C, they will be placed in the application bundle with C. The samegoes for I that is put in the application bundle with H. Classes B and F are sharedbetween multiple bundles and the Android dependent code, and therefore thesebundles are put in a shared bundle. Lastly the tool discovers a dependency of H toJ, which on its turn depends on the Android SDK. This means that J is marked asa callback class from which a callback service is formed.

A

{Android SDK}

G

C

B

H

D

E

J

I

F

@Offloadable

@Offloadable

Android

App

Bundle

App Bundle 1

App Bundle 2

Shared

Figure 5.5: Bundles are formed from the class dependency graph. Classes C and H aremarked as offloadable by the developer, and will form application bundles withthe classes that only depend on them. Classes A, G and J depend on the AndroidSDK and will be put in the Android application bundle that cannot be offloaded.Moreover J will be marked as a callback class from which a callback serviceis formed. Classes B and F are shared between the different bundles and arepackaged as a shared bundle.


The build process starts with generating a bundle activator class for each OSGibundle, that manages the lifecycle of the bundle and registers the bundle’s ser-vices with the OSGi runtime at startup. From the class dependency graph also acorrect bundle manifest is created, describing the resulting OSGi bundle and itsdependencies.

Finally, code is added to launch a Felix [29] OSGi runtime and AIOLOS frame-work together with the generated OSGi bundles, and to set up all service registra-tions when the Android application is started. The remaining Android applicationcode is built and bundled with the OSGi runtime, AIOLOS and the generated bun-dles in an Android package (.apk) file that can be installed on the Android device.

5.8 Evaluation results

We evaluate the effectiveness of mobile offloading using two example applica-tions, shown in Figure 5.6. The first one is Honza’s Chess, an existing open sourcechess game developed for Android [30]. The second application is a photo edi-tor application that enables the user to perform several image filters such as blur,sharpening or histogram equalization on his photos. When the user clicks an im-age, he first sees a preview window enabling him to adjust the filter parameters.When the parameters are correctly set, the image filter is performed on the wholeimage.

Both applications are developed as regular Android applications and can also

(a) Honza’s Chess (b) Photo Editor

Figure 5.6: Two Android applications running with the AIOLOS framework. On the leftHonza’s Chess [30] and on the right a Photo Editor application.

104 CHAPTER 5

run as such. By using the annotations and Eclipse IDE tool presented in Section5.7 these applications are integrated in the AIOLOS framework enabling to offloadcertain operations.

To show the effectiveness of mobile offloading, we have conducted experi-ments on two Android devices: a Motorola Milestone, powered by a 600 MHzARM Cortex A8 processor running Android 2.2, and an LG Optimus 2x, pow-ered by a dual-core 1 GHz ARM Cortex A9 processor connected to a WiFi accesspoint. To offload parts of the application, we used a quad core server clocked at2.4 GHz deployed close to the mobile access point in the LAN network, as well asa 2.6 GHz quad core server deployed in the Amazon EC2 Cloud.

5.8.1 Estimation accuracy

To evaluate the estimation accuracy we compare three methods for estimating theparameters α, β and γ. The first method is a moving average (MA) methodwhere for each remote method call a new value is measured for α, β and γ asnew = p × (measured) + (1 − p) × (old), with p = 0.75 for α and γ andp = 0.875 for β, as proposed in [31]. The advantage of this method is that theestimated value quickly adapts to context changes, but the disadvantage is thatoutlier measurements have a bad effect on the estimation accuracy. The other twomethods use the measurements of the latest k remote method calls. The linearleast squares (LLS) method calculates the average over the k values of measuredspeedup factors α and a linear least squares fit results in β and γ. The RANSACmethod is the approach presented in Section 5.4.

The three methods are compared in the following experiment using the photoeditor application on the LG Optimus 2x. We execute randomly a sharpen filter, acolor filter or a histogram equalization operation on a generated image with a widthand height randomly selected between 100 and 1500 pixels. First we execute 100operations locally in order to have some history to estimate the local executiontime. Next, 200 method calls are executed remotely on the server in the LANnetwork. After each method call the measured remote execution time is comparedto the estimated execution time and the parameters α, β and γ are updated. The0.1 to 0.9 percentiles of the relative error for each of the three strategies, usinga history of k = 5, 10 and 15 for the LLS and RANSAC methods are shown inFigure 5.7.

Although the LLS method uses more history for estimation, it performs worsethan the MA method. This is because an outlier measurement only affects oneestimation in the MA method, while it affects k estimations in the LLS method.The RANSAC method outperforms the MA method, as it tries to remove the effectof outliers in the measurements. When the amount of history k increases, theRANSAC method becomes better, but the gain becomes smaller as k increases:


(a) k=5

(b) k=10

(c) k=15

Figure 5.7: The 0.1 to 0.9 percentiles of the relative error of the estimated remote executiontime using a moving average (MA), a linear least squares (LLS) and a RANSACestimation strategy. The LLS and RANSAC use a history of k = 5, 10 and 15calls. RANSAC offers the best results for larger history k.

106 CHAPTER 5

Figure 5.8: Estimated versus the actual monitored execution time of a blur operation onimages of different size, using the RANSAC algorithm with k = 10.

when increasing k from 10 to 15 the gain is negligible.Our experiments show that the RANSAC algorithm with k = 10 produces

estimations within 10% of the measured values in 80% of the cases, and adaptingto a new context after five method calls. Figure 5.8 shows the estimated executiontime versus the measured execution time for the blur operation as a function of theargument size. The outliers present in the actual execution times do not influencethe estimation parameters, as they are filtered out by the random sample consensusalgorithm. As the argument size increases, there is more spread in the valuesbecause of the increase in network traffic.

Next, we also evaluate how the estimators react to a changing context. There-fore, we conduct the same experiment, but we limit the bandwidth using TC andNetem [32] on the Linux server. First we limit the bandwidth to 10 Mbit/s and ata certain moment in time, we reduce the available bandwidth to 5 Mbit/s, whichcould reflect a change in network conditions (e.g. a switch from WiFi to 3G). Fig-ure 5.9 shows how parameter β changes over time for the three methods. Becausethe MA method only uses the latest measurement, it already adapts to 5 Mbit/s af-ter one method call. The LLS approach uses all of k measurements, and thus willneed k method calls before adapting to the new bandwidth. The RANSAC methodwill choose the value with most inliers, and thus will need k/2 method calls beforeadapting.

Table 5.1 shows the different values for parameters α, β and γ when offloadingto a remote server in the LAN network or to the cloud from the Motorola Milestone


(a) k=5

(b) k=10

(c) k=15

Figure 5.9: The ability to adapt to a bandwidth change in time using a moving average(MA), a linear least squares (LLS) and a RANSAC estimation strategy. TheLLS and RANSAC use a history of k = 5, 10 and 15 calls. The MA approachadapts after one method call, the LLS and RANSAC need respectively k andk/2 method calls before the bandwidth change is taken into account.

108 CHAPTER 5

and the LG Optimus 2x. The values for parameter α show that the server in thecloud is around 25% faster than the one in the LAN network, and the Optimus 2xis around 3 times faster than the Milestone. The values for parameters β and γ areabout the same for both devices and strongly depend on the server location.

Table 5.1: Values of the parameters α, β and γ for both the Motorola Milestone and the LGOptimus 2x when offloading to a server in the LAN network and to the AmazonEC2 cloud.

α β (Mbit/s) γ (ms)

LG Optimus 2x LAN 3.35035 14.50 21.7Cloud 4.04153 6.284 469.2

Motorola Milestone LAN 11.1052 12.05 3.5Cloud 14.1115 6.509 416.5

5.8.2 Honza’s Chess

For the chess application, we annotated the class that calculates the next move ofthe AI player. The AI algorithm conducts a tree search to find the best move. Tolimit the calculation time we thresholded the maximum depth of the search tree.We executed the first 8 moves of a chess game and compared the execution timeof the AI algorithm executed on both devices locally, as well as offloaded to thenearby server and the cloud.

The experiment was repeated 10 times and the average execution times areshown in Figure 5.10. Offloading to the server in the LAN network always per-forms best. Offloading to the cloud also outperforms local execution on the Mile-stone, but the Optimus 2x can sometimes be slightly faster. The reason the cloudexecution is 1.5 to 2 seconds slower than the server in the LAN network, is thatthe Chess implementation also performs screen updates giving the user feedbackabout intermediate best found solutions, resulting in callbacks during remote ex-ecution. These callbacks involve a communication over a relatively high delayWAN network, thereby contributing to the remote execution time. On average,offloading to the LAN network leads to a 80 to 90% increase in performance, andoffloading to the Amazon EC2 Cloud offers a 20 to 65% benefit, depending on thedevice and thus in this case AIOLOS will always choose to offload.

One can also see a high fluctuation in the local execution times, while theremote execution times remain more or less constant. This is due to the fact thatfor local execution the task is CPU bound, and the calculation time will dependon the number of possible moves to consider. As the machines at the server sideare much faster, here the task becomes data bound, and the amount of data sentover the network will determine the execution time. Also note that if the player


Figure 5.10: Execution time of the Chess AI algorithm for different chess moves for localexecution on both the Motorola Milestone and the LG Optimus 2x, as wellas remote execution on a remote server deployed in the LAN network andone deployed in the Amazon EC2 Cloud. Error bars are omitted as standarddeviations are too small to be visible.

wants to increase the difficulty, he would increase the calculation depth threshold,resulting in higher calculation times and even more gain of offloading.

In this application, the execution time will depend on the pieces on the chessboard, rather than the argument size. Therefore, AIOLOS will not be able to ac-curately predict the local execution time, but rather return an average executiontime. However, for this use case this is sufficient to take the correct decision andto offload the method.

In general, this could be addressed by defining a generic metric interface, al-lowing the developer to give the middleware extra information on which metricshould be used for prediction. However, since our objective was to keep the pro-gramming model as simple as possible, this is omitted.

5.8.3 Photo Editor

For the Photo Editor application, all image filters are annotated for possible remoteexecution. We performed an experiment using the sharpening filter, which is oneof the more complex implemented filters. Again we measure the local executiontime on both mobile devices and the remote execution time when offloaded to aserver in the LAN network and one in the cloud. The sharpening filter is executed30 times on three different image sizes: 500×500, 1000×1000 and 1500×1500

110 CHAPTER 5

Figure 5.11: Execution time of a sharpening filter on a 500×500, 1000×1000 and1500×1500 RGB image for local execution on both the Motorola Milestoneand the LG Optimus 2x, as well as remote execution on a remote server de-ployed in the LAN network and one deployed in the Amazon EC2 Cloud. Inthe case of the 1500x1500 image local execution on the Milestone is omittedfor the clarity of the figure and was more than 30s.

pixels. The images are raw RGB images, thus a large amount of data has to be sentback and forth in the case of remote execution.

The average execution times are shown in Figure 5.11, together with the errorbars indicating the standard deviation. For the Milestone, remote execution is al-ways more beneficial than local execution. For the 1500×1500 image, we omittedthe bar for the Milestone, as the execution time is over 30 seconds. The Optimus2x is faster than execution on the cloud, but benefits from remote execution on theserver in the LAN network as the image size gets higher (e.g. for the 1500×1500remote execution is 23% faster). In this case, as AIOLOS uses the input size of themethod call to estimate local and remote execution time, the method call will onlybe offloaded when the image size is large enough to offer a significant gain whenexecuted remotely.

Also note that the standard deviation increases with the latency to the remoteserver. In order to make the application more suitable for (cloud) offloading, onecould try to lower the data communication, e.g. by sending the JPEG imagesinstead of the raw images.


5.8.4 AIOLOS overhead

The dominant overhead of AIOLOS is introduced when the application starts upfor the first time. Depending on the application, three to five seconds are added tothe startup time, measured on the LG Optimus 2x, to initialize the OSGi frame-work and to install and setup all the framework and application bundles. Eachmethod call of an offloadable method passes through a proxy, which will recordthe execution time and calculate the size of the arguments and return value. Thiscauses a few additional milliseconds of overhead per method call, which is negli-gible in the presented use cases. A history based profile of previous local methodcalls is maintained for each offloadable method, and the size can be limited by apredefined threshold. For the photo editor application 20 to 30 local method callswere already sufficient to accurately predict the local execution time.

5.8.5 Development tools

The development tools described in Section 5.7 greatly simplify the developmentprocess. We only had to add six annotations to the photo editor application, andone to the chess application, from which all components are generated automat-ically. After preprocessing, the tools add 574 lines of code to the photo editorapplication, and 332 to the chess application, which accounts for the extractionof service interfaces, the registering and lookup of services and the initializationof the OSGi framework. Each project is also split up into different sub-projectsfor each generated bundle, and the Felix OSGi runtime and AIOLOS frameworkbundles are automatically added to the project. The application developer needsno knowledge about the OSGi specification and programming model, as this ishandled transparently by the development tools.

5.9 Conclusion

In this paper we presented AIOLOS, a cyber foraging framework for Android, builton the OSGi module system and service platform. Given a number of candidatemethods for offloading provided by the application developer, AIOLOS decideswhich of these methods should be executed remotely in view of the current context,and migrates the code at runtime to a discovered server. For each method call,AIOLOS decides whether the call is better executed remotely or locally, using ahistory based estimation model of the network and the processing speed of theremote server.

In order to transparently develop cyber foraging enabled applications, we pro-vide the necessary tools that allow application developers to benefit from the AIO-LOS framework only by annotating their existing Android applications.

112 CHAPTER 5

To show the effectiveness of our approach, we annotated two Android appli-cations for AIOLOS: Honza’s Chess, an existing open source chess game, and anin-house developed photo editor application. Using these applications we showthe benefits from cyber foraging by offloading to a server in the LAN network,as well as to a server deployed in the Amazon EC2 Cloud. For the specified usecases offloading offers gains of 20 to 90% in execution time depending on the mo-bile device capabilities, the network connectivity and the input parameters of themethod calls. AIOLOS is able to estimate the remote execution time upfront witha relative precision of 10% for 80% of the method calls, using a random sampleconsensus algorithm. It was shown that this algorithm outperforms methods basedon moving averages and linear least square fits.

As future work we plan to support stateful offloading, which introduces theneed for correct data synchronization. In this aspect we can also investigate oper-ations on data shared between multiple users in collaborative scenarios. Anotherresearch direction is to apply cyber foraging for energy saving. Here we can reuseour approach, but an accurate energy model of the mobile device is needed.

AcknowledgmentTim Verbelen is funded by PhD grant of the Fund for Scientific Research, Flanders(FWO-V).


References[1] Gartner Group. Press Release: Gartner Says Android to Command Nearly

Half of Worldwide Smartphone Operating System Market by Year-End2012, April 2011. Available from: http://www.gartner.com/it/page.jsp?id=1622614.

[2] M. Butler. Android: Changing the Mobile Landscape. IEEE Pervasive Com-puting, 10:4–7, 2011.


[4] R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic. Cloud comput-ing and emerging IT platforms: Vision, hype, and reality for delivering com-puting as the 5th utility. Future Generation Computer Systems, 25(6):599–616, 2009.




[8] T. Verbelen, T. Stevens, P. Simoens, F. De Turck, and B. Dhoedt. Dynamicdeployment and quality adaptation for mobile augmented reality applica-tions. Journal of Systems and Software, 84:1871–1882, November 2011.


[10] R. Kemp, N. Palmer, T. Kielmann, and H. Bal. Cuckoo: a ComputationOffloading Framework for Smartphones. In ICST Conference on MobileComputing, Applications and Services, MobiCASE ’10, 2010.




114 CHAPTER 5


[13] D. Narayanan and M. Satyanarayanan. Predictive Resource Managementfor Wearable Computing. In Proceedings of the 1st international conferenceon Mobile systems, applications and services, MobiSys ’03, pages 113–128,2003.

[14] R. K. Balan, D. Gergle, M. Satyanarayanan, and J. Herbsleb. Simplifying cy-ber foraging for mobile devices. In Proceedings of the 5th international con-ference on Mobile systems, applications and services, MobiSys ’07, pages272–285, 2007.

[15] X. Gu, A. Messer, I. Greenberg, D. Milojicic, and K. Nahrstedt. AdaptiveOffloading for Pervasive Computing. IEEE Pervasive Computing, 3(3):66–73, 2004.



[18] I. Giurgiu, O. Riva, D. Juric, I. Krivulev, and G. Alonso. Calling the cloud:enabling mobile phones as interfaces to cloud applications. In Proceedingsof the 10th ACM/IFIP/USENIX International Conference on Middleware,Middleware ’09, pages 83–102, 2009.

[19] T. Verbelen, R. Hens, T. Stevens, F. Turck, and B. Dhoedt. Adaptive OnlineDeployment for Resource Constrained Mobile Smart Clients. In Proceedingsof the 3rd Conference on Mobile Wireless Middleware, Operating Systems,and Applications, volume 48, pages 115–128, 2010.





[23] B.-G. Chun, S. Ihm, P. Maniatis, and M. Naik. CloneCloud: Boosting Mo-bile Device Applications Through Cloud Clone Execution. Technical ReportarXiv:1009.3088, 2010.

[24] K. Kumar and Y.-H. Lu. Cloud Computing for Mobile Users: Can OffloadingComputation Save Energy? Computer, 43(4):51–56, April 2010.

[25] L. Zhang, B. Tiwana, Z. Qian, Z. Wang, R. P. Dick, Z. M. Mao, and L. Yang.Accurate online power estimation and automatic battery behavior basedpower model generation for smartphones. In Proceedings of the eighthIEEE/ACM/IFIP international conference on Hardware/software codesignand system synthesis, CODES/ISSS ’10, pages 105–114, 2010.

[26] M. A. Fischler and R. C. Bolles. Readings in computer vision: issues,problems, principles, and paradigms, chapter Random sample consensus: aparadigm for model fitting with applications to image analysis and automatedcartography, pages 726–740. Morgan Kaufmann Publishers Inc., 1987.


[28] ProSyst. mBS Mobile for Android. Available from: http://www.prosyst.com.

[29] Apache Felix. Available from: http://felix.apache.org/site/index.html.

[30] Honza’s Chess. Available from: http://honzovysachy.sourceforge.net.

[31] B. D. Noble, M. Satyanarayanan, D. Narayanan, J. E. Tilton, J. Flinn, andK. R. Walker. Agile application-aware adaptation for mobility. In Pro-ceedings of the sixteenth ACM symposium on Operating systems principles,SOSP ’97, pages 276–287, 1997.

[32] S. Hemminger. Network Emulation with NetEm. In Linux Conf Au,April 2005. Available from: http://developer.osdl.org/shemminger/netem/LCA2005_paper.pdf.

http://www.prosyst.com

http://felix.apache.org/site/index.html

http://honzovysachy.sourceforge.net

http://developer.osdl.org/shemminger/netem/LCA2005_paper.pdf

http://developer.osdl.org/shemminger/netem/LCA2005_paper.pdf

6Adaptive Deployment and

Configuration for Mobile AugmentedReality in the Cloudlet

“Before software should be reusable, it should be usable.”

–Ralph Johnson (1955 - )

The platforms described in chapters 3 and 5 focused mainly on offloadingbetween one device and one server. In reality however one has to cope with amultitude of interconnected, heterogeneous devices. Therefore, in this chapterwe present a hierarchical management platform, which allows for more globaloptimization and offers better scalability. Building upon the same technologiesas previous chapters, the middleware enables both deployment and configurationof application components at runtime. Also a semantically richer programmingmodel is introduced, allowing the application developer to declaratively specifysoftware components, configuration options and performance constraints.

? ? ?


Submitted to Journal of Network and Computer Applications, April, 2013.

118 CHAPTER 6

Abstract Despite recent advances in mobile hardware, most mobile devices stillfall short to execute complex multimedia applications with real-time requirementssuch as augmented reality (AR). Because offloading the application to the cloud isnot always an option due to the high and often unpredictable WAN latencies, theconcept of cloudlets has been introduced: nearby infrastructure offering virtualmachines for remote execution.

In this paper we present a cloudlet platform, providing two important contribu-tions. First, the platform allows cloudlets to be formed in a dynamic way, including(fixed) virtualized infrastructure co-located with the wireless access point, as wellas any device in the LAN network supporting the platform. The approach can alsobe extended towards the cloud, facilitating distribution of applications over threetiers (i.e. the device, the cloudlet and the cloud). Second, instead of moving a com-plete virtual machine to the cloudlet, we propose a more fine-grained approach, bymanaging and deploying applications on the component level. Application com-ponents are declared by the developer, together with their real-time constraints andconfiguration parameters. In order to meet these constraints and to optimize theuser experience, the platform distributes these components among the cloudlet atruntime while also dynamically configuring parameters.

An OSGi-based prototype implementation on the Android platform is high-lighted and evaluated using a mobile AR use case, which shows the need for acomponent-based approach for the cloudlet.

6.1 Introduction

As smartphones and tablets are gaining more and more capabilities (in terms ofCPU power, network connectivity and sensors), they are becoming the preferreddevice for daily tasks such as browsing the web, e-mail, etc. Although high-endsmartphones are equipped with dual- or quad-core processors, their processingcapabilities remain an order of magnitude lower than their desktop counterparts,due to the limited battery capacity, form factor and passive cooling. Therefore,these mobile devices still fall short to execute future resource intensive multimediaapplications, that involve real-time processing of video or audio, such as objectrecognition or mobile augmented reality [1].

To enable such resource-intensive applications, the concept of cyber foragingwas introduced by Satyanarayanan et al. [2], where parts of the application are off-loaded to nearby resources – called surrogates – connected via a wireless networkto the mobile terminals. Such surrogates are recently defined as “cloudlets" [3],virtualized infrastructure co-located with the wireless access point, which allowsmobile devices to offload applications to nearby resources in the LAN network.

Today many such cyber foraging systems exist, offloading applications eitheron a method [4] [5], a software component [6] [7] [8] or a virtual machine (VM)

ADAPTIVE DEPLOYMENT AND CONFIGURATION FOR MOBILE AR 119

Figure 6.1: Static cloudlets can be provided by a corporation in a corporate cloudlet, or bya service provider in the mobile network. Ad hoc cloudlets can be formed in thehome network or within a railway carriage.

level [3] [9]. Although a lot of research has been done on profiling and partition-ing applications and decision algorithms to decide when to offload, two importantissues still remain.

First, almost all systems focus on the scenario where one mobile device of-floads to one surrogate, called the cloudlet. However, in a realistic scenario mul-tiple devices will be connected to the wireless network, some of them requiringresources to offload to (i.e. mobile devices or tablets) and other devices offeringresources (i.e. servers or laptops). In this paper we present a cloudlet managementplatform, that extends the cloudlet to all devices in the network. This allows takingdecisions optimizing behavior for all devices in the network, taking into accountshared resources such as the network bandwidth, instead of optimizing on eachdevice separately.

In addition to cloudlets solely consisting of fixed infrastructure (e.g. a corpo-rate cloudlet), also ad hoc cloudlets can be formed with all devices available inthe LAN network (e.g resources discovered in the home network or within a rail-way carriage), as depicted in Figure 6.1. The platform can also easily be extendedtowards the cloud, allowing application components to be distributed in a 3-tierfashion: locally on the mobile device, nearby on the cloudlet infrastructure or on adistant datacenter.

A second issue is that current cyber foraging systems aim to optimize a per-formance goal from a user perspective, such as energy usage, processing time orthroughput. However, from a developer perspective, a more important goal is toachieve good performance on a wide range of devices, in which case “good" per-

120 CHAPTER 6

formance is an application specific metric. Such a metric can often be formulatedas a number of constraints (e.g. maximum allowed execution time for a specificmethod call), especially in applications with real-time requirements.

To address this issue, our cloudlet platform manages applications on the com-ponent level, where the application developer declares application componentswith their offered functionality, configurable parameters and performance con-straints. Application components are managed and configured at runtime, andcan be distributed among resources in the cloudlet depending on the current con-text, in order to achieve the desired performance. In order to ease developmentfor such constraint aware applications, we provide a programming model based onannotations, minimizing the burden put on the application developer.

We present a detailed architecture of the cloudlet platform. To show the needand effectiveness of the platform, we focus on an augmented reality use case. AnOSGi based prototype implementation is presented and evaluated on the Androidmobile platform.

The remainder of this paper is structured as follows. The next section gives anoverview of related work in the field of application offloading and cloudlets. Sec-tion 6.3 presents our mobile augmented reality use case that illustrates the needfor a component-based approach. This use case has been fully implemented toevaluate the cloudlet platform. In Section 6.4 we propose a general architecturefor a cloudlet framework, and in Section 6.5 we discuss in more detail the im-plementation of a prototype based on OSGi, and the integration with the AndroidOS. To ease component-based application development supporting the platform,an annotation-based programming model and corresponding developer tools arepresented in Section 6.6. In Section 6.7 we show the need and effectiveness of ourapproach for the mobile augmented reality use case, while Section 6.8 concludesthis paper and discusses future work.

6.2 Related work

Offloading computation from mobile devices to remote surrogates, known as cy-ber foraging [10], has been subject for over a decade. Different approaches exist,offloading parts of the application either using a method, component or virtual ma-chine level granularity. Kristensen et al. [5] propose Scavenger, a cyber foragingsystem in Python which outsources Python methods. A scheduler predicts the ex-ecution time of a method as a function of the input parameter values and/or sizesusing monitoring information from previous executions, and selects the device of-fering the shortest execution time. MAUI [4] offloads methods of Microsoft .Netapplications in order to save energy. At initialization time, MAUI measures the en-ergy characteristics of the device, and at runtime the application and network char-acteristics are monitored. From this monitoring information, a call graph of the


application is constructed, where the nodes are weighted with the energy used forprocessing the task, and the edges are weighted with the energy needed for sendingthe arguments and return value over the network. An integer linear programming(ILP) solver then decides at runtime whether or not a method is offloaded, bycalculating a program partitioning that minimizes the smartphone’s energy con-sumption, subject to latency constraints.

Using software components as unit of deployment rather than methods resultsin less fine-grained optimization options, but reduces the overhead of monitoring.Giurgiu et al. [6] and Verbelen et al. [7] use OSGi components offloadable units.Monitoring information is used to build a graph model of the application com-ponents, from which graph cutting algorithms calculate the optimal deployment,i.e. minimizing the required bandwidth or execution time. Odessa [8] is a cyberforaging system that focuses on data flow applications. The system is built onSprout, a distributed stream processing framework that provides a programmingmodel to develop parallel processing pipelines, hiding much of the complexity ofparallel and distributed programming from the application developer. Odessa usesa greedy algorithm that estimates the bottleneck in the processing pipeline, andtries to optimize the makespan and throughput of the application by exploitingoffloading and data-parallelism.

Goyal et al. [11] propose the usage of virtual machine technology as surro-gates for remote execution, which provides isolation and security. Because thedeployment of virtual machines in a cloud can lead to high WAN latencies, Suet al. present Slingshot [12], where the VMs are co-located with the wireless ac-cess point. Satyanarayanan et al. [3] proposed the concept a of cloudlet: a trusted,resource-rich computer or a cluster of computers well connected to the Internetand available for use by nearby mobile devices. Cloudlets offer their resourcesto mobile devices by dynamic VM synthesis, where small VM overlays are sentto the cloudlet from which a complete VM is created running the mobile appli-cation. Because of the coarse granularity of VMs, these systems do not containany partitioning logic, but rather an all-or-nothing approach is used, execution theapplication either locally, either completely on the VM using a thin client proto-col. In order to do enable some more fine grained partitioning, Chun et al. presentCloneCloud [9], where a set of possible partition choices is made at build time,using static code analysis. Different binaries of the application are generated anddedicated VM instructions are added at migration points for selected methods. Atruntime, a clone VM is instantiated at the server side, and the application trans-parently switches between execution at the device or at the clone. In an offlineprofiling stage, monitoring information is collected for each partitioning identi-fied, which is used to determine the best partitioning at runtime.

All these cyber foraging systems address the situation where the applicationis distributed between one mobile device and the cloud. However, in a wireless

122 CHAPTER 6

network context, network bandwidth is a shared medium, and thus there is needfor a global management entity that optimizes the bandwidth for all users. Alsothey all try to optimize generic metrics, such as energy usage, processing time, andit is often left undefined what should happen when no surrogate is available. Fromthe application developer viewpoint, one would prefer to optimize the perceivedquality of the end user, and to gracefully degrade when no remote resources areavailable, or conversely, treat the situation when parts are offloaded as a qualityenhancement on top of a more basic application functionality. To achieve this, theoffloading framework should also be able to reconfigure the application at runtime.

To tackle these issues, we propose a more fine grained approach for the cloudletdefined by Satyanarayanan [3], where the problem is tackled at the componentlevel through a middleware approach. A cloudlet platform configures and dis-tributes the application components at runtime, alleviating the developer from thetask to decide which components to offload and how to configure the applica-tion for the current context. To facilitate application development on our frame-work, we present an intuitive programming model based on code annotations. Theframework also handles resource discovery, enabling the dynamic formation ofcloudlets, and has a global overview of all resources, which makes global opti-mization possible.

In short, the main contributions of this paper are as follows: (i) a middlewareplatform for discovering and managing devices in the local network, bundling theirresources as a cloudlet, (ii) a component-based offloading framework, able to dis-tribute application components on the resources in the cloudlet, as well as config-ure them at runtime, in order to optimize the QoS offered to the end user, and (iii)an annotation-based programming model, facilitating application development ofcloudlet applications.

6.3 Use case: Mobile Augmented Reality

As a use case, we present a mobile augmented reality application featuring marker-less tracking as described by Klein et al. [13], combined with an object recognitionalgorithm presented in [14]. The application is shown in Figure 6.2. On the righta grayscale video frame is shown with the tracked feature points, from which thecamera position is estimated. The left shows the resulting overlay with a 3D object,and a white border around the recognized book.

Starting from the source code from [13], we have redesigned the parallel track-ing and mapping algorithm as a set of loosely coupled components. These arewrapped in Java code, in order to obtain software components that are able to runon multiple platforms and hence can be offloaded by our platform, as is discussedmore thoroughly in Section 6.5. We also extended the application with an objectrecognition component, as shown in Figure 6.3. The applicaiton consists of the


Figure 6.2: The augmented reality application tracks feature points in the video frames(right) to enable overlaying of 3D objects (left).

following six components:

VideoSource The VideoSource fetches video frames from the camera hardware.These frames are analyzed by the Tracker, and rendered together with anaugmented reality overlay by the Renderer.

Renderer Each camera frame is rendered on screen together with an overlay of3D objects. These 3D objects are aligned according to the camera pose asestimated by the Tracker.

Tracker The Tracker analyses video frames and calculates the camera pose bymatching a set of 2D image features to a known map of 3D feature points.The map of 3D points is generated and updated by the Mapper.

Mapper From time to time the Tracker sends a video frame to the Mapper formap generation and refinement. By matching 2D features in a sparse set ofso-called keyframes, the Mapper can estimate their 3D location in the sceneand generate a 3D map of feature points.

Relocalizer When no 2D image features are found in the video frame, the Relo-calizer tries to relocate the camera position until tracking resumes.

Object Recognizer In the keyframes of the Mapper the Object Recognizer triesto locate known objects. When an object is found, its 3D location is notifiedto the Renderer that produces an overlay.

These components are not only very CPU intensive, some of them also havestrict real-time constraints. The VideoSource and the Renderer have to be exe-cuted on the mobile device, as they access device specific hardware. In order toachieve an acceptable performance, the Tracker and the Relocalizer should be ableto process frames within 30-50 ms, which translates into a frame rate of 20-30

124 CHAPTER 6


Tracker Mapper

Relocalizer

VideoListener

TrackerService

MapperService

MapListener

Relocalizer

Service MapListener

Object

Recognizer MapListener

RendererService

VideoListener

Figure 6.3: The component diagram of the AR application. The VideoSource and the Ren-derer (dark gray) are fixed on the mobile device. The Relocalizer and Tracker(gray) have real-time constraints (≤ 50 ms).

frames per second. As the Mapper runs as a background task, constantly refiningand expanding the 3D map, this component preferably runs on a device with con-siderable CPU resources. However the Mapper does not impose strict real timerequirements. The Object Recognizer also has more relaxed requirements, as de-lays in the order of a second before recognizing an object are still tolerable toachieve an acceptable user experience.

The goal now is to run this application on a mobile device, while meeting allthe required constraints, which have to be declared by the application developer.This use case shows that offloading the whole application to the cloudlet on aVM level is not sufficient, as sending frames to the cloudlet for processing andreceiving rendered frames in return would require too much bandwidth, which isa scarce resource in current wireless access networks. Therefore we propose acloudlet architecture that will manage the application on a component level, beingable to distribute application components within the cloudlet or to other cloudlets.

In addition to component (re)distribution, QoS can also be assured by dynami-cally reconfiguring components, as shown in [15]. By adopting a component-basedapproach for the cloudlet, real-time constraints of applications can be achieved us-ing combined dynamic reconfiguration and distribution. In our use case for exam-ple, the developer defined a parameter configuring the number of tracked features,which influences the frame processing rate and which can be adapted at runtime.

6.4 Cloudlet architectureWe adopt a component-based cloudlet architecture as proposed in [16], which isshown in Figure 6.4. In this paper, we define a cloudlet as a collection of inter-



C1 C2 C3

Operating System

Node

Agent

NACloudlet

Agent

Cloudlet

Agent

Ad hoc cloudlet

in the LAN network

Elastic cloudlet (e.g. in a public cloud

infrastructure)

...

VM1

VM2NA

NA

NAEE

EE

EE

EE

OS

OS

OS

OS

Figure 6.4: The application components are distributed among nodes in two cloudlets. Anad hoc cloudlet consisting of a mobile phone, a laptop and a desktop computer,and a distant elastic cloudlet in a public cloud infrastructure.

connected resources in each other’s vicinity, for example all devices connected bya LAN network (e.g. your mobile phone and your laptop connected to your homeWiFi network). Our architecture has three interrelated management levels: thecomponent level, the node level and the cloudlet level.

Components (units of deployment specified by their providing and required in-terfaces [17]) are managed by an Execution Environment (EE), that can start andstop components, resolve component dependencies, expose provided interfaces,etc. To support distributed execution, dependencies can be resolved with other(remote) Execution Environments. In that case, proxies and stubs are generatedand the components can communicate by remote procedure calls (RPCs). Compo-nents can also define performance constraints (e.g. the maximum execution time ofa method), and expose configuration parameters to the EE. By monitoring the re-source usage of each component, the EE can detect violations of the performanceconstraints and actions can be taken such as calculating a new deployment (i.e.offloading some resource intensive components) or adapting component configu-rations (i.e. lowering component quality).

One or more Execution Environments run on top of an operating system (OS),which in turn can run on either virtualized or real hardware. The (possibly virtual-ized) hardware together with the installed OS is called a node, and is managed bya Node Agent (NA). The Node Agent manages all the EEs running on the OS, andcan also start or stop new Execution Environments, for example for sandboxingcomponents. The NA also monitors the resource usage of the node as a whole,and has a view on the (maybe virtualized) hardware it runs on (e.g. the number ofprocessing cores, processing speed, etc.).

Multiple nodes that are in the physical proximity of each other (i.e. low la-tency) form a cloudlet. The cloudlet is managed by a Cloudlet Agent (CA), that

126 CHAPTER 6

communicates with all underlying Node Agents. Cloudlet Agents of differentcloudlets can also communicate with each other, for example to migrate com-ponents between cloudlets. Within a cloudlet, the node with the most resourcesis chosen to host the Cloudlet Agent. The Cloudlet Agent has a global overviewof all available resources, and is able to calculate a deployment, which should beglobally optimal for all devices in the cloudlet.

There are two types of cloudlets, as shown in Figure 6.4: the ad hoc cloudletand the elastic cloudlet. The ad hoc cloudlet consists of dynamically discoverednodes in the LAN network. These nodes run a Node Agent that can spawn Ex-ecution Environments to deploy components in. When nodes join or leave thecloudlet, the Cloudlet Agent will recalculate the deployments, migrating com-ponents if needed. The elastic cloudlet runs on virtualized infrastructure, wherenodes run in virtual machines. Here, the Cloudlet Agent can spawn new nodeswhen more resources are needed, or stop nodes when too much resources are al-located. This type of cloudlet comes close to the VM based cloudlet envisionedby Satyanarayanan [3], the main distinction being the additional platform compo-nents running in the VM (i.e. the Node Agent and the Execution Environment)managing the applications. An elastic cloudlet could run on virtualized hardwareco-located with the wireless access point, or on a private cloud within the workenvironment.

By also deploying the cloudlet platform in the cloud, components can also beoutsourced to the public cloud, which allows applications to be outsourced in a3-tier fashion: hardware dependent components are deployed on the mobile de-vice, latency constrained components are distributed on a nearby cloudlet and un-constrained components, exhibiting challenging CPU and/or data storage require-ments, can be deployed on EEs in the public cloud.

6.5 Cloudlet platform design and implementation

The prototype implementation of the cloudlet platform is realized in OSGi [18],an industry standard module management system in Java. This technology wasselected as the OSGi standard already specifies the software component concept,and the OSGi platform can run on multiple underlying platforms. In addition,a substantial amount of middleware building blocks (e.g. Configuration Admin,Declarative Services, ...) are standardized in the OSGi Compendium Specifica-tion [19]. In this section, we describe in detail the design and implementation ofthe Execution Environment and the Node and Cloudlet Agent, together with theirintegration in the Android platform. First the most important OSGi concepts arehighlighted, on which the cloudlet platform is based.


6.5.1 OSGi

The OSGi core specification defines a service oriented module management sys-tem for Java, allowing to dynamically load and unload software modules – calledbundles – at runtime. OSGi bundles can expose a service interface by register-ing an implementation of this interface with the OSGi service registry. When abundle needs to call another service, the service registry is queried for availableimplementations. The portability of Java enables the execution of the same codeon different platforms and architectures, facilitating remote execution and codemigration. OSGi also allows to incorporate native libraries in a bundle, which areloaded at runtime depending on the underlying platform. Because OSGi was firstdesigned for embedded devices, the incurred overhead is limited.

The OSGi module management system constitutes the core of the ExecutionEnvironment of the cloudlet platform. In addition to the core specification, wealso use four compendium specifications: Declarative Services, the ConfigurationAdmin Service, the Metatype Service and Remote Service Admin.

6.5.2 Execution Environment

An overview of the Execution Environment is shown in Figure 6.5. The EE bundleproxies the components to gather monitoring information (e.g. time spent execut-ing a method) as well as to transparently forward method calls to remote instances

OSGi

Configuration

Admin ServiceMetatype

Service

Declarative

Services

Remote Service

Admin


Component

metatype

descr.

service

descr.

Service Interface

Exposed (proxied)

Service Interface

Component

Proxy

local

call

remote

call

constraint

descr.

Figure 6.5: The Execution Environment proxies all component interfaces in order to mon-itor all (local and remote via RemoteServiceAdmin) service method calls. TheConfiguration Admin and Metatype service enable the EE to dynamically con-figure components, which are defined using the Declarative Services specifica-tion.

128 CHAPTER 6

through the Remote Service Admin. The latter component implements a remoteprocedure call protocol, such as R-OSGi [20]. Components come with three de-scriptors: the metatype descriptor exposes the configurable parameters, the servicedescription declares the offered and required service interfaces and the SLA de-scriptor defines imposed constraints. The Declarative Services bundle ensures thatcomponent interfaces are registered with the OSGi runtime, and the componentis bound to all its dependencies. The Metatype and Configuration Admin serviceallow the EE to discover the available configuration options and to set these con-figurations dynamically to meet the imposed constraints.

The Declarative Services specification presents a declarative model for pub-lishing, finding and binding OSGi services. Instead of registering and looking upservices programmatically in the source code, this is now done at runtime by theOSGi framework using an XML description that is bundled with the component,which describes the provided service interfaces, the service dependencies and howthese dependencies should be bound to the component. For lifecycle managementthe developer can also define methods to be called when a component is activatedor deactivated. This provides a simplified programming model to the developer,which has no more code dependencies to the OSGi core APIs, and does not haveto handle with the complex dynamic service concept. An example of such a com-ponent description XML is shown in Listing 6.1.

The ConfigurationAdmin (CA) decouples a component from its configurationparameters. The CA maintains a persistent repository of configuration data (a

<component name=" ptam . t r a c k e r . T r a c k e r ">< implementat ion c l a s s =" ptam . t r a c k e r . T r a c k e r " / >< s e r v i c e >

< provide i n t e r f a c e =" ptam . t r a c k e r . a p i . T r a c k e r S e r v i c e " / >< provide i n t e r f a c e =" ptam . v i d e o . a p i . V i d e o L i s t e n e r " / >

. . .< / s e r v i c e >< r e f e r e n c e name=" mapper "

i n t e r f a c e =" ptam . mapper . a p i . MapperSe rv ice ". . .

bind=" se tMapper "unbind=" unse tMapper " / >

. . .< / component>

Listing 6.1: Example snippet of a component description, defining the Tracker component.The Tracker component implements the TrackerService and VideoListenerinterfaces, and needs a reference to the MapperService, which is injected bythe framework using the bind/unbind methods.


dictionary of key-value pairs) for all components. When a component is started,the CA provides the necessary parameters to correctly configure the component.When the configuration data is changed, components are updated accordingly, al-lowing at runtime reconfiguration.

Because the EE has to be able to adapt the component configuration at runtimeusing the ConfigurationAdmin, it should also know which configuration parame-ters are available and which values are allowed. This is where the Metatype Servicecomes in, which allows developers to describe attribute types in an XML format.An example metatype XML is shown in Listing 6.2. The metadata consists of anObject Class Definition (OCD), which defines a number of Attribute Definitions(ADs), and a Designate, which binds the OCD to a component.

To be able to distribute components among multiple resources in the network,we use R-OSGi as provider for our Remote Service Admin, which enables remoteservice binding and remote service calls.

In order to monitor all application components, the Execution Environmentbundle creates a proxy for each service interface provided by a component. Us-ing the OSGi hooks API from the core specification, the original service is hiddenfor other components, that can only bind to the proxied one. This way, all ser-vice calls pass through the proxy, which is able to monitor all service calls. Theproxy forwards the call to an actual component interface, either locally or remotevia R-OSGi. Using the Metatype service, the EE can discover the configurableparameters for each component, and modify these using the Configuration Admin.The configuration is adapted in order to meet constraints that are declared by the

<MetaData><OCD d e s c r i p t i o n ="PTAM T r a c k e r C o n f i g u r a t i o n "

name="PTAM T r a c k e r "id =" ptam . t r a c k e r . TrackerOCD "><AD name=" F ine P o i n t s "

id =" t r a c k e r . f i n e P o i n t s "d e f a u l t =" 1000 "min=" 200 "max=" 1000 ">

< /AD>< /OCD>< Des ignate pid=" ptam . t r a c k e r . T r a c k e r ">

<Object oc d re f =" ptam . t r a c k e r . TrackerOCD " / >< / Des ignate >

< / MetaData>

Listing 6.2: Example metatype information for the Tracker component that marks thenumber of tracked points as a configurable parameter.

130 CHAPTER 6

< s l a >< s e r v i c e name=" ptam . v i d e o . a p i . V i d e o L i s t e n e r ">

<method name=" p r o c e s s F r a m e ">< c o n s t r a i n t type =" maxTime ">50< / c o n s t r a i n t >

< / method>< / s e r v i c e >

< / s l a >

Listing 6.3: Example constraint description for the Tracker component. A video frameshould be processed within 50 ms.

developer in the constraints descriptor as shown in Listing 6.3.Here a constraint is defined that a video frame should be processed by the

Tracker component within 50 ms. The Execution Environment implements a feed-back loop that will constantly monitor the time to process a video frame. When theprocessing time exceeds 50 ms, the Tracker configuration parameters are changedin order to lower the processing time. The EE also notifies the Node Agent, whichwill ask the Cloudlet Agent whether also components need to be offloaded to en-hance the application quality.

In order to create an application that exploits the cloudlet platform, the devel-oper needs to design the application as a number of OSGi bundles. For each bundlehe has to define the component descriptor that describes the provided and requiredservices of the component. Optionally, the developer can also declare the config-urable parameters in a metatype descriptor, and required constraints in a constraintdescriptor. Because this can be a tedious and error prone task, we propose anannotation based programming model in Section 6.6 to alleviate development.

6.5.3 Node and Cloudlet Agent

The Node and Cloudlet agents are also implemented as OSGi bundles, runningon an OSGi runtime in a separate process, as shown in Figure 6.6. When an EEis started, it registers with the Node Agent running on that node via R-OSGi onlocalhost.

When first started on a device, both the Node Agent and the Cloudlet Agentbundle are active. Other devices are discovered using jSLP, a Java implementationof the SLP discovery protocol [21]. When another CA is discovered in the LANnetwork, the CA on one of the two devices (the weakest of the two) is stopped, andthe remaining CA becomes the master of the cloudlet. If the connection to the CAis lost, the Cloudlet Agent bundle is started again locally. This way, a cloudlet canbe formed in an ad hoc manner, without the need for virtualized hardware. Whenthe discovered Cloudlet Agent is not deployed nearby, both CAs communicate as


OSGi

Cloudlet

Agent

Node

Agent

JSLP

Discovery

Remote Services

(R-OSGi)

Figure 6.6: The Node Agent and Cloudlet Agent bundles run on an OSGi runtime in aseparate process. The Cloudlet Agent bundle is only present when the currentnode is also the master of the cloudlet.

peers, allowing for inter-cloudlet component offloading.

6.5.4 OSGi and Android integration

To deploy our cloudlet platform on the Android platform, the platform needs tostrictly comply with Android’s application model. Android applications are writ-ten in Java and compiled to Dalvik bytecode (which run on the Dalvik VirtualMachine), and are composed of different components: Activities, Services, Con-tent Providers and Broadcast Receivers. An Activity provides the basic interactionlogic with the user, containing a user interface and offering basic computing capa-bilities. An Android Service is a component that runs in the background, mainlyused for long-running background processes (e.g. playing music) without block-ing the user interface. Content Providers are used for managing a shared set ofapplication data, and Broadcast Receivers are small components that respond tosystem-wide broadcast announcements (e.g. announcement when running low onpower).

The OSGi runtime with the Node and Cloud Agent are embedded in a separateCloudlet Android application that starts an Android Service at boot time, whichallows the Node Agent to run in the background. The Execution Environment isembedded within an Android application as depicted in Figure 6.7. Within theAndroid application, components are identified that should be offloadable or thatshould be monitored and configured by the EE. The Android application calls thesecomponents by looking up their services with the OSGi runtime, and can alsoregister services to enable callbacks from the components. Because the offloadablecomponents can be executed both on the Android device as on a remote server, thecode is compiled in both the Dalvik bytecode format and the regular Java bytecodeformat.

Figure 6.7 also shows how the use case components are embedded in the An-droid application. The Renderer and VideoSource component reside within theAndroid activity, as they need to render to screen, or need access to the Androidinternals to fetch video frames from the hardware. They expose their interfaces to

132 CHAPTER 6

Tracker

OSGi RuntimeAndroid

Activity

Mapper

Application .apk

EEOSGi Runtime

Android Service

Cloudlet service .apk

Node

Agent

Cloudlet

Agent

VideoSource

Renderer

proxy

proxy

proxy

proxy

Android OS

.jar

.jar

...

Figure 6.7: On the Android platform a separate application hosts the Node Agent and theCloudlet Agent bundles, that run in a background process using an Android Ser-vice. Applications having offloadable and/or configurable components deploythese on an OSGi runtime hosting the EE, which proxies the provided inter-faces. Components requiring the Android internals (e.g. drawing to screen) areembedded in the Android Activity and also provide their interfaces through theEE.

the other components through OSGi and the EE. The other components such asthe Tracker and the Mapper are packaged within the .apk as separate jars, whichallows these components to be offloaded at runtime, and are deployed on the OSGiruntime.

The resulting application acts as a regular Android application, and no addi-tional SDK is needed. When the application starts, the EE is created which reg-isters with the Node Agent running on the device in the background. However, ahuge burden is put on the application developer when he wants to develop sucha component-based application. Necessary steps to take include: identifying thecomponents, implementing these separately as OSGi-bundles together with the ac-companying descriptors, embedding an OSGI runtime together with the EE, andbuilding and packaging the components as an Android application. To simplifythis process, a lightweight programming model is presented in the following sec-tion.

6.6 Programming model

To ease the development of cloudlet enabled Android applications, we present astraightforward programming model based on Java annotations. These annotationsare analyzed in a preprocessing step in the build process, from which the requiredXML descriptions are generated. In the case of an Android application, the build


system will also generate the code necessary to embed the OSGi runtime and thecloudlet platform, and create the desired components.

Listing 6.4 shows the annotated source code of the Tracker class, from whichthe Tracker component is generated, as well as the descriptors as described inSection 6.5. Classes annotated by @Component are handled as component im-plementation classes, providing all implemented interfaces. A reference to anotherservice is created by declaring a variable of the desired type and annotate it withthe @Reference annotation. This will generate a set and unset method to inject areference to the required service at runtime, and create the reference attribute in theXML description. Configurable properties are annotated by @Property, fromwhich the metatype description is generated. To declare constraints on servicemethods, they can be annotated with the @Constraint annotation, describingthe type constraint and the imposed threshold.

package ptam . t r a c k e r ;

@Componentp u b l i c c l a s s T r a c k e r implements T r a c k e r S e r v i c e , V i d e o L i s t e n e r {

@Proper ty ( min =200 , max=1000)p r i v a t e i n t f i n e P o i n t s = 1000 ;

@Reference ( p o l i c y =dynamic )p r i v a t e MapperSe rv ice mapper ;

@ C o n s t r a i n t ( maxTime =50)p u b l i c vo id p r o c e s s F r a m e ( byte [ ] d a t a ) {

/ / p r o c e s s f rame. . .

}. . .

}

Listing 6.4: Example annotated Java source file from which a component is generated,together with the required XML descriptors.

The resulting build process is shown in Figure 6.8. First the source files arepreprocessed and XML descriptors are generated from the detected annotations.In the second step, the source code of the components is compiled and packagedtogether with the XML descriptors. Finally the build system combines the appli-cation components with the middleware components and packages the resultingAndroid .apk.

134 CHAPTER 6

annotated

source code

preprocessing

+

source code

+ XML descriptors

build application

componentsbuild android .apk

OSGi


+ middleware components

OSGi

Components

Figure 6.8: Overview of the steps done by the build system. Annotated source code ispreprocessed and application components are generated. These are bundledwith the middleware components to create the resulting Android .apk whichcan be deployed on the Android OS.

6.7 Evaluation

To show the need for the cloudlet platform presented in this paper, we implementedthe augmented reality use case presented in Section 6.3 using the proposed com-ponent model. The AR code was executed on an ad hoc cloudlet consisting of amobile device and a laptop connected via WiFi. The laptop is equipped with anIntel Core 2 Duo CPU clocked at 2.26GHz. As mobile device we use a HTC De-sire, with a single core Qualcomm 1 GHz Scorpion CPU, and an LG Optimus 2xpowered by a dual core Nvidia Tegra 2 CPU, also clocked at 1GHz. Both devicescapture camera frames at a 800x480 resolution, which is the same size as theirscreen resolution. The results show both the need for dynamically configurablecomponents in the case of the Tracker, and the benefits of offloading resource in-tensive components such as the Mapper and Object Recognizer.

6.7.1 The need for dynamic configuration

The performance of the Tracker component, which has to process all video frames,is critical to the user experience, as too few frames processed per second deterio-rates the application quality. However, in this case offloading is not an option, asnot enough bandwidth is available to send and receive the video frames in a timelymanner. The Tracker exposes one configuration parameter representing the maxi-mum number of feature points that can be tracked. A high number of points willincrease the tracking accuracy, making the Tracker more resilient to fast cameramovements or motion blur, at the cost of more processing power.

For both mobile devices, the average processing time (and standard deviation)to track one video frame as a function of the number of tracked features is shownin Figure 6.9. It is clear that in order to meet the constraint of 50 ms to process aframe, the number of tracked features for the HTC desire has to be limited to 200,while the LG Optimus 2x can process up to 450 feature points, resulting in a more


Figure 6.9: In order to meet the constraint of 50 ms to process a frame, the HTC Desire canprocess at most 200 feature points per frame, while the LG Optimus can handle450 feature points.

robust tracking.In order to illustrate the dynamic adaptation of the configuration, we imple-

mented a negative feedback loop in the EE bundle: when the moving average ofthe monitored execution time of tracking one frame is smaller than 50-t ms, thenumber of tracked feature points is increased, where t = 10 is chosen. Similarlywhen the execution time rises above 50 ms, the amount of tracked feature points isdecreased. This enables the framework to automatically adjust the configuration tothe device, by monitoring the execution time at runtime, as shown in Figure 6.10.Here, the processing time per frame is shown as well as the value of the config-urable parameter (= maximum number of feature points tracked) when running theapplication on the LG Optimus 2x.

The initialization of the map (1) results in high processing times, and thus thevalue of the parameter is reduced to the minimum. When tracking is started, theparameter value is increased to have the best tracking accuracy possible within the50 ms imposed.

When the tracking is lost due to a too quick camera movement, no featurepoints are found, and the applications starts relocalizing (2). As no feature pointsare found, the processing time lowers and the parameter value is increased. Whentracking resumes (3), the processing time is too high due to the increased parame-ter, and the parameter value is adapted again to meet the imposed constraint.

This shows that the framework not only adapts the configuration depending onthe device capabilities at initialization, but can also adapt to different application

136 CHAPTER 6

Figure 6.10: The time needed to track a frame as a function of the number of feature points.Using a negative feedback loop, the framework adapts the parameter configu-ration at runtime in order to meet the maximum processing time constraint.

scenarios at runtime. In this case it is beneficial to increase the number of trackedfeature points when tracking is lost, as this gives more robust tracking when therelocalizer finds a reasonable estimate of the camera position. Once tracking isstable again, the number of feature points can be reduced to meet the imposedconstraint.

These results illustrate the benefits of dynamic configuration adaptation, butthe implemented solution is very application specific. Future work is needed tosearch for more generic solutions that adapt the application without a priori knowl-edge, for example using self-learning techniques.

6.7.2 The benefits of offloading

The goal of the cloudlet is of course to offload application components from thedevice to discovered resources nearby. In this experiment, an ad hoc cloudlet isformed between the laptop and the mobile devices. The Cloudlet Agent then se-lects resource intensive components suitable for offload, and migrates those to thelaptop. In the case of the AR application presented in Section 6.3, the Mapper andObject Recognizer are selected for outsourcing.

When a new keyframe is added, all positions of the points in the map areoptimized. This optimization problem scales with the total number of keyframes inthe map. Figure 6.11 shows the time needed for this map refinement as a functionof the keyframes in the map for one run, both executed locally on the LG Optimus2x, and remote when the Mapper component is outsourced to the laptop. As the


Figure 6.11: The time to refine the map when a new keyframe is added, as a function ofthe total number of keyframes in the map. Offloading the Mapper componentresults in a speedup factor of 10.

number of keyframes increases, the refinement process takes up to 10 seconds onthe mobile device, while remote execution takes less than a second. Overall wewitness a speed up of a factor 10 when outsourcing to the laptop.

For the Object Recognizer, larger differences are perceived. When performingan object search on a videoframe, this takes up to 30 seconds on the HTC Desire,and around 18 seconds on the LG Optimus 2x. When outsourcing the ObjectRecognizer to the laptop, the processing time is reduced to on average 1.5 seconds.

To decide automatically at runtime which components should be outsourcedand which should be executed on the mobile device, partitioning algorithms couldbe used based on the monitoring information gathered by the ExecutionEnviron-ment [22].

6.8 Conclusions and future work

In this paper, we present a cloudlet architecture that not only provides fixed in-frastructure co-located with the WiFi access point, but also enables ad hoc dis-covery of devices in the vicinity to share resources among each other. Instead ofproviding infrastructure on a virtual machine level, a more fine-grained approachis presented, where the cloudlet framework manages applications on a componentlevel. By adaptively configuring and outsourcing application components, the plat-form optimizes the application depending on the mobile device capabilities and the

138 CHAPTER 6

available resources in the cloudlet.A prototype implementation of the framework based on the OSGi industry

standard is presented, together with an annotation-based programming model whichenables easy application development. To evaluate the cloudlet platform we im-plemented a component-based augmented reality application. Using a negativefeedback loop mechanism, the framework dynamically adapts the configuration ofthe application, in order to perceive a predefined frame rate. By offloading CPUintensive components a speed up factor of 10 is achieved.

As future work, more advanced algorithms are to be investigated in order tomanage all resources and application components in the cloudlet. For exampleself-learning techniques could be used in the Execution Environment to identifywhich parameters to adapt, and the Cloudlet Agent could analyze monitoring in-formation from all nodes to calculate an optimal deployment for all applicationcomponents. To manage the increasing complexity, our architecture allows forhierarchical autonomic control loops, which is already applied in the topic of net-work management [23].

With respect to commercial deployment of such a platform, different scenariosare possible. A first scenario is to rely on the network operators to provide com-puting infrastructure in the edge network, for example co-located with wirelessaccess points. This of course requires a high initial investment and managementcost, and thus some kind of pay-per-use model could be adopted similarly to cloudcomputing, offering on-demand nearby computing resources. When the cost ofproviding infrastructure is too high, another option could be a FON-based sce-nario, where instead of users sharing their wireless connection, users also partiallyshare their home computing resources to mobile users in the vicinity. In return,they can also use shared resources of other FON users when on the go. In orderto avoid infrastructure and/or management costs, a final scenario could be a peer-to-peer scenario, where devices connect in an ad-hoc manner. This scenario couldbe successful when all users involved gain by sharing resources, for example in acollaborative augmented reality application, i.e. where multiple devices share 3Dmap information to improve tracking quality. However, in all three scenarios ef-fort has to be spent on security issues, e.g. authentication, authorization and properisolation to prevent malicious code from executing.

AcknowledgmentTim Verbelen is funded by PhD grant of the Fund for Scientific Research, Flanders(FWO-V).This project was partly funded by the UGent BOF-GOA project “Autonomic Net-worked Multimedia Systems".


References

[1] P. Jäppinen, R. Guarneri, and L. M. Correia. An applications perspectiveinto the Future Internet. Journal of Network and Computer Applications,36(1):249–254, January 2013.





[6] I. Giurgiu, O. Riva, D. Juric, I. Krivulev, and G. Alonso. Calling the cloud:enabling mobile phones as interfaces to cloud applications. In Proceedingsof the 10th ACM/IFIP/USENIX International Conference on Middleware,Middleware ’09, pages 83–102, 2009.

[7] T. Verbelen, T. Stevens, P. Simoens, F. De Turck, and B. Dhoedt. Dynamicdeployment and quality adaptation for mobile augmented reality applica-tions. Journal of Systems and Software, 84(11):1871–1882, November 2011.

[8] M.-R. Ra, A. Sheth, L. Mummert, P. Pillai, D. Wetherall, and R. Govindan.Odessa: enabling interactive perception applications on mobile devices. InProceedings of the 9th international conference on Mobile systems, applica-tions, and services, MobiSys ’11, pages 43–56, 2011.

[9] B.-G. Chun, S. Ihm, P. Maniatis, M. Naik, and A. Patti. CloneCloud: elas-tic execution between mobile device and cloud. In Proceedings of the sixthconference on Computer systems, EuroSys ’11, pages 301–314, 2011.


140 CHAPTER 6



[13] G. Klein and D. Murray. Parallel Tracking and Mapping for Small ARWorkspaces. In Proceedings of the 2007 6th IEEE and ACM InternationalSymposium on Mixed and Augmented Reality, ISMAR ’07, pages 1–10,2007.

[14] D. G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints.International Journal of Computer Vision, 60(2):91–110, 2004.

[15] W. Li. QoS Assurance for Dynamic Reconfiguration of Component-BasedSoftware Systems. IEEE Transactions on Software Engineering, 38(3):658–676, May-June 2012.

[16] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt. Cloudlets: bringingthe cloud to the mobile user. In Proceedings of the third ACM workshop onMobile cloud computing and services, MCS ’12, pages 29–36, 2012.

[17] C. Szyperski. Component Software: Beyond Object-Oriented Programming.Addison-Wesley Longman Publishing, 2nd edition, 2002.


[19] The OSGi Alliance. OSGi Service Platform, Service Compendium, Release4, Version 4.2. aQute, September 2009.


[21] E. Guttman, C. Perkins, J. Veizades, and M. Day. Service Location Protocol,Version 2. 1999.

[22] T. Verbelen, T. Stevens, F. D. Turck, and B. Dhoedt. Graph partitioningalgorithms for optimizing software deployment in mobile cloud computing.Future Generation Computer Systems, 29(2):451–459, February 2013.


[23] J. Famaey, S. Latre, J. Strassner, and F. De Turck. A hierarchical approach toautonomic network management. In Network Operations and ManagementSymposium Workshops, pages 225–232, 2010.

7Model-driven Configuration and

Deployment Optimization in MobileCloud Computing

“Knowing yourself is the beginning of all wisdom.”

–Aristotle (384 BC - 322 BC)

In the previous chapter we introduced a hierarchical management platform forruntime deployment and configuration adaptation. Subsequently, in this chapter,we present search heuristics to co-optimize the deployment and configuration ofapplications, given all constraints imposed by either the application developer orthe infrastructure capabilities. We start by extracting an abstract model of both theapplication behavior and the infrastructure, based on gathered monitoring infor-mation. Using this model, we revisit the graph partitioning algorithms presentedin chapter 4, and extend them to take into account configuration parameters andthe various constraints. We evaluate the heuristics by comparing their solution tothe brute force calculated optimum for small problem instances, and show theireffectiveness using the augmented reality application described in chapter 2.

? ? ?


Submitted to IEEE Transactions on Software Engineering, April, 2013.

144 CHAPTER 7

Abstract Over the years, mobile device capabilities in terms of CPU power, mem-ory, connectivity, etc, have evolved tremendously. However, these devices still fallshort to execute complex media rich applications. In addition, in view of the het-erogeneity of mobile devices, the application developer is faced with many chal-lenges to create applications that offer acceptable Quality of Service (QoS) on awide range of devices.

In this paper we present a framework for co-optimizing the deployment andconfiguration of component-based mobile applications, maximizing the QoS of-fered to the end user, while adhering to all constraints imposed by the infrastruc-ture or declared by the application developer. A formal model is presented, captur-ing essential application and infrastructure behavior. To solve the co-optimizationproblem at runtime, two search heuristics are designed and evaluated by comparingtheir performance to a brute force approach. We developed a prototype implemen-tation of the framework, and show the need and effectiveness of our approach usingan augmented reality use case. To aid the developer in creating applications tar-geted to our platform, a programming model based on source code annotations isproposed, allowing to specify software components, their configurable parametersand/or performance constraints that influence the QoS.

7.1 Introduction

Rapid advances in micro-chip and wireless technologies thrive the innovation ofmobile devices such as tablets and smartphones. These devices become increas-ingly popular, as they provide mobile users with more computational resources inlighter and smaller form factors. Gartner reports that although worldwide sales ofmobile phones declined by 2% during the first quarter of 2012, smartphone salesincreased by 44.7% [1]. People no longer use their mobile device only for tele-phony, but also for a myriad of other mobile applications, such as location basedservices, multimedia applications, games and many more, which makes smart-phones as a popular target platform for software developers.

However, when developing for mobile devices, new software engineering chal-lenges have to be addressed [2]. A first challenge arises from the resource limi-tations of mobile devices. As restrictions on weight, size, battery life, and heatdissipation impose limitations on computational and memory resources, mobiledevices will always be more resource constrained than their non-mobile counter-parts [3]. Therefore, mobile devices still fall short to execute many media rich anddata analysis applications that have large computation and memory requirements.Second, mobile application developers have to cope with a high level of deviceheterogeneity. While differences in target software platform can be addressed byintelligent cross compiling [4], differences in hardware capabilities make it diffi-cult to guarantee an optimal Quality of Service (QoS) on a wide range of devices.

MODEL-DRIVEN CONFIGURATION AND DEPLOYMENT OPTIMIZATION 145

Third, due to the user’s mobility, applications have to operate in different contexts,and thus the developer often has to take into account context parameters such asnetwork connectivity, location and nearby devices [5].

To address the resource limitations of mobile devices, the concept of “cyberforaging” was introduced [6], where compute-intensive tasks are offloaded to in-frastructure in the network. With the emergence of cloud computing, resourcesin a datacenter can be offered on an on-demand basis for offloading [7]. How-ever, Hassan et al. [8] show that cloud computing does not provide a silver bullet,and is outperformed by outsourcing to nearby residential computers. Dependingon the use case, outsourcing to the cloud can even be slower than local executionon the mobile device due to limited bandwidth and high WAN latencies. There-fore, Satyanarayanan [3] introduced the concept of VM based cloudlets: trusted,resource rich computers in the near vicinity of the mobile user (e.g. near or co-located with the wireless access point), on which virtual machines (VMs) are in-stantiated for remote execution. The goal is then to find the optimal deployment,deciding which application components to offload given the current context [9].

Current cyber foraging systems focus on optimizing generic metrics such asenergy usage [10] [11] or execution time [12] [13]. These systems address deviceheterogeneity and changing device contexts by dynamic (re)deployment of parts ofthe application, but do not take into account application specific QoS metrics (e.g.resolution, frame rate, ...). From a developer’s perspective, these QoS parametersare more relevant to address, especially considering media processing applicationsthat often have (soft) real-time constraints. For example, in an augmented real-ity (AR) application, the goal would be to maximize the image resolution, whileprocessing at a sufficient frame rate. In order to take into account this type ofQoS considerations, we have to combine dynamic deployment with dynamic QoSmanagement techniques [14] by also (re)configuring the application at runtime.

In this paper we present a framework addressing both component offloadingand configuration of QoS parameters at runtime. First, component-based appli-cations (including QoS parameters, runtime dependencies, resource requirementsetc.) and the available infrastructure are described in a formal model. An objectivefunction and constraints are defined to find an optimal deployment and configura-tion adhering to the infrastructure limitations and the QoS constraints imposed bythe application developer. In order to solve this co-optimization problem at run-time, two search heuristics are proposed and compared. To show the effectivenessof our approach, we integrated our algorithms in a prototype implementation of ourframework, which is evaluated through a mobile augmented reality application.

The remainder of this paper is structured as follows. In the next section, wediscuss related work in the domain of code offloading and QoS management. Sec-tion 7.3 presents the overall architecture of the component-based framework envi-sioned in this paper, together with the programming model offered to the developer

146 CHAPTER 7

in order to define application components. In Section 7.4 a model is presented forthe infrastructure, the application behavior and the application constraints. Twoheuristic algorithms are proposed to search for the global optimum in Section 7.5,which are evaluated by comparing their solutions for a number of synthetic prob-lems. In Section 7.6 we also show the effectiveness of our approach using a proto-type implementation in Java of the proposed framework and a mobile augmentedreality application. We conclude this paper in Section 7.7 and discuss future work.

7.2 Related work

7.2.1 Cyber foraging

Offloading computation from mobile devices to remote resources – also known ascyber foraging – has been a research topic for over a decade [6]. Several systemsexist, offloading either at class, method, component or virtual machine level [15].

One of the first proposed cyber foraging systems is Spectra [16], which fo-cused on offloading tasks for reducing latency and energy consumption. Usinga linear regression model, the system models the application resource demandsfrom monitoring information. Spectra heavily relies on application developers tomanually change the code to enable remote execution. Chroma [17] reduces theburden on the developer by introducing so-called “tactics”, a more declarative wayfor developers to define the application partitioning, and a brute force method isused to calculate the best partitioning.

Gu et al. [18] present an adaptive offloading framework for offloading Javaclasses, in combination with a fuzzy control model. Ou et al. [19] also address Javaclass offloading, proposing a (k+1) partitioning algorithm that splits an applicationinto one unoffloadable partition and k disjoint offloadable partitions, while min-imizing the communication between the partitions. The fine granularity of classoffloading however requires extensive monitoring and causes significant overhead.

Other systems use methods as units to outsource, such as the Scavenger cyberforaging system [12], which outsources Python methods. A dual-profile sched-uler is used, weighting tasks according to their parameter input sizes and run time.MAUI [10] outsources method calls on the Microsoft .Net runtime environment.This platform generates a program partitioning by formulating and solving an in-teger linear programming problem to maximize energy savings.

A more coarse-grained approach is to outsource software components. Zhanget al. [20] offloads platform independent software components – called weblets – tothe cloud using a Bayesian learning scheduler. Giurgiu et al. [7] and Verbelen et al.[13] use OSGi components as units to outsource. To distribute these components,a graph model of the software is built and graph cutting algorithms are used tocalculate the most appropriate deployment.


Goyal et al. [21] propose the use of virtualization on the infrastructure for re-mote execution. Here a client can request a virtual machine (VM) with specificresource guarantees to offload services to. Su et al. present Slingshot [22], wherethe VMs are co-located with the wireless access point to overcome the WAN la-tency. Chun et al. present CloneCloud [11], where virtualized clones of the mobiledevice are executed in the cloud. Different binaries of the application are gener-ated in an off-line profiling stage, with special VM instructions added at migrationpoints for selected methods. At runtime a clone VM is instantiated at the serverside, and the application transparently switches between execution at the device orat the clone.

Satyanarayanan et al. [3] introduce the concept a of cloudlet: a trusted, resource-rich computer or a cluster of computers well connected to the Internet and availablefor use by nearby mobile devices. Cloudlets offer their resources to mobile devicesby dynamic VM synthesis, where small VM overlays are sent to the cloudlet fromwhich a complete VM is created.

All these systems aim to optimize application execution solely by offloading.In this paper however, we combine the offloading problem with dynamic con-figuration adaptation, which allows the application to gracefully degrade whenno or insufficient remote resources are available. Existing approaches also tacklethe case of one mobile device offloading to one or more remote devices. In thiscontribution, we state a general optimization problem that also takes into accountmultiple mobile users sharing the same network and CPU resources.

7.2.2 Dynamic QoS adaptation

QoS characteristics define the non-functional properties of a system, affectingthe perceived quality, for example response time, image resolution, frame rateetc. When dealing with mobile computing, QoS management plays an importantrole, as heterogeneous mobile devices have to operate in highly variable environ-ments [23]. The main challenge then is to autonomically adapt the system to thisvariability, maximizing the QoS in each context [24].

One domain where software adaptation for QoS optimization is used is livevideo streaming [25], where the transcoding parameters of the video are changedat runtime to adapt the bitrate to variable available bandwidth [26], using a feed-back control loop. As the limited number of parameters is known upfront and theirinfluence on the QoS is well-defined, this allows for highly efficient and fast de-cision algorithms. In our case however, the decision algorithm should work forgeneric applications without a priori knowledge on the configurable parametersand their relationship with the QoS.

Another related domain focuses on service-based systems, where autonomiccomputing techniques are used for service deployment, selection and configura-

148 CHAPTER 7

tion [27]. In order to find an optimal solution, different optimization techniques areused such as genetic algorithms [28], tabu search [29], simulated annealing [30],Markov decision models [31], etc. Chuang et al. [14] discusses QoS adaptation ofservices for mobile computing over a wireless environment, and presents a fuzzycontrol model for resolving various QoS rules. In this work we combine serviceconfiguration with component offloading, and exploit these well-known optimiza-tion techniques, i.e. simulated annealing.

In previous work [32] we combined component offloading and service selec-tion for QoS optimization, where the software developer had to declare a limitedset of valid service combinations. For each configuration, a software partitioningwas calculated among which the one offering the best QoS was chosen. In this pa-per, we adopt a more flexible approach, where the developer declares a number ofconfigurable parameters and a set of possible parameter values. Our contributionalso presents new search heuristics that optimize both deployment and configu-ration simultaneously. This is a very complex problem, as the solution space in-creases exponentially with the number of configurable parameters and the numberof application components.

7.3 Architecture

The overall architecture of our Java framework is shown in Figure 7.1. We assumeapplications are component-based, where a component is the unit of deploymentand is specified by its providing and required interfaces. Components are managedby an Execution Environment (EE) that can start and stop components, resolvecomponent dependencies, expose provided interfaces, etc. To support distributed


Operating System

EE

EE

OS

OS

C1 C2 C3

C4 C5

Monitor

Analyze Plan

Execute

Autonomic Manager

C4

reconfigure

component

offload

component

Model

Figure 7.1: The overall architecture of our adaptive framework. Application componentsare deployed in containers – called Execution Environments (EE) – that run ondifferent devices. Adaptation is achieved by monitoring the EEs and take actionthrough (re)configuration or migration of components at runtime.


execution, dependencies can be resolved by other (remote) Execution Environ-ments. In that case, proxies and stubs are generated automatically and the com-ponents communicate by remote procedure calls (RPCs). Components can alsodefine performance constraints (e.g. the maximum execution time of a method),and expose configuration parameters to the EE.

In order to allow application developers to easily define application compo-nents used by the framework, we adopt a programming model based on codeannotations. Listing 7.1 shows an example annotated source code file. A com-ponent is declared by annotating the class implementing the service interface, andattributes can be annotated as configurable properties, or define dependencies toother component interfaces. Methods can be annotated with constraints, for ex-ample constraining the maximum time a method call can take. Similarly, the de-veloper can define which configurable properties affect the QoS and how. Fromthese code annotations, software components can be created automatically at buildtime [13].

Different EEs are managed by an Autonomic Manager, implementing the au-tonomic computing monitor-analyze-plan-execute (MAPE) loop [33]. At runtime,method calls between application components are monitored by the EE, record-ing the size of the arguments, the size of the return value and the execution time.Together with data about the infrastructure capabilities and the current resourceusage, this monitoring information from all connected EEs is gathered and con-densed into a model of the infrastructure and the runtime behavior of the applica-

@Componentp u b l i c c l a s s C1 implements S e r v i c e I n t e r f a c e {

@Proper ty ( v a l u e s = 1 : 5 )p r i v a t e i n t p a r a m e t e r = 1 ;

@Reference ( p o l i c y =dynamic )p r i v a t e O t h e r S e r v i c e s ;

@ C o n s t r a i n t ( maxTime =100)p u b l i c vo id doSomething ( ) {

. . .}. . .

}

Listing 7.1: Example annotated Java source file defining a component, a configurableproperty and a timing constraint (i.e. calling doSometing() should takeless than 100 ms).

150 CHAPTER 7

tion. This model then is used in the planning stage, where a decision algorithmcalculates necessary adaptations in order to optimize the quality of service. Twopossible adaptation strategies are considered. First, application components canbe migrated between different devices, i.e. improving performance by offloadingfrom a resource-constrained mobile device to a more powerful server in the net-work. Second, application components can also be reconfigured at runtime, e.g.lowering a quality parameter in order to meet a minimum frame rate. The remain-der of this paper is on the “analyze” and “plan” phases of the autonomic controlloop. First, we describe how to model the application behavior and infrastruc-ture from runtime monitoring information. Next, we also present and evaluate twosearch heuristics to find an optimal deployment and configuration of the compo-nents.

7.4 Application and infrastructure model

In this section, we build a model of the application and infrastructure based on run-time monitoring information. These models capture the application behavior andconstraints (both infrastructure and QoS constraints), and are used in Section 7.5as input for the decision algorithms.

7.4.1 Application model

We consider applications that consist of a number of components. Each com-ponent can provide and consume service interfaces, and can offer configurablediscrete parameters. Components communicate by calling methods provided inthe service interfaces. The developer’s view of the application can then be repre-sented using an UML component diagram. An example application consisting offive components is shown in Figure 7.2. Although this component diagram showsthe dependencies and configurable parameters of each component, this does notreflect the application behavior at runtime.

The actual control flow of the application can be represented in UML sequencediagrams for all the use case scenarios of the application. For example, the se-quence diagrams of the application presented in Figure 7.2 are shown on Fig-ure 7.3.

However, the sequence diagrams depicted in Figure 7.3 still fall short to de-scribe the application behavior in sufficient detail. For example, in Figure 7.3(a)the total execution time before the call of method m2 by component C5 returns,depends on the number of times the loop is executed, and in Figure 7.3(b) theexecution depends on the conditional path taken.

Therefore, sequences are represented as shown in Figure 7.4. The condi-tional sequence in Figure 7.3(b) is split up in multiple sequences (Fig. 7.4(a) and


C1

C5

C2

C4

C3

p1

p2

Figure 7.2: An example of a component-based application. Each component consumesand/or provides service interfaces. Components 1 and 4 also offer a parame-ter that can be configured at runtime.

Fig. 7.4(b)), each representing one conditional path. To model the loop, the methodcalls in a sequence are annotated with the number of times they are called withinthe sequence as shown in Figure 7.4(c). To capture the overall application behav-ior, we also keep track of the number of times each sequence is called per timeunit.

More formally, let C and M represent the set of application components andthe set of public methods offered by all components. A sequence s ∈ S(C,M)

represents a sequence of calls of methods m ∈ M between components ci, cj ∈C. Next, mscicj denotes a call to method m of component cj in sequence s bycomponent ci. To further define the application behavior fs is the number of timessequence s is executed per time unit, and fmscicj

is the number of times methodcall mscicj is executed in sequence s.

For each callmscicj we track the size of the arguments of the methodAmscicj,p,

C5 C2 C4m2()

m6()

m5()

loop

(a)

C1 C2 C3m2()

m4()

alt m3()[condition1]

[condition2]

(b)

Figure 7.3: The actual behavior of the application is captured in UML sequence diagrams,showing the actual method calls, loops and conditional flows.

152 CHAPTER 7

as well as the size of the return valueRmscicj,p and the relative CPU loadLmscicj

,p

of the method call. The argument size, return size and CPU load of a method callmscicj are expressed as a function of the vector p, representing the values of theconfiguration parameters pi ∈ P . These functions can be given by the developer,or can be estimated from monitoring information.

C1 C2 C3m2()

m3()

[condition1]

(a)

C1 C2 C3m2()

m4()

[condition2]

(b)

C5 C2 C4m2()

m6()

m5()#calls = N

#calls = N

(c)

Figure 7.4: The two UML sequence diagrams shown in Figure 7.3 are split up in threesequences. The conditional sequences are split up in a separate sequence foreach condition, and loops are modeled by an annotation stating how many timeseach method is called within the sequence.

7.4.2 Infrastructure model

The infrastructure model consists of a number of interconnected devices d ∈ D.Each device processor has a rate at which load can be processed Speedd and anumber of cores Coresd.

The devices are connected by a (wireless) network, that is characterized byits bandwidth BW and latency Lat. The bandwidth denotes both the capacity(maximum number of bytes that can be sent per time unit) and the speed (the rateat which bytes are sent) of the network. The latency is the round trip delay of thenetwork connecting two devices.


7.4.3 Constraints

A number of constraints are defined that restrict the allowed deployments and con-figurations. The network is limited in capacity by the maximum number of bytesthat can be sent per time unit, and also the devices have a maximum load that canbe processed per time unit. In addition to the constraints imposed by the infrastruc-ture, the application developer can also define constraints on the execution time ofmethods, for example restricting the maximum execution time of a method.

Let Xid be defined as

Xid =

{1 if component ci is deployed on device d0 otherwise (7.1)

with ∑d

Xid = 1 ∀i (7.2)

requiring that each component is deployed on exactly one of the devices, and

hij = 1−∑d

Xid ×Xjd (7.3)

meaning that hij equals 1 when ci and cj are deployed on a different device.The bandwidth used (the number of bytes sent over the network per time unit)

should be less than BW or

bandwidth =∑s

∑m

∑i

∑j

hij × fmscicj× fs

×(Amscicj,p +Rmscicj

,p)

≤ BW (7.4)

Next, for each device, the total load generated by the sequences should not exceedthe maximum load that can be processed per time unit on the whole device. Letloadsd be the load generated per time unit by sequence s on device d, defined as

loadsd =∑m

∑i

∑j

Xjd × Lmscicj,p

×fmscicj× fs (7.5)

The total load generated on device d is defined as loadd and the load constraintbecomes

loadd =∑s

loadsd

≤ Speedd × Coresd ∀d (7.6)

154 CHAPTER 7

Note that this is only an approximation of the load generated on the device. Inreality, different sequences are not necessarily parallelizable, in which case thetotal device capacity can not be utilized. Therefore, this constraint is probably atoo optimistic approximation. However, when the application is not parallelizableat all, extra constraints could be added restricting the load of all sequences to onlyone processor core.

Finally for each time constrained method m the execution time of a methodcall Tmscicj

should be lower than the imposed threshold or

Tmscicj= (

∑d

Xjd × Lmscicj,p ×

1

Speedd)

+hij × ((Amscicj,p +Rmscicj

,p)× 1

BW+ Lat)

+∑

m∈children(mscicj)

Tmc

≤ thresholdm ∀s, ci (7.7)

Here the first term accounts for the CPU time spent executing method call mscicj ,and the second term accounts for the network time required to send the argumentand return values over the network in the case components ci and cj are deployedon different devices. The final term accounts for the execution time of all submethod calls.

7.5 Decision AlgorithmsIn order find the optimal configuration and deployment, the goal is to search for anallocation of each component to a device, and for a value for each configurationparameter that optimizes a given utility function, while adhering to all constraintsimposed. In the situation of d devices, c components, p parameters and vi possiblevalues for parameter i ∈ p, the number of possible solutions is dc ×

∏i∈p vi.

Therefore, a brute force search for the optimum is inappropriate for use at runtimedue to the size of the solution space. To find a valid (although possibly suboptimal)solution in acceptable time, we propose two heuristics.

7.5.1 Optimization goal

The objective is to maximize the utility of all components, where the utility func-tion reflects the quality of the application perceived by the end user as a functionof the configuration parameters:

max∑j

utilitycj (p) (7.8)


This utility function can be provided by the application developer. When no in-formation is given by the developer, we use the aggregated load generated by allmethods of the component as utility measure, assuming that more work done bythe component results in a better quality or

utilitycj (p) =∑s

∑m

∑i

Lmscicj,p ∀cj (7.9)

Other utility functions could be used, for example an utility function for mini-mizing the energy usage, when the devices energy characteristics are known (i.e.energy usage per CPU load, energy usage per byte received/sent, etc.).

7.5.2 Objective function

In order to maximize the utility function while adhering to all defined constraints,we define the following objective function to maximize

objective = +w1 × S(bandwidth

BW)

+w2 ×∑d

S(loaddLmax,d

)

+w3 ×∑

constrained m

S(Tmscicj

thresholdm)

+w4 × (∑j

utilitycj (p)

max_utilitycj) (7.10)

For each constraint, we calculate the ratio of the actual value to the constrainedthreshold value, resulting in a positive value. Values larger than one indicate thatthe constraint is violated. We want to add a penalty factor when a constraint isviolated, thus we scale this ratio using the function S(x) with parameters a, b andt, defined as:

S(x) =

{1− ea×(x−b) if x < ln(1+t)

a + b−t− x otherwise

(7.11)

This function is shown in Figure 7.5. When x < b the result is a positive contri-bution to the objective function, and when x > b, the result is a negative penaltyfactor, decreasing exponentially with parameter a. By choosing b ≈ 1, a positiveterm is added to the objective function when the constraint is satisfied, and a largepenalty factor is added when the constraint is violated. When a threshold of −tis reached, a linear decrease is further used instead of an exponential decrease.This is done to prevent too excessive penalty factors, in order to allow heuristicsto escape from local optima.

156 CHAPTER 7

1

-t

b

S(x)

y

x

Figure 7.5: The weight function S(x) used for constructing the objective function. Theweight function introduces a large penalty factor when x > b.

Each term is weighted with wi, which allows to prioritize between differentconstraints. By choosing w4 >> w1, w2, w3 we make sure that the utility is op-timized. In our experiments we choose w4 = 1000, w3 = 100, w2 = 20 andw1 = 10. For S(x) we set a = 10, b = 0.95 and t = 1000. Note that t ≈ w4,which assures that a violated constraint leads to an overall negative objective value.

7.5.3 KL-based search heuristic

The first algorithm is inspired by the Kernighan-Lin (KL) graph partitioning algo-rithm [34], often used for calculating the optimal layout in digital circuit design.The algorithm consists of two loops. The outer loop continues until no better solu-tion is found. The inner loop searches the solution space by performing a numberof “moves”.

In the context of our problem there are two types of moves. One type of moveschanges the deployment location of a component ci ∈ C to another device dj .The other type of moves changes the configuration value of one of the parameterspi ∈ P to another value vj . For each move the “gain” g is calculated, as the changein objective function when performing this move.

Subsequently, the move with the highest gain is selected and a new solution isfound. The performed move is kept in an ExploredMoves list, that ensures thatthis move is not retried later on in the loop.

Note that also moves with a negative gain are performed when no better movesare found. This enables the heuristic to escape from local optima. At the start ofthe inner loop, the ExploredMoves list is also initialized with all moves that leadto the current solution (InitialMoves), in order to prevent the algorithm to getstuck in the current solution when a local optimum is reached.


Algorithm 7.1 KL-based algorithm

CurrentSolution← StartSolutionBestSolution← StartSolutionrepeatExploredMoves← InitialMovesrepeat

Calculate possible moves K such that∀k ∈ K : k 6∈ ExploredMovesCalculate objective gain g, ∀k ∈ KPerform move kbest with maximum gain g to get NewSolutionCurrentSolution← NewSolutionAdd kbest to ExploredMovesif obj(CurrentSolution) > obj(BestSolution) thenBestSolution← CurrentSolution

end ifuntil no more moves possible

until no better solution foundreturn BestSolution

7.5.4 Simulated annealing (SA)

A second search heuristic is based on simulated annealing (SA), a well-knowncombinatorial optimization technique inspired by the cooling process of metal,which has already been used in the context of optimizing service compositions [30]or graph partitioning [9].

The SA algorithm moves from one solution in the solution space to anotherby performing a random move, and the gain in objective is calculated. Again amove can represent the migration of a component ci ∈ C to another device dj ,or a change in the configuration value of one of the parameters pi ∈ P . If thegain is positive, the move is accepted and a new solution is found. When the gainis negative, the move is only accepted with a probability of exp(g/T ), in whichT is the temperature, that is lowered gradually over time. Thus, the longer thealgorithms runs, the smaller the chance of accepting a move with negative gain.The complete algorithm is shown in pseudocode as Algorithm 7.2.

At the start of the algorithm, an initial temperature T1 has to be determined. Asfirst suggested by Park et al. [35], T1 is chosen such that the fraction of acceptedmoves with negative gain at the initial temperature is equal to a parameter P1,called the initial acceptance probability. The initial temperature is then calculatedas T1 = ∆

P1, where ∆ is the moving average of the negative gains occurred during

the first iteration.At each temperature level the number of times moves explored is called the

epoch length L, which should be chosen proportional to the size of the solution

158 CHAPTER 7

Algorithm 7.2 Simulated annealing algorithm

CurrentSolution← StartSolutionBestSolution← StartSolutionTemperature← T1

repeatcounter ← 0repeatcounter + +Calculate objective gain g of a random moveif g ≥ 0 then

Perform moveelse

Perform move with probability eg

Temperature

end ifif obj(CurrentSolution) > obj(BestSolution) thenBestSolution← CurrentSolution

end ifuntil counter ≥ L (epoch length)Temperature← α.Temperature

until stopconditionreturn BestSolution

space. In our case we found good results setting the number of iterations as twentytimes the product of the number of components, the number of devices and thenumber of parameters. After each epoch the temperature is gradually decreaseduntil the algorithm reaches its stop condition. We adopt a simple geometric coolingfunction, in which the temperature at the kth epoch is given by Tk = α × Tk−1

where α(0 < α < 1) is the cooling ratio. As stopping condition we use Johnson’sstopping rule [36] that ends when the fraction of accepted moves during one epochis below a predefined limit Fmin for I consecutive epochs.

We chose parameters P1 = 0.627, α = 0.908, Fmin = 0.02 and I = 5 asproposed in [9], which leads to close to optimal results as will be discussed in thenext section.

7.5.5 Evaluation

To evaluate the accuracy of the algorithms, we first compare their heuristic so-lution to the theoretical optimum for a number of benchmark problems. Thesebenchmark problems are generated as follows. For the infrastructure model onlytwo devices are considered, one “mobile” device which has two cores and tentimes lower Speed than a “server” device modeled with four cores. Next, an ap-plication model is generated parameterized by the number of components (5,10,15


and 20) and the number of parameters (1 to 5) of the application. Also, one extracomponent is introduced, which is not allowed to move and is bound to the mo-bile device. This component models components that are tied to the hardware andcannot migrate, such as the user interface.

Each parameter can take either 2, 3, 4 or 5 different values, and the utility ofa parameter configuration is determined by the highest value of the first parame-ter. If this value is the same then the second value is considered and so on. Then,sequences are randomly generated such that each component is present in at leastone method call. For each method call, a value for the argument size Amscicj

,p,the size of the return value Rmscicj

,p and the relative CPU load Lmscicj,p is gen-

erated for each parameter configuration, such that these are always less or equal tothe configuration with a higher quality. This reflects that to realize a lower qual-ity, less processing power and/or communication is required. Finally one or moreconstraints are randomly added restricting the execution time of one of the meth-ods. The value of the constraint is chosen between the minimum and maximumpossible execution time of the method on the mobile device.

This way relatively small problem instances are created, which enables us tocalculate the theoretical optimum using a brute force (BF) algorithm, and evaluatethe effectiveness of the proposed search heuristics. For each number of compo-nents and number of parameters, we generated hundred problem instances. Fig-ure 7.6 shows the relative number of the cases that the KL algorithm finds theoptimal solution, i.e. where the resulting objective function value equals the valuereturned by the brute force algorithm. The white bar shows the extra number of the

Figure 7.6: The KL algorithm produces close to optimal results for small problem instances,but solution quality quickly degrades as the problem size increases.

160 CHAPTER 7

Figure 7.7: SA finds the optimal solution in more than 85% of the cases for problem sizes upto 15 components. Performance degrades slowly as the problem size increases.

cases where the resulting objective function was suboptimal, but the same utilitywas achieved. This is for example the case when the KL solution achieves the op-timal quality, but uses more bandwidth (but still less than the available bandwidth).

For problem instances of ten or less components and two or less parameters theKL algorithm finds the maximum utility in 85% of the cases, but the performancequickly degrades as the number of components increases.

The results for SA are shown in Figure 7.7. As SA is a non-deterministic al-gorithm, we repeated the experiment ten times and the averages are plotted. Againwe see that the performance degrades as the problem size increases, but muchslower than in the case of KL. In the case of 20 components and 5 parameters, theSA algorithm still finds the optimal utility in 69% of the cases, compared to only17% for KL.

The difference in performance of both search heuristics is explained whenlooking at the execution times. The average execution time of both algorithmsand the brute force (BF) approach are shown in Figure 7.8 for the problem in-stances with 3 parameters. All experiments are executed on a server equippedwith a Xeon Nehalem CPU clocked at 2.26 GHz. KL is the fastest algorithm, withexecution times of about 10 ms, at the cost of suboptimal solution quality. The SAalgorithm is about hundred times slower than KL, and is even slower than the bruteforce approach for small problems. However for the larger problem instances, theSA algorithm is much faster than the BF approach, while still finding the optimalsolution in more than 80% of the cases.


Figure 7.8: The KL algorithm is about hundred times faster than SA, but at the cost ofsolution quality.

7.6 Experimental results

In order to validate our approach, we implemented a prototype of the frameworkpresented in Section 7.3 in Java, which enables us to run the framework on desk-top operating systems, as well as on Android. Our prototype is built on OSGi [37],a component-based service platform in Java, which enables transparent monitor-ing and mobile code [13]. We implemented a component-based augmented realityapplication on top of this framework, and generated an application and infrastruc-ture model from this monitoring information. This way, we can validate whetherour model sufficiently captures the application, and our decision algorithm leadsto optimal deployment and configuration for different mobile devices, taking intoaccount application specific constraints and network connectivity.

7.6.1 AR use case scenario

As use case, we present an augmented reality application featuring markerlesstracking as described by Klein et al. [38], combined with an object recognitionalgorithm presented in [39]. The application is shown in Figure 7.9. On the righta grayscale video frame is shown with the tracked feature points, from which thecamera position is estimated. The left part shows the resulting overlay with a 3Dobject, and a white border around the recognized book.

The component-based implementation of this application is shown in Fig-ure 7.10. The runtime behavior of these components is captured in a number ofsequences, illustrated in Figure 7.11. The first sequence (Fig. 7.11(a)) shows the

162 CHAPTER 7

Figure 7.9: The augmented reality application tracks feature points in the video frames(right) to enable the overlay of 3D objects (left).

tracking and rendering thread: the VideoSource component periodically fetches acamera frame from the hardware, which is processed by the Tracker component.The Tracker estimates the current camera position from a number of recognizedfeature points. The camera frame is also passed to the Renderer, which queriesthe Tracker for the current camera position, in order to render the correct over-lay. From time to time the Tracker sends a video frame to the Mapper for map


TrackerMapper

Object

Recognizer

resolution (800x480,400x240)

#features[200..1000]

Feature

Detector

Feature

Matcher

Figure 7.10: The AR application is implemented as a set of loosely coupled software com-ponents. The VideoSource and Tracker component each expose a configura-tion parameter affecting the QoS of the application.


generation and refinement, which is shown in the second sequence (Fig. 7.11(b)).By matching 2D features in a sparse set of so-called keyframes, the Mapper canestimate their 3D location in the scene and generate a 3D map of feature points,which is notified to the Tracker. Finally, in a third sequence (Fig. 7.11(c)), thekeyframes are analyzed for SIFT features, which are more complex to calculate,but can be used for object recognition by matching them against a database of SIFTfeatures of known objects. When objects are recognized and localized in the map,their position is notified to the Renderer, that can use this information to render anoverlay.

Video Tracker RendererprocessFrame()

processFrame()getCamPosition()

(a)

Tracker MapperaddKeyframe()

notifyMapUpdate()

(b)

Mapper ObjectRecFeature

DetectorsearchObjects()

searchFeatures()

Feature

Matcher

matchFeatures()

Renderer

updateRenderObjectPosition()

(c)

Figure 7.11: The augmented reality application consists of three sequences. In (a) the track-ing and rendering sequence is shown, which processes the video frames. Themap refinement sequence is shown in (b), and (c) depicts the object recognitionsequence.

164 CHAPTER 7

Two crucial configuration parameters affecting the application quality wereidentified: the camera resolution and the number of tracked features. Both devicessupport two resolutions: 800x480 and 400x240. The number of features to trackaffects the processing time of a frame by the Tracker (which is crucial to achievean acceptable frame rate). Typical values for this parameter are 1000, 950, ..., 200.When more features are used for tracking the frame, a better estimate of the cameraposition can be made, resulting in more robust tracking, at the cost of increasedprocessing time.

We evaluated the AR use case on two mobile devices, both connected to alaptop via WiFi. The laptop is equipped with an Intel Core 2 Duo CPU clocked at2.26 GHz. As mobile devices we use a HTC Desire, with a single core Qualcomm1 GHz Scorpion CPU, and an LG Optimus 2x powered by a dual core Nvidia Tegra2 CPU, also clocked at 1 GHz.

The monitored execution times of the tracker and object recognition sequencesfor different configurations are shown in Figure 7.12. Figure 7.12(a) shows thatthe time to process a frame increases linearly with the number of feature pointstracked. It also shows that the LG Optimus is 2 to 2.5 times faster than the HTCDesire. Figure 7.12(b) shows the processing times for object recognition, andagain the Optimus is 2 to 3 times faster than the Desire, but the only acceptableprocessing times are achieved with the laptop, which is about 10 times faster thanthe Optimus. Therefore we set the relative Speed parameter as 0.4, 1 and 10 forthe Desire, Optimus and laptop respectively.

Based on the monitoring information, we have set values forLmscicj,p,Amscicj

,p,Rmscicj

,p for each method call. In this case each method call is executed only oncein the sequence (fmscicj

= 1). Every five seconds one frame is added to the mapand searched for objects (#callss = 0.2). For the tracker sequence, the developerwants a minimal frame rate of 15 frames per second (fs = 15), meaning that aframe should be processed within 60ms, and objects should be recognized within3 seconds. The devices are connected using a WiFi network of 10 Mbps and alatency of 1 ms.

Using this information, we can now calculate the optimal deployment and con-figuration. The Mapper, ObjectRecognizer, FeatureDetector and FeatureMatchercomponents are offloaded to the laptop. The Tracker components run on the mo-bile device, because of the limited bandwidth. Depending on the CPU capacity, theconfiguration is adapted to achieve the required frame rate. For the HTC Desireimages are captured in 400x240 resolution and only 250 feature points are tracked,the Optimus captures frames in 800x480 resolution and tracks 500 points, as couldbe expected from Figure 7.12(a).


(a)

(b)

Figure 7.12: Monitored execution times of the tracker (a) and object recognition (b) se-quences, for different configurations.

166 CHAPTER 7

7.7 ConclusionIn this paper we present a framework for adapting both the deployment and theconfiguration of software components, in order to optimize the Quality of Serviceof the end user. We presented an application and infrastructure model, to captureruntime monitoring information, and to model application specific constraints. Wecompared two decision algorithms to optimize the application configuration anddistribution, taking into account the network connectivity, the available resourcesand application constraints imposed by the application developer.

Comparing both search heuristics shows that a Simulated Annealing approachresults in better solution quality, at the cost of more processing time. A Kernighan-Lin based heuristic finds a solution in tens of milliseconds, but the solution qualityquickly degrades as the problem size increases.

For a mobile augmented reality application we show that our model capturesthe application runtime behavior, and our algorithm is able to find the optimal de-ployment and configuration depending on the device capabilities and the networkcontext.

As future work, the KL based algorithm will be further refined in order to geta better solution quality for larger problem instances, for example by adopting amultilevel partitioning approach [9].

AcknowledgmentTim Verbelen is funded by PhD grant of the Fund for Scientific Research, Flanders(FWO-V).This project was partly funded by the UGent BOF-GOA project “Autonomic Net-worked Multimedia Systems".


References

[1] Gartner Group. Press Release : Gartner Says Worldwide Sales of MobilePhones Declined 2 Percent in First Quarter of 2012, May 2012. Availablefrom: http://www.gartner.com/it/page.jsp?id=2017015.

[2] A. I. Wasserman. Software engineering issues for mobile application devel-opment. In Proceedings of the FSE/SDP workshop on Future of softwareengineering research, FoSER ’10, pages 397–400, 2010.


[4] A. Puder and O. Antebi. Cross-Compiling Android Applications to iOS andWindows Phone 7. Mobile Networks and Applications, pages 1–19, 2012.

[5] P. Makris, D. Skoutas, and C. Skianis. A Survey on Context-Aware Mobileand Wireless Networking: On Networking and Computing Environments’ In-tegration. IEEE Communications Surveys and Tutorials, pages 1–25, 2012.

[6] R. Balan, J. Flinn, M. Satyanarayanan, S. Sinnamohideen, and H. Yang. Thecase for cyber foraging. In Proceedings of the 10th workshop on ACMSIGOPS European workshop, EW ’10, pages 87–92, 2002.

[7] I. Giurgiu, O. Riva, D. Juric, I. Krivulev, and G. Alonso. Calling the cloud:enabling mobile phones as interfaces to cloud applications. In Proceedingsof the 10th international conference on Middleware, Middleware ’09, pages83–102, 2009.

[8] M. A. Hassan and S. Chen. An Investigation of Different Computing Sourcesfor Mobile Application Outsourcing on the Road. In Proceedings of the Con-ference on Mobile Wireless Middleware, Operating Systems, and Applica-tions, pages 153–166, 2011.

[9] T. Verbelen, T. Stevens, F. D. Turck, and B. Dhoedt. Graph partitioningalgorithms for optimizing software deployment in mobile cloud computing.Future Generation Computer Systems, 29(2):451–459, 2013.

[10] E. Cuervo, A. Balasubramanian, D. Cho, A. Wolman, S. Saroiu, R. Chandra,and P. Bahl. MAUI: making smartphones last longer with code offload. InProceedings of the 8th international conference on Mobile systems, applica-tions, and services, MobiSys ’10, pages 49–62, 2010.


168 CHAPTER 7

[11] B. Chun, S. Ihm, P. Maniatis, M. Naik, and A. Patti. CloneCloud: elas-tic execution between mobile device and cloud. In Proceedings of the sixthconference on Computer systems, EuroSys ’11, pages 301–314, 2011.

[12] M. Kristensen. Scavenger: Transparent development of efficient cyber for-aging applications. In Proceedings of the IEEE International Conferenceon Pervasive Computing and Communications, PerCom ’10, pages 217–226,2010.

[13] T. Verbelen, P. Simoens, F. De Turck, and B. Dhoedt. AIOLOS: Middle-ware for improving mobile application performance through cyber foraging.Journal of Systems and Software, 85(11):2629–2639, November 2012.

[14] S.-N. Chuang and A. Chan. Dynamic QoS Adaptation for Mobile Mid-dleware. IEEE Transactions on Software Engineering, 34(6):738 –752,November-December 2008.

[15] M. Sharifi, S. Kafaie, and O. Kashefi. A Survey and Taxonomy of CyberForaging of Mobile Devices. IEEE Communications Surveys and Tutorials,14(4):1232 –1243, 2012.

[16] J. Flinn, S. Park, and M. Satyanarayanan. Balancing Performance, Energy,and Quality in Pervasive Computing. In Proceedings of the 22 nd Inter-national Conference on Distributed Computing Systems, ICDCS ’02, pages217–226, 2002.

[17] R. K. Balan, M. Satyanarayanan, S. Y. Park, and T. Okoshi. Tactics-basedremote execution for mobile computing. In Proceedings of the 1st interna-tional conference on Mobile systems, applications and services, MobiSys’03, pages 273–286, 2003.

[18] X. Gu, K. Nahrstedt, A. Messer, I. Greenberg, and D. Milojicic. Adaptiveoffloading for pervasive computing. IEEE Pervasive Computing, 3(3):66–73, July-September 2004.





[22] Y. Su and J. Flinn. Slingshot: deploying stateful services in wireless hotspots.In Proceedings of the 3rd international conference on Mobile systems, appli-cations, and services, MobiSys ’05, pages 79–92, 2005.

[23] N. Davies. The impact of mobility on distributed systems platforms. InProceedings of the IFIP/IEEE International Conference on Distributed Plat-forms, pages 18 –25, 1996.

[24] D. Chalmers and M. Sloman. A survey of quality of service in mobile comput-ing environments. IEEE Communications Surveys and Tutorials, 2(2):2–10,1999.

[25] L. De Cicco, S. Mascolo, and V. Palmisano. Feedback control for adaptivelive video streaming. In Proceedings of the second annual ACM conferenceon Multimedia systems, MMSys ’11, pages 145–156, 2011.

[26] M. Prangl, I. Kofler, and H. Hellwagner. Towards QoS Improvements of TCP-Based Media Delivery. In Fourth International Conference on Networkingand Services, ICNS ’08, pages 188 –193, March 2008.

[27] F. Cuadrado, J. Duenas, and R. Garcia-Carmona. An Autonomous Enginefor Services Configuration and Deployment. IEEE Transactions on SoftwareEngineering, 38(3):520–536, May-June 2012.

[28] G. Canfora, M. Di Penta, R. Esposito, and M. L. Villani. A framework forQoS-aware binding and re-binding of composite web services. Journal ofSystems and Software, 81(10):1754–1769, October 2008.

[29] B. Liu, Y. Shi, and H. Wang. QoS oriented web service composition andoptimization in SOA. In Proceedings of the Joint Conferences on PervasiveComputing, JCPC ’09, pages 605–610, December 2009.

[30] F. Rosenberg, M. MuÌLller, P. Leitner, A. Michlmayr, A. Bouguettaya, andS. Dustdar. Metaheuristic Optimization of Large-Scale QoS-aware ServiceCompositions. In Proceedings of the IEEE International Conference on Ser-vices Computing, SCC ’10, pages 97–104, July 2010.

[31] R. Calinescu, L. Grunske, M. Kwiatkowska, R. Mirandola, and G. Tambur-relli. Dynamic QoS Management and Optimization in Service-Based Sys-tems. IEEE Transactions on Software Engineering, 37(3):387–409, May-June 2011.

170 CHAPTER 7

[32] T. Verbelen, T. Stevens, P. Simoens, F. D. Turck, and B. Dhoedt. Dynamic de-ployment and quality adaptation for mobile augmented reality applications.Journal of Systems and Software, 84(11):1871–1882, 2011.

[33] J. Kephart and D. Chess. The vision of autonomic computing. IEEE Com-puter, 36(1):41–50, January 2003.

[34] B. Kernighan and S. Lin. An efficient heuristic procedure for partitioninggraphs. Bell System Technical Journal, 49(2):291–307, 1970.

[35] M.-W. Park and Y.-D. Kim. A systematic procedure for setting parame-ters in simulated annealing algorithms. Computers & Operations Research,25(3):207–217, March 1998.

[36] D. S. Johnson, C. R. Aragon, L. a. McGeoch, and C. Schevon. Optimiza-tion by Simulated Annealing: An Experimental Evaluation; Part I, GraphPartitioning. Operations Research, 37(6):865–892, November 1989.


[38] G. Klein and D. Murray. Parallel Tracking and Mapping for Small ARWorkspaces. In Proceedings of the 6th IEEE and ACM International Sympo-sium on Mixed and Augmented Reality, ISMAR ’07, pages 1–10, 2007.

[39] D. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. Inter-national Journal on Computer Vision, 60(2):91–110, 2004.

8Conclusions and perspectives

“I have not failed. I’ve just found 10,000 ways that won’t work.”

–Thomas A. Edison (1847 - 1931)

Mobile devices have become part of modern-day life. Smartphones and tabletsdrive the electronic industry, and mobile app markets are flooded with hundreds ofthousands of applications. Notwithstanding the tremendous technical evolutions,we are still facing the same challenges as back in 1983 when the first handset wasreleased. Although advances in micro chip technology have driven the develop-ment of smartphone, hardware vendors are still on a quest for longer battery life,more processing power and thinner devices. These hardware limitations, togetherwith the highly heterogeneous mobile device market, impose many challenges onmobile software developers, that have to tweak their application for a wide rangeof devices with different capabilities.

Throughout this dissertation, we have addressed these challenges by lettingthe application be self-adaptive. Two adaptation mechanisms are discussed. Onthe one hand, parts of the application are offloaded from the mobile device to lessconstrained infrastructures in the network. Our experiments in Chapters 5 and 6have shown that this indeed can lead to a speedup of up to a factor 10, but ofcourse this depends on many factors, such as the characteristics of the offloadedpart, the network conditions and the infrastructure used. On the other hand, wealso considered configuration adaptation at runtime, either by switching betweendifferent component implementations at runtime as proposed in Chapter 3, or bytuning configurable parameters as shown in Chapter 6.

172 CHAPTER 8

In order to apply these adaptation mechanisms, we have designed and im-plemented a middleware platform that is able to transparently offload and con-figure application components at runtime. Our middleware runs on mobile plat-forms such as Maemo (Chapter 3) and Android (Chapter 5, 6) as well as desktopLinux platforms and server machines in the Amazon EC2 Cloud (Chapter 5). Be-sides component adaptation, the middleware framework also handles discoveryand management of resources in the network, as well as monitoring and profilingapplication components at runtime. This monitoring information can be used forpredicting execution times as illustrated in Chapter 5, and results show that ourapproach using a random sample consensus algorithm offer a relative precision of10% or less for 80% of the method calls.

Of course, these adaptation mechanisms are still of no use when they are notadopted in an intelligent way. Therefore, this dissertation presents a number ofoptimization algorithms for optimizing component deployment and/or componentconfiguration. In the case of component offloading, the network bandwidth isoften a limiting factor. To address this issue, Chapter 4 focuses on partitioningalgorithms for minimizing the bandwidth required for communication, inspired bythe Kernighan-Lin (KL) graph partitioning algorithm and random search heuristicssuch as Simulated Annealing (SA). In Chapter 7, the same techniques are appliedfor co-optimizing the deployment as well as the configuration of the application, inorder to maximize the Quality of Service perceived by the end user. In both cases,our search heuristics provide near to optimal results for small problem instances,but scale better in terms of execution time than optimal search algorithms. In gen-eral, the SA based algorithms lead to better results than the KL based techniques,at the cost of longer execution time.

Still, the middleware platform and accompanying optimization techniques areof little value without applications making use of the possibilities offered. Tovalidate the middleware, common smartphone applications such as a photo edi-tor and a chess game are used in Chapter 5. However, the real benefits of theframework only become apparent when dealing with complex media processingapplications, that have stringent resource requirements and often soft real-timeconstraints. Therefore, we developed a mobile augmented reality (AR) applicationimplementing state-of-the-art computer vision techniques, introduced in Chapter2. We have shown the need and effectiveness for deployment and configurationco-optimization for this application in Chapter 6 and 7, offloading CPU intensivecomponents for speed up, and configuring constrained components depending onthe device capabilities. To reduce the burden put on the application developer, weproposed an annotation-based programming model. In Chapter 5, a single anno-tation suffices to define an offloadable component. In Chapter 6 we extended theprogramming model with semantically richer annotations, allowing the developerto specify dependencies, configurable parameters and performance constraints.

CONCLUSIONS AND PERSPECTIVES 173

In retrospect, one could ask which are the most valuable lessons learned andthe key insights from this research. The main contribution of this dissertation isthe combination of dynamic offloading and configuration which enables highlyadaptive application scenarios, necessary to cope with the vastly increasing deviceheterogeneity. As discussed in Chapter 7, taking also the configuration into ac-count tremendously increases the complexity of the optimization problem at hand,and optimal solution techniques are no longer feasible to use at runtime. Anotherimportant lesson learned is the impact of the applications and their characteristicson the applicability of offloading and configuration techniques. Applications thatare driven by user input, as discussed in Chapter 5, often have other characteris-tics and requirements than data-driven applications such as the augmented realityapplications discussed in chapters 3 and 6. More research is required on how tocapture the characteristics of the application, either at runtime or given by the ap-plication developer, and to decide which metrics should be optimized. Also, whenrelying on input from the developer, care should be taken to minimize the bur-den and to automate this proces as much as possible. Possible approaches couldbe to more thightly integrate component-based concepts into the mobile platformAPIs, or to employ model-driven engineering techniques for mobile applicationdevelopment.

Another important question to ask is whether the research results from thisdissertation are sustainable, given the current mobile computing trends. As theprocessing power of mobile devices continue to increase exponentially, one couldargue that the need for offloading components to increase speed will become obso-lete. Moreover, applications that are currently most popular comprise simple webbrowsing, e-mail, social media, etc. which require few resources of the mobileuser. Also more demanding applications such as basic speech recognition for en-abling voice commands are already implemented on many current systems. On theother hand, many resource intensive components such as complex object or speechrecognition can already be identified upfront, and are often deployed statically asa service in the Cloud.

However, as illustrated in this dissertation, for data-driven applications such asaugmented reality, that need to process various sensor data streams in near real-time, no one size fits all solution will become apparent in the near future. As hard-ware capabilities increase, so does the size of the sensor information (e.g. cameraresolution) that needs to be processed. As mentioned in Chapter 2, an importantuse case is defined by collaborative applications, where the input of many can beprocessed distributed in order to gain a better end result for all. This tend to bea “chicken or the egg” dilemma, as no applications exploiting adaptive offloadingwill arise when no such adaptation system is deployed, and vice versa. Therefore,in future perspective an important goal is to collaborate with other disciplines,working on both the next generation applications, together with the frameworks to

174 CHAPTER 8

support these applications.Another incentive is energy, which forms a fundamental constraint for any

battery-powered device. As battery capacity improves at a slower rate than forexample processor speed or memory capacity, energy management is becomingincreasingly important in mobile computing. By offloading parts of the applica-tion to fixed infrastructure, or by degrading the application to a lower quality, theenergy used by the mobile device could be reduced. An important challenge in thisrespect is to trade off the energy used by processing versus the energy required forsending data over the network. Research is ongoing at the INTEC BroadbandCommunication Networks group to construct an energy model of current mobiledevices, in order to apply the techniques presented in this dissertation for energysaving.

Finally, current advances in mobile hardware lead to the development of newperipherals such as smart watches, head-mounted displays or smart textiles. Newdisplay technologies such as Google Glass will empower a new kind of compleximmersive applications. Due to the even more stringent restrictions on weight,form factor and heat dissipation, these wearable devices will be even more resourceconstrained, and adaptive configuration and deployment could prove a key enablerfor the next generation mobile applications.

Documents

Adaptief uitbesteden en configurerentverbele/PhD_Verbelen.pdf · 2013-06-26 · Adaptief uitbesteden en configureren van complexe mobiele toepassingen Adaptive Offloading and Configuration