Badoo: Cross platform Mobile Test Automation and Continuos Delivery

Badoo: Cross pla,orm mobile test automa3on and con3nuous delivery

Sathish Gogineni Mobile QA Automa4on Lead

Raise your hands if…. •  You’re working in Mobile pla=orm?

•  You have apps on mul4ple mobile pla=orms?

•  You’re aware of the test automa4on tools in Mobile?

•  You are using func4onal test automa4on?

•  You’re using “Cross pla=orm” test automa4on?

A bit of self promo3on

Who is Badoo?

•  A social networking service

•  241 million users

•  Available in 190 countries.

•  Support 45 languages on mobile

•  250 employees based in London and Moscow

More about Badoo

•  Badoo has 2 popular products

•  Apps support 4 mobile pla=orms

– Badoo – HotOrNot

Mobile Test Automa3on at Badoo •  A single test automa4on code base

–  Run across two products –  Run on three pla=orms –  Run on phones and tablets

•  We have –  68% func4onal coverage for Badoo in Android & iOS ( 390 scenarios )

–  76% func4onal coverage for HotOrNot in Android & iOS ( 250 scenarios )

–  52% func4onal coverage in MobileWeb ( 165 scenarios ) –  And in Windows pla=orm we have 0%

A bit about Cross pla,orm test

Mobile cross pla,orm test automa3on?

•  Write once , Run anywhere(?)

•  Write test specifica4ons

•  Implement the tests at high level

•  Implement the pla=orm/applica4on UI code

Cross pla,orm stack in BDD

Scenarios

Feature

Step defini3ons

Android Page iOS Page

Test Specifica4on

High level implementa4on Pla=orm specific

Why cross pla,orm test automa3on?

•  Business logic is same. It is only interac4on with UI is change

•  Pooled resources

•  More reusable code. Less maintenance

What should be considered?

•  Who is wri4ng the test cases? App developers / QA developers?

•  Which mobile pla=orms?

•  Wri4ng in language other than pla=orm specific

A bit about Cross Pla,orm test tools

Mobile Cross Pla,orm Tools

•  Instrument based tools

•  Image Recogni4on tools

– Calabash – Appium – Monkey Talk

– eggPlant – seetest

A bit about Calabash

What is Calabash?

•  A framework wriben in “Ruby”

•  Has client-‐server architecture using JSON

•  Integrated with cucumber

•  Supports Android and iOS pla=orms

Calabash : Pros & Cons •  Pros – Has well structured client framework –  Supports backdoor opera4ons to interact with Applica4on code

•  Cons –  Can’t run on “Produc4on” build in iOS – Have to use Ruby programming language –  Forced to write test cases in BDD ( ‘Given When Then’ )

A bit about Appium

What is Appium? •  Is a Server wriben in Node.js using JavaScript

•  Uses pla=orm-‐provided frameworks under the hood

•  Supports Web driver JSONWire protocol

•  Supports iOS, Android, Mobile Web and Hybrid applica4ons

Appium : Pros & Cons •  Pros – Tests can be wriben in any ( most of ) programming languages

– Good community support – Can use same produc4on builds for tes4ng

•  Cons – Lible bit “buggy”

A bit about Con4nuous Delivery

Con3nuous Delivery is…

A prac4ce of crea4ng, tes3ng and releasing stable applica4on frequently in smaller chunks

What is Con3nuous Delivery?

Con3nuous Delivery =

Con3nuous Integra3on +

Con3nuous Test Automa3on


Development Branch


Development Branch


Development Branch

Implement Feature tests


Development Branch

Feature Tests Implement

Feature tests


Development Branch

Feature Tests

Smoke Tests



Development Branch

Feature Tests

Smoke Tests



Development Branch

Feature Tests

Smoke Tests

Nightly Builds

Nightly Builds



Development Branch

Feature 2

Feature 1


Development Branch

Master

Feature 2

Feature 1


Development Branch

Integra4on Branch

Master

Feature 2

Feature 1


Development Branch

Integra4on Branch

Integra4on Tests

Master

Feature 1

Feature 2

Con3nuous Test Automa3on Development Branch

Integra4on 2

Integra4on 1

Master Branch


Integra4on 2

Integra4on 1

Master Branch

Nightly Builds

Nightly Builds


Integra4on 2

Integra4on 1

Master Branch

Release Branch


Integra4on 2

Release Tests

Integra4on 1

Master Branch

Release Branch


Integra4on 2

Integra4on 1

Master Branch

Release Branch

Release Tests

Release Tests

Scaling build infrastructure

How we started?

•  A single build machine with Jenkins

•  Running on Android emulator and iOS simulator

•  Run the tests only on “master” branch every night

What we have today?

•  Distributed CI server with 12 mac-‐mini agents

•  Each Agent has Android and iOS device connected

•  Running the tests on Real devices

•  Integrated with JIRA

What we will do in future?

•  More devices to each agent and Parallelizing test suite

•  Using VMs for scaling build agents

Summary

•  Worth wri4ng cross pla=orms tests. It Ensure behavior is same in every pla=orm

•  Mobile Test Automa4on tools are s4ll evolving

•  And, they have limita4ons

Nevertheless… •  We reduced Regression tes4ng to a day for every release ( 66% 4me save )

•  3X more releases to the app stores

•  More stable applica4on. Crash rate reduced from 6% to 2%

hQps://techblog.badoo.com/

Sathish Gogineni [email protected]

Documents

Badoo: Cross platform Mobile Test Automation and Continuos Delivery