mobile Automation in the cloud
Hello!Dmitry Lemeshko
QA Automation @ Ciklum Minsk / eBuilder Sweden
PLAN for TODAY’s EVENING:× Automation without a cloud: any problems?
× Execution in cloud: server-side vs client-side
× Device labs, frameworks and integrations
× What may go wrong a.k.a limitations
× Tool set and Use case on real project
× Your Questions
LOCALRUN
Who took the device?
UPDATE device os VERSION?APPIUM Server errors?
Device cleaning? Xcode 7 or xcode 8?
Who is responsible for maintenance?Parallel run?
WITHCLOUD
Who took the device?
UPDATE device os version?APPIUM Server errors?
DEVice Cleaning? Xcode 7 or xcode 8?
Who is responsible for maintenance?Parallel run?
LET’S START the JOURNEY
POPULAR Mobile Device Clouds
Client-side vs Server side cloud execution
SERVER-side CLOUD execution
PROS
× Tests execution self-managed× Parallel run out-of-box× Add extra data and apps× Automatic results aggregation
CONS
× Limited language support× Follow code structure× No control on test execution× Limited integration
CLient-side cloud execution
PROS
× Language flexibility× Drive your own framework× Code privacy× More ways for integration
CONS
× Lots to manage yourself: device availability, test execution, parallel run, results
Device labs
Android/iOS 143 / 36 357 / 42 176 / 176
Android 4.4+/6.x/7.x 67 / 31 / 7 137 / 35 / 5 150 /16 / 8
iOS 9.x/10.x 23 / 3 5 / 14 100 / 10
Manufactures 20 32 22
Tablets 18% 30% 30%
Framework support
Android AppiumEspressoRobotium
Appium CalabashEspresso UI AutomatorRobotium
iOS Appium Appium CalabashUI Automation XCTest / XCUITest
Integration support
CLOUD API JAVA, RUBY CLIENTS JAVA, RUBY, PYTHON CLIENTS
AWS API / AWS CLI
PLUGINS JENkiNS, GRaDLE JENkiNS, GRaDLE
App release tools Hockey App
Bug Trackers JIRA, GITHUB JIRA
URLhttp://appium.testobject.com/wd/hub
Desired capabilitiestestobject_api_key = 73A**********12Testobject_device = Samsung_Galaxy_S7_real
TEST OBJECT Appium Setup
testobject_appium_version = 1.6.3testobject_app_id = 92testobject_suite_name = Smoke Testtestobject_test_name = Navigation via menutestobject_cache_device = truetestobject_session_creation_timeout = 30Returntestobject_test_live_view_urltestobject_test_report_url
TEST OBJECT OPTIONAL CAPABILITIES
test execution output in cloud1. Device logs
2. Appium logs
3. Screenshots
4. Test run recording
5. Cpu/Memory usage
What may go wrong a.k.a limitations
1. Appium server update frequency
2. test runs fail on device setup stage
3. No login with custom google account
4. No device settings change
5. No wifi turn on/off
Moved to testobject.. still have limitations1. No device storage access
2. Sms, contacts & call history cleaning
3. No push notifications for iOS
4. No Geo location setup
5. Camera usage
Tool set and Use case on real project× Appium× Java/TestNG× HockeyApp API× TestObject API× Team City CI
× Smoke: random in group× Regression: top 10 devices
(covers 70% Android users)× Feature test (on request):
50+ devices
BACKUP & RESTORE FUNCTIONALity TESTING
Can you try it?
GO AHEAD!
250 min trial
5 free devices
3 free devices
Recommended
