Top100summit 跨平台自動化測試與工具徐翔智

揭示研发管理白金定律，分享那些激动人心的创新与变革，使得团队获得过多源动力与更大的推动力！

跨平台的自动化测试与工具

Tony Hsu徐祥智Sr Quality Manager

Tony_Hsu@trend.com.tw

趨勢科技

http://top100summit.com/content.php?id=365

跨平台自动化测试与工具

案例背景实践

案例ROI分析

• UI Error handled • Multi-threads • Reboot scenario

失敗與放棄

•Automation Framework •Build Portal •L10n Automation •Tools

案例启示

未来想尝试

•Automation Framework •Win Browser UI Automation •iOS Mobile Automation •Android Mobile Automation

參考資料

Manual Automation

30.84 hr/man 1.1 hr/man

案例背景 http://top100summit.com/content.php?id=365

• Launch in 2012 March

• Windows, iPhone, iPad, Android Phone and Android Pad. Support Browser IE, FireFox and Chrome.

• 20+ team members development team size (QA/RD)

• Cross-site testing with localization team in JP/China & support team in every region.

• Multilingual User Interface (MUI) build.

10 + Years

87600 hours 2102400 Min In software development

About Me

• Sr Quality Manager / Sr Project Manager •Consumer product development group • TrendMicro

跨平台的自动化测试与工具

Automation Framework

Testing Report

Build Portal for Cross-platform

L10n Testing Automation

目標

实践 1.1 - Automation Framework – Testing Flow

B1001 B1002 B1003 B1004….

•Build monitoring •Build/Testing Deployment •Auto-Testing Execution •Sending Report

•QA Report

实践 1.2 – Automation Framework Key Component

•Daily Build Monitor

•Information for automation server

•Testing target machines

•Source location

•Server

•Revert the client VM

• package the Build and Testing script

• Deploy resource to client

• Trigger automation testing

• Collect testing result and send email

•Client

• Testing scripts Execution

• Generate testing report

实践 1.3 - Automation Framework –Infrastructure Blocks

Product Library

Testing Script

Add-on Module

Common Library

Python STAF Core Basis

Add-on

Application Testing

实践 1.4 - Automation Framework – Infrastructure

Windows Auto Testing

Android Auto Testing

Common lib

Test suite

Test Runner

Test Cases

iOS Auto Testing

Linux Auto Testing

TestResult.txt

Common lib

Plink Tool

iOS Auto Testing

Profile

UIAutomation

Instruments

iOSview001.js

实践 1.5 - iOS

实践 1.6 – Android

Common lib

Android project

Psexec.exe

Android Test project - Robotium

Instruments

TestView01.apk

Androidview001.java

Python & ADB

androidview001_001.py

PC for Android

Android emulator

实践 2.1 – Automation Testing Report – Key Info

•Build Number •Testing Machine Name/IP •Testing Time •Platform •Results •Crashes •Testing Cases •Further Link for logs/screenshot

实践 2.2 – Automation Testing Report–Link to details

•Testing Steps of failure • Logs • Screenshot

实践 3.1 – Build Status Portal

Problem to Solve under cross regional team Environment.

1. How and Where to download Build?

2. What’s change in the build?

3. What’s the build configuration Status?

实践 3.2 – Build Status Portal

1. Build Number

2. Build Result

3. FTP download URL for Windows

4. SAMBA download URL for Mac OS

5. What's New in the build

6. Build Time

7. Configuration Highlight

8. Build Log

实践 4.1 – Localization Automation Testing

The problem to Solve for L10n UI verification

1. UI changes impact on other languages?

2. How to efficiently validate OEM customized UI changes?

3. How to validate UI truncation issue after language translation?

实践 4.2 – Localization Automation Testing – UI Truncation

1. Source code parser for • “SetText” with Android • “NSString” with iOS 2. Word counts of the parsing results 3. Compare the UI width 4. List of potential UI truncation ISsue

实践 4.3 – Localization Automation Testing – UI Screenshot Automation

•Key Technology •iOS UIAutomation •Robotium

失败经验 • UI Error handled

• Multi-threads testing execution

• Reboot scenario

未来尝试

• More Testing Cases.

• MetroApp Automation

效益分析

Before (Manual) After (Automation)

Testing Efficiency 30.84 hr/man 1.1 hr/man

Testing Coverage

Platform Win7_x32 W7_x32 , Vista_x32 , W7_x64 + Vista_x64 + WinXp and so on.

Browser IE, Firefox, Chrome = 3 IE, Firefox, Chrome with different versions

Language EN = 1 EN, JP, FR, TW, DE = 5

成功要素

1. Clearly Define the automation testing scope and staged-goal

2. Uses of Tools

3. Plan automation tasks as parts of QA testing cycle

4. Testing Report links to further info/logs to trace the issue

成功经验总结

1. Automation Framework (Build Monitoring, Testing Server and Client)

2. Integration with iOS/Android and Windows

哪些技术促使项目成功。

1. For automation framework, Python and Software Testing Automation Framework (STAF)

2. For Android UI Automation, Robotium

3. For Browser UI control, Selenium and JsonWireProtocol

4. For Testing Client environment, VM workstation, ESXi, Lab Manager

案例启示 -跨平台的自动化测试与工具

•Automation Framework •Consolidated reporting •VM workstation, ESXi, LabManager •Software Testing Automation Framework (STAF) •Python •Build Portal •Consolidated build status and download for cross platform build.

•L10n Automation •UI verification •Tools •Selenium UI control •Robotium •iOS UIAutomation •iOS/Android emulator

參考資料

•Automation Framework •http://www.python.org/download/ •http://sourceforge.net/projects/pywin32/ •http://staf.sourceforge.net/getcurrent.php •Windows Browser UI Automation •http://code.google.com/p/selenium/ •iOS Mobile Automation •Introduction to Automating UI Testing in iOS •iOS Automated Tests with UIAutomation •UI Automation JavaScript Reference •Android Mobile Automation •http://code.google.com/p/robotium/

www.top100summit.com

跨平台自动化测试与工具

案例背景实践

案例ROI分析

• UI Error handled • Multi-threads • Reboot scenario

失敗與放棄

•Automation Framework •Build Portal •L10n Automation •Tools

案例启示

未来想尝试

•Automation Framework •Win Browser UI Automation •iOS Mobile Automation •Android Mobile Automation

參考資料

Manual Automation

30.84 hr/man 1.1 hr/man

Top100summit 跨平台自動化測試與工具徐翔智

Documents

Ryan Hsu 徐培恩

徐悲鸿之女徐静斐离世 - wxrb.comszb.wxrb.com/jnwb/pc/attachment/201911/28/043b8640-5795...工、纪录片《悲壮惊鸿——徐悲鸿的抗战岁月》在宜兴徐悲鸿

TAIWAN ASSOCIATION OF THORACIC & … · cardiac hypertrophy towards heart failure(三總楊翔宇) ... (奇美吳南鈞) ... (北榮徐博奎)

9928134李志翔 1209

Top100summit用友池建强-构建企业级应用开发平台

Top100summit 谷歌-scott-improve your automated web application testing

Top100summit 互联网发布与实验体系建设

開幕典禮程序表 - npust.edu.t · 林忠孝. 林芳銘邱秋霞柯冠銘洪仁杰洪國翔徐秀如徐睿良馬上閔張金龍. 張莉毓張慧珍許衷源連一洋

徐偉初教授 20170324

10110842 洪敏翔

Top100summit 林曙涌-测试卓越驱动电信领域持续集成

Top100summit ibm-董国兴-系统架构优化与技术规划

The Nobel Prize in Physiology or Medicine 1998 Group 6 B9502002 王元欽 B9502013 吳明諴 B9502016 呂宗哲 B9502040 徐鴻麟 B9502042 屠冠翔 B9502049 莊千儀 "for their

Top100summit前端的云时代支付宝前端平台架构王保平

城西小学徐玲玲

第四組胡惇棊楊淳翔魏子翔

再别康桥徐志摩

制作：徐芳吴丹

Top100summit 闵刚金蝶自动化测试发展之路 2

Prof. Dr. YU Xiang ( 余翔 )

Top100summit 跨平台自動化測試與工具 徐翔智

Top100summit 跨平台自動化測試與工具徐翔智