View
1.767
Download
0
Category
Preview:
DESCRIPTION
讲师 :张克冰 2011年加入SAP 中国研究院,担任质量管理专员, 主要负责软件开发过程中的质量标准和测试策略的制定,公司级探索性测试培训师,教练。 十年软件从业经验,一致专注于软件测试和软件质量管理领域。2009年开始接触敏捷开发,热衷于敏捷, 热衷于提高软件测试在scrum中的工作有效性。 话题介绍: 敏捷开发模式下, 虽然自动化测试的作用日益强大,但是手工测试始终是软件测试不可缺少的一部分。 手工测试该如何去适应敏捷开发的步伐?是墨守成规的脚本化测试,还是漫无目的的随机测试?探索性测试给敏捷开发团队提供了一种轻快,有效的测试方法。 在实战中,我们把探索性测试跟scrum结合起来,让每个scrum的成员都有机会参与手工测试。 在测试的过程中,大家感受到探索性测试的乐趣, 更重要的是,我们在摸索探索性测试的过程中,通过不断的尝试,探索出适合我们团队的探索性测试的模式。 作为公司的探索性测试培训师和教练,我跟大家经历了从认识它,学习它, 到不断改进它, 直到这个方法可以适应复杂大型的ERP软件开发的过程。对于我来说, 探索性测试不但是一种新的测试方法,更加可以促进scrum对已有的测试和质量管理流程进行改进。 非常希望有机会跟大家分享我们在探索性测试的道路上收获的经验和反思。 Agenda: 1. 敏捷模式下的手工测试。2. 探索性测试简介。3.探索性测试之旅。4. 实战中的探索性测试。 5.探索性测试的反思和未来之路
Citation preview
© 2011 SAP AG. All rights reserved. 1 Public
Our Exploratory Testing Journey
Zhang Kebing
SAP Labs China
June, 2012 Public
© 2011 SAP AG. All rights reserved. 2 Public
Disclaimer:
This presentation outlines our general product direction and should not be relied on
in making a purchase decision. This presentation is not subject to your license
agreement or any other agreement with SAP. SAP has no obligation to pursue any
course of business outlined in this presentation or to develop or release any
functionality mentioned in this presentation. This presentation and SAP's strategy
and possible future developments are subject to change and may be changed by
SAP at any time for any reason without notice. This document is provided without a
warranty of any kind, either express or implied, including but not limited to, the
implied warranties of merchantability, fitness for a particular purpose, or non-
infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally or grossly
negligent.
© 2011 SAP AG. All rights reserved. 3 Public
The role of manual testing in Scrum
We use manual test when
1. Some features are too costly to automate.
2. Some features are not automatable.
3. Find the complicated bugs.
4. Explore new ways of testing a feature.
Shipping new features after each sprint
© 2011 SAP AG. All rights reserved. 4 Public
Ad hoc vs. Scripted Test Case Design
Scripted
Testing
Ad hoc
Testing Design effort Low High
• No test strategy
• No focus
• No traceability
• High
planning/maintenance
effort
• No variations
• Kills creativity
© 2011 SAP AG. All rights reserved. 5 Public
Exploratory Software Testing with Tours
A lightweight approach to test case design and execution
[1]
© 2011 SAP AG. All rights reserved. 6 Public
Exploratory Testing
Interaction
Creativity
[2]
[3]
© 2011 SAP AG. All rights reserved. 7 Public
Analogy: A software tester is like a tourist…
• You can not see everything
• There are lots of different ways of exploration
• You want to make sure to see the most
important things
[4]
© 2011 SAP AG. All rights reserved. 8 Public
Feature: Campaign Management in Real Estate
© 2011 SAP AG. All rights reserved. 9 Public
Garbage Collector Tour Garbage men go street by street, house by house. They stay only
a few moments but they crisscross the neighborhood in a
methodical manner.
[5]
© 2011 SAP AG. All rights reserved. 10 Public
Garbage Collector Tour 1
Example for Campaign in Real Estate
1. Open the UI of add a new Campaign.
2. Execute all main sub-frames.
3. Make sure you go to each sub-frame and
create a new item.
4. Don’t test every detail.
5. Test document
© 2011 SAP AG. All rights reserved. 11 Public
Garbage Collector Tour 2
Example for Campaign in Real Estate
1. Open the UI of add a new Campaign.
2. Only test one campaign type, but check all the buttons on the menu.
3. Don’t touch other sub features.
4. Test Document
Anti-Social Tour
You are not like everybody else.
[6]
© 2011 SAP AG. All rights reserved. 13 Public
Anti-Social Tour
[7]
Example
If valid input is a number enter a
character
if valid input is a picture file use a word
file
if you are on the last page of a wizard
hit “Back” and change something, …
© 2011 SAP AG. All rights reserved. 14 Public
Supermodel Tour
It’s all about the look…
[8]
© 2011 SAP AG. All rights reserved. 15 Public
Supermodel Tour
Design and execution of the tour in general:
1. Pick a feature
2. Make sure it follows the UI guidelines
3. All icons are correct
4. All texts contain no spelling mistakes
5. …
© 2011 SAP AG. All rights reserved. 16 Public
Standard Tours
Business District • Guidebook Tour (F1
Tour) • Money Tour • Landmark Tour • Intellectual Tour • FedEx Tour • After-Hours-Tour • Garbage Collector's
Tour1
Tourist District • Collector's Tour • Lonely Businessman
Tour • Supermodel Tour • TOGOF Tour
(TestOneGetOneFree) • Scottish Pub Tour
Seedy District • Saboteur • Antisocial Tour
• Opposite • Crime Spree • Wrong Turn
• Obsessive-Compulsive
Historical District • Bad-Neighborhood • Museum Tour • Prior Version Tour
Entertainment District • Supporting Actor Tour • Back Alley Tour • All-Nighter Tour
Hotel District • Rained-Out Tour • Couch Potato Tour
James Whittaker
Exploratory Software Testing: Tips, Tricks, Tours, and Techniques to Guide Test Design
Addison Wesley Pub Co Inc, 2009
© 2011 SAP AG. All rights reserved. 17 Public
Exploratory Testing with Tours!
Failure!
Failure!
Failure!
Failure!
[14]
© 2011 SAP AG. All rights reserved. 18 Public
Exploratory Software Testing Ingredients
Test Tour Time Box Protocol
© 2011 SAP AG. All rights reserved. 19 Public
Time Box
Helps you to
stay focused
It should
between 30 – 60
minutes per tour
You must take
note, record
your test within
time box
Prepare the test
before you start
to test
© 2011 SAP AG. All rights reserved. 20 Public
Protocol – Test Document
Written on the fly by the
tester
Documents what has
been tested
Documents which bugs
have been found
Documents which test
opportunities are still
available
Documents open
questions (bug or
feature?)
Examples
© 2011 SAP AG. All rights reserved. 21 Public
Test Tour
• Defines test approach
(How)
• Leaves the tester enough
freedom to explore
• Ensures tester stays
focused, not testing
everything
[9]
© 2011 SAP AG. All rights reserved. 22 Public
Integration into the Scrum process
- Test Strategy
Team A:
Scripted Test
ET
Team B:
Unit Test
Auto-System Test
ET
© 2011 SAP AG. All rights reserved. 23 Public
Integration into the Scrum process
- Which tour?
Reports
• Intellectual Tours : ask the hardest
questions to the reports.
• Monthly close?
• Enter transactions with huge
amount?
• View reports with multiple
currencies?
• …
© 2011 SAP AG. All rights reserved. 24 Public
Integration into the Scrum process
- Which tour?
Invest more than one tours for a feature.
• Garbage collector tour - Test
Coverage
• Anti-social tour - invalid inputs or
operations.
• Landmark tour - Features in different
sequences.
© 2011 SAP AG. All rights reserved. 25 Public
Integration into the Scrum process
1. Define Tasks – Planning Meeting
2. Test Prepare - Tester
3. Who perform – any team member
4. Rules – Cross test, start test when all development tasks is done.
Story Tasks To-Do In Process Completed Done
Story
Bug
Story
© 2011 SAP AG. All rights reserved. 26 Public
What ET bring to our scrum team?
Long document Just Enough VS
[10]
© 2011 SAP AG. All rights reserved. 27 Public
What ET bring to our scrum team?
• Enlarge testing resource
• Involve everyone
[11]
© 2011 SAP AG. All rights reserved. 28 Public
What ET bring to our scrum team?
• Better test coverage
• Each of tester has their own way
to explore the software
[12]
© 2011 SAP AG. All rights reserved. 29 Public
Current Challenges
• Try more tours with scrum team
• Try more different approaches
• Follow up on bug reports and find out whether your test strategy
works or not.
© 2011 SAP AG. All rights reserved. 30 Public
Try!
Contact information:
Zhang Kebing
Quality Engineer/ Scrum Master
KebingZhang@hotmail.com
© 2011 SAP AG. All rights reserved. 31 Public
References
[1] http://www.flickr.com/photos/statelibraryofnsw/2876551803/
[2] http://www.quardev.com/articles/exploratory_testing
[3] http://www.flickr.com/photos/bobcatrock/2862617356/
[4] http://www.flickr.com/photos/gamene/4341746457/
[5] http://www.flickr.com/photos/nasacommons/4858566574/
[6] http://www.flickr.com/photos/foxypar4/1852428153/
[7] http://www.flickr.com/photos/judepics/1950452493/
[8] http://www.flickr.com/photos/istolethetv/4045140609/
[9] http://www.nipic.com/show/1/66/71cbc36ac3b50aa9.html
[10] http://www.superstock.com/stock-photos-images/1775R-6937
[11] http://www.allmoviephoto.com/photo/2003_poster_smith_international.html
[12] http://www.fs3000.com.cn/bbs/read.php?tid=610
[14] http://blog.sysomos.com/2010/05/19/twittera-as-a-resource-tool/question-
mark/
© 2011 SAP AG. All rights reserved. 32 Public
No part of this publication may be reproduced or transmitted in any form or for any
purpose without the express permission of SAP AG. The information contained
herein may be changed without prior notice.
Some software products marketed by SAP AG and its distributors contain
proprietary software components of other software vendors.
Microsoft, Windows, Excel, Outlook, and PowerPoint are registered trademarks of
Microsoft Corporation.
IBM, DB2, DB2 Universal Database, System i, System i5, System p, System p5,
System x, System z, System z10, System z9, z10, z9, iSeries, pSeries, xSeries,
zSeries, eServer, z/VM, z/OS, i5/OS, S/390, OS/390, OS/400, AS/400, S/390
Parallel Enterprise Server, PowerVM, Power Architecture, POWER6+, POWER6,
POWER5+, POWER5, POWER, OpenPower, PowerPC, BatchPipes,
BladeCenter, System Storage, GPFS, HACMP, RETAIN, DB2 Connect, RACF,
Redbooks, OS/2, Parallel Sysplex, MVS/ESA, AIX, Intelligent Miner, WebSphere,
Netfinity, Tivoli and Informix are trademarks or registered trademarks of IBM
Corporation.
Linux is the registered trademark of Linus Torvalds in the U.S. and other
countries.
Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or
registered trademarks of Adobe Systems Incorporated in the United States and/or
other countries.
Oracle and Java are registered trademarks of Oracle and/or its affiliates.
UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.
Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and
MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.
HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®,
World Wide Web Consortium, Massachusetts Institute of Technology.
© 2011 SAP AG. All rights reserved.
SAP, R/3, SAP NetWeaver, Duet, PartnerEdge, ByDesign, SAP BusinessObjects
Explorer, StreamWork, and other SAP products and services mentioned herein as
well as their respective logos are trademarks or registered trademarks of SAP AG
in Germany and other countries.
Business Objects and the Business Objects logo, BusinessObjects, Crystal
Reports, Crystal Decisions, Web Intelligence, Xcelsius, and other Business
Objects products and services mentioned herein as well as their respective logos
are trademarks or registered trademarks of Business Objects Software Ltd.
Business Objects is an
SAP company.
Sybase and Adaptive Server, iAnywhere, Sybase 365, SQL Anywhere, and other
Sybase products and services mentioned herein as well as their respective logos
are trademarks or registered trademarks of Sybase, Inc. Sybase is an SAP
company.
All other product and service names mentioned are the trademarks of their
respective companies. Data contained in this document serves informational
purposes only. National product specifications may vary.
The information in this document is proprietary to SAP. No part of this document
may be reproduced, copied, or transmitted in any form or for any purpose without
the express prior written permission of SAP AG.
Recommended