ASTQB w/ XBOSoft CEO Phil Lew: Agile and Waterfall - What Do Testers Do Differently?

WhatAreTheDifferencesBetweenAgileTes4ngand

“Other”Tes4ng

Philip Lew

©XBOSo=,Inc.2016AllRightsReserved

1

http://www.astqb.org/agile-tester/

@xboso=@philiplew

Applyingthe7Habits–WhattoDo?

•  Last4mewewentoverAgile7Effec4veHabits

•  Today,we’llcoverwhatyoudoin‘AgileTes4ng’,whichisamaVeroffocusand4ming

•  Asopposedtowaterfallwhereso=warewas‘readyfortes4ng’,weare‘readyfortes+ng’(doingtes4ngac4vi4es)allthe4me,butwhatexactlyarewedoingwhen?

©XBOSo=,Inc.2016AllRightsReserved 2@xboso=@philiplew

A=erWorkingWithManyofOurClientsinBothWaterfallandAgile

•  I’vefoundthatfocusand4mingarethekeydifferences– Eitherintheirsuccess– Orintheirfailure

•  Notrocketscience•  Mostlynochangesintechnology,butchangesinfocusand4ming

©XBOSo=,Inc.2016AllRightsReserved 3

“Having the Right People In the Right Place Doing the Right Thing At the Right Time.”

@xboso=@philiplew

WhyAreYouHere?


•  Everwonderhowdotes4ngprac4cesdifferwhetheryouronanagileversusawaterfallteam?

•  Doyous4llhavestep-by-steptestscriptsorjusthavegeneraltestscenariosoruserstories.

•  Doyouhaveatestplan?•  You’reintherightplace!

@xboso=@philiplew

SedngExpecta4ons…

•  Interac4ve-askques4onsviaGoto-WebinarPanel

•  Iwon’treadtheslides…•  Slidesforyouasatake-away

– Sessionisrecordedandavailablea=erwards

5©XBOSo=,Inc.2016AllRightsReserved@xboso=@philiplew



0

20

40

60

80

100

1202004-01-04-2004-01-10

2004-04-04-2004-04-10

2004-07-04-2004-07-10

2004-10-03-2004-10-09

2005-01-02-2005-01-08

2005-04-03-2005-04-09

2005-07-03-2005-07-09

2005-10-02-2005-10-08

2006-01-01-2006-01-07

2006-04-02-2006-04-08

2006-07-02-2006-07-08

2006-10-01-2006-10-07

2006-12-31-2007-01-06

2007-04-01-2007-04-07

2007-07-01-2007-07-07

2007-09-30-2007-10-06

2007-12-30-2008-01-05

2008-03-30-2008-04-05

2008-06-29-2008-07-05

2008-09-28-2008-10-04

2008-12-28-2009-01-03

2009-03-29-2009-04-04

2009-06-28-2009-07-04

2009-09-27-2009-10-03

2009-12-27-2010-01-02

2010-03-28-2010-04-03

2010-06-27-2010-07-03

2010-09-26-2010-10-02

2010-12-26-2011-01-01

2011-03-27-2011-04-02

2011-06-26-2011-07-02

2011-09-25-2011-10-01

2011-12-25-2011-12-31

2012-03-25-2012-03-31

2012-06-24-2012-06-30

2012-09-23-2012-09-29

2012-12-23-2012-12-29

2013-03-24-2013-03-30

2013-06-23-2013-06-29

2013-09-22-2013-09-28

2013-12-22-2013-12-28

2014-03-23-2014-03-29

2014-06-22-2014-06-28

2014-09-21-2014-09-27

2014-12-21-2014-12-27

2015-03-22-2015-03-28

2015-06-21-2015-06-27

2015-09-20-2015-09-26

2015-12-20-2015-12-26

2016-03-20-2016-03-26

2016-06-19-2016-06-25

"agiletes*ng"searchterm

2004 2016

@xboso=@philiplew


0

20

40

60

80

100

1202004-01-04-2004-01-10

2004-03-28-2004-04-03

2004-06-20-2004-06-26

2004-09-12-2004-09-18

2004-12-05-2004-12-11

2005-02-27-2005-03-05

2005-05-22-2005-05-28

2005-08-14-2005-08-20

2005-11-06-2005-11-12

2006-01-29-2006-02-04

2006-04-23-2006-04-29

2006-07-16-2006-07-22

2006-10-08-2006-10-14

2006-12-31-2007-01-06

2007-03-25-2007-03-31

2007-06-17-2007-06-23

2007-09-09-2007-09-15

2007-12-02-2007-12-08

2008-02-24-2008-03-01

2008-05-18-2008-05-24

2008-08-10-2008-08-16

2008-11-02-2008-11-08

2009-01-25-2009-01-31

2009-04-19-2009-04-25

2009-07-12-2009-07-18

2009-10-04-2009-10-10

2009-12-27-2010-01-02

2010-03-21-2010-03-27

2010-06-13-2010-06-19

2010-09-05-2010-09-11

2010-11-28-2010-12-04

2011-02-20-2011-02-26

2011-05-15-2011-05-21

2011-08-07-2011-08-13

2011-10-30-2011-11-05

2012-01-22-2012-01-28

2012-04-15-2012-04-21

2012-07-08-2012-07-14

2012-09-30-2012-10-06

2012-12-23-2012-12-29

2013-03-17-2013-03-23

2013-06-09-2013-06-15

2013-09-01-2013-09-07

2013-11-24-2013-11-30

2014-02-16-2014-02-22

2014-05-11-2014-05-17

2014-08-03-2014-08-09

2014-10-26-2014-11-01

2015-01-18-2015-01-24

2015-04-12-2015-04-18

2015-07-05-2015-07-11

2015-09-27-2015-10-03

2015-12-20-2015-12-26

2016-03-13-2016-03-19

2016-06-05-2016-06-11

“Software Testing” search term

@xboso=@philiplew


0

20

40

60

80

100

120

2004-01-04-2004-01-10

2004-04-11-2004-04-17

2004-07-18-2004-07-24

2004-10-24-2004-10-30

2005-01-30-2005-02-05

2005-05-08-2005-05-14

2005-08-14-2005-08-20

2005-11-20-2005-11-26

2006-02-26-2006-03-04

2006-06-04-2006-06-10

2006-09-10-2006-09-16

2006-12-17-2006-12-23

2007-03-25-2007-03-31

2007-07-01-2007-07-07

2007-10-07-2007-10-13

2008-01-13-2008-01-19

2008-04-20-2008-04-26

2008-07-27-2008-08-02

2008-11-02-2008-11-08

2009-02-08-2009-02-14

2009-05-17-2009-05-23

2009-08-23-2009-08-29

2009-11-29-2009-12-05

2010-03-07-2010-03-13

2010-06-13-2010-06-19

2010-09-19-2010-09-25

2010-12-26-2011-01-01

2011-04-03-2011-04-09

2011-07-10-2011-07-16

2011-10-16-2011-10-22

2012-01-22-2012-01-28

2012-04-29-2012-05-05

2012-08-05-2012-08-11

2012-11-11-2012-11-17

2013-02-17-2013-02-23

2013-05-26-2013-06-01

2013-09-01-2013-09-07

2013-12-08-2013-12-14

2014-03-16-2014-03-22

2014-06-22-2014-06-28

2014-09-28-2014-10-04

2015-01-04-2015-01-10

2015-04-12-2015-04-18

2015-07-19-2015-07-25

2015-10-25-2015-10-31

2016-01-31-2016-02-06

2016-05-08-2016-05-14

Software testing

Agile testing

@xboso=@philiplew


India

UnitedKingdom

UnitedStates

Canada

Relative Searches By Geography for Either Term

@xboso=@philiplew

So=wareisNew•  So=waredevelopmentisnew–notlongenoughtoknowhowtodoit.

•  Wearenotbuildingdoorsormakingtoilets.•  Whenwedonotknowwhatwearedoing,thebeststrategyistodo‘liVlebyliVle’approachandgetfeedbackassoonaspossible.

•  Weusethesametechniqueinoureverydaylives,andAgileisanapplica4onofitinso=ware.


WhatGotUsHere

•  Smallerteams•  Fasteritera4ons•  Listeningtotheuser

– Con4nuousbeta– Datacollec4on&analy4cs


•  Communica4on•  Workingsmarter•  Analysis,adap4onandimprovement

1.  Changes in technology (mobile, cloud) 2.  Changes in business models 3.  Demand changes in the way we work

@xboso=@philiplew

AgileQuality

•  Nopolice•  Sharedteamresponsibility•  Tes4ngisdonebyeveryone?

– Tes4ngisdonebyqualifiedteammembers– Developerscouldtakeonsometes4ngtasks

•  Developmentisdonebyeveryone?– Developmentisdonebyqualifiedteammembers– Testerscouldtakeonsomedevelopmenttasks



OneoftheMainAgileFlavors:Scrum

1 2

3

@xboso=@philiplew

AGILEPLANNING


1

@xboso=@philiplew

TesterPairing•  WithTestSpecialists

– Security– Performance– Usability– Automa4on

•  WithDomainExperts– Businessperspec4ve– Auto-correct/complete/verifystories– Benefitsinbothdirec4ons


1

@xboso=@philiplew

Tes4ngisCri4calPartofthePlan

•  Tes4ngisnotana=erthought– Clarifystorytestswithcustomerandenduser

•  Thinkfromtheirpointofview–  Iden4fytes4ngac4vi4estosupportdevelopment

• Whiteboxtes4ng•  Integra4ontes4ng

– Backloggrooming•  Improveandclarifystories

17©XBOSo=,Inc.2016AllRightsReserved

1

@xboso=@philiplew

WorkloadManagement•  “Sustainable”-thisAgileprinciplehelpstopreventtestteamburnout

•  Sincesometes4ngtasksareoutsidethesprints(suchastestautoma4oncodedevelopment),sometestersshouldworkoutsidethesprintsontheselong-termprojects

•  Properes4ma4onduringtheini4aldaysofeachitera4onneededtoavoidpressureontestersduringtheendoftheitera4on

•  Properes4ma4onalsoincludeses4ma4onoftes4ng

1


SizetheJobProperlyandPrio4ze

•  SomeAgileprojectsritualizesprint4medeadlines,whilenotfollowingotherrulesrelatedtosustainableworkload

•  Whentheteamcon4nuallyover-commits,thetestteamgetssqueezedonthelastweekendofeverysprint

•  Resolvethischallengethroughrisk-basedtes4ng•  Reducethescopeoftes4ngbasedonrisk

– Movelow-riskteststonextsprint

1

©XBOSo=,Inc.2016AllRightsReserved@xboso=@philiplew 19

PlanningforTechnicalDebt

•  Agileteamsshouldhavea“fixbugsfirst” approachtopreventalargebacklogofbugs– Reduce‘4metofix’beyondthesprintlength

•  Periodicstabiliza4onor‘fixdefects’sprintsshouldbeplanned

•  Withpropermetrics,tes4ngcanhelpmeasureandmanagetechnicaldebt

1


UnderstandingTestBasis

•  Testbasisisconstantlychangingeachitera4onandsome4meswithin– Stories– Acceptancetests– Environment– Productusage


1

@xboso=@philiplew

SprintBacklog–SprintPlanning•  Eachbacklogentrybrokendownintotasks

•  Tasksincludedevelopment,tes4nganddocumenta4on– allplannedtogether


1

@xboso=@philiplew

UserStories

•  Ideallyreviewedbybusinessandtechnicalstakeholdersearlyinthesprint

•  Testersshouldbeinvolvedinthesereviews•  Highqualityuserstorieswithtestoracles,leadtoearlydefectremovalanddefectpreven4on

•  Testersshouldinspectforcompletenessandclarity

1


StoryGrooming•  Clarifydescrip4onvialanguageandexamples

•  Predictandavoidalternateinterpreta4ons•  Elicitacceptancetestsfromproductownerorcustomer/enduser

•  Predictalternateorerroneousflows•  Relatenewstoriestoalreadyunderstoodconceptsandfunc4ons


1

@xboso=@philiplew

Infrastructure

•  Developingandmaintainingtakes4me,some4mesoverlookedandnotsmall– Tools– Frameworks– Data– Environments


1

@xboso=@philiplew

TestData

•  Automa4ngtestdatachoresiscri4cal– Generateandcapture– Maintainandorganize– Validate– Changeproceduresandaccess


1

@xboso=@philiplew

Func4onalTes4ng

•  Canshi=focusdependingonitera4on– Stories

• Crossfunc4onal• Storyinterac4ons• Experiencebasedandexploratory


1

@xboso=@philiplew

NonFunc4onalTes4ng•  O=enforgoVenun4ltheend,planitinasastoryasintegraltothetestplan– Usability– Security– Performance– Sta4c-Whitebox– Inspec4ons


1

@xboso=@philiplew

ReviewTests

•  Inaddi4ontorequirements,businessandtechnicalstakeholderscanreviewtestcondi4ons(acceptancecriteria)– Reducesthenumberoffalseposi4vesandfalsenega4vesassociatedwithtes4ng

–  Improvestestresults–  Increasesconfidenceinquality– Gaininsight/reviewwhat’simportantandwhat’snot

1


PlanningforChange

•  SomeAgiletes4ngrequiresdifferentorchangeinfocusintechnicalskills

•  Testersareequippedwhentheyknowhowtoprogramoratleastunderstandthelanguagebeingused

•  Duetoshortitera4onsandcompressedregression,Agiletes4ngrequirestestautoma4on

•  Testersarebestequippediftheycaneffec4velyimplementtestautoma4on

1


DURINGTHESPRINT


2

@xboso=@philiplew

TesterPairing–HeatoftheSprint•  Withtestdomainexperts

– Security– Performance– Usability– Automa4on

•  Withothertesters– “ExtremeTes4ng”--XT(XP)

•  Withprogrammers– Whitebox– Programminglogicversustes4nglogic


2

@xboso=@philiplew

DuringtheSprint–WhatDoYouDo?

•  TestDrivenDevelopment– AcceptanceTestDrivenDevelopment– BehaviorDrivenDevelopment

•  AutomatedTes4ng•  ExploratoryTes4ng•  Con4nuousIntegra4on


2

@xboso=@philiplew

TestDrivenDevelopment•  Definethetest

– Thisiswheretesterscanhelpindesignandcri4queofdatadriventestsandunittests

•  Implementthetest•  Developcodetopassthetest


2

@xboso=@philiplew

AutomatedUnitTes4ng

•  Automatedunittestslimitregressionrisk•  Canberunaso=enaseveryfewhours,oronce/day

•  Automatedunittestscanbecombinedwithsta4ccodeanalysisandcodecoveragetomaximizeeffec4venessandcompletenes– Don’tusuallyfinddefects

2


AutomatedFunc4onalTes4ng

•  Automatedfunc4onaltes4ngcanbeintegratedwithcon4nuousintegra4on,automatedunittes4ng,codecoverage,andsta4canalysisframeworks

•  Furtherminimizesregressionriskandincreasestes4ng‘velocity’.

•  SeleniumandFinesse,amongstmanyotheropensourcetoolscandothis.

2


Sta4cAnalysis

•  Sta4ccodeanalysiscaniden4fymanyproblemspriortotes4ng– Complexity– Deadcode– Reusability

•  Somesta4canalysiscaniden4fysecurityandmaintainabilityproblemsthattes4ngcannotiden4fyeasily

2


ExploratoryTes4ng

•  Notfreeformdowhateveryouwant•  Structuredexperiment•  Time-boxed•  Purposefulandfocused

– Charter


Design

Execu4onLearning

2

@xboso=@philiplew

ExploratoryTes4ngCharters•  Mission,whatyousetouttodo•  Whatyouwanttoaccomplish,learn•  Ensuresharedunderstandingofscope•  Examples:

– Failures,errors,andrecovery– Endtoendsystem– Emergingbehaviors(analy4cs)– Non-func4onal

•  Security–aVacks•  Performance


2

@xboso=@philiplew

ExploratoryTes4ngTime-Boxes•  Varieddependingonscopeandmission

– Typical–2hourstohalfday– Short–1hourorless– Long–1dayormore


2

@xboso=@philiplew

ExploratoryTes4ng–MaximizingOutput

•  Duringthesession,takenoteson– Decisionsyoumade– Whatyoudid– Whatyoudiscovered

•  A=erthesession– Stopatthedesignated4me-box– Reviewfindings

• Whatworkedandwhatdidn’t• Whatdidwelearnandshouldwedoanothersame/diff


2

@xboso=@philiplew

Con4nuousIntegra4on

•  Detectintegra4onproblemsimmediatelya=ercodecheck-in

•  Brokenbuildscanbereducedoreliminated•  Combinecon4nuousintegra4onwithautomatedunittes4ngandautomatedfunc4onaltes4ngasthefounda4onofyour‘smoke’regression.

2


THERETROSPECTIVE


3

@xboso=@philiplew

Tes4ngImprovements

• Wherewas4mewasted?• Whatwasnottestable?• Whattasksfellofftheplateandwhy?• Whattasksweretotallyoverlooked?•  Isautoma4ondoingwhatwewant?

3


AnalyzeDefects

• Whatnewdefectswereintroduced– Whatarethesourcesofdefects– Wherewerethedefectsintroduced

•  Commoncauses•  Howtoavoid?

3


Collabora4on

• Withclient/user-Didwedeliverwhatwasexpected?

• Withdevelopers• Withbusinessanalysts• Withotherteams• Whatinforma4onwasmissed/notcommunicatedandwhy?

3


THEEND


4

@xboso=@philiplew

Successfully‘GoingAgile’•  Agileisnotatechniqueortechnology•  Youcanbeagilewithoutabook,withoutacoach,justlikeyoucanloseweightifyouwantto,withoutWeightWatchers

•  Withadiet,youwantalifestylechange– Notjusta3weekeffort,butforlongtermhealth– Some4mesyouhavesetbacks

•  WithAgile,youwantawork-stylechange


Summary•  Agilepresentsasignificantshi=in4mingofwhenand

whatyoudoasatester•  Testersshouldembracethesechanges(opportuni4es)

–  Learnnewskills–  Takeonadifferentmindsetincollabora4on–  “Everyoneisresponsibleforquality”

1.  Agiledoesn’tmean‘notestplan’asyou’veseen,butit’sadifferenttestplan.

2.  Beflexibleintheheatofthesprint,followtheplanand“Gowhereyouarewantedandneeded.”

3.  Agilegivesyoutheopportunityforates4ngretrospec4vethatyoudidnothavebefore.Useittoimprove.


Thanks! For those knowledge seekers, download:

Agile Metrics Whitepaper Agile Test Plan

http://xbosoft.com/knowledge-center/software-testing-white-paper/

Philip Lew @philiplew [email protected]

www.xbosoft.com ©XBOSo=,Inc.2016AllRightsReserved 50