Embed Size (px)
DESCRIPTION
Cory von Wallenstein walks through some best practices for using Selenium to test a large web site like DynDNS.com
Citation preview
SeleniumSeminar
CoryvonWallenstein
February12,2009
Overview
• WhatisSelenium?Whatisitnot?
• Seleniumasasupportandbug‐reporttool
• High‐levelbestpracCces• UnderthehoodofourimplementaCon
UnigressificaConalTesCng
• TermsIuse,exactdefiniConsdebatable
• UnittesCng– “TestonethinginisolaCon”
• RegressiontesCng– “Testonethingthatbroke”
• FuncConaltesCng– “Testthingsworkingtogether”
• VerificaContesCng– “Testallofthethingsworkingtogether”
What’stheproblem?Iwantto…
• Testthecode• Testthatthecodeproducesawebpage• Testthecontentonthewebpage• TestthesemanCcmeaningofthecontentonthewebpage
• Testthedisplayoftheformthatisbelowthecontentonthewebpage
• Testthedisplayoftheformthatisabovethecontentonthewebpage
What’stheproblem?Iwantto…
• Testthesubmissionoftheformwithinvaliddatasuchthatanerrorisdisplayedabovetheform(serversidevalidaCon)andtheformisabovethecontentonthewebpage.
• TestthesubmissionoftheformwithinvaliddatasuchthatanerrorisdisplayedabovetheformwithclientsidevalidaConiftheclienthasJavaScriptandserversidevalidaConifnotandtheformisabovethecontentonthewebpage.
What’stheproblem?Iwantto…
• TestthesubmissionoftheformwithinvaliddatasuchthatanerrorisdisplayedabovetheformwithclientsidevalidaConiftheclienthasJavaScriptandserversidevalidaConifnotandtheformisabovethecontentonthewebpage,exceptforIE6,wheretheformerrordisplaysbelowtheform.
What’stheproblem?Iwantto…
• TestthesubmissionoftheformwithinvaliddatasuchthatanerrorisdisplayedabovetheformwithclientsidevalidaConiftheclienthasJavaScriptandserversidevalidaConifnotandtheformisabovethecontentonthewebpage,exceptforIE6,wheretheformerrordisplaysbelowtheform.forFirefox,Safari,Opera,andIE7,butdisplayamessageforIE6userstellingthemtoswitchbrowsers.
What’stheproblem?Iwantto…
• TestthesubmissionoftheformwithinvaliddatasuchthatanerrorisdisplayedabovetheformwithclientsidevalidaConiftheclienthasJavaScriptandserversidevalidaConifnotandtheformisabovethecontentonthewebpage,exceptforIE6,wheretheformerrordisplaysbelowtheform.forFirefox,Safari,Opera,andIE7,butdisplayamessageforIE6userstellingthemtoswitchbrowsersandmakesure*nixusersgetavirtualhigh‐fivewhentheysubmittheform.
What’stheproblem?Iwantto…
• MakesureitallsCllworkssixmonthsfromnowwhenIforgetwhattheheckthisformwassupposedtodo.
TakealookatSelenium.
h'p://seleniumhq.org
Whatisitnot?
SeleniumArchitecture
• Recordworkflows• Interactwithcontentdisplayedinthebrowserinafairlyrobustway
• Createtestsfromworkflowsinthelanguageofyourchoice
• SametestcodecanbebrowserandOSagnosCc
• Built‐inremotecontrolandgridfuncConality
SeleniumIDE
• FirefoxPlugin• Recordworkflows• Playbackworkflows• Exchangeworkflows• AgoodstarCngpointfortestcode
BestPracCces
High‐levelBestPracCces
• TheIDEgeneratedcodeisgarbage,butyoufullyexpectedthat.:‐)
• Verydependentonscreenscraping• VerydependentonelementnesCng
• SuscepCbletoCmingissues
• Butit’sagoodstarCngpointforlearning!• (BrushuponyourXpath)
GoalsforSeleniumTesCng
• Ifstuffmovesaround,butthefuncConalityisthere,thetestshouldnotbreaknorrequiremodificaCon.
• Ifcontentischanged,butthefuncConalityisthere,thetestshouldnotbreaknorrequiremodificaCon.
• Ifyournetworkislatentoryourcomputerisunderload,thetestshouldnotbreak.
UseSpecificDOMIDs
• Trouble://div[@id='sidenav‐content']/table[1].0.2
• Lesstrouble://div[@id=’bu;on_next’]
• Designfortest• IDisunlikelytochange(ifyouhaveagoodnamingscheme!)
• Moveitemtotestanywhereonpage;acrosspages
• Easytoseparate“isitthere?”from“doesitcontainwhatIexpectitto?”
AvoidStringMatching
• Troubleinscrapingscreenforstring:Anerroroccurred.Sorry.Toughnoogies.
• Lesstroubleincheckingpresenceofelement://label[@for='username'and@class='error']
• UsetheDOMagributesasmuchaspossible.
• Matchlongstringsasligleaspossible.
• Bestkepttokeywords…doeselementcontain:– “required”,“orderfailed”,”error”
IsThePageReadyToTest?
• Built‐in“ispagedoneloading”methodlessthanperfect.
• Lookforsomethingonthepage– Header– Footer– Ajax–Noswirlyicon
GoHigherLevel
• Startrefactoringintohigherlevelmethodssoonerratherthanlater
• Commonones:– isPageReallyReady– clickAndWait
– createNewAccount– login– assertBlahAndCaptureScreenshot– populateFormAndAssertDataAppearsInDatabase
GoHigherLevel
• Organizebaseclassesaroundworkflows– Why?Commontestfixtures!
• Commonfixtures:– Testsrequiringanaccount– Testsrequiringnoaccount– Testsrequiringsigningupforanaccount– Testsrequiringapaidforitem
– Testsrequiringdatatodisplaythatisdifficulttoproduce
MakeChangesEasy
• Organizeunittestclasses/filesaroundURIs• EdgecasetesCng• DatavalidaCon• Loggedinversusloggedout
MakeChangesEasy
• OrganizefuncConaltestclassesaroundworkflows• Createanaccount(mulCplesteps)
• Buysomething
• RMA/cancelsomething
• Provisionsomething
• Thing1interactswithThing2
PlanforFailure
• Keepunit,regression,funcConal,andverificaContestsseparate
• IdealisfuncConaltestfailed,andaunittesttoldyouexactlywhatcausedthefailure
• OrafuncConaltestfailed,andaregressiontesttoldyouyou’veseenthisbugbefore
• Nodebuggingnecessary
PlanforFailure
• Keepunit,regression,funcConal,andverificaContestsseparate
• ShortenCmebetweenfailurecreaConandfailurediscovery.
• Runtestsuitesatdifferentfrequencies/events.
AvoidWorkflowOverlap
• Doasmuchsetupbehindthescenesaspossible– Createanaccountinthedatabaseratherthanform– Addanorderiteminthedatabase…
– Etc.• Aimtotestjustasingle“thing”pertestcase
• Refactorprerequisiteworkflowstobaseclasses
SeleniumisSlow
• Raretohaveatestcaseexecuteinlessthan5‐10seconds.
• Cleanbrowserforeachtestcase.IsolaCon.– HumanCmemorevaluablethanmachineCme.
• Planaccordingly.• Openexasperatedbyexternaldependenciesandsystems…
Mockup!Unplug!
• EverythingbutaverificaContestshouldusemocking
• Mocking– Mockbehaviorofexternalsysteminafastandreproducibleway
– Example:DynDNS.comandcreditcardprocessor– Beabletoreproducespecificerrorresponses!!!– Anyonecanmakesomethingthatworksonsuccess.OurjobistomakeitworkonparCalfailure.
