Sikuli Google

8/2/2019 Sikuli Google

1/54

SIKULIUSING SCREENSHOTS FOR GUIAUTOMATION AND TESTING

Tsung-Hsiang (Sean) Chang, Rob Miller

MIT CSAIL

Collaborator : Tom Yeh

1

12010611


2/54

How totell computers what to do?

2

22010611


3/54

SHELL SCRIPT

case $rtval in

1) rm -f /tmp/dirip.$$ ; return ;;

255) rm -f /tmp/dirip.$$ ; return ;;

esac

mfile=`cat /tmp/dirip.$$`

if [ -z $mfile ]

then

mfile=`pwd`/*

else

grep "*" /tmp/dirip.$$

if [ $? -eq 1 ]then! mfile=$mfile/*fi

fi

for i in $mfile

do

if [ -f $i ]then! echo "$i Delete?" >> /tmp/finallist.$$fi!

done

3

32010611


4/54

4

42010611


5/54

GUI AUTOMATION

5

52010611


6/54

1

2

3

6

62010611


7/54

7

72010611


8/54

click click

7

72010611


9/54

SIKULI SCRIPT

8

82010611


10/54

SIKULI TEST

9

92010611


11/54

OUTLINE

Problem - How to refer to a GUI widget?

Sikuli Script - examples and demos

Sikuli Test - demo

Test By Demonstration

Support of testing practices

Conclusions

10

102010611


12/54

HOW TO REFER TO A GUI WIDGET?

11

112010611


13/54


Abbot

by pre-programmed references

11

112010611


14/54


Abbot

by pre-programmed references

Autoit

by absolute locations

11

112010611


15/54

Use Screenshots!

12

122010611


16/54

TEMPLATE MATCHING

click()

13

132010611


17/54

TEMPLATE MATCHING

click()

13

132010611


18/54

1

2

3

EXAMPLE 1:

EMPTY TRASH CAN

14

142010611


19/54

EX2: FACEBOOK STALKER

15

152010611


20/54

EX2: FACEBOOK STALKER

15

152010611


21/54

16

162010611


22/54

RESTRICT SEARCH REGION

17

172010611


23/54

EX3: MUTE SPEAKERS

18

182010611


24/54

EX3: MUTE SPEAKERS

18

182010611


25/54

SPATIAL OPERATORS

19

192010611


26/54

SPATIAL OPERATORS

19

192010611


27/54

SPATIAL OPERATORS (2)

20

202010611


28/54

EX4: SET IP ADDRESS

21

212010611


29/54

VISUAL-EVENT-DRIVEN

PROGRAMMING

22

222010611


30/54

VISUAL-EVENT-DRIVEN

PROGRAMMING

22

222010611


31/54

SIKULI TEST

23

232010611


32/54

VISUAL ASSERTION

Verify appearance

24

242010611


33/54

VISUAL ASSERTION

Verify replacement

25

252010611


34/54

TEST BY DEMONSTRATION

click

detect changes

26

262010611


35/54

Demo

27

272010611


36/54

GENERATE ACTION

COMMANDS

mousedown

mouseup

Adown

Aup

bdown

bup

28

282010611


37/54

GENERATE ACTION

COMMANDS

low-level events

mousedown

mouseup

Adown

Aup

bdown

bup

28

282010611


38/54

GENERATE ACTION

COMMANDS

low-level events

high-levelcommands

click type Ab

mousedown

mouseup

Adown

Aup

bdown

bup

28

282010611


39/54

GENERATE ACTION

COMMANDS

low-level events

high-levelcommands

click type Ab

mousedown

mouseup

Adown

Aup

bdown

bup

combine withscreenshots

click

28

282010611


40/54

GENERATE ASSERTIONS

click

29

292010611


41/54

GENERATE ASSERTIONS

click

1. find differences / manually hint

29

292010611


42/54

GENERATE ASSERTIONS

2. group together

click


29

292010611


43/54

GENERATE ASSERTIONS

2. group together3. combine with assertExist

click


29

292010611


44/54

GENERATE ASSERTIONS

2. group together3. combine with assertExist

click


29

292010611


45/54

jUNIT INTEGRATION

30

302010611


46/54

TEST-DRIVEN DEVELOPMENT

31

312010611


47/54

READABILITY OF CODE

32

322010611


48/54

PLATFORM INDEPENDENCE

Works on ANY GUI that can be displayed onWindows/Linux/Mac

Virtual machines

Remote desktop

Mobile simulators: Android, iPhone

Web: Flash, HTML+Javascript

33

332010611


49/54

34

342010611


50/54

34

342010611


51/54

LIMITATIONSTheme Variations

Visibility Constraints

35

352010611


52/54

DISCUSSION

Sikuli programs are written against the user interfaceinstead of an API

UI: visible, familiar, always exists API: faster, probably more stable

36

362010611


53/54


54/54

Thanks!

http://sikuli.csail.mit.edu

New Sikuli 0.10 released.

Follow us on twitter @sikuli

Th k t f di NSF d Q t C t ( t f TP t j t )

or, just google sikuli
http://sikuli.org/http://sikuli.org/

Documents

Sikuli Google