Testing your stuff (the hard way)

@fribulusxax / wojtek@endpoint.com

The easy way

Webservice/website case

- if it’s a website (frontend + backend)

- it has to run on different browsers/clients- usually runs on a single server role- usually the infrastructure is complex, managed and

you dont have an access so you dont care

http://example.com

Physical website/webservice- if it’s not only a website/webservice (e.g. digital signage)

- it’s run on your infrastructure- you are the designer- you are the manager and maintainer

- it’s distributed- you care about more than about HTTP - e.g. WS, SPDY,

HTTP, UDP, TCP, ROS, ARP - those protocols differ a lot and are both synchronous and asynchronous

- testing gets more complex

S

The hard way

ws://example.com

DB

- leap motion- mouse- space navigator- touchscreen

ROS

server A (presentation) server B (content)

server C (control)

server D (I/O)

STATE

DBinput devices

Rollback time

Automation

Dependencies

Continuous integration/builds

Test scenario1. User submits a pull request2. CI server checks out the PR code3. CI server runs first unit tests (less than a minute)4. CI server creates master VM on the fly and converges it with chef-zero (approx 20mins)5. CI server builds tells master VM to build other 3 VMs (30 minutes)

a. build ISOb. create slave VMsc. bootd. provision

6. Run BATS/serverspec/minitest a. ensure that all services are up and running (e.g. ROS stack, Chrome extensions etc)

7. On the top of everything run Selenium + ROS testsa. two browsers are tested synchronusly via Selenium Server (Json Wire protocol)b. User input is replayed via ROS bag (simulation of USB frames)c. Assertions are mad in browsers, on the system level and on the ROS bus

More info: http://blog.endpoint.com/2015/02/testing-your-chef-repo-pull-requests.html

Thank you