Getting Started with callstats.io

Varun Singh CEO, callstats.io

07. 07. 2015

http://geek-and-poke.com/

B r i n g i n g h i g h q u a l i t y o f experience to every interaction.

It is not all about throughput*• … it is about latency or delay!

*)capacity bandwidth speed

audio only call

Gameplan

• Do Nothing

• Wait for emails from customers

Gameplan• Do nothing

—> It is someone else’s problem

• use the following reasons in support: 1.Insufficient  Capacity    (upgrade  plan)  

2.Misbehaving  Browser/Device  (update  or  change)  

3.Or…  NATs/Firewall              (change?)  

• Bonus tip: you can use the same reasons for the other customer’s ISP (Internet Service Provider).

Manual Data Gathering

Gameplan

• Do Nothing

• Wait for feedback

• Active Monitoring

• endpoint monitoring

• network monitoring

Low hanging fruit

• Endpoint monitoring (Traffic lights)

• colors to show connection is active/inactive

• off/online

• good/bad/ugly

HTML APIs• HTMLVideoElement Extensions

• http://www.w3.org/TR/2014/CR-media-source-20140717/#htmlvideoelement-extensions

• droppedVideoFrames,  corruptedVideoFrames,  totalFrameDelay

• WebRTC Statistics API

• (work in progress) http://www.w3.org/TR/2015/WD-webrtc-stats-20150206/

• PeerConnection Metrics: RTP, datachannel

• Transport Metrics: active and passive candidates

• Security cipher suite

Monitoring What?

• Annoyances

• Transport quality

• Per-stream media quality

Aggregate Dashboard

Totally failed: Sessions failed for all participants.

Partially failed: some participants were not able to join the conference.

Filter Dashboard by Time

What was the main cause of failure?

ICE connection failure occurs when the endpoints encounter a NAT/Firewall.

Session Details

hidden

Conference Events

visualises media events like muted audio, pausing video. Additionally, events like call hold and resume

Annoyances Setup times: waiting for the conference to begin: NAT traversal, negotiation, etc.

Note: gathering and connectivity checks can happen concurrently

Annoyances

Disruption: loss of connectivity when network interfaces change, low available capacity,or high delay

The light grey vertical lines show disruption, highlighted by the red bounding boxes.

User Behaviour: The user tries to correct for the disruption by turning on and off video

Zoom in to the events to view disruptions

Transport Metrics

Quality Metrics

In this case, it shows two users have comparable quality of experience, meanwhile the third user has one order lesser q u a l i t y a n d a l s o indicative that the user drops and rejoins.

Also shows that the congestion control has a late joiner problem. As the quality improves as when one user drops off at the end of the call.

higher value indicates better user experience.

Search

• for a particular conferenceID

• by criteria: failed calls, TCP calls, relayed calls

• monitor ongoing calls in real-time

Notifications

• WebRTC Clients Notifications

• (e.g., modify settings to deliver optimum quality)

• Dashboard/Operation Notifications

• (e.g., aggregate quality or number of completed session dropped below a threshold, etc)

Gather

Report

Analyze

FIX

Deploy?

webrtc-internal?

via email?

Has

this

bee

n re

port

ed b

efor

e?

Did it solve the issue?Until next report?

Collect

Diagnose

Fix

Deploy

• Analytics for WebRTC

• 3 lines of code —> 5 minutes.

1. include <script>

2. initialize() —> needs registration keys

3. addNewFabric(pc,  …) —> call started

• optionally: send user events: audio muted/unmuted, video paused/resumed, call terminated, call held.

• optionally: reportError() —> call failed to be setup due to SDP or Firewalls.

• optionally: associateMstWithUserID() —> bridges using multiple media streams in a single peerconnection may want to correlate mediastream quality across endpoints.

• optionally: collect and send user feedback

http://www.callstats.io/api/