Real time voice call integration - Confoo 2012

1 Dealing with Continuous Data Processing, ConFoo 2012

REAL TIME VOICE CALL AND MOBILE INTEGRATION

Michael Peacock, February, 2012


WHAT’S IN STORE

• Telephony integration• Providers• Features and workings• Sample applications:• Phone number verification• Lead tracking• Event feedback

• How we use it


Live Coding


Live & Semi-live Demos


MICHAEL PEACOCK

Web Systems DeveloperTelemetry TeamSmith Electric Vehicles Corp

Lead Developer / Technical Author / Occasional Speaker

@[email protected]


SMITH ELECTRIC VEHICLES


NO. NOT MILK FLOATS (ANYMORE)

ALL ELECTRIC, COMMERCIAL VEHICLES.

Photo courtesy of kenjonbro: http://www.flickr.com/photos/kenjonbro/4037649210/in/set-72157623026469013


ALL-ELECTRIC COMMERCIAL VEHICLES


TELEPHONY INTEGRATION

• Calls• Make out bound calls• Receive incoming calls• Record calls • Transcribe calls• Log caller inputs

• SMS (mobile)• Send • Receive


TELEPHONY INTEGRATION

• Numbers• Provisioning

• Other• Text to speech• Speech to text*• Accents


MY FIRST EXPOSURE AS A DEVELOPER

• 6 Years ago• Gambling website:• Land line phone number verification as part of

identity and fraud screening• Expensive service• Complex with tonnes of complicated

documentation

• SMS Parsing• A little bit easier• Still quite a clunky process


PROVIDERS


TWILIO VS TROPO

Twilio Tropo

30$ Free Credit Free for development

Incoming and outgoing calls

Send and receive SMS messages

IVR: Interactive Voice Response

DTMF: Record key presses

Number provisioning

Open VBX (Twilio project, with a Tropo fork)

Twimlets File hosting

Connect: for apps SIP: Additional Integration

iOS Integration W3C: Speech Recognition Grammar

Browser Integration IM Service integration


WHY?

Most people I talk to are in one of two camps when they learn about Web Apps + Phone calls:• Cool! I’ve got a million ideas• Why?So let’s answer the why


WHY: CUSTOMER SERVICE

• Add that personal touch• Text or call to confirm order status change

• Your order has been dispatched

• Thank you for your recent order• Please take a moment to rate your recent purchase

• How was our service• How quickly did we deliver• Any comments you want to leave us

• Request a call back• Customer enters their details and automatically

they get a call from the company in question• Integrate with existing CRM solutions


WHY: LEAD TRACKING

• Advertising is all about conversions• A lot of that is about clicks to websites, which

is easy to track• Some of it is about calls to numbers• Many businesses still, rightly so, rely on

customers finding their information and calling them

• Big advertising networks can track this• Now we can too


WHY: VERIFICATION & SECURITY

• Phone number verification• Do we need to verify a users contact details

• We often verify email addresses

• Some business cases require phone number verification too

• Two factor authentication• Username + Password + SMS code

• Confirmation• Is it definitely you transferring your savings to

help unlock the assets of a dead Nigerian businessman?


WHY: MARKETING

• Text based marketing• Pizza discount SMS anyone?


HOW? INBOUND CALLS

• User makes a call

• Twilio looks up the in bound number; makes a request to our server

• Our server generates some XML determining what to do, what to say, and where to redirect when the caller does something

•Caller does something, Twilio calls the appropriate URL relating to the action


XML


HOW: OUTBOUND CALLS

• We send a REST request with some basic details

• Twilio connects the call

• Calls back to our app to read call-flow XML


RestPOST/GETXML


XML: HEADER AND WRAPPER

• Set the standard XML headers: mime type • Wrap everything in a response• <Response>• The root XML element: everything belongs in

here


XML: SPEAK

•<Say>Welcome Conference Delegates</Say>• Say it again?

• <Say loop=“2”>We can also say it again</Say>

• Male or female• <Say voice=“woman”>Or perhaps as a woman instead</Say>

• Accent• <Say language=“en-gb”>British instead of American English</Say>

• Language• <Say language=“fr”>Ou même en français au lieu</Say>


XML: PLAY

• <Play>http://url-to-an-audio-file

</Play>• loop

http://url-to-an-audio-file/


PLAY / HOLD

• https://labs.twilio.com/twimlets/holdmusic• Creative Commons Hold Music

https://labs.twilio.com/twimlets/holdmusic


XML: ASK

• <Gather>• action: URL to submit response to• method (GET/POST)• timeout – how long to wait for a response• numDigits – number of digits to wait for• finishOnKey – key to indicate end of entering

digits


XML: RECORD

• <Record>• action, method, timeout and finishOnKey• maxLength – maximum length of recording• transcribe• transcribeCallback• playBeep


TRANSCRIPTIONS

• Part of call recording• Paid for service• Maximum duration of transcription: 2 minutes• Post transcription Callback:• Text of the recorded call• Status: Completed or Failed• Properties for accessing the transcription over

the REST API


XML: SEND A TEXT

• <SMS>• to• from• statusCallback• action and method


XML: TRANSFER THE CALL

• <Dial>• action, method and timeout• hangupOnStar• timeLimit• callerId

• Statuses• <Number>, <Client>, <Conference>


MORE XML

• <Hangup>• <Redirect>• <Reject>• <Pause>


XML IS GOOD...BUT...

• Its tedious to write

• We don’t really want XML lying around in our application

• Do we really want views / templates just for our twilio XML?

• Client Library• Generates it for us


HOW: REST

• Phone numbers• Searching• Purchasing• Updating• Deleting

• Calls• Making

• SMS• Sending


SECURITY

• Is that you Twilio?

•X-Twilio-Signature

• Contains a signed hash of various paramaters

• Read the manual• I didn’t spot this gem initially, and was writing

all sorts of hacks / tokens into my apps


DEBUGGING


SANDBOX

• Don’t need a number• Call their sandbox number• Enter your sandbox PIN• Call is transferred to your application• Free...well it costs you to call from your phone• Or call from the browser using the client



HOW MUCH?

Twilio Tropo

Phone Numbers $1/month $3/month

Inbound minutes 1c 3c / minute

Outbound minutes 2c 3c / minute (international+)

Toll free numbers $2/month* $5/month*

SMS In 1c 1c

SMS Out 1c 1c


MY APPLICATIONS

• Field service technicians• Travelled half way around the country to install

a GPRS enabled device to a vehicle• Can’t access Internet at customer location• How do they know it works?• Simple! Phone up and ask

• Alerts• Driver deviates from route

• Send an SMS back to the vehicle• Call his fleet manager, warn him that the route

deviation is costing him in terms of battery capacity


WHY YOU MIGHT WANT TO USE IT


WHAT WE ARE GOING TO BUILD

• Phone number verification• Verify a users phone number• Confirm transactions: some online banking

transactions

• Lead tracking• Conference talk feedback


PHONE NUMBER VERIFICATION

Requirement: You need to verify the validity of a users phone number.1. Ask the user for their number and tell

them an access code2. Call the number3. Ask for the access code4. Check the access code5. Verify the number


PNV: GET THE DETAILS


PNV: CREATE THE UN-VERIFIED USER

// unverified user; default verified status of false

$user = new UnverifiedUser();

$user->setFullName( $_POST['full_name'] );$user->setFullPhoneNumber( $_POST['number'] );

$user->generateRandomVerificationCode();

$user->save();

$userID = $user->getID();


PNV: MAKE THE CALL

$twilioClient = new Services_Twilio( $sid, $token );

$call = $twilioClient->account->calls->create($ourPhoneNumber,$countryCode . $telephoneNumber,'http://my-url/verify/&user=' . $userID

);


PNV: ASK FOR MORE

// create a response / twiml object$response = new Services_Twilio_Twiml();

$gather = $response->gather( array( 'numDigits' => 4,

'action' => $url . '/verify/' . $userID ) );

$gather->say('This is the ' . $siteName . ' telephone verification service. To verify

your telephone number, please enter the four digit code shown on your screen now.'); print $response;


PNV: VERIFY

$code = (int) $_REQUEST['Digits'];

if( $user->getVerificationCode() == $code ){

$user->verify();}else{

$this->askAgain();}


PNV: ASK AGAIN

$gather = $response->gather( array( 'numDigits' => 4,

'action' => $url . /verify-again/' . $userID ) );

$gather->say('Sorry, the code you entered was incorrect, please try again.');

print $response;


PNV: VERIFIED

$response->say('Thank you, your phone

number has been verified');

print $response;


HEARTBEAT

• Javascript timer to check for verification status• Redirects the page when verification is

complete


PNV: LIVE SAVED DEMO


LEAD TRACKING

1. Advertiser signs up2. A number is automatically assigned to

them3. Calls to that number route to our main

tracking application4. Call is logged5. Caller is transferred


LEAD TRACKING: NUMBER PROVISIONING

$twilioClient = new Services_Twilio( $sid, $token );// search for a local canadian number$numbers = $twilioClient->account->available_phone_numbers->getList('CA', 'Local');if( count( $numbers->available_phone_numbers ) > 0 ){

// buy the first we find$purchased = $twilioClient->account->incoming_phone_numbers-

>create(array('PhoneNumber' => $numbers->available_phone_numbers[0]->phone_number ));

$advert->setPhoneNumber( $numbers->available_phone_numbers[0]->phone_number );

$advert->setPhoneNumberSid( $purchased->sid );}else{

throw new Exception('No numbers available');}


LEAD TRACKING: POINT THE NUMBER

$purchased->update( array(

'FriendlyName' => 'Lead tracking for

Michaels advert', 'VoiceUrl' => 'http://my-

application-url',

'VoiceMethod' => 'POST‘)

);


NEWLY PROVISIONED NUMBER


LEAD TRACKING: INBOUND CALLS

• Code at http://my-application-url

$dialledNumber = $_POST['To'];

http://my-application-url/


CALLER ID?

For a fee we can lookup the Caller ID automatically on inbound calls.

If enabled, it comes in the POST data from Twilio


LEAD TRACKING: MAKE A CONNECTION

• We have the number• Look it up in the database• Find the actual advertisers / lead owners

number• Log the lead conversion (call)• Connect the call


LOOKUP THE NUMBER


LOG THE LEAD


TRANSFER THE CALL

$response = new Services_Twilio_Twiml();

$response->dial( $advertisersNumber, array('callerId' => $ourVerifiedNumber ));

print $response;


APP: CONFERENCE SESSION FEEDBACK

• In bound number to ask audience members to rate a talk• Option to leave feedback at the end


LIVE SAVED DEMO: CONFERENCE FEEDBACK


LIVE DEMO: YOUR TURN

Give it a try:

+1 514-418-0598


APPS: CONNECT

• oAuth Link• User gives your application permission to access their account for numbers and billing• You build the App• Set your pricing model• Let the end user worry about the usage costs• Utility pricing without the pain!


BROWSER: CLIENT

• JavaScript API to connect with Twilio• Let’s you make and receive calls from the

browser• Redirection can be to a client login

• Call centre setup:• No need for expensive hardware



MOBILE

• In addition to SMS integration• iOS SDK Available• Use WiFi or Mobile Data• Lots of out of the box features

• Android on the way


TWIMLETS

• Set of hosted pre-defined basic applications• You just stick some GET parameters on the end• Lets you quickly test and experiment with

features without needing to write a full app


OPEN VBX


A LOOK BACK AT TROPO

• Two key additional features• Speech recognition

• Basic• W3C: Speech Recognition Grammar

• SIP Integration


TROPO: W3C: SPEECH RECOGNITION GRAMMAR

What we have discussed so far is really great stuff, but sometimes you want that little bit more.

• Examples• Branch Finder• Flight booker• Tourist information

• Your caller really needs to be able to search with their voice• Enter: W3C Speech Recognition Grammer

• Supported by Tropo


W3C SPEECH RECOGNITION GRAMMER

• Too much to discuss in detail here...• XML based way of defining potential requests and processing natural speech from callers• Very powerful!


BASIC SPEECH RECOGNITION

• Please enter your account number

Tropo supports both touch tone entering and speech recognition


TROPO: SIP INTEGRATION

• Integrate with your existing VOIP solution• Call other SIP based accounts• Receive calls to a SIP account


WHICH TO USE

• IMHO...• Twilio in most cases

• Really well documented• Lots of feature support• Lots of example code and client libraries

• Tropo if you need a little bit more “enterprise-y”• SIP integration• Speech recognition

• Clearly both are striving to win – adding more features and value for customers like us


FINAL THOUGHTS


DEMO APPLICATIONS

• I’ve previously released some, incredibly hacky, versions of some of the demos given here today, on my GitHub profile• I promise, I will clean these up and update them with what you have seen today• Keep an eye on my twitter @michaelpeacock for updates


THANKS & QA

Michael [email protected]@michaelpeacock

mailto:[email protected]

Technology

Real time voice call integration - Confoo 2012