Embed Size (px)
Citation preview
Do You See What I See? The Effect of Gaze Tracking
on Task Space Remote Collaboration
KunalGupta1,GunA.Lee2,andMarkBillinghurst21HITLabNZ,UniversityofCanterbury
2EmpathicCompuIngLab,UniversityofSouthAustralia
ISMAR2016,September20th,2016
Mo#va#on
• Improvingremoteassistanceofexpertuser(e.g.maintenance)• SupporIngrichcommunicaIoncues
Task Space Teleconferencing
• Focusonsharingviewofremotetaskspace• Methods
• Handheldtabletswithcameras+ARcues,Fixedcamerasinworkspace• LimitaIons
• Fixedviewpoint,Difficulttoknowwhereuserlooking
Head Worn Collabora#ve Systems
• Placecameraonhead+useheadmounteddisplay• HWC+HMD+remotepoinIngimprovescollaboraIon
• LimitaIons• Remoteviewfixed,Expertdoesn’tknowexactlywhereworkerlooking
Gaze Tracking in Teleconferencing
• Monitorbased(Brennan2008),(Carle[a2010)• Gazeprovidesa[enIoncue,significantlyimprovedperformance
• Headmounted(Fussell2003),(Ou2005)–noHMD• Noperformanceimprovement,focusofa[enIoncanbepredicted
Comparison to Previous Work
Rem=remotecollaboraIon,FtF=facetofacecollaboraIonGaze=eyetracking,HWC=headworncamera,HMD=headmounteddisplay
Key Research Ques#ons
• Q1:Willsharingofgazeandpointercuesaffectthefeelingofco-presencebetweenusers?
• Q2:WillsharingofgazeandpointercuesimproveperformanceinaremotecollaboraIvetask?
Prototype Design
• Combiningthefollowing(1)aheadmountedeye-tracker(2)headmountedcamera(3)headmounteddisplay(4)remoteviewingsohware
SystemDiagram
Local Worker
BrotherAirScouter
MicrosohLifecamHD5000
LogitechWebcamc920
Pupil Labs Eye Tracking
• Opensourceeye-tracking• UseIRreflecIonintoeye• ImageprocessingonPC• Trackseyeat30fps• Providesrawdata• www.pupil-labs.com
Remote Expert Desktop
• Livecameraview• Gazeshownasreddot• Canaddpointercues
• Mouseinput• Greendot
• ShowninHMD
User Experiment
• In remote expert collabora#on ...
• Does Pointer / Eye tracker cues have significant effect on co-presence?
• Does Pointer / Eye tracker cues have significant effect on task performance?
Experimental Design
Eye tracker cue
Pointer cue No Yes
No NONE E
Yes P Both
Experimental Design – Setup
Experimental Design – Task
• Blockassembly• Fourdifferentstructures• 17piecesineach• Pilottestedtobalancedifficultylevel
• AssignedtocondiIonswithcounterbalancing• AcIveheadmovementencouragedthroughsecondarytask(Imer)andL-shapedesksetup
Experimental Design – Procedure
• PracIcetrialinface-to-facecollaboraIon• ParIcipantsseparatedfortheexperimentaltrials• ForeachcondiIon:
• RemotehelpercreatesstructurebasedoninstrucIon• Performexperimentaltask• Answerper-condiIonquesIonnaire
• Post-experimentalquesIonnaire&debriefing
Experimental Design – Par#cipants
• Within-subject• BalancedLaInsquaredesign• 30parIcipants(15pairs)recruited,26retained• 21-33yearsold,73%male• FluentEnglishspeaking• Noonehaddoneblockassemblyovervideoconferencingbefore
Results - Summary • BoththePOINTERandEYETRACKINGvisualcueshelpedparIcipantstoperformthetasksignificantlyfaster.
• ThePOINTERcuesignificantlyimprovedbothlocalandremoteusers’perceivedqualityofcommunicaIon,collaboraIon,andco-presence.
• TheEYETRACKINGsignificantlyimprovedthecommunicaIonandcollaboraIonquality,andsenseofbeingfocusedforlocalworkers,andenjoymentforremotehelpers.
• TheBOTHcondiIonrankedasthebestinmostoftheaspectsofuserexperience,whiletheNONEcondiIonwasrankedastheworst.
• VisualcuesmadetheconversaIonmoreefficient,changedthechoiceofwordingindeicIcexpressions,andhelpedparIcipants’feelmoreconnected.
Results – Task comple#on #me • Repeatedmeasuretwo-wayANOVA(α=0.05)
• POINTERcue• F(1,12)=4.908,p=.047*• 15%lessIme
• EYETRACKERcue• F(1,12)=5.811,p=.033*• 10%lessIme
• InteracIon• F(1,12)=0.566,p=.466
sec.
Results – Per-condi#on ra#ng ques#onnaire • Q1 Ifeltconnectedwithmypartner.• Q2 IfeltIwaspresentwithmypartner.• Q3 Mypartnerwasabletosensemypresence.• Q4 Mypartner(orforRemoteHelper:I)couldtell
whenI(orforRemoteHelper:mypartner)neededassistance.• Q5 Ienjoyedtheexperience.• Q6 IwasabletofocusonthetaskacIvity.• Q7 Iamconfidentthatwecompletedthetaskcorrectly.• Q8 MypartnerandIworkedtogetherwell.• Q9 Iwasabletoexpressmyselfclearly.• Q10Iwasabletounderstandpartner’smessage.• Q11Informa9onfrompartnerwashelpful.
Adopted from [Kim et al. 2014]
Results – Per-condi#on ra#ng ques#onnaire
• 7-pointLikertScale• 1:totallydisagree~7:totallyagree
• Internalconsistency:Cronbach’sα=.937• Aggregatedinto0~100scale
Results – Per-condi#on ra#ng ques#onnaire
• AlignedRankTransform(ART)+RepeatedmeasureANOVA(α=0.05)
• POINTERcue• F(1,12)=7.414,p=.019*
• EYETRACKERcue• F(1,12)=26.822,p<.001*
• InteracIon• F(1,12)=2.023,p=.180
Results – Per-condi#on ra#ng ques#onnaire
• AlignedRankTransform(ART)+RepeatedmeasureANOVA(α=0.05)
• POINTERcue• F(1,12)=11.914,p=.005*
• EYETRACKERcue• F(1,12)=15.929,p=.002*
• InteracIon• F(1,12)=5.157,p=.042*
Results – Per-condi#on ra#ng ques#onnaire
• LocalWorkers
Results – Per-condi#on ra#ng ques#onnaire
• RemoteHelpers
Results – Ranking
Results – Ranking
• Friedmantest(α=0.05)
• Localworker• E>NoneonC5
• Remotehelper• E>NoneonC2,4,5
1: the best ~ 4: the worst
Results – Preference and qualita#ve feedback
• Understandingpartner• Localworkers:85%preferredcondiIonsincludingPOINTERcue“WithPointer,Icanrelatetowhatheistalkingabout,becauseIcouldunderstandhimmore.”
• Remotehelpers:70%preferredtheBOTHcondiIon“Theeyetrackerhelpsmetolookinthesameviewofmypartner.”
• Performingtaskefficiently• 77%ofLocal&85%ofRemoteuserspreferredtheBOTHcondiIon“TheeyetrackerwasgivingmypartnermoreinformaFonaboutwhereIlookedat,whilethepointerwasforgivingmetheinstrucFonfrommypartner.”
Results – Behaviour observa#on
• Pointercuereducednumberofphrasessaid• LocalworkerF(1,11)=6.532,p=.027*• RemotehelperF(1,11)=8.479,p=.014*
• Referringtoobjects&direcIng
WithoutPointer WithPointer describefeatures
(colour,size,shape,...) “thisone”
“moveleG/right”,“infrontof” “putithere”,“nexttothis”
“that”object “this”object
Discussion • Performanceimprovedbyusingeye-trackerandpointer
• Pointer–givingdirectguidance• Eye-tracker–showedworkersfocusofa[enIon
• Eyetrackerprovidedbenefitevenwithoutpointer• SameviewcondiIonforlocalworker,butimprovedcommunicaIonquality
• BenefitofgazeinformaIon• ImprovedcommunicaIon,moreenjoyable,focus,reduceinterrupIon
• Benefitofvirtualpointercue• EaseofdirecIon,increasedsenseofpresence,increasedpartnerawareness
• Differentuserneeds• Localworker–understanding/empathy–benefitsfromeye-tracking• Remoteexpert–givinginstrucIon–benefitsfromusingpointercue
Implica#ons
1. Eye-trackingcanbeusedtochangethenatureofremotecollaboraIonwithheadwornsystems• MakeremoteuserawareofimplicitintenIons
2. ProvidinggazecuesalonecansignificantlyimprovetheremotecollaboraIonevenwithoutremotepoinIng• eye-trackingjustasbeneficialasusingremotepoinIngbyitself
3. CommunicaIoncueslikegazeandpoinIngplayaveryimportantroleincreaIngasenseofco-presenceanddeeperunderstanding• Mostoftheuserspreferredgaze+pointerduetoconnecIoncreated
Limita#ons
1. PrototypetoobulkyandtetheredtoPC,notwearable• NewHMDswithintegratedcamerascouldovercomethis
2. TasklimitedcomparedtorealisIcremotecollaboraIon• DidhavekeyelementssuchasobjectidenIficaIon
3. Experimentalmeasuresnotsodetailed• Detailedbehavioralanalysis,conversaIonalanalysis
Conclusions
• DoessharingeyetrackinginformaIonbetweenlocaluserandremoteexperthelpintermsofco-presenceandperformance?
• Usinggazeandpointera[enIoncuesimprovedperformanceIme.• Gazeandpointercuesimprovedthefeelingofco-presence
• Manyareasforfuturework• Exploreparalleltask–bothpeoplewithsameroles• ProvidesymmetriccommunicaIoncues–gazebothways• Useotherphysiologicalcues,GSR,heartrate,EEG
Empathy Glasses (CHI 2016)
• Combinetogethereye-tracking,display,faceexpression• Implicitcues–eyegaze,faceexpression
++
PupilLabs EpsonBT-200 AffecIveWear
Masai,K.,Sugimoto,M.,Kunze,K.,&Billinghurst,M.(2016,May).EmpathyGlasses.InProceedingsofthe34thAnnualACMConferenceExtendedAbstractsonHumanFactorsinCompuFngSystems.ACM.
Affec#veWear – Emo#on Glasses
• Photosensorstorecognizeexpression• UsercalibraIon• Machinelearning• Recognizing8faceexpressions
Remote Collabora#on
• EyegazepointerandremotepoinIng• Faceexpressiondisplay• ImplicitcuesforremotecollaboraIon
