Detecting Eye Contact Using Wearable Eye-Tracking Glasses

• Eye contact is important aspect in face to face interaction

In case of children• Atypical pattern of gaze and eye contact • Early sign of autism

Goal:Eye contact detection between two Individuals

Single wearable eye tracking device

Clinical settings

Multiple static camera-distant, problem with frontal view

Mutual eye tracking – Use EOG

Face analysis • Problem of finding and analysing faces

in video • Computer vision • Localize the face and facial parts

(eyebrow, eye, nose, mouth etc)• Gaze estimation using 2D appearance

of eye • Estimate the 3D gaze direction

based on a single image of an eye • Key idea: Learn the appearance model

of the eye for different gaze directions • Large number of samples

OMRON OKAO• Commercial vision library• Detect and analyse child’s face• Takes the video as input • Localizes all facial parts in video

• Estimates 3D head pose • If eyes in the correct position,

• Gaze direction

• Promising result for frontally presented faces

Face analysis result by OKAO

Bounding boxFacial parts Head poseGaze direction

Experimental setup• Objectives

– Record the video and gaze data with minimum obtrusiveness for children

– Allow the analysis of the data for eye contact detection

• Protocol– Interactive session

• 5 to 8 min• Examiner wears the eye tracker glass• Interact with the child• Sitting in front

Provide online annotation by pressing foot pedal----error prone

Experimental setup• During interaction

– Eye gaze was tracked and egocentric video was recorded

• OKAO vision library is applied – Obtain face information of the child

• Location/orientation of face • 3D gaze direction

– Adult gaze information• Provided by eye tracker

Experimental setup• ParticipantsFemale subject of age 16 months Recoded session – 7 min

Method• Combines

– Eye gaze of the examiner– Face information of the child (gaze direction)

• Extract features from gaze and face information

• Train the classifier – Detect existence of eye contact in a specific

frame

Features • Relative location (RL)

– Examiner’s gaze point with respect to child’s eye center

• 3D Gaze direction of the child (GD)– With respect to image plane

• Head orientation (HO)– 3D head position of the child

• Confidence of eye detection (CE)

Method• Detect Eye contact

– Binary classification with ground truth• For each frame in the video and given

feature – Detect the eye contact

• Simple rule works– Fix threshold for RL and GD

• Examiner’s gaze point is close to child’s eye• Child’s gaze facing towards examiner

Gaze direction

Face orientation

Results • Randomly select 60% as training; rest as

testing • 5 trees ---- depth 6

1. RL is more reliable 2. Vertical is more

frequent than horizontal

Performance

OKAO vision library fails to detect correct gaze direction