#3208 Learning to Estimate Driver Drowsiness from Car ......– AUC2) classifying each sample with a drowsy timestamp or normal timestamp based on the score. §We compared the results

1

IBM Research – Tokyo

#3208Learning to Estimate Driver Drowsiness fromCar Acceleration Sensors using Weakly Labeled DataTakayuki Katsuki, Kun Zhao, and Takayuki YoshizumiIBM Research – Tokyo

ICASSP 2020

2


Compute drowsiness score from car accelerations sensor data for each time § Our method detect driver’s drowsiness that may cause an accident.§ Input: three-axis acceleration (from acceleration sensor), speed, and direction (from GPS)§ Output: drowsiness score

Drowsiness score

Accident

Obtain sensor data from an edge device (such as a smartphone or a drive recorder),

and compute a scoreAlert

Acceleration sensor etc.

Threshold

Time

3


Driver’s status can be estimated by sensors mounted on a car

AbnormalHigh risk!

Normal

Acceleration sensor in a car can capture car behavior

Analyze difference between them

Driver’s status will affect car behavior through driving operations.

Car behavior is unstable

Car behavior is stable

4


§ Too difficult (Ordinal setting)

§ Easy (Our approach assigning label to whole trip)

Since obtaining label for each timestamp is not a realistic goal,we use weak labels for whole trip not for timestamp

Drivers do not remember exact time when they felt drowsy after driving.

Drowsy

Drivers can not label their own drowsiness in a timely manner while drowsy driving.

Drivers can answer whether they felt drowsy while they were driving after the trip!!

TimeDrowsy timestamp (label for timestamp)

Drowsy trip (label for whole trip)

??

“I felt drowsy!”

5


To learn the estimation function from weak labels, we impose two assumptions

1) Drowsiness is measured on an ordinal scale

2) Drowsiness increases as the trip gets longer due to tiredness

We can consider which of two different data points is drowsier

Time

Time

Tiredness

Time

DrowsinessDrowsiness is

partly due to tiredness

Tiredness monotonically increase over time

Some component of drowsiness also increases over time

6


We learn the estimation function for driver drowsiness such that estimated drowsiness score becomes large as driving time increases

Time

If u < t, our learning algorithm adjusts f as

Time

Lear

ned

scor

e

<

tu

Drowsiness increases over time

Compare is input features from acceleration sensors at each time

7


Learning ordinal scale of drowsiness from weakly labeled data§ Optimize the following expected loss (objective function) in data labeled for the whole trip as drowsy

§ : D-dimensional input features

§ : Estimation function for drowsiness

§ : Loss function

§ , : Timestamps

It does not require to know absolute label’s value and we just compare which data is drowsier

Timeu t

It returns larger loss when score with shorter driving time is greater than that with longer driving time

8


Stochastic Optimization for Scalable Learning § Let be all candidate pairs of samples in -th trip ;

we empirically approximate the expectation in the objective function, as

§ For stable learning, we add a regularization term, ,and then derive the gradients as

§ : Model parameters

§ We can estimate the drowsiness score for the new data as

9


The proposed method achieves better estimation performance for drowsiness than anomaly detection and classification approaches. § The driving data consisted of 11 drowsy trips and 94 normal trips, including highway and ordinary road trips. There were about 40

driving hours (over 100,000 samples)

§ The performance of our drowsiness score was evaluated regarding two types of the area under the curve (AUC) in cross-validation :– AUC1) classifying each trip as either a drowsy trip or normal trip based on the maximum score during the trip– AUC2) classifying each sample with a drowsy timestamp or normal timestamp based on the score.

§ We compared the results of the proposed method with anomaly detection and classification approaches.

Proposed method were closest to the upper left corner

10


Important features for estimating drowsiness.

§Positive effect on drowsiness– Forward-backward jerk

• Unstable driver’s acceleration and braking– Forward-backward acceleration– Vertical jerk

§Negative effect on drowsiness– Lateral acceleration

• Route or vehicle’s steering characteristics which are not directly related to driver operation

– Magnitude of acceleration– Angle-acceleration

11


Conclusion § We formulated a learning problem for estimating driver drowsiness from car-acceleration sensor data using a weakly labeled data. The

data for the whole trip rather than at each timestamp were labeled as drowsy or normal.

§ We proposed a learning algorithm based on the assumption that some aspects of driver drowsiness increase over time due to tiredness.

§ An experimental evaluation on real driving datasets demonstrated that the proposed method outperformed the baseline methods.

12


Thank you

Takayuki Katsuki, Kun Zhao, and Takayuki YoshizumiIBM Research – Tokyo

Documents

#3208 Learning to Estimate Driver Drowsiness from Car ......– AUC2) classifying each sample with a drowsy timestamp or normal timestamp based on the score. §We compared the results