Kaggle: Whale Challenge

張智星jang@cs.nthu.edu.tw

http://www.cs.nthu.edu.tw/~jang

多媒體資訊檢索實驗室台灣大學 資訊工程系

-2-

Whale Challenge

Problem definition Identify the existence of whales from sensor recordings

Characteristics: Imbalance data Some recordings are hardly recognizable by non-experts

-3-

Dataset

Training set 47,844 recordings of 2 seconds

88.97% (42,565 recordings): w/o whales11.03% (5,276 recordings): with whales

Test set 25,468 recordings of 2 seconds

Recording format 2000-Hz sample rate, 16-bit resolution

-4-

Preprocessing

Potential preprocessing Trend removal

Trend estimation via polynomial fitting

Noise removalBand-pass filter

Removal of “non-whale” partLinear prediction?

-5-

Spectrogram

W/o band-pass filter

W/ band-pass filter

kwcPreprocess.m

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-0.05

0

0.05

Time (sec)A

mpl

itude

Original waveform and its trend

Waveform Trend

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-0.02

0

0.02

0.04

Time (sec)

Am

plitu

de

After trend removal and band-pass filter

Trend-subtracted waveform After band-pass filter

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-0.05

0

0.05

Time (sec)

Am

plitu

de

Original waveform and its trend

Waveform Trend

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-0.02

0

0.02

0.04

Time (sec)

Am

plitu

de

After trend removal and band-pass filter

Trend-subtracted waveform After band-pass filter

-6-

Potential Features

Acoustic features Volume Pitch Spectrum MFCC …

Visual features (obtained from spectrogram) Radon transform Hough transform Gabor filters …

-7-

Pitch Tracking

kwcPitchTracking.m

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

-2

0

2

x 104 Waveform of 20090328_000000_236s4ms_TRAIN25_1.aif

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.820

30

40

50

60Desired (circle) and computed (dot) pitch

Pitc

h (s

emito

ne)

-8-

Volume

kwcVolume.m

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2-0.02

0

0.02

0.04

Am

plitu

de

Waveform

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

Time (sec)

absS

um

Volume

-9-

Spectrogram

kwcSpectrogram.m

Time (sec)

Fre

q (H

z)

20090328_000000_236s4ms_TRAIN25_1.aif

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

100

200

300

400

500

600

700

800

900

1000

-10-

Visual Features viaRadon Transform

Radon transform Projection onto lines at various angles

For grayscale images only

Detection objects at a specific angle

-11-

Example of Radon Transform

Source http://www.mathworks.com/help/images/ref/radon.html

Output Code: goRadon.m

20 40 60 80 100

20

40

60

80

100

(degrees)

x'

0 20 40 60 80 100 120 140 160

-60

-40

-20

0

20

40

6010

20

30

40

50

60

70

0

50100

150

-50

0

50

0

20

40

60

(degrees)x'

10

20

30

40

50

60

70

-12-

Example of Radon Transform (2)

Source image Output Code: goRadon2.m

Time (sec)

Fre

q (H

z)

20090328_000000_236s4ms_TRAIN25_1.aif

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

100

200

300

400

500

600

700

800

900

1000

(degrees)

x'

0 20 40 60 80 100 120 140 160

-20

0

205

10

15

20

25

-13-

Visual Features viaHough Transform

Hough transform Commonly used for detection lines and circles

For BW images only (after edge detection)

-14-

Visual Features viaHough Transform (2)

Hough transform Point to curve mapping

Two points Two sine curvesThe intersection is the right θ and ρ

for the line connecting these two points

sincos, iiii yxyx

-15-

Example of Hough Transform

Source http://www.ebsd-image.org/documentation/reference/ops/hough/op/houghtransform.html

Image Hough space and its maxima Detected lines

-16-

Example of Hough Transform (2)

Source http://www.mathworks.com/help/images/analyzing-images.html (MATLAB code available)

Image Detected linesEdge image Hough space and its maxima

-17-

Methods

Thresholding Volume variance Pitch variance

Static classifiers Naïve Bayes classifiers

GMM SVM …

Sequence classifiers HMM CRF …

-18-

HMM Training

kwcHmmTrain.m

1 1.5 2 2.5 3 3.5 4 4.5 5-2385

-2384

-2383

-2382

-2381

-2380

-2379

-2378

-2377

Training iterations

Mea

n fil

e-ba

sed

log

likel

ihoo

d

-19-

HMM Evaluation

kwcHmmEval.m

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

-1000

-500

0

Am

plitu

de

D:/dataSet/kaggle-whaleChallenge/train2/20090328_000000_236s4ms_TRAIN25_1.aif

Time (sec)

Fea

ture

dim

. in

dice

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8

5

10

15

-20-

HMM

Basic models Class 1: sil

Class 2: sil-whale-sil

Advanced models sil sil-whale-sil-whale-sil

…

sil

0.9

w 0.6sil 0.1

1.0

sil

0.4 1.0

-21-

HMM (2) Other approach

Train HMM models Align each recording with the HMM

Extract features from the whale part for other static classifiersDuration (no. of frames)Average log likelihood per frame

sil

0.9

w 0.6sil 0.1

0.4 1.0

-22-

Performance Evaluation

Performance evaluation of methods based on thresholding (http://en.wikipedia.org/wiki/Receiver_operating_characteristic): ROC, DET AUC