1
(Jeremy)Behavioral Informatics and Interaction Computation Lab (BIIC)
:
()
2016
2016.07.17
2
3
(BSP)
BSP INGREDIENTS
4
()
: +
I. II.
III. IV.
5
BSP INGREDIENTS
6
. . .
7
:
8
(20133 13 )
(20135 29 )
9
10
200/
:
?
11
62.5 ():" "
89 ():
12
:
13
:
14
:
:
-frame Dense Points Tracking
TRAJ
MBHxy
Each = A Unit-level (66ms) -length Derived Video features
: Dense Trajectory Fisher-
1
2
3
1
2
Acoustic LLDs
Each : = A Unit-level (200ms)-length Dense Acoustic Features
Functionals
1: {1, 1}1
1:1
2:1
:1
1:
: Dense Unit Acoustic Features
2: {1, 2}
3: {1, 3}
4: {1, 4}
K-Means Bag-of-word
15
:
late fusion technique
Support vector regression
Support vector regression
+
16
:
1
2
Spearman correlation()
= .
17
:
2
1
2
2
1
10
= .
= .
= .
18
?
19
20
Word2Vec
Hierarchical Probabilistic
21
Word2Vec
22
...
N-gram K-meansAll Documents
BOWper Document
Word2vec
N
23
()
Average support vector regression
Support vector regression
+
= . .
24
()
25
?
26
multi-task learning
()task
Task 1 - feature
Task 2 - feature
Task 8 - feature
.
.
.
Kernel
Multi-task learning
27
= . .
?
28
- . . .
29
30
31
(Taiwan Triage and Acuity Scale, TTAS)
32
(=)
(~200)
(=)
(=)
(=)
33
:
()
SpeakerDiarization
Raw audio-videorecording
S1
S2
Sk
. . . MFCCPitch
Intensity
1 : [1,1]
2 : [1, 2]
: [1,]
34
:
:
S1
35
:
Support vector classification
Support vector classification
Fusion
36
:
72.3%
51.6%
gold standard
37
(: 0-3, : 4-6, : 7-10)
: :
: :
: :
Poker face Talk with smiling
Trembling voice
38
database
Before After
:
:
:
39
Before After
: :
: :
40
Pilot work ()
- . . .
41
( ~ 2-5s)
Global label ()
3-5 minutes
42
Thin-slice
Naumann et al. : Personality
Ovies et al. : Affect style
Oltmanns et al. : Personality disorders
:
Motion Capture(Avatar)
43
The USC CreativeIT database
44
:
:
: 45: 90
45
(multimodal)
(density-weighted)
(mutualinformation)
thin-slice
46
Activation 0.384 0.722
Dominance 0.675 0.834
Valence 0.571 0.822
(Global)
(Spearman )
47
91%
9%
Act.(10% data remain)
Including
Reduced 98%
2%
Dom.(70% data remain)
Including
Reduced
95%
5%
Val.(20% data remain)
Including
Reduced
thin-slice?
48
49
- :
1. 2.
(10)
Activation:
4.4 ()3.8 ()4.6 ()
0
1
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
51
53
55
57
59
61
63
65
67
69
71
73
75
77
79
81
83
85
87
89
91
93
95
97
99
TIME SEGMENTS
Emotion-Rich behaviors
1
0
1
1 3 5 7 9
11
13
15
17
19
21
23
25
27
29
31
33
35
37
39
41
43
45
47
49
51
53
55
57
59
61
63
65
67
69
71
73
75
77
79
81
83
85
87
89
91
93
95
97
99
TIME SEGMENTS
Emotion-Rich behaviors
2
Valence: 4.0 ()3.7 (4.3 ()
52
Assumption: Gold Standard
53
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Act. Val.
Agreement
Entire Slice
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Act. Val.
Correlation
Entire Slice
thin slice
thin slice
54
55
56
()
Pattern
Contextualize
57
Data
evaluation
Always look for insights
58
59
ASD ADOS
Couple Therapy
Affective Computing
Oral Evaluation
Stroke Prediction
BiiC: BSP
fMRI Analysis
Pain Scale
60
61
:
application domain
62
Challenging the status quoMaking a positive impact
63
BiiC lab @ NTHU EEhttp://biic.ee.nthu.edu.tw