ChatBot 的智慧與靈魂YUN-NUNG (VIVIAN) CHEN陳縕儂

Apr 27th, 2017@iThome #ChatBot Day

HTTP://VIVIANCHEN.IDV.TW

Intelligent Assistants 2

Apple Siri (2011) Google Now (2012)Google Assistant (2016)

Facebook M & Bot (2015)Google Home (2016)

Microsoft Cortana (2014)

Amazon Alexa/Echo(2014)

Why We Need?

Get things done

E.g. set up alarm/reminder, take note

Easy access to structured data, services and apps

E.g. find docs/photos/restaurants

Assist your daily schedule and routine

E.g. commute alerts to/from work

Be more productive in managing your work and personal life

3

“Hey Assistant”

Why Natural Language?

Global Digital Statistics (2015 January)

4

Global Population

7.21B

Active Internet Users

3.01B

Active Social Media Accounts

2.08B

Active Unique Mobile Users

3.65B

The more natural and convenient input of devices evolves towards speech.

Intelligent Assistant Architecture 5

Reactive Assistance

ASR, LU, Dialog, LG, TTS

Proactive Assistance

Inferences, User Modeling, Suggestions

Data Back-end Data

Bases, Services and Client Signals

Device/Service End-points(Phone, PC, Xbox, Web Browser, Messaging Apps)

User Experience“restaurant suggestions”“call taxi”

Dialogue System / ChatBot Dialogue systems are intelligent agents that are able to help users finish tasks

more efficiently via spoken interactions.

Dialogue systems are being incorporated into various devices (smart-phones, smart TVs, in-car navigating system, etc).

6

JARVIS – Iron Man’s Personal Assistant Baymax – Personal Healthcare Companion

Good dialogue systems assist users to access information conveniently and finish tasks efficiently.

App Bot A bot is responsible for a “single” domain, similar to an app

7

Seamless and automatic information transferring across domains reduce duplicate information and interaction

愛食記

Map

LINE

Goal: Schedule a lunch with Vivian

KKBOX

GUI v.s. CUI (Conversational UI) 8

https://github.com/enginebai/Movie-lol-android

GUI v.s. CUI (Conversational UI)

網站/APP內GUI 即時通訊內的CUI

情境 探索式，使用者沒有特定目標 搜尋式，使用者有明確的指令

資訊量 多 少

資訊精準度 低 高

資訊呈現 結構式 非結構式

介面呈現 以圖像為主 以文字為主

介面操作 以點選為主 以文字或語音輸入為主

學習與引導使用者需了解、學習、適應不同的介面操作

如同使用即時通訊軟體，使用者無需另行學習，只需遵循引導

服務入口 需另行下載App或進入特定網站與即時通訊軟體整合，可適時在對話裡提供服務

服務內容 需有明確架構供檢索 可接受龐雜、彈性的內容

人性化程度 低，如同操作機器 高，如同與人對話

9

Two Branches of Bots

Task-Oriented

Personal assistant, helps users achieve a certain task

Combination of rules and statisticalcomponents

POMDP for spoken dialog systems (Williams and Young, 2007)

End-to-end trainable task-oriented dialogue system (Wen et al., 2016)

End-to-end reinforcement learning dialogue system (Li et al., 2017; Zhao and Eskenazi, 2016)

Chit-Chat

No specific goal, focus on natural responses

Using variants of generation model

A neural conversation model (Vinyals and Le, 2015)

Reinforcement learning for dialogue generation (Li et al., 2016)

Conversational contextual cues for response ranking (AI-Rfou et al., 2016)

10

Dialogue System FrameworkMODULAR SYSTEM

11

System Framework 12

Speech Recognition

Language Understanding (LU)• Domain Identification• User Intent Detection• Slot Filling

Dialogue Management (DM)• Dialogue State Tracking• Policy Optimization

Generation

Hypothesisare there any action movies to see this weekend

Semantic Framerequest_moviegenre=action, date=this weekend

System Action/Policyrequest_location

Text responseWhere are you located?

Screen Displaylocation?

Text InputAre there any action movies to see this weekend?

Speech Signal current bottleneck error propagation

Backend Database

1 2

3 4

Interaction Example

Intelligent AgentQ: How does a dialogue system process this request?

Good Taiwanese eating places include Din Tai Fung, etc. What do you want to choose?好的台式餐廳包含鼎泰豐…，你想要選哪一間?

find a good eating place for taiwanese food我想找好的台式食物

14

Language Understanding

System Framework 16

Speech Recognition

Language Understanding (LU)• Domain Identification• User Intent Detection• Slot Filling

Dialogue Management (DM)• Dialogue State Tracking• Policy Optimization

Generation

Hypothesisare there any action movies to see this weekend

Semantic Framerequest_moviegenre=action, date=this weekend

System Action/Policyrequest_location

Text responseWhere are you located?

Screen Displaylocation?

Text InputAre there any action movies to see this weekend?

Speech Signal

Backend Database

1. Domain IdentificationRequires Predefined Domain Ontology

17

Organized Domain Knowledge (Database)

Restaurant DB Taxi DB Movie DB

Classification!

find a good eating place for taiwanese food我想找好的台式食物

find a good eating place for taiwanese food我想找好的台式食物

2. Intent DetectionRequires Predefined Schema

18

Restaurant DB

FIND_RESTAURANTFIND_PRICEFIND_TYPE

:

Classification!

find a good eating place for taiwanese food我想找好的台式食物

3. Slot FillingRequires Predefined Schema

19

Restaurant DB

Restaurant Rating Type

Rest 1 good Taiwanese

Rest 2 bad Thai

: : :

FIND_RESTAURANTrating=“good”type=“taiwanese”

SELECT restaurant {rest.rating=“good”rest.type=“taiwanese”

}Semantic Frame Sequence Labeling

O O B-rating O O O B-type O

Dialogue Management

State Tracking

Dialogue Management

Policy Optimization

System Framework 21

Speech Recognition

Language Understanding (LU)• Domain Identification• User Intent Detection• Slot Filling

Dialogue Management (DM)• Dialogue State Tracking• Policy Optimization

Generation

Hypothesisare there any action movies to see this weekend

Semantic Framerequest_moviegenre=action, date=this weekend

System Action/Policyrequest_location

Text responseWhere are you located?

Screen Displaylocation?

Text InputAre there any action movies to see this weekend?

Speech Signal

Backend Database

Dialogue State TrackingRequires Hand-Crafted States

22

location rating type

loc, rating rating, type loc, type

all

i want it near to my office我希望可以離我辦公室近一點

NULL

find a good eating place for taiwanese food我想找好的台式食物

i want it near to my office我希望可以離我辦公室近一點

find a good eating place for taiwanese food我想找好的台式食物

Dialogue State TrackingRequires Hand-Crafted States

23

location rating type

loc, rating rating, type loc, type

all

NULL

i want it near to my office我希望可以離我辦公室近一點

find a good eating place for taiwanese food我想找好的台式食物

Dialogue State TrackingHandling Errors and Confidence

24

FIND_RESTAURANTrating=“good”type=“taiwanese”

FIND_RESTAURANTrating=“good”type=“thai”

FIND_RESTAURANTrating=“good”

location rating type

loc, rating rating, type loc, type

all

NULL

?

?

find a good eating place for taiwanese food我想找好的台式食物

rating=“good”, type=“thai”

rating=“good”, type=“taiwanese”

?

?

Dialogue Policy for Agent Action

Inform(location=“Taipei 101”)

“The nearest one is at Taipei 101”

Request(location)

“Where is your home?”

Confirm(type=“taiwanese”)

“Did you want Taiwanese food?”

Task Completion / Information Display

ticket booked, weather information

25

Generation

System Framework 27

Speech Recognition

Language Understanding (LU)• Domain Identification• User Intent Detection• Slot Filling

Dialogue Management (DM)• Dialogue State Tracking• Policy Optimization

Generation

Hypothesisare there any action movies to see this weekend

Semantic Framerequest_moviegenre=action, date=this weekend

System Action/Policyrequest_location

Text responseWhere are you located?

Screen Displaylocation?

Text InputAre there any action movies to see this weekend?

Speech Signal

Backend Database

Output / Natural Language Generation

Goal: generate natural language or GUI given the selected dialogue action for interactions

Inform(location=“Taipei 101”)

“The nearest one is at Taipei 101” v.s.

Request(location)

“Where is your home?” v.s.

Confirm(type=“taiwanese”)

“Did you want Taiwanese food?” v.s.

28

Challenges and Recent TrendsDEEP LEARNING / MULTIMODALITY/ DIALOGUE COVERAGE / DIALOGUE COMPLEXITY

29

Challenges

Predefined semantic schemaChen et al., “Matrix Factorization with Knowledge Graph Propagation for Unsupervised Spoken Language Understanding,” in ACL-IJCNLP, 2015.

Data without annotationsChen et al., “Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models,” in ICASSP, 2016.

Semantic concept interpretationChen et al., “Deriving Local Relational Surface Forms from Dependency-Based Entity Embeddings for Unsupervised Spoken Language Understanding,” in SLT, 2014.

Predefined dialogue statesChen, et al., “End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding,” in Interspeech, 2016.

Error propagationHakkani-Tur et al., “Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM,” in Interspeech, 2016.

Cross-domain intention/bot hierarchySun et al., “An Intelligent Assistant for High-Level Task Understanding,” in IUI, 2016.Sun et al., “AppDialogue: Multi-App Dialogues for Intelligent Assistants,” in LREC, 2016.Chen et al., “Leveraging Behavioral Patterns of Mobile Applications for Personalized Spoken Language Understanding,” in ICMI, 2016.

Cross-domain information transferringKim et al., “New Transfer Learning Techniques For Disparate Label Sets,” in ACL-IJCNLP, 2015.

30

FIND_RESTAURANTrating=“good” rating=5? 4?

HotelRest Flight

Travel

Trip Planning

Deep Learning for LU

IOB Sequence Labeling for Slot Filling

Intent Classification

31

𝑤0 𝑤1 𝑤2 𝑤𝑛

ℎ0𝑓 ℎ1

𝑓ℎ2𝑓

ℎ𝑛𝑓

ℎ0𝑏 ℎ1

𝑏 ℎ2𝑏 ℎ𝑛

𝑏

𝑦0 𝑦1 𝑦2 𝑦𝑛

(a) LSTM (b) LSTM-LA (c) bLSTM-LA

(d) Intent LSTM

intent

𝑤0 𝑤1 𝑤2 𝑤𝑛

ℎ0 ℎ1 ℎ2 ℎ𝑛

𝑦0 𝑦1 𝑦2 𝑦𝑛

𝑤0 𝑤1 𝑤2 𝑤𝑛

ℎ0 ℎ1 ℎ2 ℎ𝑛

𝑦0 𝑦1 𝑦2 𝑦𝑛

𝑤0 𝑤1 𝑤2 𝑤𝑛

ℎ0 ℎ1 ℎ2 ℎ𝑛

Deep Learning for LU

Joint multi-domain intent prediction and slot filling

Information can mutually enhanced

32

Hakkani-Tur, et al., “Multi-Domain Joint Semantic Frame Parsing using Bi-directional RNN-LSTM,” in Interspeech, 2016.

ht-1 ht+1ht

taiwanese

B-type

food please

O O

hT+1

EOS

FIND_REST

Slot Filling Intent Prediction

Contextual LU (Chen et al., 2016) 33

just sent email to bob about fishing this weekend

O O O OB-contact_name

O

B-subject I-subject I-subject

U

S

I send_emailD communication

send_email(contact_name=“bob”, subject=“fishing this weekend”)

are we going to fish this weekend

U1

S2

send_email(message=“are we going to fish this weekend”)

send email to bob

U2

send_email(contact_name=“bob”)

B-messageI-message

I-message I-message I-messageI-message I-message

B-contact_nameS1

Domain Identification Intent Prediction Slot Filling

Chen et al., "End-to-End Memory Networks with Knowledge Carryover for Multi-Turn Spoken Language Understanding,” in Interspeech, 2016.

Supervised v.s. Reinforcement

Supervised

Reinforcement

Hello

Agent

……

Agent

……. ……. ……

Bad

“Hello” Say “Hi”

“Bye bye” Say “Good bye”Learning from teacher

Learning from critics

34

Scenario of Reinforcement Learning

Agent learns to take actions to maximize expected reward.

35

Environment

Observation ot Action at

Reward rt

If win, reward = 1

If loss, reward = -1

Otherwise, reward = 0

Next Move

Dialogue Policy Optimization

Dialogue management in a RL framework

36

U s e r

Reward

Observation Action

Environment

Agent

Natural Language Generation Language Understanding

Dialogue Manager

Slides credited by Pei-Hao Su

Optimized dialogue policy selects the best action that can maximize the future reward.Correct rewards are a crucial factor in dialogue policy training

Reinforcement Learning Based BotUser goal: Two tickets for “the witch” tomorrow 9:30 PM at regal meridian 16, Seattle.

37

RULE BASED SYSTEMUser: Can I get 2 tickets for the witch?Agent: Which theater do you want?User: Regal meridian 16Agent: What time would you like to see it?User: 9:30 pmAgent: What date would you like to see it?User: Tomorrow!Agent: How many people are you?User: Just two adults.Agent: What is your location?User: Seattle!Agent: What kind of video format do you like?User: Oh, any kind is okay. Agent: Do you have any price constraint?User: No, I don’t care.Agent: Great - I was able to purchase 2 tickets for you to see the witch tomorrow at regal meridian 16 theater in Seattle at 9:30 pm.User: Thank you.Agent: Thank you!

REINFORCEMENT LEARNING SYSTEMUser: Can I get 2 tickets for the witch?Agent: What date would you like to see it?User: TomorrowAgent: Which theater do you prefer?User: Regal meridian 16Agent: What is your location?User: SeattleAgent: What time would you like to see it?User: 9:30 pm.Agent: Great - I was able to purchase 2 tickets for you to see the witch tomorrow at regal meridian 16 theater in Seattle at 9:30 pm.User: Thanks.Agent: Thanks!

The system can learn how to efficiently interact with users for task completion

Li et al., "End-to-End Task-Completion Neural Dialogue Systems,” in arXiv, 2017.

Brain Signal for Understanding

Misunderstanding detection by brain signal

Green: listen to the correct answer

Red: listen to the wrong answer

38

Detecting misunderstanding via brain signal in order to correct the understanding results

Sridharan et al., “NeuroDialog: An EEG-Enabled Spoken Dialog Interface,” in ICMI, 2012.

Video for Intent Understanding 39

Proactive (from camera)

I want to see a movie on TV!

Intent: turn_on_tv

Sir, may I turn on the TV for you?

Proactively understanding user intent to initiate the dialogues.

App Behavior for Understanding Task: user intent prediction

Challenge: language ambiguity

User preference Some people prefer “Message” to “Email” Some people prefer “Outlook” to “Gmail”

App-level contexts “Message” is more likely to follow “Camera” “Email” is more likely to follow “Excel”

40

send to vivianv.s.

Email? Message?

Communication

Considering behavioral patterns in history to model understanding for intent prediction.

Chen et al., “Leveraging Behavior Patterns of Mobile Applications for Personalized Spoken Language Understanding,” in ICMI, 2015.

Evolution Roadmap 41

Single domain systems

Extended systems

Multi-domain systems

Open domain systems

Dialogue breadth (coverage)

Dia

logu

e d

epth

(co

mp

lexi

ty)

What is influenza?

I’ve got a cold what do I do?

Tell me a joke.

I feel sad…

Intent Expansion

Transfer dialogue acts across domains

Dialogue acts are similar for multiple domains

Learning new intents by information from other domains

CDSSM

New Intent

Intent Representation

12

K:

Embedding Generation

K+1

K+2<change_calender>

Training Data<change_note>

“adjust my note”:

<change_setting>“volume turn down”

The dialogue act representations can be automatically learned for other domains

postpone my meeting to five pm

Chen et al., “Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models,” in ICASSP 2016.

42

Policy for Domain Adaptation

Bayesian committee machine (BCM) enables estimated Q-function to share knowledge across domains

QRDR

QHDH

QL DL

Committee Model

The policy from a new domain can be boosted by the committee policy

#dialogues

rew

ard

Gašić, et al., “Policy Committee for Adaptation in Multi-Domain Spoken Dialogue Systems,” in ASRU, 2015.

43

Evolution Roadmap 44

Knowledge based system

Common sense system

Empathetic systems

Dialogue breadth (coverage)

Dia

logu

e d

epth

(co

mp

lexi

ty)

What is influenza?

I’ve got a cold what do I do?

Tell me a joke.

I feel sad…

High-Level Intention for Dialogue Planning

High-level intention may span several domains

Schedule a lunch with Vivian.

find restaurant check location contact play music

What kind of restaurants do you prefer?

The distance is …

Should I send the restaurant information to Vivian?

Users can interact via high-level descriptions and the system learns how to plan the dialogues

Sun, et al., “An Intelligent Assistant for High-Level Task Understanding,” in IUI, 2016.

45

Empathy in Dialogue System

Embed an empathy module

Recognize emotion using multimodality

Generate emotion-aware responses

46

Emotion Recognizer

vision

speech

text

Fung et al., “Towards Empathetic Human-Robot Interactions,” in arXiv, 2016.

Conclusion

47

Conclusion

The conversational systems can manage information access via spoken interactions

A domain is usually constrained by the backend database

Semantic schema should be predefined

Cross-domain knowledge and intention is difficult to handled

Human-like dialogue system

Robustness error handling via interactions / reasoning

Dialogue coverage domain switching / open-domain system

Dialogue complexity common sense and empathy

48

Q & ATHANKS FOR YOUR ATTENTION!

49