From Smart House to Smile House:

Robo Remote for HEMS and its Controlling of Open IoT Space

based on ECHONET Lite

Dept. of Home Electronics yamalab.com

Yoichi Yamazaki 山崎 洋一

Dept. of Home Electronics Faculty of Creative Engineering

KAnagawa Institute of Technology

神奈川工科大学

創造工学部ホームエレクトロニクス開発学科

yamazaki@he.kanagawa-it.ac.jp

URL: http://yamalab.com 1/19

Dept. of Home Electronics yamalab.com

KAnagawa Institute of Technology (KAIT)

“The Sagami Robot Industry Special Zone

http://sagamirobot.pref.kanagawa.jp/ 2/19

Dept. of Home Electronics yamalab.com

KAnagawa Institute of Technology (KAIT)

Students undergraduate graduate

Engineering 1,302

Information Technology 1,934

Creative Engineering 852

Applied Bioscience 877

Nursing Care 81

Total 5,046 184

Staffs

Prof. 101

Associated Prof. 65

Lecturer / Assistant Prof. 27

Other educational staff 10

Total 203

(2015)

http://www.kait.jp/english/

3/19

IoT R&D center

Dept. of Home Electronics yamalab.com

Mission:

Create human-and-environmentally-friendly smart life

with Leading-edge (Home) Energy Management System

and Next Generation Home Appliances

Dept. of Home Electronics

Bed roomLiving roomKitchen

Smart House

Center

Open IoT Space

http://www.he.kanagawa-it.ac.jp/

4/19

Dept. of Home Electronics yamalab.com

IoT (Internet of Things)

User InternetInformation

terminal

IT: User-driven service

User InternetInformation

appliances

IoT: Things-driven service

5/19

Dept. of Home Electronics yamalab.com

What is Interaction?

message

message

Interaction

Cognition

Behavior / Expression

6/19

Dept. of Home Electronics yamalab.com

Minimal Expression: verbal & nonverbal

verbal

nonverbal

speech

facial expressioneye expression

gesture posture distance

Unconscious

Conscious

7/19

Dept. of Home Electronics yamalab.com

Qualitative expressions

of quantitive information

Casual Visualization

Behavior expressions

of Robot and systems

8/19

Dept. of Home Electronics yamalab.com

AHOGE:

Antenna Hair-type Object for Generating Empathy

dissimilar from human

intuitive understandable expression

robotic element

AHOGE9/19

Dept. of Home Electronics yamalab.com

Qualitative Expressions for HEMS

angersurprisehappiness sadness

WARNING

DANGER

CAUTION

SAFETY

relax

NEUTRAL

3 scale1 scale

positive negative

10/19

Dept. of Home Electronics yamalab.com

monthly electricity statuselectricity consumption

high-power appliance

Qualitative expressions using an AHOGE

are effective for human memory11/19

Speech

Voice

Facial expression

Eye expression

Gesture

Posture

Distance

Individual characteristic

Emotion understanding

Intension understanding

Atmosphere understanding

Universal knowledge

Customized knowledge

Deep level inference

Emotion

Intension

Personality

Existence

Atmosphere control

Dept. of Home Electronics yamalab.com

Deep Level Recognition / Deep Expression

Deep level

Surface level

Speech

Voice

Facial expression

Gesture

Emotion understanding

Intension understanding

Atmosphere understanding

Universal knowledge

Customized knowledge

Deep level inference

Recognition

[Y. Tang, 2013] [Y. Yamazaki, 2014]

Expression

12/19

Dept. of Home Electronics yamalab.com

Smile House and its Application for Smile Life

Atmosphere

control

Robo RemoteHome appliances

cooperation

Tele-communicative

Anywhere Home

Home security

Mental & physical

support for Elderly

based on Memory Key

Robotic

Visualization

Deep Expression

Attentive

Energy saving

13/19

Dept. of Home Electronics yamalab.com

Robo Remote for HEMS

Echonet Lite base Robot Remote system

Casual Control

Wi-Fi rooter

Air conditioner ×4

Room lighting ×6

TV

HEMS server（tablet）

Sensor unit ×3 14/19

Air conditionerHue

(bridge)

Hue

(lighting)

Kinect v2

[Gesture Remote]

ECHONET Lite-enabled device

laptop

Home network router

JSONP ECHONET Lite

IR

Remote

[Smart house control]

Dept. of Home Electronics yamalab.com

Casual Remote for HEMS

[Robo Remote]

TV

Kadecot API server

(tablet PC)

15/19

Dept. of Home Electronics yamalab.com

Robo Remote for HEMS

https://www.youtube.com/watch?v=hxEGOGVckj4

16/19

Dept. of Home Electronics yamalab.com

Remote living connection for elderly of nursing home relocation

Casual Connection

“Anywhere Home”: Remote Home Connection

Casual swich

Environment information by robot

Facial expression /Motion/ Speech (dispose raw data)

Casual &Deep Expression

Remote accessible feeling

Nursing facility

Motion sensor

Companionrobot

Uncompressed environment

Presence compression

Multiplex-image hidingdisplay

Robot Media/ Sensor

Elderly

Home

Family

NAO(Aldebaran Robotics）

（14,000, 000 JPY FY2014-2015, KAIT ）

17/19

Dept. of Home Electronics yamalab.com

Personal characteristic

motion expression of working

Deep Expression for Compression of Existence

Combinative expression

of gestures and sounds

for Deep emotional Expression

NAO(Aldebaran Robotics）

18/19

Dept. of Home Electronics yamalab.com

Conclusion

Casual

Visualization

Casual

Control

Casual

Communication

Casual

Connection19/19