Development of an AIoT Senior Care Bedroom · Internet of Things, Artificial Intelligence, Big Data… Wi-Fi • Publish / subscribe / topic IoT is a Line/WeChat group • M2M (Machine

© GRC, YZU. All Rights Reserved. 1

Development of an AIoT Senior Care Bedroom

RESEARCH SEMINAR SERIES

MONASH UNIVERSITY MALASYSIA, SCHOOL OF BUSINESS

CEO

Yeh-Liang Hsu, 徐業良Professor, Mechanical Engineering Department

Director, Gerontechnology Research Center

Yuan Ze University, Taiwan 元智大學

Editor-in-Chief, Gerontechnology

2019/12/04

© GRC, YZU. All Rights Reserved.

3G’s for the aged society: Gerontology, Geriatrics, Gerontechnology• It is only natural to consider applying technologies for the care of older adults…

• First International Congress on Gerontechnology, 1991

• International Society for Gerontechnology, 1997

“Design technology and environment for independent living and social participation of

older persons in good health, comfort and safety”

• Scope of Gerontechnology: Health, Housing, Mobility, Communication, Leisure, Work

2


A design approach to gerontechnology:

Creativity➔Prototype➔Products➔Sales

3

Est. 2003

Research & Development

Industry-academia collaboration

Commercialization

GerontechnologyIndustry-Academia

Alliance

SEDA G-Tech

Smart LivingIoTGerontechnology

TransdisciplinaryEducation

Internationalacademic exchange

International Society for

Gerontechnology

New Taipei City

Design ThinkingTransdisciplinary

education program

Innovative Product Development Program

福祉科技與服務管理學刊

GerontechnologyOfficial Journal of the ISG

Mission: To carry out a successful industrial and educational experiment

Gerontechnology is only valuable if we can bring research to daily applications


Persuasive technology: How technology is designed to change behaviors of the users

• A new behavior was created in 2007

• User Centered Design: NeedMotivation

4

More fun than functions…


Wi-Fi

Bluetooth

Android App

Wi-Fi

3G/4G

Cloud Server

MQTT Broker

Bluetooth

IoT gateway

If telehealth systems connect older adults with family members

5

BP/BG Measurement

The son is the trigger… Care, interaction,

connection, in addition to health monitoringtemperature/humidity

Indoor

localization

https://www.facebook.com/enmeilai?fref=ts


Bringing research to daily applications: More DESIGN THINKING!

• Empathy: User centered, persona…

• Define the right problem: What, how, Why

• Think like a boss… (not like a professor…)

6

IDEO (http://www.ideo.com) CEO Tim Brown

Proud to be a designer!

PhD 1992, Stanford, ME, Design Division


Internet of Things, Artificial Intelligence, Big Data…

Wi-Fi

• Publish / subscribe / topic➔

IoT is a Line/WeChat group

• M2M (Machine to Machine) communication

• The light started to have

some AI…

• The bed started to collect

big data…


Supporting the caregivers --Use smart technologies to provide real-time status and

long-term patterns of the older adults

日本総務省の「就業構造基本調査」によると、介護・看護を理

由に離職した人の数は、2016年10月～2017年9月の1年間で、9万

9100人

According to the “Basic Survey on Employment Structure,” by

Ministry of Internal Affairs Japan, the number of employees who left

work because of caring or nursing was 99,100 during the year from

October 2016 to September 2017. 2019/06/06 FNN PRIME

8


Transfer familiar artifacts in the homes into IoT devices

• WhizPad is an extremely comfortable

mattress capable of motion sensing

• Pressure-relieve material for prevention

of pressure ulcer

9

WhizPadWeb UI

Mobile App Taipei Veteran Hospital

• In Taiwan 2017, 22.5% hospital falls

occurred when patients leave the beds

• Three-stage leave bed alert (sit-up, sitting at

the side, leave bed) for fall prevention.

• Four sensing areas / rule based

• Private cloud IoT


Field trial result in a medical center – paper publication to support the product

• We recruited 254 participants who were at

risk of bedsores from the ICU of a hospital,

with 127 in the control group and 127 in the

intervention group.

• In addition to the standard nursing care,

participants in the intervention group used

the motion-sensing mattress, which can

detect the bed activities during their

hospitalization.

• 13 (10.2%) participants in the control group

developed bedsores while only 2 (1.6%) in

the intervention group developed bedsores,

showing a significant reduction of 8.6%

(P=0.003).

• After adjusted for some confounding

variables, we found that the likelihood

of development of bedsore in the

intervention group was reduced by 90%

(OR=0.10, 95% CI: 0.02, 0.51, P=0.005)

compared with that in the control group.

• The average timing to develop bedsore

in participants using the motion-sensing

mattress was 7.5 days and was

postponed by 2.3 times than that in

participants who received standard

nursing care which was 3.3 days.

10


2nd generation: Machine Learning to capture the intention of leaving bed

• 1000+ learning samples from 31

testers in 7 weight groups

• 30 sensing areas, classification of

Lying/sitting/left edge/right edge

• Edge AI: NeuralNet model

deployed on chip

Testing accuracy: 96.0% (360

test examples)

• Individual patient management

3-stage leave bed alert

Long pressure area alert

• Using 2 AAA batteries

• Amazon Web Service (AWS) for

data storage and Cloud AI: Daily living pattern, sleep quality…11

© GRC, YZU. All Rights Reserved. 12

43.05%

55.55%

59.72%

42.36%

46.52%

47.22%

64.58% Daily living pattern obtained from WhizPad


Accumulating the norm for in-bed activities

• 1 week

• In-bed 51.09%

• 2 weeks

• In-bed 45.88%

13


Use 70% as threshold to define “normal activities”

14

Normally, the older adult

1. Gets up no earlier than 05:50

2. Gets up no later than 07:30

3. Not in bed during the day

4. Goes to bed no earlier than 19:00

5. Goes to be no later than 20:40

1. In-bed longer (64.58%)

2. Got up later than normal at 07:50

3. Back to bed multiple times during

the day

4. Went be bed earlier at 17:30

5. Left bed multiple times during the

night

1 32 4 5

9/22: 64.58%

1

32 4 5

In-bed 45.88%

AI: Learning ➔ Prediction


0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

00

:00

01

:00

02

:00

03

:00

04

:00

05

:00

06

:00

07

:00

08

:00

09

:00

10

:00

11

:00

12

:00

13

:00

14

:00

15

:00

16

:00

17

:00

18

:00

19

:00

20

:00

21

:00

22

:00

23

:00

00

:00

2018/06/21甘泉伯伯活動頻率及模式

Motion sensing WhizCarpet: Fall detection and mobility monitoring

15

week2

5 0

39 0

117651 6677 42 0

55424 4310 37 0

88 532 10648 3152 571 1590

106 136 166 309 369 493

10 92 104 27 38 124

10 58 38 220 410 156

0 14 19 56

1 0 34 143

0 28 71 436

2 10 50 368

Easy and flexible set up;

soft surface reduces risk

of fracture when falling

Real time position display,

fall detection and alert ➔Mobility and pattern analysis,

abnormal pattern alert

Wandering behavior, path, hot spot

analysis of demented patients➔


Bluetooth devices cannot M2M: WhizConnect to integrate Bluetooth products

16

WhizConnect

Long term health

and living pattern

Vital signs

Environmental data

WhizSeriesReal time status

and event alerts


WhizTouch: IoT call button for

bedridden older adults

• Big button; mobile phone as receiver;

green light to confirm the call

• But…

• Internet usage is low for older adults

(31.1% /8.6%@2018). High monthly fee

for Internet service is not acceptable.

• Connecting the IoT product to home Wi-

Fi AP has been a major usability issue;

Wi-Fi coverage is often poor in bedroom

• LPWAN: Sigfox, LoRa?

Narrow Band IoT: the last mile

• Sim card connects directly to NB-IoT base

station; used anywhere without setup

• NB-IoT based WhizConnect to bring smart

homecare products to the homes…

17

IoT care products at home?

Speech videos/(2019-10-23) Whiztouch 40 sec video with EN subtitles.mp4


2015: Internet of Things! 2017: NB-IoT 2019: Big data / AI

• Consider B2B2C business model

18

Edge AI:

event identification

Cloud AI:

pattern recognition

3rd Party service cloud

Commercial Bluetooth

healthcare products

© GRC, YZU. All Rights Reserved. 19ACTIVITY ROOM

OLD THEATRE

OLD DAYS

MUSIC CORNER

INTERACTIVE WALL

LIFE TREES

TRAIN STATION

Banciao veteran dementia

nursing home

Speech videos/(2018-01-24) 板橋火車站_02.mp4

Speech videos/(2018-01-24) 板橋火車站_01.mp4


We are designers!UX: User Experience Design

20

The world needs transdisciplinary talents

Find that Trigger－Caregivers are the best designers!

Yeh-Liang Hsu, [email protected]

Thank you

Transdisciplinary learning

Real world problem and experience

“Design” is our common language!Design starts with a problem. Design ends with a solution.

DESIGN IS A PROBLEM SOLVING PROCESS

Documents

Development of an AIoT Senior Care Bedroom · Internet of Things, Artificial Intelligence, Big Data… Wi-Fi • Publish / subscribe / topic IoT is a Line/WeChat group • M2M (Machine