Grounding to the Real World - Architecture for Ubiquitous...

Grounding to the Real WorldGrounding to the Real World-- Architecture for Ubiquitous Architecture for Ubiquitous

Computing Computing --ISMIS 2003ISMIS 2003

Hideyuki NakashimaHideyuki Nakashima中島秀之中島秀之

Cyber Assist Research Center, AISTCyber Assist Research Center, AIST

h.nakashima@aist.go.jp 2

Background

• Information technologies are advanced to the point where it becomes feasible to assist ordinary people enjoy the full capability of information processing powers.– Although the Internet opened up huge

possibilities for daily use, users still have to learn a lot of special concept to use it.

Background 2

• With the help of various sensing technologies, it is time to design arich, keyboard-less interface and provide an ubiquitous computing environment for ordinary people in everyday life. Ubiquitous agent technology plays essential role here.

IntroductionIntroduction

TICKET

Take the next train.Change to A line at Ueno station.The fare is YY yen.

I want to go to Maebashi station.

Good Old DaysGood Old Days

券売機券売機券売機

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

券売機

・・・・・

When she getsolder...

券売機券売機券売機券売機

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

How much does it cost?

What is the best route to Maebashistation?

券売機券売機券売機券売機

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

●●●●

××××

△△△

○○○○○

▲▲▲▲

□□□ 210

※※※※ 400

※●※●※

×○×

△■△■

○□○□

▲×▲×▲

□◎□◎ 210

■※■ 400

あっ！

Please go through..

The next train will leave from platformNo 2 to your right.

TURN RIGHT TURN RIGHT

speaker

Connection to Internet

Time table server

Seat reservation system

My-button

LED display

Station master computer

Station LAN

TokyoRF tag

Unawareable environmentUnawareable environmentIR communicator

Position sensor

Ubiquitous Computing• Omnipresence of computing power

• Human centered services– The opposite direction from virtual reality where the

reality is reconstructed in the digital world

– Computers come out to the real world

• Gadgets are hidden– calm computing– disappearing computer

– oxygen

– unawarable computers

MIT Oxygen: Intelligent Space• Context aware environment: computation “in the air”

Stereo Cameras: to track persons and objects

Active Cameras: for video-conferencing and meeting archive support

Microphone Arrays: beam-forming for untethered spoken interface

Before:People had to learn to use machines

Computers in the Digital WorldComputers in the Digital World

People in the Real WorldPeople in the Real World

Internet schoolInternet schoolPC schoolPC school

After:Grounding is the key technology

Digital WorldDigital World

Real WorldReal World

groundinggrounding

“Grounding”

• Connection between digital data and real things– similar to “symbol grounding problem”

– (but not the same)

• Total grounding is not necessary nor possible– utilization of artificial means or human aid

From WWW to HNM

• Wherever, Whenever, Whoever

→ Here, Now, Me (situatedness)

• At the same time:

Just as much as you want to know

→ intelligent information filter

• And,

Just as much as you want to be known

→ privacy

Location based communicationLocation based communication

Human Centered Infrastructure

ID-based communication

Location-based communication

Broadcasting satellite３６０００ｋｍ～７８０ｋｍ

High-altitude aeroplane/ship２０ｋｍ

broadcastbroadcast

Sensor networkSensor network

Communication Methods

Local 1-10 meters global

Magnetic cardMagnetic cardICIC--cardcard

NetworksNetworksPhonesPhones

BroadbandHigh-speed

LowLow--powerpowerShortShort--rangerange

LocationLocation--basedbased

IDID--basedbased

Advantages of Location Based Communication

<x, y, z, t>

• Location is one of the essential keys for grounding the digital (logical) world to the real world.– Situated information processing and services

• Interaction without ID– Possibility of PRIVACY protection

– For better human interface

– Application to emergency situations

Various location-based communication(addressing) methods

(cricket)*(mobile phones)*

(cricket)*(mobile phones)* carrier pigeoncarrier pigeon

Logical addressing“passenger at 12C”

Positioning by server

Self-Positioning

Physical addressing“you”

(bluetooth)*DSRC

(bluetooth)*DSRC CoBITCoBIT

*positioning only

Security vs. Privacy vs. Convenience

securitysecurity privacyprivacy

convenienceconvenienceThey basically contradict each other with some exceptions

Needs technology for privacy protection

amountofservice

privacy protection

not zero

Dynamic/Voluntary Control of the Balancing Point is Necessary

• Trade-off between the amount of services and the amount of personal information

• Let the user set the point!

Privacy Security

AgentAgent--based Network based Network ArchitectureArchitecture

Ubiquitous Computing

How do we access and control devices

Intuitive User Interfacethat uses real world propertiessuch as location(point and click)

Why Agents?

• Aids for human cognition– interface

• Resource control– cognitive resources

– devices• multiple users and• multiple devices

Cognitive Resource Control

• SmartKom example:– cognitive resource of human

• when running terse• when walking• when standing verbose

DFKI: SmartKom

• positioning– indoor: light beacon

– outdoor: GPS

・Maintains who reserved which time slot・Controls remote access・The reserver has the access right

homeVCR

toilet

bathliving

air conditionerthermometer

thermometer

・The access right belongs to who reserved・Access right can be delegated・The device may act as a mediator

homeserver

RFID tagRFID tab

RFID tag

OK Dad, go ahead...

Access Control of Home AppliancesBased on Location and Priority

My son has a reservation. Let’s ask him.

someone is using the air conditioner・change reservation

or, delegate authority

・Deny remote access when someone is near-by

・communication center for the family

CONSORTS

InferenceEngines

Personal Agents

ServiceAgentsUser1 Usern

Communication and censoring

devices

Physical WorldContentServer

Devicewrapper

Agent communicationAPIphysical link

Spatial information database- sensor-based geometrical representation- ontology- location information

Spatio-Temporal Reasoning Agent

Camera, Wireless LANCoBIT, etc.

User modelmanager

Physical Grounding

Web Services

Spatio-temporal Inference Agent in Consorts

spatio-temporal map(Segment Representation)

Inference engine:spatio-temporal relations and their changes (distance, phase, connection etc.)

device wrapper agents

personal agents,service agents

request, reply, inform

sensor data

Spatial informationdatabase

interpretation

Sensor information at location (x,y) mapped onto a map

Geometric representation of sensor information

Museum

1st Floor

Entrance

3rd Floor2nd Floor

Room CRoom A Room B

Painting BPainting A Painting C

Hierarchical representation of physical space

Painting D Painting D

Ubiquitous Computing with FIPA-Agents

• Direct connections to sensors are not good

• Agents mediate communication of sensor information

• Agentcities Network• http://www.agentcities.net/

– EU centered worldwide research network based on FIPA standard

– 134 platforms• EPFL, BT, Motorola， etc.

– Tokyo.Agentcities.Net (CARC)• http://www.carc.aist.go.jp/agentcities/

A Snapshot of Our Museum Information Support System

Main Window (Museum Map)

Information Window

Message WindowMessage Window

Human Icon: Current Location

Yellow Area: Service Zones

Blue Line: Users’ Trajectories

UBKit: Software construction tool kits

middleware

infrared

SOAP DNS

network

IEEE802.11 BluetoothMicroservers

voice positioning

real world

CONSORTSCONSORTS

Micro servers + Optical Communication forIntegrated Remote Control System

Ultra small low energy micro server(Linux) coin size

Self configuring plug and play network

positioning system

voice recognition

sensor networkinfrared

Lightweight middleware (service directory, SOAP)

sensorgateway

UBKit (Ubiquity Building Toolkit)

Integrated control of all home applianceswith intuitive interface

Connection toConnection tothe Semantic Webthe Semantic Web

Annotation as Grounding

Content(Real World)

SemanticWeb

AnnotationAnnotation

Semantics forComputers

Sharing semanticsbetween computers

and human(Grounding)

Multimodal Content Processing

• Grounding linguistic content to the physical world– annotation with location– interaction through CoBIT

• User modeling– record of movement– record of inquiries

• Summarization and interactive presentation

Cyber Assist ProjectCyber Assist Project

The goal:Grounding the digital world to the real worldfor “here, now, and to me” service

informationinformation

Digital worldDigital world

Real world

goodsgoods

humanhumanSemantic WebSemantic Web

Ubiquitous ComputingUbiquitous Computing

My Button• One button

– Do what I want here, now.

– No button?Activation by

brain wave?

• The ultimate man-machine interface？– Intelligent

personal agent– Sharing

situations

command button

fingerprint recognizer

reflectors photo detector

microphone

speaker

Optical Bi-directional Communication

server

cameraLED

terminal

•Download•voice

•power supply

•Upload•position•direction•motion•(ID)

My Button V1: CoBIT

• Compact Battery-less Information Terminal

• Terminal: solar cell + reflector + earphone

My Button V1

• Server (environment) side: LED + camera(s)

青森産のリンゴです。一個200円です。これ買いますか？

ありがとうございます。

アラスカ産のシャケです。

これを買いますか？

となりにはサンマがあります。

Application to Markets

Environment

The Image of the Target System

device

Intelligent Content

Sensors and actuators

Situationrecognition

Query/orders

information

request

Information retrievalInformation processing

Voicerecognition

Semantic query

User modeling

Photo/radio communication

vision

Multiagent technology

Application toEmergency rescue

ＷｉｒｅｌｅｓｓＬＡＮ

Ｍｙ-Ｂｕｔｔｏｎ

i-lidar

camera

intelligent contentsemantic web

voicecommand/query

voiceinformation

ＩＲ

The next train will leave from platformNo 2 to your right.

ＣｏＢＩＴ

The evolution of the communication technologies in Star Trek

CoBITmy buttons

conventional

ApplicationsApplications

Mass Support

• Information support for the whole city

• Examples– Global Car Navigation

• Optimal route planning using the current positions and destinations of all cars in the area

• Coordinated signal control

– Theme Park total coordination

– On-line Government• Direct democracy with the help of discussion and

decision support system

Special project for earthquake disaster mitigation in urban areas(Ministry of Education and Science)

• Ad hoc network for emergency communications

• Multiagent simulation for– evacuation

– rescue

Real-time dynamic ad-hoc network for large-scale disasters (earthquake)– Effective in

congested areas

– Alternative to other communication infrastructures• First 10-30 min.

– Connection to other lines when possible

Grounding the Simulation

KernelGIS

Agents

Viewers

ComponentSimulators

Portable devices Robots

Rescue team

Real-worldsensors

Rescue HQ

Summary

• We have to connect the digital world to the real world→ GROUNDING

• Multiagent concept is the key to ubiquitous computing– Grounding

– The core architecture

– Ubiquitous agents for human support– Many MA applications

Grounding to the Real World - Architecture for Ubiquitous...

Documents

Grounding Guide

Grounding Kits - RODAN · Grounding kits. FIMO Grounding Kits offer multiple solutions for the earthing of coa-xial cables, hybrid cables and elliptical and rectangular waveguides

Grounding and Grounding Fault analysis

Grounding System - electric-factories.com · • Grounding System • Lightning Protection • Conduit Accessory. 3 บริษัท วสุธร ไฟฟ้าโรงงาน

20. GROUNDING SISTEM.doc

Ubiquitous learning

Membuat Sistim Grounding Sederhana

Grounding System

Ubiquitous approfondimento

GROUNDING FOR OFFSHORE WIND FARM ELECTRICAL SYSTEMvbn.aau.dk/files/31870118/GROUNDING FOR OFFSHORE WIND FARM... · institute of energy techonology grounding for offshore wind farm

Perencanaan Grounding Gardu Induk

Grounding Kits Catalog

Tentang Grounding

Ubiquitous Environment

Sistem Pembumian (Grounding System)

Grounding Pada Teg.tinggi

Sistem Proteksi Petir Dan Grounding

BAB v Grounding

Application Development for Mobile and Ubiquitous …ts2/admuc/lecture1415/9. Cross...Department of Computer Science Institute for System Architecture, Chair for Computer Networks

Basic Grounding