Embed Size (px)
Citation preview
Robots Learning Like Babies
Andrea Censi
Laboratory for Information and Decision SystemsMassachusetts Institute of Technology
http://censi.mit.edu - [email protected]
時代基⾦金會暨⿇麻省理⼯工學院MIT全球產研計劃台灣年會 2015 MIT ILP-EPOCH TAIWAN SYMPOSIUM
Taipei, July 28, 2015Meet the Future of Robotics and Machine
Learning in Innovation Economy
Outline
1. What is learning?
2. What is the value proposition for robotics?
3. What are the challenges in designing learning systems?
2
What is learning?
‣ It’s about using data- …to make predictions- …to make decisions
3
Examples of learning
4
Examples of learning
‣ Netflix
5
Examples of learning
‣ Netflix
5
Examples of learning
‣ Netflix
5
Examples of learning
‣ Netflix
5
Examples of learning
‣ Netflix suggests you a good movie to watch.
6
Examples of learning
‣ Netflix suggests you a good movie to watch.
6
‣ Example of a “recommender” system- other classical example: shopping suggestions
Examples of learning
‣ Netflix suggests you a good movie to watch.
6
‣ Example of a “recommender” system- other classical example: shopping suggestions
‣ Netflix Challenge (2007-9): $1M for help in making the learning algorithm better.
7
algorithminput output
8
learning algorithm
dataa decision rule
9
learning algorithm
data
9
learning algorithm
dataJohn, “Godfather”,
Jane, “Godfather”,
Jane, “Terminator”,
...
9
learning algorithm
dataJohn, “Godfather”,
Jane, “Godfather”,
Jane, “Terminator”,
...
you
rating?
a movie
10
learning algorithm
data
10
learning algorithm
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.30
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.010.30
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.010.30 0.50
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.010.30 0.50
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.250.010.30 0.50
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.250.010.30 0.50
parameters (high dimensional)
data
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.250.010.30 0.50
parameters (high dimensional)
data best* decision rule
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.250.010.30 0.50
parameters (high dimensional)
data
* in the family considered
best* decision rule
10
learning algorithm
candidates decision rules
error on test data 0.10 0.15 0.250.250.010.30 0.50
parameters (high dimensional)
data
* on the data available* in the family considered
best* decision rule
11
Examples of learning
12
‣ Nest Learning Thermostat
Examples of learning
‣ Nest Learning Thermostat - acquired by Google ($2B)- see also: Ecobee in the Apple ecosystem
13
Examples of learning
‣ Nest Learning Thermostat - acquired by Google ($2B)- see also: Ecobee in the Apple ecosystem
13
~ NT 300
Examples of learning
‣ Nest Learning Thermostat - acquired by Google ($2B)- see also: Ecobee in the Apple ecosystem
13
+ learning =
~ NT 300
Examples of learning
‣ Nest Learning Thermostat - acquired by Google ($2B)- see also: Ecobee in the Apple ecosystem
13
+ learning =
~ NT 300 NT 8000
14
March 2015 in Boston
14
March 2015 in Boston
Boston, March 2015
14
March 2015 in Boston
Boston, March 2015
“snow”fun only on the first day
14
March 2015 in Boston
Boston, March 2015
“snow”fun only on the first day
15
16
17
learning algorithm
data
17
learning algorithm
datasensor data time series
17
learning algorithm
data
“I’m cold”, “I’m hot”sensor data time series
17
learning algorithm
data
turn on/off
temperature date
“I’m cold”, “I’m hot”sensor data time series
18
learning algorithm
data
a control policy
turn on/off
temperature date
“I’m cold”, “I’m hot”sensor data time series
18
learning algorithm
data
a control policy
turn on/off
temperature change
a model of the system
turn on/off
temperature date
“I’m cold”, “I’m hot”sensor data time series
18
learning algorithm
data
a control policy
date
desired temperature
a model of the task
turn on/off
temperature change
a model of the system
turn on/off
temperature date
“I’m cold”, “I’m hot”sensor data time series
19
‣ What is a robot?
19
‣ What is a robot?- has sensors
19
‣ What is a robot?- has sensors- has actuators
19
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
19
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
‣ QUIZ: Which one is the robot?
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
A
‣ QUIZ: Which one is the robot?
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
A B
‣ QUIZ: Which one is the robot?
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
A B C
‣ QUIZ: Which one is the robot?
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
A B C
‣ QUIZ: Which one is the robot?
D
20
‣ What is a robot?- has sensors- has actuators- interacts with the physical world
A B C E
‣ QUIZ: Which one is the robot?
D
21
learning algorithm
data
21
learning algorithm
datasensorimotor experience
21
learning algorithm
data
task examples (given by user )
sensorimotor experience
21
learning algorithm
data
a control policy
command
observations desired state
task examples (given by user )
sensorimotor experience
21
learning algorithm
data
commands
change in state
a model of the system
a control policy
command
observations desired state
task examples (given by user )
sensorimotor experience
21
learning algorithm
data
observations
desired state
a model of the task
commands
change in state
a model of the system
a control policy
command
observations desired state
task examples (given by user )
sensorimotor experience
Examples of learning
‣ Kiva Systems (now AmazonRobotics)
22
Examples of learning
‣ Kiva Systems (now AmazonRobotics)
22
Examples of learning
‣ Kiva Systems (now AmazonRobotics)
22
Examples of learning
‣ Kiva Systems (now AmazonRobotics) - Learning allows high performance
with very cheap components
23
Examples of learning
‣ Kiva Systems (now AmazonRobotics) - Learning allows high performance
with very cheap components
23
learning algorithm
data
Examples of learning
‣ Kiva Systems (now AmazonRobotics) - Learning allows high performance
with very cheap components
23
recorded commands
recorded observationslearning
algorithm
data
Examples of learning
‣ Kiva Systems (now AmazonRobotics) - Learning allows high performance
with very cheap components
23
recorded commands
recorded observationslearning
algorithm
datacommands
change in state
a model of the system
Examples of learning
‣ Kiva Systems (now AmazonRobotics) - Learning allows high performance
with very cheap components
24
recorded commands
recorded observationslearning
algorithm
data
a model of the system
‣ Learning makes systems more robust.
25
interferencesabotage faults / wear
‣ Learning makes systems more robust.
25
interferencesabotage faults / wear
Examples of learning
‣ Dyson 360eye- learns a 3D map of your house
26
Examples of learning
‣ Dyson 360eye- learns a 3D map of your house
26
Examples of learning
‣ “Learning by demonstration”
27
Examples of learning
‣ “Learning by demonstration”
27
[Muelling, Peters]
Examples of learning
‣ “Learning by demonstration”
27
[Muelling, Peters]
learning algorithm
data
Examples of learning
‣ “Learning by demonstration”
27
[Muelling, Peters]
user demonstrations
robot’s own experience
learning algorithm
data
Examples of learning
‣ “Learning by demonstration”
27
[Muelling, Peters]
user demonstrations
robot’s own experience
learning algorithm
data
control policy
commands
current state
Learning
‣ It’s about using data- …to make predictions- …to make decisions
28
Learning
‣ It’s about using data- …to make predictions- …to make decisions
28
‣ Value proposition for robotics:- add value: more complex functionality- reduce design cost- reduce building cost- reduce operating cost
Learning
‣ It’s about using data- …to make predictions- …to make decisions
28
‣ Value proposition for robotics:- add value: more complex functionality- reduce design cost- reduce building cost- reduce operating cost
but…
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
29
30
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
30
more adaptive less adaptive
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
30
more adaptive less adaptive
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
Human
30
more adaptive less adaptive
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
Fruit flyHuman
30
Dyson 360eye
more adaptive less adaptive
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
Fruit flyHuman
30
Dyson 360eye
more adaptive less adaptive
Downsides of Learning
‣ Adaptivity comes at a (computational) cost.
Fruit flyHuman iRobot’s Roomba
Downsides of Learning
‣ It might be harder to understand your design.
31
Downsides of Learning
‣ It might be harder to understand your design.
31
‣ Just like “genetic algorithms”…
Downsides of Learning
‣ It might be harder to understand your design.
31
2006 NASA ST5 spacecraft antenna.
‣ Just like “genetic algorithms”…
Downsides of Learning
‣ It might be harder to understand your design.
32
Downsides of Learning
‣ It might be harder to understand your design.
32
33
Downsides of Learning
(image, label)(image, label)(image, label)(image, label)(image, “gorilla”)(image, label)
learning algorithm
data
‣ It might be harder to understand your design.
33
Downsides of Learning
(image, label)(image, label)(image, label)(image, label)(image, “gorilla”)(image, label)
learning algorithm
data
‣ It might be harder to understand your design.
image of person
“gorilla”
33
Downsides of Learning
(image, label)(image, label)(image, label)(image, label)(image, “gorilla”)(image, label)
learning algorithm
data
‣ It might be harder to understand your design.
image of person
“gorilla”
evidence: + eyes + hair - clothes
Downsides of Learning
‣ Machine learning systems are hard to maintain.
34
Downsides of Learning
‣ Machine learning systems are hard to maintain.
34
Learning
‣ It’s about using data- …to make predictions- …to make decisions
35
‣ Value proposition for robotics- add value: more complex functionality- reduce design cost- reduce building cost- reduce operating cost
Learning
‣ It’s about using data- …to make predictions- …to make decisions
35
‣ Value proposition for robotics- add value: more complex functionality- reduce design cost- reduce building cost- reduce operating cost
‣ But:- systems need more computational resources- systems are hard to trust- systems are less maintainable
Learning
‣ It’s about using data- …to make predictions- …to make decisions
35
‣ Value proposition for robotics- add value: more complex functionality- reduce design cost- reduce building cost- reduce operating cost
‣ But:- systems need more computational resources- systems are hard to trust- systems are less maintainable
‣ Learn about learning: “The Master Algorithm” by Domingos (Sep’15)
