The iCub humanoid robot: An open-systems platform for research in cognitive development Giorgio Metta et al., Neural networks 2010 윤성재

The iCub humanoid robot: An open-systems plat-form for research in cognitive development

Giorgio Metta et al., Neural networks 2010

윤성재

S FT COMPUTING @ YONSEI UNIV . KOREA16

Outline

• Motivation• Design goals• Foundations of human development• Specific results

– Mechatronics of the iCub– Software architecture– Sensorimotor coordination models– Object affordances– Imitation and communication

• Conclusion

2


Motivation

• Consider to be a humanoid: – Biological-like system which takes decisions and acts in the environment.

• Adapts and learns how to behave in new situations.• Invents new solutions on the basis of the past experience.

– Possibility to interact with it: • To teach, to demonstrate, even to communicate.

– The developmental process leading to a mature humanoid.• Based on a road map of human development.

• The main idea is

to be able to test the iCub in the same manner as a developmental psychologist would test an infant in a laboratory experiment.

3


Design goals

• The construction of cognitive systems could not progress without a certain number of ingredients :

– Development of a sound formal understanding of cognition – Natural cognition and study of the development of cognition – Action in humans by using neuroscience methods – Physical instantiation of these models in a behaving humanoid robot

• Initial period of human cognitive development and its implementation on the iCub

– Not to preprogram the cognitive skills but, similarly to what happens in humans to implement them into a system that can learn much like a hu-man baby does.

4


Design goals

• Considering action and embodiment and how perception and cogniti-on are intertwined into development

– Exemplar experimental scenarios are discovering the action possibilities of the body (the so called body map)

– Learning to control one’s upper and lower body (crawling, bending the tor-so) to reach for targets

– Learning to reach static and moving targets, and learning to balance in order to perform stable object manipulations when crawling or sitting

• Interaction with other agents – Recognizing manipulation abilities of others and relating those to one’s

own manipulation abilities – Learning new motor skills and new object affordances by imitating manip-

ulation tasks performed by others– Learning what to imitate and when to imitate others gestures– Learning regulating interaction dynamics.

5


Foundations of human development

• To provide a description of human development, we have to consider three basic elements:

– What is innate, where do we start from? – What drives development? – How is new knowledge incorporated?

• What is innate, where do we start from? – Developmental psychologists, typically refer to innate elements in terms of

prenatal prestructuring or the so-called core abilities. – Motor system requires constraints in order to reduce the large number of

effective degrees of freedom • These constraints come in the form of muscular synergies. To facilitate

control, the activation of muscles is therefore organized into functional synergies at the beginning of life

• Prestructuring comes also in the form of specific core abilities.

6



• What drives development? – Motivations come in different forms in the newborn: social and explorative.

• Social motive is providing further possibilities for learning, safety, com-fort, etc.

• How is new knowledge incorporated?– Final product is shaped by the dynamical interaction with the environment.

The brain is only one side of this process– Factors like exposure or deprivation to the environment are also important

• The body biomechanics and body growth are all fundamental to the development of cognition.

7



• Neurophysiology is also helping to show the complexity of the brain. – Canonical and mirror neurons are indistinguishable.

Their visual responses, however, are quite different. • The canonical type is active in two situations:

(1) When grasping an object and

(2) When fixating that same object• Mirror neuron, becomes active under either of two conditions:

(1) Manipulating an object (e.g. grasping it, as for canonical neurons)

(2) Watching someone else performing the same action on the same object

– Two visual pathways• Dorsal & Ventral

– Dorsal deals with information required for action – Ventral is important for more cognitive tasks such as maintaining

an object’s identity and constancy

8


Mechatronics of the iCub

• 1m tall and weighs 22 kg• # of DOF(degrees of freedom)

– For the upper body is 38– Each leg has 6 DOF (for crawling, sitting and squatting)

• Equipped with digital cameras, gyroscopes and accelerometers, micro-phones, and force/torque sensors.

%Distributed sensorized skin is under development using capacitive sensors technology

• A set of DSP-based control cards. Takes care of the low-level control loop in real-time.

– DSPs communicate with each other via a CAN bus. – Four CAN bus lines connect the various segments of the robot.

• All sensory and motor-state information is transferred to an embedded Pentium based PC104 card that handles synchronization and reforma-tting of the various data streams.

9

• Specific results


Sensorimotor coordination model

• Cognitive capabilities depend greatly on the development of sensorimotor coordination and sensorimotor mapping.

– Experience enters into the process in forming reliable and sophisticated tools for exploring and manipulating the outside world.

• The research on sensorimotor coordination has two distinct themes. – Modeling Sensorimotor systems evolve from sets of relatively independe-

nt mechanisms to unified functional systems. – Modeling the role of motor representation as tools serving not only action

but also perception.

• Layered controller system for the iCub including: 1. Spinal behaviours

2. Eye movements and attention

3. Reacting and body schemes

4. Grasping

10



Object affordances

• The term affordance is – ‘‘Action possibilities’’ on a certain object– E.g.) A chair is ‘‘sit-able’’ for a perceiver

• Humans learn to exploit object affordances throughout their entire life but not all are learnt autonomously.

– A large set is conveyed by social means either by communication or by observing others actions.

– Difficult to separate the importance of learning by exploration and learning from others.

• Distinguish two means of acquisition of object affordances:– Self-exploration (autonomous learning)– By observation (learning from examples)

11



Object affordances

1. System has acquired the capability to coordinate movements with respect to sensory information.

2. It can start interacting with objects and understanding its interface (autonomous learning)

– How to grab the object, what are the effects of certain applied actions.

3. System may start recognizing and interpreting other agents (learni-ng from examples)

– Interacting with similar objects, learning other object affordances and inte-rpreting activities.

12



Object affordances

• Use Bayesian Networks (BN) for learning affordances.– Model the dependencies between robot actions, object characteristics,

and the resulting effects

• Affordances can be learned autonomou-sly by experience and by self-observa-tion. Features can be either selected or ignored

13



• Experiments– Playground scenario consisting of several objects with two shapes

(box and ball), different sizes. – iCub was able to perform three different actions: grasp, tap and touch.

• Results show how the model is able to capture the basic object beha-viour under different actions.

– Relations are shaped by the experience of the robot and by its current skills.

Object affordances

14



Imitation and communication

• Imitation plays a central role and communication.– Particular concern here are cognitive skills required for communicating

through body gestures.

• Large part of this iCub work took a human–robot interaction perspec-tive to analyzing and developing controllers to enhance human–robot communication.

• Relies on sophisticated cognitive skills which requires– Recognize and interpret somebody else’s gestures in terms of its own

capabilities (mirror effects), – Learn new gestures– Recognize the purpose of other people’s gestures– Predict the result of a demonstrated manipulation task– Finally, the ability to decide what part of the demonstration is relevant to

imitation

15



Conclusion

• Forefront research in developmental robotics, the iCub was designed completely from scratch of

– Mechanics, electronics, firmware, and software • Multisensory neurons connection

– Brain is not made of a set of isolated areas dealing with perception or mo-tor control but rather that multisensory neurons.

• Computational model of affordances – Quintessential primitives of cognition by mixing perception and action in a

single concept– Includes the possibility of learning both the structure of dependences bet-

ween sets of random variables • Facilitating the creation of a computation model of imitation and inter-

action between humans and robots – By evaluating the automatic construction of models from experience, their

correction via feedback. – Explores the domain between mere sensorimotor associations and the

possibility of true communication between robot and people.16

Documents

The iCub humanoid robot: An open-systems platform for research in cognitive development Giorgio Metta et al., Neural networks 2010 윤성재