Robotics for AI - csc.villanova.edumatuszek/fall2015/RoboticsCAI2015text.pdf · CSC 8520 Fall 2015. Paula Matuszek Slides based in part on and -part2.ppt…

Robotics for AI

Dr. Cynthia Matuszek

Paula Matuszek Fall 2015

Cynthia Matuszek, Fall 2015

2

General Topics

◆ Hardware Design◆ Sensing

◆ Environment Models

◆ Actuation◆ Mobility

◆ Mapping◆ Localization

◆ Manipulation

◆ Motors

◆ Control◆ (Inverse) Kinematics◆ Dynamics◆ Motion planning

◆ Cognition◆ Machine learning◆ Classic AI◆ Others

◆ Human-robot interaction


3

Wall•E: 2008

Data. Star Trek: TNG: 1987

Sentinel. X-Men, Days of Future Past: 2014

Fictional Robots

Robby. Forbidden Planet: 1956

Eva. Ex Machina, 2015

CSC 8520 Fall 2015. Paula Matuszek Slides based in part on www.jhu.edu/virtlab/course-info/ei/ppt/robotics-part1.ppt and -part2.ppt !4 3

Some21stcenturyrobots

http://www.pleoworld.com/pleo_rb/eng/lifeform.php

Slides based in part on www.jhu.edu/virtlab/course-info/ei/ppt/robotics-part1.ppt and -part2.ppt

http://www.pleoworld.com/pleo_rb/eng/lifeform.php


5

◆ Autonomous?◆ Physical?◆ Human-friendly?

What is a Robot?

“A robot is a reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through variable programmed motions for the performance of a variety of tasks.” (Robot Institute of America)

◆ Humanoid?◆ Sensory?◆ Intelligent?

◆ Mobile?◆ Manipulative?◆ What else?


• What is hard for humans is easy for robots.– Repetitive tasks. – Continuous operation. – Complicated calculations. – Refer to huge databases.

• What is easy for a human is hard for robots. – Reasoning. – Adapting to new situations. – Flexible to changing requirements. – Integrating multiple sensors. – Resolving conflicting data. – Synthesizing unrelated information. – Creativity.

Whatarerobotsgoodat?


• Dangerous– space exploration – chemical spill cleanup – disarming bombs – disaster cleanup

• Boring and/or repetitive– welding car frames – part pick and place – manufacturing parts.

• High precision or high speed– electronics testing – surgery – precision machining.

Whattaskswouldyougiverobots?


8

Robots Up to Now

◆ Robots now:◆ Expensive◆ Complex◆ Special-purpose

◆ Environments◆ Dedicated◆ Constrained

◆ Use and Management◆ Controlled by trained experts◆ Slow and expensive to reconfigure/repurpose


9

Robots Now

◆ As technology improves:◆ Smaller◆ Cheaper◆ More broadly capable

◆ Can consider deploying in human-centric environments◆ Homes◆ Schools◆ Care facilities

◆ Requires: flexibility and human-robot interaction (HRI).


10

What Should They Do?

◆ Robots are moving away from factory floors to…◆ Entertainment, toys ◆ Homes (personal robotics) ◆ Medical, surgery ◆ Industrial automation (mining, harvesting, warehouses, …) ◆ Hazardous environments (space, underwater, battlefields, …) ◆ Roads

◆ Research Trends◆ Manipulation of everyday objects◆ Complex household tasks◆ Object recognition, mapping, interaction◆ Human robot interaction


11

What Subsystems Are There?

◆ Or: what does a robot need to know?◆ Where am I?◆ What’s around me?◆ How am I posed?

◆ How do I change it?◆ What do I want to do?

◆ With respect to the environment?◆ Where do I go, and how?◆ What do I want to change?

◆ How do I do that?◆ Who needs to be involved?

◆ People? Other robots?


12

What Subsystems Are There?

◆ Sensing◆ Perceiving the world◆ Creating a world model

◆ Actuation◆ Doing something in the (physical) world◆ Mobility, manipulation, …

◆ Control◆ Navigation, motion planning, kinematics, dynamics

◆ Cognition and Learning◆ Interfaces


13

Sensor data

Actions

High Level View

World Model


14

Sensors◆ Perceive the world

◆ Passive sensors capture signals generated by environment. ◆ Background, lower power. E.G.: cameras.

◆ Active sensors probe the environment. Explicitly triggered, ◆ More info, higher power consumption. Example: lidar

◆ What are they sensing?◆ The environment: e.g. range finders, obstacle detection◆ The robot's location: e.g., gps, wireless stations◆ Robot's internals: joint encoders

◆ Proprioception◆ Close your eyes - where's your hand?


15

◆ Optical◆ Laser / radar◆ 3D◆ Color spectrum

◆ Pressure, temperature, chemical◆ Motion & Accelerometer◆ Acoustic

◆ Sonar, ultrasonic

◆ E-field Sensing

Some Typical Sensors

CSC 8520 Fall 2015. Paula Matuszek Slides based in part on www.jhu.edu/virtlab/course-info/ei/ppt/robotics-part1.ppt and -part2.ppt !1610

• Use sensors for feedback • Closed-loop robots use sensors in

conjunction with actuators to gain higher accuracy – servo motors.

• Uses include mobile robotics, telepresence, search and rescue, pick and place with machine vision, anything involving human interaction

Whatusearesensors?


17

◆ Manipulators◆ Anchored somewhere: assembly lines, ISS, hospitals.◆ Common industrial robots

◆ Mobile Robots◆ Move around environment◆ UGVs, UAVs, AUVs, UUVs◆ Mars rovers, delivery bots,

ocean explorers

◆ Mobile Manipulators◆ Both move and manipulate◆ Packbot, humanoid robots

Robot Systems


18

◆ Take some kind of action in the world: affect it◆ Involve movement of robot or subcomponent of robot

◆ Robot actions can include◆ Pick and place: Move items between points◆ Path control: Move along a programmable path◆ Sensory: Employ sensors

for feedback (e-field sensing)◆ Manipulation: interact with

objects in the world

Actuators/Effectors


19

Some Typical Actuators

◆ Pneumatic◆ Hydraulic◆ Electric solenoid ◆ Motors

◆ Analog (continuous)◆ Stepping (discrete increments)◆ Gears, belts, screws, levers

◆ What’s missing?


20

Mobility

◆ Legs◆ Wheels◆ Tracks◆ Crawls◆ Rolls◆ Treads◆ …


21

Mobile Robots

◆ Space Rovers◆ Key issues: mobility in rough terrain, time delay, temperatures,

maintenance, joint infiltration

◆ Autonomous Robotic Cars◆ Key issues: dynamic environments, safety

◆ Flying Robots◆ Key issues: limited computation power and payload

◆ Personal Robots◆ Key issues: safety, human-friendliness


22

Mobile Robots: Space


23

Mobile Robots: Cars

Mobility Legs, Wheels, and Wings

Many slides adapted from slides © R. Siegwart, ETH Zürich – Autonomous Systems Laboratory


25

Characterizing Locomotion

◆ Locomotion:◆ Physical interaction between robot and environment.◆ Locomotion is concerned with:

◆ Interaction forces; mechanisms and actuators that generate them

◆ The most important issues in locomotion are:◆ Stability

◆ Center of gravity◆ Static/dynamic stabilization◆ Inclination of terrain

◆ Type of environment◆ Water, air, soft or hard ground

◆ Contact◆ Contact point(s)◆ Contact area◆ Angle of Contact◆ Friction

Adapted from © R. Siegwart, ETH Zürich – ASL


• Degrees of freedom = Number of independent directions a robot or its manipulator can move

• 3 degrees of freedom: 2 translation, 1 rotation • 6 degrees of freedom: 3 translation, 3 rotation

• How many degrees of freedom does your knee have? Your elbow?

• Effective DOF vs controllable DOF: • Underwater explorer might have up or down, left or right,

rolling. 3 controllable DOF. • Position includes x,y,z coordinates, yaw, roll, pitch.

(together the pose or kinematic state). 6 effective DOF. • Holonomic: effective DOF = controllable DOF.

DegreesofFreedom(DOF)

http://www.nasm.si.edu/exhibitions/gal109/newhtf/htf541b.htm


27

Degrees of Freedom

https://en.wikipedia.org/wiki/Degrees_of_freedom_%28mechanics%29

◆ DoFs◆ Number of independent parameters that define the

state (not location!) of a physical system.


28

Passive Dynamic Walking◆ Gravity-powered walking with bipedal gait

https://www.youtube.com/watch?v=CK8IFEGmiKY


29

◆ Boston Dynamics BigDog

Dynamic Quadruped

https://www.youtube.com/watch?v=cNZPRsrwumQ


30

Basic Wheel Types

https://www.youtube.com/watch?v=8sH1a511_q4


31

Walking Wheels

HallucII,ChibaInst.OfTech.


32

◆ Passive locomotion on rough terrain

◆ 6 wheels◆ One fixed wheel in the rear◆ Two bogies on each side◆ Front wheel, spring suspension

◆ ≈ 60 cm long, 20 cm tall◆ Highly stable in rough terrain◆ Climbs obstacles up to 2x wheel diameter

Passive: the Shrimp

Adapted from © R. Siegwart, ETH Zürich – ASL


33

◆ JPL Lunar rover◆ Walking, rolling or

combination◆ 6 DoF legs with

1lockable 1 DoF wheels

◆ Requires a substantial planning component

◆ Under development

Active: Athlete


34

Flying

◆ Advantages◆ Rough terrain◆ Ground-inaccessible areas◆ Z-axis maneuverability◆ Perspective for mapping & sensing◆ Flying is cool

◆ Disadvantages◆ Control problems◆ Z-axis controllability◆ Weight & scaling laws◆ Flying is dangerous


35

Types of Flyers

◆ Fixed wing (sometimes with flaps)◆ Flapping wing◆ Rotors/props

◆ Axial (single)◆ Coaxial (reversed)◆ Tandem (two non-coaxial)◆ Quadcopters (+)

◆ Lighter-than-air

https://robotics.eecs.berkeley.edu/~ronf/Ornithopter/index.html


36

Manipulators


37

◆ Actuator◆ Generates motion or force; usually a motor◆ “Drive type”

◆ End Effector◆ Device at the end of an arm; interacts with environment

◆ DoFs◆ Gripper

◆ What it sounds like; a type of manipulator

◆ Actuation◆ How are the individual parts made to move?

Terminology


38

◆ Modeled as a chain of rigid links connected by joints

◆ Links: unjointed length of robot◆ Joints: translational or rotational movement

◆ Joints have DoFs◆ How many to describe its position and orientation in space?

◆ Sliding or jointed

◆ Manipulator / End Effectors◆ Grippers / Tools◆ Sensors

Manipulator Robot

Links

JointsEndEffector(gripper)


39

◆ Types◆ By drive◆ By actuation

◆ Tendons◆ Direct servoing◆ Underactuation

◆ By motion◆ Prismatic (linear)◆ Revolute (rotational)

◆ By Characteristics◆ Payload◆ Working area/radius

Characterization


40

◆ Prismatic: sliding / translational

◆ Revolute: rotational

Joints


• Joint play, compounded through N joints. • Accelerating masses produce vibration,

elastic deformations in links. • Torques, stresses transmitted depending on

end actuator loads. • Feedback loop creates instabilities.

• Delay between sensing and reaction. • Firmware and software problems

• Especially with more intelligent approaches

Whataresomeproblemswithcontrolofrobotactions?

Localization where am I? – errors and odometry

“Position”: Global map

Perception Motion Control

Cognition

Real WorldEnvironment

Localization

PathEnvironment Model Local Map


43

Localization: Where am I?

◆ Given:◆ A map (which MAY be being found on the fly)◆ A set of sensor readings

◆ Where am I in that map?

◆ Things to consider :◆ Belief representation◆ Map representation◆ Types of sensor data◆ Probabilistic representations


44

Challenges of Localization

◆ Knowing absolute position (e.g. GPS) is not sufficient◆ Localization in human-scale as relates to environment

◆ Planning in Cognition needs >1 position as input

◆ Perception and motion plays an important role◆ Sensor noise◆ Sensor aliasing◆ Effector noise◆ Odometric position estimation◆ Probabilities and uncertainty management


• Sensing isn’t enough: need to act on data sensed • Hard because data are noisy; environment is dynamic

and partially observable. • Must be mapped into an internal representation

• state estimation • Good representations

• contain enough information for good decisions • structured for efficient updating • natural mapping between representation and real

world. • Cognition!

RoboticPerception

46

Cognition and Mobility

“Position”: Global map

Perception Motion Control

Cognition

Real WorldEnvironment

Localization

PathEnvironment Model Local Map

Sensors Actuation Kinematics

Path Planning

Mapping Localization

Representation

Route planning Task planning

…


47

Control: The Brain

◆ Open loop, i.e., no feedback◆ Instructions◆ Rules

◆ Closed loop, i.e., feedback◆ Adapts◆ Can learn


• Belief state: model of the state of the environment (including the robot) • X: set of variables describing the environment • Xt: state at time t • Zt: observation received at time t • At: action taken after Zt is observed

• After At, compute new belief state Xt+1 • Probabilistic, because uncertainty in both Xt and

Zt.

BeliefState


• Low-level, reactive control • bottom-up, sensor results directly trigger

actions • Model-based, deliberative planning

• top-down, actions are triggered based on planning around a state model

SoftwareArchitectures


• Augmented finite state machines • Sensed inputs and a clock determine next state • Build bottom up, from individual motions • Subsumption architecture synchronizes AFSMs,

combines values from separate AFSMs. • Advantages: simple to develop, fast • Disadvantages: Fragile for bad sensor data, don’t

support integration of complex data over time. • Typically used for simple tasks, like following a

wall or moving a leg.

Low-Level,ReactiveControl


• Belief State model • Current State, Goal State • Any of planning techniques • Typically use probabilistic methods

• Advantages: can handle uncertain measurements and complex integrations, can be responsive to change or problems.

• Disadvantages: slow; current algorithms can take minutes. Developing models for the number of actions involved in driving a complex robot too cumbersome.

• Typically used for high-level actions such as whether to move and in which direction.

Model-BasedDeliberativePlanning


• Usually, actually doing anything requires both reactive and deliberative processing.

• Typical architecture is three-layer: • Reactive Layer: low-level control, tight senso-

action loop, decision cycle of milliseconds • Deliberative layer: global solutions to complex

tasks, model-based planning, decision cycle of minutes

• Executive layer: glue. Accepts directions from deliberative layer, sequences actions for reactive layer, decision cycle of a second

HybridArchitectures

53

◆ Autonomous robots…◆ Perform their tasks in the world by themselves◆ Do not require human control / intervention◆ Learn about their environment and tasks◆ Avoid damage (to themselves, people, property)◆ Adapt to changing situations◆ Make and execute decisions◆ Possess some degree of self-sufficiency

◆ Intelligently and safely perform tasks◆ Without direct human control

What Is Autonomy?

All of these?

54

◆ Key types◆ Strong AI: mental/thought capabilities

equal to (or better than) human◆ Weak (bounded) AI: intelligent actions or

reasoning in some limited situations

◆ These are problematic◆ How do we measure it?◆ What’s an ‘intelligent action’?

◆ In practice, ‘previously human only’◆ Is there something ineffable missing?

◆ How does it change when it’s a robot?◆ Focus is on Autonomy

Artificial Intelligence

55

◆ Many subtasks◆ Understanding and modeling of the mechanism

◆ Kinematics, dynamics, odometry◆ Reliable control of actuators

◆ Understanding and modeling the environment◆ Integration of sensors

◆ Understanding and modeling the task◆ Generation of task-specific motions◆ Creation of flexible control policies, new situations

◆ Coping with noise and uncertainty◆ Probably physical tasks!

Autonomous Task Performance

56

◆ Knowledge Representation◆ Search◆ Planning◆ Learning◆ Inference

Intelligent Action Needs…

57

DARPA Grand Challenge 1

58

DGC 1 Challenges

◆ Localization◆ But not mapping

◆ Sensor management◆ What sensors?◆ Where’s the road?

◆ Narrow pass◆ Switchbacks, turns◆ Tunnels

◆ Actuator management

✓ Knowledge Representation

✓ Search ✓ Planning✗ Learning ✗ Inference

59

DARPA Grand Challenge 2

60

DGC 2 Challenges

◆ All of the above, plus…◆ Visual parsing of traffic elements

◆ Sensing + Knowledge

◆ Awareness of other cars◆ Sensing + Planning

◆ Non-3D-guided tasks◆ Faster speeds◆ Safety


✓ Search ✓ Planning✗ Learning ✓ Inference

61

Google Self-Driving Car

https://www.youtube.com/watch?v=uCezICQNgJU

62

Google Challenges

◆ All of the above, plus…◆ Sublimely oblivious drivers◆ Full-speed actuator management◆ Legal management◆ Ethical management


✓ Search ✓ Planning? Learning ✓ Inference

63

DARPA Robot Challenge 1

64

DRC 1 Challenges

◆ All of the above, plus…◆ Non-designed environment

◆ Balancing◆ (Much) harder actuation◆ Bipedal motion

◆ Yet more sensor hassles◆ Weight, power◆ Manipulation

◆ !!!


✓ Search ✓ Planning ✓ Learning ✓ Inference ✗ Autonomy

65

DRC 1

66

Robocup

67

Robocup

◆ All of the above, plus…◆ Object interaction

◆ Balls, walls, ...

◆ Deliberately difficult goal task◆ Streets designed for easy driving◆ S&R not designed

◆ Enormous robot design space◆ Antagonistic agents


✓ Search ✓ Planning ✓ Learning ✓ Inference ✓ Autonomy

68

Other Robot Tasks

69

◆ Knowledge Representation◆ Search◆ Planning◆ Learning◆ Inference

Intelligent Action Needs…

70

◆ Background Knowledge◆ Task-Specific Knowledge◆ Explicit vs. Implicit◆ Representation Choices

◆ Probabilistic?◆ Human-understandable?

Knowledge Representation


• Social robots • In care contexts • In home contexts • In industrial contexts

• Comprehension • Natural language • Grounded knowledge acquisition • Roomba: “Uh-oh”

Human-RobotInteraction


• Robots are getting smaller, cheaper, and more ubiquitous

• Humans need to interact with and instruct them, naturally

• Language, gesture, demonstration, …

• Key requirements: • Language understanding

learned from data • Follow instructions in a

previously unseen world • Learn to parse natural

language into robot-usable commands

Motivations


• Social robots • Safety/security

• Ubiquitous Robotics • Small, special-purpose robots

Human-RobotInteraction


• How do humans handle it? • Assumptions about retention and understanding • Anthropomorphization

• How do robots make it easier? • Apologize vs. back off • Convey intent • Cultural context (implicit

vs. explicit communication)

MoreHuman-RobotInteraction

http://www.tweenbots.com/

http://www.tweenbots.com


• Healthcare and personal care • surgical aids, intelligent walkers, eldercare

• Personal services • Roomba! Information kiosks, lawn mowers,

golf caddies, museum guides • Entertainment

• sports (robotic soccer) • Human augmentation

• walking machines, exoskeletons, robotic hands, etc.

Wherearerobotsworkingnow?


• Industry and Agriculture assembly, welding, painting, harvesting, mining, pick-and-place, packaging, inspection, ...

• Transportation Autonomous helicopters, pilot assistance, materials movement

• Cars (DARPA Grand Challenge, Urban Challenge)

Antilock brakes, lane following, collision detection

Andmore...Exploration and Hazardous environments

Mars rovers, search and rescue, underwater and mine exploration, mine detection

MilitaryReconnaissance, sentry, S&R, combat, EOD

HouseholdCleaning, mopping, ironing, tending bar, entertainment, telepresence/surveillance


• Medical robotics • Autonomous surgery • Eldercare

• Biological Robots • Biomimetic robots • Neurobotics

• Navigation • Collision avoidance • SLAM/Exploration

Andmore...


• Robots that can learn. • Robots that interact smoothly with people. • Robots with artificial intelligence. • Robots that make other robots • Swarms

• ..?

TheFutureofRobotics

62


• Mechanisms • Morphology: What should robots look like? • Novel actuators/sensors

• Estimation and Learning • Reinforcement Learning • Graphical Models • Learning by Demonstration

• Manipulation (grasping) • What does the far side of an object look like? How

heavy is it? How hard should it be gripped? How can it rotate? Regrasping?

Tomorrow’sproblems


• Which decisions can the machine make without human supervision?

• May machine-intelligent systems make mistakes (at the same level as humans)?

• May intelligent systems gamble when uncertain (as humans do)?

• Can intelligent systems exhibit personality? Should they?

• Can intelligent systems express emotion? Should they?

• How much information should the machine display to the human operator?

Willrobotstakeovertheworld?

HAL - 2001 Space Odyssey

Documents

Robotics for AI - csc.villanova.edumatuszek/fall2015/RoboticsCAI2015text.pdf · CSC 8520 Fall 2015. Paula Matuszek Slides based in part on and -part2.ppt…