Agent-Based Modellingand Simulation - UPpaginas.fe.up.pt/~lpreis/mams0910/MAMS_Aula3_AgentBasedSimulation.pdf · • Agent-Based Modelling and Simulation (ABMS) – Agent Based Modelling

12-03-2010

1

MAMS MAMS –– MetodologiasMetodologias AvançadasAvançadas de de ModelaçãoModelação e e SimulaçãoSimulação 11

Agent-Based Modelling and

SimulationLuís Paulo Reis

[email protected]

Http://www.fe.up.pt/~lpreisAssistant Professor at FEUP – Faculty of Engineering of the University of Porto

Member of the Directive Board of LIACC – Artificial Intelligence and Computer Science Lab.

Rosaldo [email protected]

Http://www.fe.up.pt/~rossettiAssistant Professor at FEUP – Faculty of Engineering of the University of Porto

Researcher at LIACC – Artificial Intelligence and Computer Science Lab.


Presentation Outline• Motivation

• Simulation

– Introduction

– A Few Examples

– Simulation Advantages

– Simulation Project Life-Cycle

• Agents and Multi-Agent Systems (MAS)

– Artificial Intelligence and Autonomous Agents

– Multi-Agent Systems, Cooperative and Competitive MAS

– Coordination in MAS/Multi-Robot Systems

• Agent-Based Modelling and Simulation (ABMS)

– Agent Based Modelling and Simulation

– ABMS Tools and Applications

– Creating an Agent Based Model

• Some ABMS Projects involving Coordination at LIACC

– Costal Ecosystems Simulation

– Heterogeneous Robotic Teams Simulation

– Simulation of Soccer Games

– Flexible Aerial Mission Simulation

– Simulation of Intelligent Wheelchairs

– Simulation of Poker Games

• Conclusions

Motivation | Simulation | Multi Agent Systems | ABMS | Projects at LIACC | Conclusions

12-03-2010

2


Motivation• Traditional Simulation Drawbacks:

– Systems are getting more complex

– Complex systems are difficult to model as a whole

– Higher level tools available

– Human behaviour is often neglected or over simplified in the simulation process

• Distributed Applications Challenges:

– Need for coordination of heterogeneous entities

– Entities with local processing/decision capabilities

– Human vs Artificial entities

• Agent Based Modeling and Simulation:

– Entities represented by Agents with Autonomous Behaviour



Simulation• Simulation is the imitation of some real thing, state of affairs, or

process, over time, representing certain key characteristics or

behaviours of the selected physical or abstract system

• Simulation:

– Modeling of natural systems or human systems in order to gain

insight into their functioning

– Simulation of technology for performance optimization, safety

engineering, testing, training and education

– Widely used tool for decision making, what if analysis

• Applied to complex systems that are impossible to solve

mathematically


12-03-2010

3


A Few Examples of Applications• Games

• Film Industry

• Manufacturing

• Bank operations

• Airport and Airlines

• Flight Simulation

• Military Operations

• Transportation

• Satellite Navigation

• Robotics

• Biomechanics

• Molecular Dynamics

• Logistics, supply chain, distribution

• Hospitals: Emergency, operation, admissions...

• Computer networks

• Business processes

• Chemical plants

• Fast-food restaurants

• Supermarkets

• Stock Exchange

• Theme parks

• Emergency-response systems

• Sports



A Few Examples of Applications

Flight SimulatorsWar gaming

Games & Sports

Transportation systems

RoboticsAerodynamics: Wind Tunnel


12-03-2010

4


Simulation is not Appropriate if?• Problem can be solved by:

– Common sense

– simple calculations

– Analytical methods

– Direct experiments

• Simulation costs exceed savings

• Resources & time are not available

• Data is not available

• Verification & validation are not practical due to limited resources

• System behavior is too complex (essential model is not easy to

capture)?



Simulation Advantages• Advantages of Simulation:

– When mathematical analysis methods are not available, simulation

may be the only investigation tool

– When mathematical analysis methods are available, but are so

complex that simulation may provide a simpler solution

– Provides practical feedback when designing real world systems

– Time compression or expansion

– Higher Control

– Lower costs

– Comparison of alternative designs or alternative operating policies

– Sensitivity Analysis

– Training tool

– Doesn’t disturb real system


12-03-2010

5


Life-cycle of a Simulation Project



Life-cycle of a Simulation Project1. Problem Formulation

– Statement of the problem

2. Set Objectives & Project Plan

– Questions to be answered

– Is simulation appropriate?

– Methods, alternatives

– Allocation of resources

3. Model Conceptualization

– Requires experience

– Begin simple and add complexity

– Capture essence of system

– Involve the user

4. Data Collection

– Time consuming, begin early

– Determine what is to be collected


12-03-2010

6


Life-cycle of a Simulation Project5. Model translation

– Computer form

– general purpose vs. special purpose lang.

6. Verification

– Does the program represent model and run properly? Common sense

7. Validation and Calibration

– Compare model to actual system

– Does model replicate system?

– How to calibrate the model?

8. Experimental Design

– Determine alternatives to simulate

– Time, initializations, etc.

9. Production & Analysis

– Actual runs + Analysis of results

– Determine performance measures

10. More Runs?



Presentation Outline• Motivation

• Simulation

– Introduction

– A Few Examples

– Simulation Advantages

– Simulation Project Life-Cycle

• Agents and Multi-Agent Systems (MAS)

– Artificial Intelligence and Autonomous Agents

– Multi-Agent Systems

– Cooperative and Competitive MAS

• Agent-Based Modelling and Simulation (ABMS)

– Agent Based Modelling and Simulation

– ABMS Tools and Applications

– Creating an Agent Based Model

• Some ABMS Projects at LIACC

– Costal Ecosystems Simulation

– Flexible Aerial Mission Simulation

– Heterogeneous Robotic Teams Simulation

– Simulation of Intelligent Wheelchairs

– Simulation of Soccer Games

– Simulation of Poker Games

• Conclusions


12-03-2010

7


Artificial Intelligence

• Intelligence

– “Capacity to solve new problems through the use of

knowledge”

• Artificial Intelligence

– “Science concerned with building intelligent machines,

that is, machines that perform tasks that when performed

by humans require intelligence”



Autonomous Agents

Computational System, situated in a given environment,

that has the ability to perceive that environment using

sensors and act, in an autonomous way, in that

environment using its actuators to fulfill a given function.”


12-03-2010

8


Agent Requisites

• Traditional definition include to much or leaves “holes”!

• Requisites:

– Perceive its environment (sensors)

– Decide actions to execute (“think”)

– Execute actions in environment using its actuators

– Communicate?

– Perform a complex function?

• Agents vs Objects:

– Agents decide what to do

– Object methods are called externally

– Agents react to sensors and control actuators

“Objects do it for free; agents do it for money”



Multi-Agent System (MAS)• Composed by multiple agents that:

– Exhibit autonomous behavior

– Interact with the other agents in the system

• MAS Motivation:– Problem Dimensions

– Legacy Systems

– Natural Solution (distributed problems)

– Distributed knowledge or information

– Human-machine interface

– Project Clarity and simplicity

– Efficiency

– Robustness and Scalability

– Problem division

– Information privacy


12-03-2010

9


Multi-Agent System (MAS)

• To build individual autonomous intelligent agents is important

• However:

– Agents don’t leave alone…

– Necessary to work in group..

– Multi-Agent Applications…

– Coordination in necessary: “to work in harmony in a group to

achieve a given goal”



Reasons for Coordination

• Dependencies in agent actions

• Need to respect global constraints

• No agent, individually has enough resources, information or capacity to execute the task or solve the complete problem

• Efficiency:

– Information exchange or tasks division

• Prevent anarchy and chaos:

– Partial vision, lack of authority, conflicts, agent’s interactions


12-03-2010

10


Cooperative vs Competitive MAS

• Cooperative MAS:

– Usually projected by a single entity

– Global utility and global performance

• Competitive MAS (“self-interested agents”):

– Each agent has a distinct designer

– Agents have their own motivation and agenda

– Agents are interested in their own utility

– Usual in negotiation, electronic commerce, internet



Intelligent Robotics and Multi-Robot Systems

• Robotics

– Science and technology for projecting, building, programming and using Robots

– Study of Robotic Agents (agents with body)

– Increased Complexity:

• Environments: Dynamic, Inaccessible, Continuous e Non Deterministic!

• Perception: Vision, Sensor Fusion

• Action: Robot Control

• Robot Architecture (Physical / Control)

• Navigation in unknown environments

• Interaction with other robots/humans

• Multi-Robot Systems


12-03-2010

11


The “Name” - ABMS

ABMS is known by many names:

• ABM: “Agent-based modeling” or “anti-ballistic missile?”

• ABS: “Agent-based simulation” or “anti-lock braking system?”

• IBM: “Individual-based modeling” or “International Business

Machines Corporation?”

ABM, ABS, and IBM are all widely-used acronyms, but “ABMS”will be

used throughout this talk

ABMS is not the same as “mobile agents”



Need for Agent-based ModelingWe live in an increasingly complex world!

Systems are More Complex:

– Systems are becoming more complex: more variables and interactions

– Decentralization of Decision-Making

– Increasing Physical and Economic Interdependencies

New Tools, Toolkits, Modeling Approaches:

– New tools exist to analyze complex systems

– Economic markets and the diversity among economic agents

– Social systems, social networks

– Robotic systems - interaction

Avalaible Data:

– Micro-data available in databases (finer levels of granularity) enables micro-simulations!

Computational Power :

– Computational power advancing – micro-simulations!


12-03-2010

12


ABMS as a New Field

Agents

– Discrete entity with its their own perceptions, actions, goals and

behaviors

– Autonomous, with a capability to adapt and modify its behaviors

Assumptions

– Some key aspect of behaviors can be described

– Mechanisms by which agents interact can be described

– Complex social processes and a system can be built “bottom-up.”

Agents may be diverse and heterogeneous



Example: Modeling Simple Flocking Behavior with Agent Rules

• Cohesion: Steer to move toward the

average position of local flockmates

• Alignment: Steer towards the average

heading of local flockmates

• Separation: Steer to avoid crowding local

flockmates

“Boids”by Craig Reynolds, http://www.red3d.com/cwr/boids/


12-03-2010

13


Agent Interaction Topologies:Neighborhoods

• Various Topologies Connect Agents with Their

Neighbors

• Agents can move in free (continuous) space

• Cellular automata have agents interacting in local

“neighborhoods”

• Agents can be connected by networks of various

types and be static or dynamic

• Agents can move over Geographical Information

Systems

• Sometimes spatial interactions are not important



Agent-Based Model

• An agent-based model consists of:

– A set of agents (part of the user-defined model)

– A set of agent relationships (part of the user-defined model)

– A framework for simulating agent behaviors and interactions (provided

by an ABMS toolkit or other implementation)

• Unlike other modeling approaches, agent-based modeling begins

and ends with the agent’s perspective


12-03-2010

14


ABMS Platforms• Agent-based Modeling and Simulation Toolkits

– Repast (Java) –similar to Swarm (Objective C, Java)

– NetLogo, StarLogo

– MASON

– AnyLogic(commercial)

• General Tools

– Spreadsheets, with macro programming

– Computational Mathematics Systems: MATLAB and Mathematica

• General Programming Languages (Object-oriented)

– Java, C++, Pascal

• Agent-based model development process often makes use of

several tools



When to use Agent-Based Modeling?

• Natural representation as agents

• Decisions and behaviors may be defined discretely

• Agents adapt and change their behaviors

• Agents learn and engage in dynamic strategic behaviors

• Dynamic relationships with other agents

• Organizations (adaptation and learning are important at the

organization level)

• Spatial component to their behaviors and interactions


12-03-2010

15


ABMS Applications• Business and Organizations

– Manufacturing Operations

– Supply chains

– Consumer markets

– Insurance industry

• Economics

– Artificial financial markets

– Trade networks

• Infrastructure

– Electric power markets

– Transportation

– Hydrogen infrastructure

• Crowds

– Pedestrian movement

– Evacuation modeling

• Society and Culture

– Ancient civilizations

– Civil disobedience

– Social determinants ofterrorism

– Organizational networks

• Military

– Command & control

– Force-on-force

• Biology

– Population dynamics

– Ecological networks

– Animal group behavior

– Cell behavior and subcellularprocesses



Building an ABM• Identify purpose of the model

• Identify questions the model is intended to answer

• Engage potential users in the process

• Analyze the system under study:

– Identify components

– Component interactions

– Relevant data sources

• Apply the model and conduct a series of “what-if”

experiments by systematically varying parameters and

assumptions

• Test the robustness of the model and its results by using

sensitivity analysis


12-03-2010

16


Building an ABM• Agent perspective

– (1) identify the agents and get a theory of agent behavior

– (2) identify the agent relationships and get a theory of agent

interaction

– (3) get the requisite agent-related data

– (4) validate the agent behavior models in addition to the model as a

whole, and

– (5) run the model and analyze the output from the standpoint of

linking the micro-scale behaviors of the agents to the macroscale

behaviors of the system.

• Typically:

– Unified Modeling Language (UML) for representing models

– Object-oriented languages/specific languages for implementing



Building an ABMGeneral steps in building an agent model:

– 1. Agents: Identify the agent types and other objects (classes) along

with their attributes.

– 2. Environment: Define the environment the agents will live in and

interact with.

– 3. Agent Methods: Specify the methods by which agent attributes are

updated in response to either agent-to-agent interactions or agent

interactions with the environment.

– 4. Agent Interactions: Add the methods that control which agents

interact, when they interact, and how they interact during the

simulation

– 5. Implementation: Implement the agent model in computational

software.


12-03-2010

17


ABMS – Our Approach

• Separation of the Environment and Agents

– Agents have perception of the environment

– Agents have possible actions on the environment

– Simulator (environment) manages agent communications

– High-level interaction between agents and environment

– Simulator/Visualizer separation



Main Projects• FC Portugal – New Coordination Methodologies in the Simulation

League– FCT/POSI/ROBO/43910/2002, 12 Months, 27800€ (+LIACC/FEUP, 2000 - …)

• LEMAS – Learning in Multi-Agent Systems using RoboCup Sony

Legged League– FCT POSI/ROBO/43926/2002, 12 Months, 32908€

• Portus – A Common Framework for Cooperation in Mobile

Robotics– FCT POSI/SRI/41315/2001, 30 Months, 20000€

• Rescue: Coordination of Heterogeneous Teams in Search and

Rescue Scenarios – FCT POSC/EIA/63240/2004, 24 Months, 32800€

34


12-03-2010

18


Main Projects• ABSES - Agent Based Simulation of Ecological Systems

– FCT/POSC/EIA/57671/2004, 30 Months, 75000€

• DITTY - Information Technology Tool for the Management of European

Southern Lagoons (European Project – 18 Partners)

• SIGA/SITBAL: Soccer Intelligent Game Analysis and Simulation

• IntellWheels – Intelligent Wheelchair with Flexible Multimodal

Interface

• ACORD - Adaptative Coordination of Robotic Teams

– FCT/PTDC/EIA/70695/2006, 24 Months, 95000€

• Intelligent Handball Players Tracking at F.C.Porto and FADEUP

• HuRoboT - Cooperation within teams of Humanoids and Humans



ABSES – Agent Based Simulation ofEcological Systems

• Realistic simulation of ecological models

– Difficult task

– Mixing complex biological, chemical and physical processes

– Slowness associated to each simulation

• Integrate human factor/decisions in the simulation

• Provide flexible services to help sustainable management of aquatic ecosystems

– Custom solutions to “any” aquatic ecosystem

– Environmental impact studies/water framework directive

– Aquaculture optimization/Carrying capacity


12-03-2010

19


ABSES - Technology Description

• EcoDynamo

– Simulator for aquatic ecosystems

• Intelligent Agents

– Include the human rationality in the scenarios generation and decisions

• ECOLANG

– Communication language for simulations of complex ecological systems

• EcoSimNet

– Platform that integrates all the previous

– Enables parallel simulations -clusters



FC Portugal: Coordination in Multi-Agent Systems

• Coordination in MAS using several ABS Environments

as test benches:

– Soccerserver 2d, Simpspark 3d, Rescue

• Cooperation project that started in 2000:

– LIACC/FEUP and IEETA/UA

• Research has been integrated in several teams:

– More than 25 awards in RoboCup International Competitions

• 4 FCT Funded Projects:

– FC Portugal(04/05), LEMAS(04/05), RESCUE(06/08), ACORD(08/09)


12-03-2010

20


• Joint International Project: – (Distributed) Artificial Intelligence

– Intelligent Robotics

• Soccer – Central Research Topic:– Very complex collective game

– Huge amount of technologies involved:• Autonomous Agents, Multi-Agent Systems, Cooperation, Communication, Robotics,

Sensor Fusion, Real-Time Reasoning, Machine Learning, etc

• Senior Leagues:– Soccer Simulation (2D ,3D Humanoids, Mixed Reality)– Small-Size– Middle-SIze– Sony Legged (AIBOs)– Humanoids (Small/Middle/Standard Platform -Nao Robot)– RoboCup Rescue Simulation– RoboCup Rescue Robots– RoboCup @ Home

FC Portugal: RoboCup InternationalProject



FC Portugal Project: SelectedResearch Contributions

• STRATEGY – Formalization of a Strategy for a Competition

• SBSP - Situation Based Strategic Positioning

• DPRE - Dynamic Positioning and Role Exchange

• ADVCOM – Advanced Communication (using a communicated WS)

• SLM – Strategic Looking Mechanism

• COACH Unilang – Language for a (robo)soccer coach

• SetPlay Framework and Playmaker

• Debugging Methodology for Multi-Robot Systems and Visual

Debugger


12-03-2010

21


FC Portugal Project - Formalizationof a Strategy for a Team



FC Portugal Project – ResearchContributions

• Strategic Layer


Strategy = {Tactics, Triggers};

Tactics = {Tactic1, Tactic2, ...,Tactic t}

Triggers = {Trigger1, Trigger2, ... , Trigger tg }

Tactic = {Formations, Situations, Binders, [Tactical Parameters]} ;

Formations = {Formation1, Formation2, ... , Formation f }

Situations = {Situation1, Situation2, ... , Situation s}

Binders = {Binder1, Binder2, ... , Binder b}

Tactical Parameters = {Tactical Param1,..., Tactical Param tp}

Situation = {Condition1, Condition2, ... , Condition cd }

Binder = {[Original Formations], Situations, Terminal Formation} [Original

Formations]

Formation = { Distribution, Sub-Tactics, [Agent Types] }

Sub-Tactics = {(SubTact1, m(SubTact1)), (SubTact2, m(SubTact2), … ,

(SubTact st, m(SubTact st))} ,

Distribution = {Value1, Value2, ... , Value v}

Sub-Tactic = {Amounts, Roles, [Sub-Tactical Parameters]}

Roles = {Role1, Role2, ... , Role r }

Amounts = { Amount1, Amount2, ... , Amount a}

12-03-2010

22


Texas-Holdem Poker Simulation• Poker is a game humans find fascinating

• Huge and growing market:

• Casinos, tournaments, online, television

• Challenge of Poker for DAI: Many new and interesting

problems not faced in Chess, Go, or Backgammon:

• Random, hidden information, bluffing and trapping, need

for opponent modeling

• Poker is a simple game that demands for complex strategies

• Project General Objective:

– Develop an agent capable of beating the best human players in “No Limit, Multi-Player, Texas Hold’em, Poker”



Texas-Holdem Poker SimulationAgent 1

Agent 2

Agent n

LIACC Poker

Simulator

Human Interface

Human

PlayerPoker Builder

Agent 3

AAAI Poker

Competition

Server

Online Poker

Server 1

Online Poker

Server n

StrategyOpp.Modelling

AgentOpp.

Model

St OM

POKERLANG

<STRATEGY>::= {<ACTIVATION_CONDITION> <TACTIC>}

<ACTIVATION_CONDITION>::= {<EVALUATOR>}

<TACTIC>::= <PREDEFINED_TACTIC> | <TACTIC_NAME><TACTIC_DEFINITION>

<PREDEFINED_TACTIC>::= loose_agressive | loose_passive | tight_agressive |

tight_passive

<TACTIC_NAME>::= [string]

<TACTIC_DEFINITION>::={<BEHAVIOUR> <VALUE>}

<BEHAVIOUR>::= {<RULE>}

<RULE>::= {<EVALUATOR> | <PREDICTOR>} <ACTION>

<EVALUATOR>::= <NUMBER_OF_PLAYERS> | <STACK> | <POT_ODDS> |

<HAND_REGION> | <POSITION_AT_TABLE>

<PREDICTOR>::= <IMPLIED_ODDS> | <OPPONENT_HAND> | <OPPONENT_IN_GAME> |

<STEAL_BET> | <IMAGE_AT_TABLE>

<ACTION>::= {<PREDEFINED_ACTION><PERC> | <DEFINED_ACTION><PERC>}

<PREDEFINED_ACTION>::= <STEAL_THE_POT> | <SEMI_BLUFF> |

<CHECK_RAISE_BLUFF> | <SQUEEZE_PLAY> | <CHECK_CALL_TRAP> |

<CHECK_RAISE_TRAP> | <POST_OAK_BLUFF>


12-03-2010

23


Conclusions – Benefits of ABSM• Natural representation as agents

• Decisions and behaviors may be defined discretely (with

boundaries)

• Agents adapt and change their behaviors

• Agents learn and engage in dynamic strategic behaviors

• Agents have a dynamic relationships with other agents, and agent

relationships form and dissolve

• Agents form organizations, and adaptation and learning are

important at the organization level

• Agents have a spatial component to their behaviors and

interactions (very interesting for robotics)



Conclusions – Projects at LIACC• Costal Ecosystems Simulation

– Innovative multi-agent ecological simulation system (EcoSimNet,

Ecolang, EcoDynamo, DSS, Agents)

– Agents introducing the human factor/decisions

– Environmental impact studies / bivalve production optimization

• Heterogeneous Robotic Teams Simulation

– Enabled the study of Multi-Agent coordination

– Several MAS coordination methodologies developed with huge

comepetition success

• Simulation of Poker Games

– Possible to include humans and agents in the same simulation

– PokerLang (Poker strategy definition language)


12-03-2010

24


Luís Paulo [email protected]

Http://www.fe.up.pt/~lpreisAssistant Professor at FEUP – Faculty of Engineering of the University of Porto

Member of the Directive Board of LIACC – Artificial Intelligence and Computer Science Lab.

Agent-Based Modelling and

Simulation

Documents

Agent-Based Modellingand Simulation - UPpaginas.fe.up.pt/~lpreis/mams0910/MAMS_Aula3_AgentBasedSimulation.pdf · • Agent-Based Modelling and Simulation (ABMS) – Agent Based Modelling