Software Agent- MAS: multi-agent systems-

Outline

• Definition

• Issues and elements of MAS– MAS architectures– Coordination– Collaboration– Several issues in designing competitive MAS

• Applications

• MAS research direction

• Summary

2/28

Multi-agent Systems

• A multi-agent system contains a number of agents…– …which interact through communication…– …are able to act in an environment…– …have different “spheres of influence” (which may coincide)…– …will be linked by other (organizational) relationships

• MAS as seen from distributed AI– A loosely coupled network of entities that work to-

gether to find answers to problems that are beyond the individual capabilities or knowledge of each en-tity

• A more general meaning– systems composed of autonomous components

that exhibit the following characteristics:• each agent has incomplete capabilities to solve a

problem• there is no global system control• data is decentralized• computation is asynchronous

3/28

Overview of MAS

• Aspects of multi-agent systems– Cooperative vs. competitive– Homogeneous vs. heterogeneous– Macro vs. micro– Interaction protocols and languages– Organizational structure– Mechanism design / market economics– Learning

• Types of MAS– Cooperative MAS

• Distributed problem solving: Less autonomy• Distributed planning: Models for cooperation and teamwork• Typical (cooperative) MAS domains

– Distributed sensor network establishment– Distributed vehicle monitoring– Distributed delivery

– Competitive or self-interested MAS• Distributed rationality: Voting, auctions• Negotiation: Contract nets

4/28

Comparison with Traditional Approaches

• Traditional– Client-server– Low-level messages– Synchronous– Can not do the job!

• Agent break-throughs

– Peer-to-peer topology– Blackboard coordina-

tion model– Encapsulated mes-

saging– High-level message

protocols

A14<MAS>-5

Client Server

IntelligentAgents

IntelligentAgents

IntelligentAgents

Function(Parameters)

Return(Parameters)

BlackboardMessage

Reply

Traditional Software

Agents IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

IntelligentAgentsIntelligent

Agents

5/28

Main Points in MAS

• MAS researchers develop communications languages, interaction protocols, and agent architectures that facilitate the development of multi-agent systems

• MAS researcher can tell you how to program each ant in a colony in order to get them all to bring food to the nest in the most efficient manner, or how to set up rules so that a group of selfish agents will work together to accomplish a given task

• MAS researchers draw on ideas from many disciplines outside of AI, including biology, sociology, economics, organization and manage-ment science, complex systems, and philosophy

6/28

Key Elements of MAS

• A coordination mechanism supported by a common agent communi-cation language and protocol

• A collaboration mechanism supported by agent community architec-ture (including agent and interaction architecture) to support the or-ganization goal

• A shared ontology

• Popular MAS architectures– Object Manager Group (OMG)– Foundation for Intelligent Physical Agents (FIPA)– Knowledgeable Agent-oriented System (KAoS)– Open Agent Architecture (OAA)– General Magic group

7/28

MAS Architectures (1)

• OMG’s Model– Composed of agents and agencies that collaborate using general patterns and

policies– Agents are characterized by: capabilities, type of interaction and mobility– Agencies support:

• concurrent execution of agents• security• agent mobility

• FIPA’s Model – Agents– Agent Platform (AP)– Directory Facilitator (DF)– Agent Management System (AMS)– Agent Communication Channel (ACC)– Agent Communication Language (ACL)

8/28

MAS Architectures (2)

• KAoS’s Model– An Open Distributed Architecture for Software agents– Defines various agent implementations– Uses conversation policies to elaborate on agent-to-agent communication

• OAA Model

9/28

MAS Architectures (3)

• General Magic’s Model – A commercial agent technology for electronic commerce– Views MAS as an electronic marketplace– The marketplace is modeled as a network of computers supporting a collection of

places that offer services to mobile agents– The mobile agents:

• can travel, meet other agents, create connections to other places• they have authority

• Zeus: a MAS development toolkit

A

B

C

D

Agent Facilitator

Abilit ies Database

Agent Name Server

Address Book

request

reply

Transport Protocol

MESSAGE

Common Message Format (Language)

Shared mesage content representation and ontology

Agent

Perform Task A

Agent

Perform Task C

Agent

Perform Task D

External program

10/28

MAS Architectures (4)

• Geo-Agents (GIS agents) Architecture

Geo-Agents

Domain (Service) AgentDomain (Service) Agent

Task Agent

Facilitator

Administrator UI Agent

Task Agent

Other Agent Systems UserQuery agent

Exchange reg-istry

Query agent

Query agent

Query agent Pass taskReply

Coordi-nate

Coordi-nate

Collabo-rate

Control/Re-ply

Task(GeoScrip

t)Reply

Collabo-rate

Data sources

Retrieve

11/28

Coordination

• Coordination: a process to manage dependencies among activities

• Three aspects of coordination– Activity aspect

• What activity to execute?• When an activity should be executed?• Model to coordinate distributed tasks: Statecharts, Flowcharts, Process algebra, Lotos,

SDL, Estelle …– Conversation (state) aspect

• What is the structure of the conversation among the coordinating entities?• FSM, Petri-Nets, State Transition Diagrams

– Implementation aspect• How to implement distributed software systems where software components coordinate

their actions

12/28

KQML

• Knowledge Query and Manipulation Language (KQML) is both a message format and a message-handling protocol to support run-time knowledge sharing among agents

• KQML comprise a substrate on which to develop higher-level models of inter-agent interaction such as contract nets

• KQML is a coordination mechanism from the conversation aspect

• KQML contains an extensible set of performatives, which defines the permissible speech acts agents may use

• Example performative:

Coordination

(ask-all /* message layer */ :content "price(IBM, [?price, ?time])“

/* content layer */ :receiver stock-server

/* communication layer */ :language standard_prolog :ontology NYSE-TICKS

:sender me)

13/28

KQML: Types of Performatives

• Basic informative performatives: tell, deny, … • Database performatives: insert, delete, … • Basic responses: error, sorry, … • Basic query performatives: ask-one, ask-all, evaluate,… • Multi-response query performatives: stream-all, …• Basic effector performatives: achieve, … • Generator performatives: standby, ready, next, … • Capability-definition performatives: advertise • Notification performatives: subscribe • Networking performatives: register, forward, pipe, broadcast, … • Facilitation performatives: broker-one (all), recommend-one (all), re-

cruit-one (all)

Coordination

14/28

Collaboration

• Collaboration refers to cooperative effort among agents to reach a single goal by exchanging knowledge built upon the underlying coor-dination mechanism

• Example mechanism: Contract Net Protocol (CNP)– Negotiation as a collaboration mechanism – Negotiation on how tasks should be shared

• A task (plan) may be decomposed in a hierarchy of subtasks (hierarchical planning)• An agent may subcontract another agent to perform a (sub)task.

Con-tract

Bid

agent agent

15/28

Collaboration

Contractor

Potential candidate agents

Task announcement ("broadcast")

Contractor

Candidate Candidate

Bid

Bid

Phase 1: Task Announcement- The contractor agent publicly announces a task.- Potential candidates evaluate the task according to their won skills and availability.

Phase 2: Submission of Bids / Proposals- Agents that satisfy the requiremenst, i.e., are able to perform the task, send their bid / proposal to the contractor.

16/28

Collaboration

Contractor

Selected candidate

Contractor

Contracted agent

Contract

Phase 3: Selection- The selection of the best candidate is made by the contractor based on received bids and on the CVs of the candidates.

Phase 4: Contract awarding- A contract is established between the contractor and the selected candidate.- A privileged bilateral communication channel is established between the two agents.

17/28

Several Issues in Designing Competitive MAS

• Distributed rationality

• Pareto optimality

• Stability

18/28

Distributed Rationality

• Techniques to encourage/coax/force self-interested agents to play fairly in the sandbox

– Voting: Everybody’s opinion counts (but how much?)– Auctions: Everybody gets a chance to earn value (but how to do it fairly?)– Contract nets: Work goes to the highest bidder– Issues:

• Global utility• Fairness• Stability• Cheating and lying

Competitive MAS

19/28

Pareto Optimality

• S is a Pareto-optimal solution iff– S’ ( x Ux(S’) > Ux(S) → y Uy(S’) < Uy(S))– i.e., if X is better off in S’, then some Y must be worse off

• Social welfare, or global utility, is the sum of all agents’ utility– If S maximizes social welfare, it is also Pareto-optimal (but not vice versa)

Competitive MAS

X’s utility

Y’s utility

Which solutionsare Pareto-optimal?

Which solutionsmaximize global utility(social welfare)?

20/28

Stability

• If an agent can always maximize its utility with a particular strategy (regardless of other agents’ behavior) then that strategy is dominant

• A set of agent strategies is in Nash equilibrium if each agent’s strat-egy Si is locally optimal, given the other agents’ strategies

– No agent has an incentive to change strategies– Hence this set of strategies is locally stable

• Prisoner’s dilemma– Pareto-optimal and social welfare maximizing solution: Both agents cooperate– Dominant strategy and Nash equilibrium: Both agents defect

Competitive MAS

Cooperate Defect

Cooperate 3, 3 0, 5

Defect 5, 0 1, 1

21/28

Development of MAS

• Define the organization of the MAS according to the problem specifi-cation (or solution structure)

• Decide the coordination mechanism• Select a MAS implementation framework, e.g., Zeus, that supports

the coordination mechanism• Implement the collaborative mechanism which support the MAS or-

ganization• Implement shared ontology • Implement each task agent (including customizing associated com-

munication module)• Customize middle agents

– Facilitators– Mediators– Brokers– Matchmakers and yellow pages– Blackboards

22/28

Applications of MAS

• Advanced Manufacturing Management Systems– Agents as representatives of machines, users, business processes, etc.

• Intelligent Information Search on Internet– Some agents may show learning capabilities (learn the preferences of their

users, ..)• Intelligent security enforcement on Internet

– Agents are representative of sensors or IDSs• Shopping Agents in Electronic Commerce

– With search, price comparison, and bargaining capabilities• Multi-agent auction in E-commerce• Distributed Surveillance

– For information search or to look for special events informing their users of relevant news

• Distributed Signal Processing– For problem diagnosis, situation assessment, etc. in the network

• Distributed Problem Solving– Collaborative design, scheduling, and planning

23/28

Agent Organizations

• Multiple (human and/or artificial) agents

• Goal-directed (goals may be dynamic and/or conflicting)

• Affects and is affected by the environment

• Has knowledge, culture, memories, history, and capabilities (distinct from individual agents)

• Legal standing is distinct from single agent

• Q: How are MAS organizations different from human organizations?

MAS Research Directions

24/28

Organizational Structures

• Exploit structure of task decomposition– Establish “channels of communication” among agents working on related subtasks

• Organizational structure:– Defines (or describes) roles, responsibilities, and preferences– Use to identify control and communication patterns:

• Who does what for whom: Where to send which task announcements/allocations• Who needs to know what: Where to send which partial or complete results

MAS Research Directions

25/28

Communication

• Communication models– Theoretical models: Speech act theory– Practical models:

• Shared languages like KIF, KQML, DAML• Service models like DAML-S• Social convention protocols

• Communication strategies– Connectivity (network topology) strongly influences the effectiveness of an organi-

zation– Changes in connectivity over time can impact team performance:

• Move out of communication range coordination failures• Changes in network structure reduced (or increased) bandwidth, increased (or reduced)

latency

MAS Research Directions

26/28

Learning in MAS

• Emerging field to investigate how teams of agents can learn individu-ally and as groups

• Distributed reinforcement learning– Behave as an individual, receive team feedback, and learn to individually contribute

to team performance– Iteratively allocate “credit” for group performance to individual decisions

• Genetic algorithms: Evolve a society of agents (survival of the fittest)

• Strategy learning: In market environments, learn other agents’ strate-gies

MAS Research Directions

27/28

Adaptive Organizational Dynamics

• Potential for change:– Change parameters of organization over time– That is, change the structures, add/delete/move agents, …

• Adaptation techniques:– Genetic algorithms– Neural networks– Heuristic search / simulated annealing– Design of new processes and procedures– Adaptation of individual agents

MAS Research Directions

28/28

Summary

• “Agent” means many different things

• Different types of “multi-agent systems”:– Cooperative vs. competitive– Heterogeneous vs. homogeneous– Micro vs. macro

• Lots of interesting/open research directions:– Effective cooperation strategies– “Fair” coordination strategies and protocols– Learning in MAS– Resource-limited MAS (communication, …)

• Next lecture– Communication & Platform

29/28