A Gentle Introduction to Soar, an Architecture for Human Cognition: 2006 Update

A Gentle Introduction to Soar,an Architecture for Human Cognition: 2006 Update

Soft Computing Lab.Yongjun Kim

26th Mar., 2009

Jill Fain Lehman, John Laird, Paul Rosenbloom

Outline

• Introduction

• Soar (State operator and result) Architecture– The Idea of Architecture– What Cognitive Behaviors have in common– Behavior as Movement through Problem Spaces– Tying the Content to the Architecture– Memory, Perception, Action, and Cognition– Detecting a Lack of Knowledge– Learning– Putting it all together: a Soar Model of Joe Rookie– Stepping back: the Soar Architecture in review– From Architecture to Unified Theories of Cognition

• Discussion

2/28

Introduction

• Many intellectual disciplines contribute to cognitive science:– Psychology, linguistics, anthropology, artificial intelligence, etc.

• Each discipline provides expertise and contributes microtheories.– Descriptions of regularities in behavior– Theories that try to explain those regularities

• How to know the contributions of each discipline fit in the big pic-ture?

– Go ahead and try to put the whole picture together.– Try to build unified theories of cognition (UTCs).

• A set of general assumptions for cognitive models that account for all of cognition.

• Soar was developed and used as a candidate UTC in 1980’s.– Try to find a set of computationally-realizable mechanisms and struc-

tures that can answer all the questions about cognitive behavior.

3/28

The Idea of Architecture

• Architecture– Architecture is the fixed set of mechanisms and structures.– An architecture stands as a theory of what is common among behavior.– Any complex system can be decomposed into architecture and content.

– Architecture requires content to produce behavior.

• Cognitive Architecture– A theory of the fixed mechanisms and structures that underlie human

cognition.– Soar is used as a cognitive architecture.

4/28

What Cognitive Behaviors have in common

• Soar theory assumes that cognitive behavior has at least the fol-lowing characteristics:

– It is goal-oriented.– It takes place in a rich, complex, detailed environment.– It requires a large amount of knowledge.– It requires the use of symbols and abstractions.– It is flexible, and a function of the environment.– It requires learning from the environment and experience.

• Need to explore the architecture in terms of some particular con-tent in order to see how the architecture contributes to behavior.

– Have to be goal-oriented about something.– Need a scenario.

5/28


• A Simple Scenario from Baseball.

6/28


• A Simple Scenario from Baseball.– Behave in a goal-oriented manner:

• Joe’s goal is to win the game.• A number of subgoals : get the batter out, strike the batter out with a curve ball, etc.

– Operate in a rich, complex, detailed environment:• Many relevant aspects of Joe’s environment.

– Positions of people, the number of balls and strikes, etc.– Use a large amount of knowledge:

• Need to draw on statistics about his own team, his own pitching record, etc.– Behave flexibly as a function of the environment:

• Need to respond to his own perceptions of the environment.– Is it windy?, is the batter left-or right-handed? etc.

– Use symbols and abstractions:• Need to draw on his previous experience by abstracting away from this day and place.

– Learn from the environment and experience:• Joe need to throw Sam a fast ball next time.

7/28


• A Simple Scenario from Baseball.– For Joe to act like a rookie pitcher, many different kinds of knowl-

edge should be given to it.

– Need to find some way to represent and process Joe’s knowledge in Soar.

• Assume that there is an underlying structure to behavior and knowledge.• This structure provides a means for organizing knowledge as a sequence of deci-

sions through a problem space.

8/28

Behavior as Movement through Problem Spaces

• The Space of Possible Actions for Joe.– Must make his decisions with respect to the situation at the moment.– Support two points of view : a static / a dynamic view of Joe’s life.

9/28


• The Abstract Form of a Problem Space.– Consist of states, features, values and operators.– The state is a representation of all the aspects of the situation (inter-

nal and external).– Only one state exists at any time and prior states are not directly ac-

cessible.

10/28

f2 : batter statusv6 : out


• The Abstract Form of a Problem Space.– Movement could be entirely random.– To keep behavior goal-directed, the succession of operators and the re-

sulting state transformations must be guided by the principle of ratio-nality:

• If an agent has knowledge that an operator application will lead to one of its goals then the agent will select that operator.

• Tying the Content of Joe’s World to the Soar Architecture.– Map knowledge into goals, states and operators.– Determine what knowledge becomes part of the state and the opera-

tors.– Find how to know what an operator application will do.– Find how to know when the goal has been achieved.

11/28

Tying the Content to the Architecture

• Knowledge Representation in the Architecture– Must be domain independent.– What is common across all domains and problems?

• In Soar, it is the decomposition of knowledge into goals, problem spaces, states, and operators.

12/28


• Guidelines for Tying the Domain Content to the Architecture– The domain knowledge of the objects and people in the game (K1)

=> the features and values in the state– Knowledge of actions (K5 and K7) => operators– Knowledge about objectives (K4) => goals– Knowledge of the rules of the game (K3) + K1 + K4 + K5 + K7 =>

problem space

• The Operator Selection– Operators that share common tests for goals and situations can be

considered to be part of the same problem space.• Given the initial state and the goal in the game, the operators will be the various

kinds of pitches.

13/28


• The Effect of Operators– Can be defined in two ways:

• Defined by the execution of the operator in the external world using knowledge of physical actions (K7).

• Defined by knowledge of abstract events or particular episodes (K2).

• Goal Evaluation– Determining that the current state is a desired state relies on knowl-

edge of the rules of the game (K3).– The environment can give signals of success and failure.

• Umpire’s judge (“You’re out!”)

• The Modification of Goal and Problem Space– Done by augmenting the state with goals and problem spaces.

• Ex. Joe’s team is ahead in the fifth inning, but rain is on the horizon => out quickly.• Ex. A member of the opposing team on base => short windup.

14/28

Memory, Perception, Action, and Cognition

• Soar’s Memory– Consist of long-term memory (LTM) and working memory (WM).– LTM has three different types: procedural, semantic, and episodic.

15/28


• Memory type– Long-term memory (LTM)

• Procedural : knowledge about how and when to do things.– How to ride a bike, how to solve an algebra problem, etc.

• Semantic : knowledge consists of facts about the world.– Bicycles have two wheels, a baseball game has nine innings, etc.

• Episodic : knowledge consists of things you remember.– The time you fell off your bicycle and scraped your elbow.

• LTM is not directly available, but must be searched to find what is relevant to the current situation.

• Procedural knowledge is primarily responsible for controlling behavior and maps directly onto operator knowledge.

– Working memory (WM)• Knowledge that is most relevant to the current situation.• In Soar, WM is represented as a set of the features and values that make up the cur-

rent state (and substates).• Can be used to retrieve other knowledge from LTM.• Working memory elements in Soar arise in one of two ways:

– Through perception.– Through retrievals from long-term memory. 16/28


• Examples of LTM procedural knowledge– There are dependencies between the rules. However, Soar doesn’t

recognize them.– Rules are processed by the architecture in a general way.

17/28


• The Decision Cycle– Generate behavior out of the content in LTM and WM.– Do its work in five phases:

• Input– WM elements are created.

• Elaboration– WM elements are matched against the “if” parts of the rules in LTM.

• Decision– Decide suggestions according to preferences (symbolic/numeric).

• Application• Output

– Support limited parallelism.• Multiple actions can be packaged together as a single operator.

18/28

Detecting a Lack of Knowledge

• Impasse in the Decision Cycle– Happen when the decision cycle can’t decide a single operator due

to lack of knowledge for preferences (e.g., without r5).– Soar automatically creates a substate.

• The goal is to select between two operators for the original state.– Semantic and episodic memories are usually used in substates.

• The reminding is goal-driven.

19/28


• Memory Search in Impasse– Assume that Joe has the following fact in episodic memory.

– A Cue must be created that can be used to search the memory.

– In some cases, no likely match will be returned. Then, the model can modify the cue.

– The next three rules define the evaluate operator that creates prefer-ences to resolve the tie.

20/28


• Resolving an Operator-Tie Impasse

21/28


• Working Memory Hierarchy– Working memory consists of a state/substate hierarchy.– The hierarchy grows as impasses arise and shrinks as impasses are

resolved.– If multiple changes are suggested in different states, the change to

the state that is highest in the hierarchy is selected.– If a change occurs to a context high up in the hierarchy, then all the

substates below the changed state disappear.

• Impasse Type in Soar– Other types of impasses can be occurred.

• E.g., an operator tie, an operator no-change impasse, etc.– The full set of impasses defined in Soar is fixed and domain-inde-

pendent.

22/28

Learning

• Four Learning Mechanisms– Chunking– Reinforcement learning– Episodic learning– Semantic learning

• Three Questions for Learning Systems:– What do they learn?

• Soar systems learn structures for its LTM: rules, declarative facts, and episodes.– What is the source of knowledge for learning?

• Different between the learning mechanisms.– When do they learn?

• Different between the learning mechanisms.

23/28

Learning

• Chunking– Is the most developed learning mechanism.– Is deductive and compositional.– Resolving impasses can lead to learn new rules (called chunks).

• Reinforcement Learning– Knowledge source is feedback from the environment (reward).– Learn rules that generate preferences based on future expected re-

wards.– Two parts in Soar

• Must learn rules that test the appropriate features of the states and operators.– Initially create rules based on the rules that propose operators, and specialize

them to consistently predict the same value.• Must learn the appropriate expected rewards for each rule.

– Done by comparing the prediction of a rule with what happens during the next decision.

24/28

Learning

• Episodic Memory– Knowledge source is the stream of experience.– Episodes are recorded automatically as a problem is solved.

• An episode consists of a subset of the WM elements that exist at the time of record-ing.

– Comparison with chunking and reinforcement learning• Passive learning mechanism.• The contents of an episode are determined only indirectly by reasoning.• No distinction between conditions for retrieval and what should be retrieved.

• Semantic Memory– Knowledge source is the co-occurrence of structures in WM.

• Knowledge about the rules of baseball, what is a home run, etc.– When to store a structure in semantic memory is a research issue.– Comparison with chunking and reinforcement learning

• Deals with static structure instead of derivation-based rules.– Comparison with episodic memory

• More general than episodes.• Place and time information is disassociated. 25/28

Putting it all together: a Soar Model of Joe Rookie

• To Build a Full Model.– Specify the domain knowledge and which memories it is stored in.– Tie the domain knowledge to state structures and operators.– Specify the relationships between different levels by the impasses

and the kinds of knowledge that will be missing, and learned.

26/28

Stepping back: the Soar Architecture in review

• A Cognitive Architecture– Is a fixed set of mechanisms and structures that process content to

produce behavior.– Is a theory, or point of view, about what cognitive behaviors have in

common.

• Soar Architecture– States and Operators– Working Memory– Long-term memory– The Perception/Motor Interface– The Decision Cycle– Impasses– Four Learning Mechanism

27/28

From Architecture to Unified Theories of Cognition

• The model of Joe Rookie is a content theory.– Can explain why human needs change with time.– Can explain the specific factors that motivate people.

• Soar has contributed a number of content theories to the field.– NTD-Soar, Instructo-Soar, IMPROV, TacAir-Soar, Soar MOUTBOT, etc.

• Content theories model aspects of human language (or concept learning, or multi-tasking) within a framework.

• The theory of the resulting model will be compatible with what is assumed to be architectural in the other content theories.

• Content theories constitute a burgeoning unified theory of cog-nition (UTC).

– Soar as a UTC.28/28

Discussion

• What is the similarity and difference between Soar and ACT-R?– Both try to model human.– Soar has focused on memory system, but ACT-R on brain image.

• Do you think Soar can be a UTC?– A UTC must explain the following things.

• How intelligent organisms flexibly react to stimuli from the environment.• How they exhibit goal-directed behavior and acquire goals rationally.• How they represent knowledge (or which symbols they use).• Learning.

• It is known that there is efforts to model emotions into Soar. Do you think emotions can play a big role in cognition?

29/28

Documents

A Gentle Introduction to Soar, an Architecture for Human Cognition: 2006 Update