Operant Conditioning

Operant ConditioningJanuary 26th, 2010Psychology 4851OutlineHistory & IntroductionThree major questions:What is learned?Why learn?How does learning happen?2Classical vs. OperantClassicalRequires reflex actionNeutral stimulus associated with USOutside of subjects controlOperantStrengthening/weakening of voluntary actionSubject responds or doesnt3Classical vs OperantClassical = Prediction problemWhats going to happen?

Operant = Control problemWhat to do to maximize reward?4Whats in a Name?Operant learning: subject operates on environmentInstrumental conditioning: subject is instrumental in obtaining outcome5Control of BehaviourControlELearn to control an animals behaviour through manipulation of its environmentDiscriminative Stimuli - SD

ControlAUnderstand behaviour as an agent controlling its actions based on predicted outcomes6Instrumental ConditioningE. L. ThorndikePuzzle boxesLaw of EffectAny behaviour followed by an appetitive stimulus will increase in frequencyVice versa

7Operant ConditioningB. F. SkinnerOperant boxesFree operant procedure

8Blank SlateGive me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I'll guarantee to take any one at random and train him to become any type of specialist I might select doctor, lawyer, artist, merchant-chief and, yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors.

9Blank SlateGive me a dozen healthy infants, well-formed, and my own specified world to bring them up in and I'll guarantee to take any one at random and train him to become any type of specialist I might select doctor, lawyer, artist, merchant-chief and, yes, even beggar-man and thief, regardless of his talents, penchants, tendencies, abilities, vocations, and race of his ancestors. I am going beyond my facts and I admit it, but so have the advocates of the contrary and they have been doing it for many thousands of years.

10What is learned?11What can be learned?Skinner believed any complex behaviour could be conditionedWalden TwoPigeon Project

12Contiguity & Contingency= bar press= foodPerfect contingencyStrong RespondingDegraded contingencyWeak Responding13Contiguity & ContigencySuperstitious behavioursSkinner15 s FT reinforcementReinforcement not contingent on behaviourPigeons repeat behaviour that occurs before reinforcementContiguity, but not contingency14Contiguity & ContingencyDifferent contingencies lead to different behaviour patternsSchedules of reinforcementFixed vs VariableRatio vs Interval

15Type of AssociationStimulus Response associationOutcome serves to strengthen (or weaken) associationStamps in the connectionSRO16Type of AssociationResponse Outcome associationGoal directed behaviourSRO17ResponseOutcomeColwill & Rescorla (1986)Phase 1Devaluation TestPush LeftPelletPellet+LiClRight?

Push RightSucroseSucrose+LiClLeft?# PushesPellet Devalued Sucrose DevaluedRight Pushes Left Pushes18Type of AssociationStimulus primes outcomeMotivates responding that leads to outcomeDOEPavlovian-Instrumental TransferSRO19Differential Outcomes EffectFaster learning, better accuracy & retention for DOE groupSuggests S-R-O encodingNo rewardNo rewardPeas & CornPeas & CornNo rewardNo rewardPeasCornControl GroupDOE Group20Pavlovian-Instrumental TransferPhase 1Phase 2TestLeverFoodLightFoodLight: #Presses?No Light: #Presses?# PressesLightNo CSThe presence of the CS intensifies operant responding21Why learn through operant conditioning?22Why operant?Seems obvious:Getting more reinforcement is sure to be beneficial to the organism

But, what is a reinforcer?What exactly are we working for?

Reinforcement is a difficult term to define non-circularly23Premack PrincipleBehaviors are reinforcing, not stimuliTo predict what will be reinforcing, observe the baseline frequency of different behaviorsHighly probable behaviors will reinforce less probable behaviors24Response Deprivation HypothesisLow frequency behaviors can reinforce high frequency behaviors (and vice versa)

All behaviors have a preferred frequency = the behavioral bliss point

Deprivation below that frequency is aversive, and organisms will work to remedy this25Example: How often do you scratch your leg? Not very often. But if you are put in a cast, you will come to feel that leg-scratching is very desirable. Positive & Negative ReinforcersPositive Reinforcement

Lever Press FoodPositive Punishment

Lever Press ShockNegative Reinforcement

Lever Press Shock offNegative Punishment

Lever Press Food removedIncreasesDecreasesResponse Rate:Stimulus:AddedRemoved26How does learning happen?27Choice: MatchingHow to allocate behaviors between multiple options based on the consequences of actions?

Led to behavioural & neuro-economicsProspect TheoryResp A Rf. Rate AResp BRf. Rate B=28ShapingHow to create novel responses?Skinner (1943)Pigeon bowlingresponses that more closely approximated the final formSuccessive approximationsFirst described in 1937Why would this surprise Skinner?

29LimitationsSome behaviours cannot be easily conditioningYawning, scratching

BelongingnessPresence of female wont reinforce biting

Instinctual DriftImportance of animals natural ecology30Nature vs NurtureWhich is more important?Which is stronger?A.I. built in algorithms or learning?31