Psych 156A/ Ling 150: Acquisition of Language II Lecture 2 Introduction: Mechanism of Language Acquistion

Psych 156A/ Ling 150:Acquisition of Language II

Lecture 2

Introduction:

Mechanism of Language Acquistion

Announcements

Start working on HW1 (due: 4/19/12)

Be looking over the review questions for introduction

Readings for next time:

Werker (1995) + (recommended) Swingley (2009)

What’s being learned:

Patterns or “rules” of language = grammar

A distinction: Prescriptive vs. Descriptive Grammar Rules

Prescriptive: what you have to be taught in school, what is prescribed by some higher “authority”. You don’t learn this just by listening to native speakers talk.

“Don’t end a sentence with a preposition.”“ ‘Ain’t’ is not a word.”

A distinction: Prescriptive vs. Descriptive Grammar Rules

Descriptive: what you pick up from being a native speaker of the language, how people actually speak in their day-to-day interactions. You don’t have to be explicitly taught to follow these rules.

The dwarf is who Sarah first talked with.

“You’re horrible!” “No, I ain’t - I’m Hoggle!”

Possible objections to a mental rule set

“Why should I believe I store a set of rules unconsciously in my mind? I just understand sentences because they make sense.”


But why do some sentences make sense and others don’t?

Hoggle has two jewels.*Two Hoggle jewels has.

“Why should I believe I store a set of rules unconsciously in my mind? I just understand sentences because they make sense.”


Why can we recognize patterns even when some of the words are unknown?

‘Twas brillig, and the slithy tovesdid gyre and gimble in the wabe...

Possible objections to an unconscious rule set

“When I talk, the talk just comes out - I’m not consulting any rule set.”

Possible objections to an unconscious rule set

“When I talk, the talk just comes out - I’m not consulting any rule set.”

Analogy: wiggling your fingers

When you want to wiggle your fingers, you “just wiggle them”.

But your finger-wiggling intention was turned into commands sent by your brain to your muscles, and you’re never conscious of the process unless something interferes with it. Nonetheless, there is a process, even if you’re not aware of it.

Main points

How do we explain how this process works?

Language acquisition is a process that involves inferring a structured system from the available input.

Levels of RepresentationMarr (1982)

Describing vs. Explaining in Vision

“…it gradually became clear that something important was missing …neurophysiology and psychophysics have as their business to describe the behavior of cells or of subjects but not to explain such behavior….What are the problems in doing it that need explaining, and what level of description should such explanations be sought?” - Marr (1982)

On Explaining (Marr 1982)

“But the important point is that if the notion of different types of understanding is taken very seriously, it allows the study of the information-processing basis of perception to be made rigorous. It becomes possible, by separating explanations into different levels, to make explicit statements about what is being computed and why…”

On Explaining (Marr 1982)

“But the important point is that if the notion of different types of understanding is taken very seriously, it allows the study of the information-processing basis of perception to be made rigorous. It becomes possible, by separating explanations into different levels, to make explicit statements about what is being computed and why…”

Our goal: Substitute “language learning” for “perception”.

The three levels

Computational What is the goal of the computation?

Algorithmic How can this computational theory be implemented? What is the representation for the input and output, and what is the algorithm for the transformation?

Implementational How can the representation and algorithm be realized physically?

The three levels: An example with the cash register

Computational What does this device do? Arithmetic (ex: addition).Addition: Mapping a pair of numbers to another number. (3,4) 7 (often written (3+4=7))Properties:

(3+4) = (4+3) [commutative](3+4)+5 = 3+(4+5) [associative](3+0) = 3 [identity element](3+ -3) = 0 [inverse element]

True no matter how numbers are represented: this is what is being computed


Computational What does this device do? Arithmetic (ex: addition).Addition: Mapping a pair of numbers to another number.

Algorithmic What is the input, output, and method of transformation?

Input: arabic numerals (0,1,2,3,4…)Output: arabic numerals (0,1,2,3,4…)Method of transformation: rules of addition, where least

significant digits are added first and sums over 9 have their next digit carried over to the next column

99 + 5






99 + 5

14






1

99 + 5

4






1

99 + 5

104




Input: arabic numerals (0,1,2,3,4…)Output: arabic numerals (0,1,2,3,4…)Method of transformation: rules of addition


A series of electrical and mechanical components inside the cash register.

The three levels: An example with a sandwich


Input: arabic numerals (0,1,2,3,4…)Output: arabic numerals (0,1,2,3,4…)Method of transformation: rules of addition



Computational What is the goal?

Make a peanutbutter and jelly sandwich.

Properties:- slices of bread containing both peanutbutter and

jelly- number of bread slices: 2- sandwich is sliced in half- crusts are left on- jelly type: grape- peanutbutter type: crunchyetc.







Input: ingredients (peanutbutter, jelly, bread slices), tools (knife, spoon)

Output: completed, edible sandwich with the required propertiesMethod: Use the spoon to put jelly on one slice & spread it with

the knife. Use the spoon to put peanutbutter on the other slice & spread it with the knife. Put the two slices of bread together, with the spread sides facing each other. Cut the joined slices in half with the knife.



Directing your younger sibling to follow the steps above to make you a sandwich.




Input: ingredients (peanutbutter, jelly, bread slices), tools (knife, spoon)

Output: completed, edible sandwich with the required propertiesMethod: PBJ-making steps.

Mapping the Framework:Algorithmic Theory of Language Learning

Goal: Understanding the “how” of language learning

First, we need a computational-level description of the learning problem.

Computational Problem: Divide sounds into contrastive categories

xx

x

xx

xx x

x x

x

xx

x x

x

x

x

x

x

x

xxC1 C2

C3C4

xx

x

xx

xx x

x x

x

xx

x x

x

x

x

x

x

x

xx




Computational Problem: Divide spoken speech into words

who‘s afraid of the big bad wolf




Computational Problem: Map word forms to speaker-invariant forms

fwiends

friends

friends

“friends”




Computational Problem: Identify grammatical categories

“This is a DAX.”

DAX = noun




Computational Problem: Identify the concept a word is associated with

(Word-meaning mapping)

“I love my daxes.”

Dax = that specific toy, teddy bear, stuffed animal, toy, object, …?




Computational Problem: Identify the rules of word order for sentences.

Subject Verb Object

Subject Verb Object

Subject Verb tSubject Object tVerb

EnglishGermanKannada

Subject tObject Verb Object

Jareth juggles crystals



Second, we need to be able to identify the algorithmic-level description:

Input = sounds, syllables, words, phrases, …

Output = sound categories, words, words with affixes, grammatical categories, sentences, …

Method = statistical learning, prior knowledge about how human languages work, …

Recap: Levels of Representation

Language acquisition can be viewed as an information-processing task where the child takes the native language input encountered and uses it to construct the adult rule system (grammar) for the language.

Main idea: The point is not just to describe what children know about their native language and when they know it, but also how they learned it.

Three levels:

computational: what is the problem to be solved

algorithmic: what procedure will solve the problem, transforming input to desired output form

implementational: how is that procedure implemented/instantiated in the available medium

Computational Modeling: Understanding the Mechanism

Computational Level:

Theoretical linguistic studies can often tell us what needs to be learned about language. Experimental studies can often tell us about when children seem to know different kinds of language knowledge. This defines the goal of language acquisition:

Learn the appropriate what by the appropriate when.


Algorithmic Level:

But how do we know what the input is, what the output ought to look like, and what method(s) children use to get from the input to the output?


Algorithmic Level:

Input: The CHILDES database has a wealth of child-directed speech transcripts and videos from a number of different languages. This can help us figure out what children’s input looks like.

http://childes.psy.cmu.edu/


Algorithmic Level:

Output: Theoretical linguistics and experimental studies can tell us what the output should look like by observing adult and child knowledge of various linguistic phenomena.


Example problem: word segmentation

output

input


Algorithmic Level:

Method: Computational modeling can often help us figure out how children are getting from the input to the output.


What goes here?

Computational Modeling: What a “Digital” Child Can Tell Us

We can construct a model where we have precise control over these:

• The hypotheses the child is considering at any given point

[hypothesis space]

• How the child represents the data & which data the child uses

[data intake]

How the child changes belief based on those data

[update procedure]

“I love my daxes.”





[hypothesis space]


[data intake]

How the child changes belief based on those data

[update procedure]“I love my daxes.”


daxes




[hypothesis space]


[data intake]

• How the child changes belief based on those data

[update procedure]

dax = that specific toy more probable

dax = any object less probable


Models are most informative when they’re grounded empirically.

This is why most models make use of the child-directed speech data available through databases like CHILDES.

Many models will try to make cognitively plausible assumptions about how the child is representing and processing input data:• Processing data points as they are encountered• Assuming children have memory limitations (ex: memory of data points

may decay over time)

General Modeling Process

(1) Decide what kind of learner the model represents (ex: normally developing 6- to 8-month-old child learning first language)

(2) Decide what data the child learns from (ex: Bernstein corpus from CHILDES) and how the child processes that data (ex: divide speech stream into syllables)

(3) Decide what hypotheses the child has (ex: what the words are) and what information is being tracked in the input (ex: transitional probability between syllables)

(4) Decide how belief in different hypotheses is updated (ex: based on transitional probability between syllables)

















(5) Decide what the measure of success is

ex: making correct generalizations• Knowing that dax refers to all teddy bears, even ones the

child hasn’t seen before

ex: achieving a certain knowledge state by the end of the learning period• Recognizing words in a fluent speech stream

The goal of modeling

Remember: the goal is generally to see if a particular learning strategy (as described by the hypothesis space, data intake, and update procedure) will allow the child to go from the input to the output. This then tells us about the process of language acquisition (the algorithmic level of explanation).


What goes here?

Recap: Mechanism of Acquisition

One of the main goals of the study of language acquisition is to explain it, rather than just describe it.

There are three different levels of explanation, according to Marr: the computational level, the algorithmic level, and the implementational level.

The algorithmic level focuses on the process (the “how”) of acquisition, and computational modeling is a technique that can be used to investigate different strategies a child might use to learn language.

Questions?

Use the rest of this class period to look over the review questions and work together on HW1. You should be able to

do all the introductory review questions and up through question 2 on HW1.

Documents

Psych 156A/ Ling 150: Acquisition of Language II Lecture 2 Introduction: Mechanism of Language Acquistion