Statistical Learning in Infants(and bigger folks)

Statistical Learning

• Neural network models emphasize the value of statistical information in language

– What information can be extracted from this?

– Is this sufficient to account for human performance?

– Are humans able to perform this kind of analysis?

– If so, does it contribute to an understanding of the uniquely human ability to learn language?

Saffran, Aslin, & Newport (1996)

• 8-month old infants– Passive exposure to continuous speech (2 mins)

bidakupadotigolabubidaku…– Test (Experiment #2)

bidakubidakubidakubidakubidaku…kupadokupadokupadokupadokupado…

– Infants listen longer to unfamiliar sequences– Transitional Probabilities

bi da ku pa do ti

1.0 1.0 1.0 1.0

.33

JennySaffran

DickAslin

ElissaNewport

What is it good for?

• Word Learning

– Transitional probabilities: local minima = word boundaries

– Saffran’s example: ‘pretty baby’ /prItibebi/p (ti|prI) = 0.8p (be|ti) = 0.03

• How else could children segment words?

– Words in isolation (Peters, 1983; Pinker, 1984)

– Stress-based segmentation: 90% of English words are stress-initial (Cutler & Carter, 1987)

– Phonotactic segmentation, e.g., *dnight (Gambell & Yang, 2005)

Are Local Minima Effective?

• Gambell & Yang (2005) -

– Adult input to children from 3 corpora in CHILDES

– 226,178 words, 263,660 syllables

– Precision: hits/(hits + false alarms) 41.6%Recall: hits/(hits + misses) 23.3%

More Statistical Learning

• Additional Stimulus types– Tones– Shapes– etc.

• Additional species…

Cotton-top Tamarin“Jackendoff”

Marc Hauser

Where do constraints come from?

• Substantive ConstraintsIf the statistical learning mechanism is able to pick up regularities that go beyond those found in natural languages, then there must be additional substantive linguistic constraints that provide the restrictions on natural languages

• Constraints on Learning & Processing“… some of the constraints on natural language structure might arise from constraints on the computational abilities this mechanism exhibits.” (p. 130)

Albert Bregman

k t b| | |C - V - C - V - C | | a a

Autosegmental Phonology

Where do constraints come from?

• “This compatibility between learning and languages in turn suggests that natural language structures may be formed, at least in part, by the constraints and selectivities of what human learners find easy to acquire.” (p. 159)

Where do constraints come from?

• How well does this generalize?

Where do constraints come from?

• Substantive Constraints vs. Constraints on Learning or Processing

Rather than removing the need for substantive constraints, Newport’s approach seems to shift the burden of explanation onto the theory of representations

JennySaffran

Curr. Dir. Psych. Sci., 12: 110-114 (2003)

Experiment 1 - Syllable Size

• Step 1: Pattern Induction

– Regime A: CVCV words, e.g., boga, diku

– Regime B: CVCCVC words, e.g., bikrub, gadkug

• Step 2: Segmentation

– 4 words: [baku, dola], [tupgod, girbup]

– Continuous stream: tupgodbakugirbupdolabaku…

• Step 3: Testing

– Same words used in segmentation: [baku, dola], [tupgod, girbup]

– Infants listened longer to words consistent w/ induced pattern

Experiment 2 - Phonotactics

• Step 1: Pattern Induction

– Regime A: -V+V syllables, e.g., todkad, pibtug

– Regime B: +V-V syllables, e.g., dakdot, gutbip

• Step 2: Segmentation

– 4 words: [kibpug, pagkob], [bupgok, gikbap]

– Continuous stream: pagkobbupgokgikbapkibpug…

• Step 3: Testing

– Same words used in segmentation: [kibpug, pagkob], [bupgok, gikbap]

– Infants listened longer to words inconsistent w/ induced pattern

Experiment 3 - Unnatural Phonotactics

• Experiment 2

– -V+V pattern is stated over a feature-based class: /p,t,k/ vs. /b,d,g/

• Experiment 3

– Modify segment ‘groupings’: /p,d,k/ vs. /b,t,g/

– Other details just like Experiment 2

– No listening preference at test phase

Conclusion

• “To the extent that patterns that do not occur in natural languages are more difficult to acquire, we may consider the possibility that constraints on how infants learn may have served to shape the phonology of natural languages. Patterns that are difficult to acquire are less likely to persist cross-linguistically than those that are easily learned. Thus, languages may exploit devices such as voicing regularities in part because they are readily acquired by young learners.”

[Saffran & Thiessen 2003, p. 491]

(Peña et al. 2002)

Marcus et al. (1999)

• Training

– ABA: ga na ga li ti li– ABB: ga na na li ti ti

• Testing

– ABA: wo fe wo– ABB: wo fe fe

Gary Marcus

#1: ABB vs. ABA#2: ABB vs. ABA#3: ABB vs. AAB

Marcus, Fernandes & Johnson, 2007 (Psychological Science)

So what are we learning…?