understanding action verbs- embodied verbal semantics approach pavan kumar srungaram m.phil...
TRANSCRIPT
Understanding Action Verbs- Embodied Verbal Semantics
Approach
Pavan Kumar SrungaramM.Phil Cognitive Science (09CCHL02)
2009-10Supervisor: Prof. Bapi Raju, S.
University of Hyderabad
Language in Two Different Perspectives• Chomskyan view: • Based on structures and rules. • Syntax is autonomous free of
meaning and general cognition• Human language is an abstract
formal system
Problems
• In many cases it was observed that background knowledge, concept, meaning have come into the rules governing syntax.
Language as a part of General Cognition• Lakoff, Fauconnier – Language not to
be studied in terms of structure/ Grammar but the way it is used
• Language and thought are embodied, concepts arise from our sensorimotor experience and the neural substrates that give rise to it.
• Metaphors We Live By !
Embodiment
• Acknowledge the role of body and its sensorimotor processes can and do play in cognition.
• Understanding cognition in the context of biological function to support the activities of the body.
• Cognition as real-time situated activity, inseparable from perception and action.
Behavioural Tests
• Considered the experiments by Bergen et al. (2010) and tried to replicate three of those tests in Telugu.
• In addition to these tests, we tried to verify whether the results produced in those experiments were modality specific
Behavioural Experiment
• There are 4 tasks in this experiment• Each subject participates only in one
task• Task 1: Image-Verb Matching Task• Task 2: Verb-Image Matching Task• Task 3: Verb-Verb Matching Task• Task 4: Image-Verb (auditory)
Matching Task
Hypothesis
• If verb comprehension requires the activation of the motor areas related to the same effector with which the action is performed.
• Windhorst Hypothesis: Motor and perceptual systems display lateral inhibitions among neural structures responsible for related but incompatible functions
Materials and Methods
• Computer Based Test • Two Stimuli presented at specific
times• Subjects are to judge as quickly as
possible whether the two stimuli match
• The response times of the subjects are recorded and analyzed
Materials and Methods
• The stimuli used in the tasks are: a) Images: These contain a single
character performing some action b) Action Verbs written (or) typed both
in program compatible formats. c) Action Verbs (audio) • All these three are recorded and used
in program compatible formats.
Algorithm
• First Stimuli: Presented for 1000 milliseconds
• Inter stimulus interval: 500 milliseconds a) 450 milliseconds: Visual Mask
b) 50 milliseconds: Blank• Second Stimuli presentation: This is
until the subject makes his decision by pressing the designated key.
Task 1: Image-Verb Matching Task• Image is shown for a period of 1000
milliseconds
• 500 milliseconds inter stimulus interval
• This is followed by a verb in typed format
• Show images and verbs!
Results
• The observation is in accordance with the hypothesis.
• Subjects took longer time to reject the image-verb pairs of non-matching same effector condition compared to those in non-matching different effector condition.
Means Table
ConditionAverage RT in milliseconds Standard Deviation
Matching Condition 1281.34 444.4169
Non-Matching Same effector Condition 1502.599 431.3717
Non-Matching Different effector Condition 1394.467 386.6818
ANOVA: Analysis of Variance
RT
Sum of Squares df Mean Square F Sig.
Between Groups 1175122.403 2 587561.202 3.306 .040
Within Groups 2.506E7 141 177703.579
Total 2.623E7 143
Condition 1: Matching ConditionCondition 2: Non-Matching Same effector ConditionCondition 3: Non-Matching Different effector Condition
Results
• The results obtained in the test provide evidence that understanding action verbs, language users recruit neural resources that are normally used for performing an action. This also supports the claim that understanding language might require simulation of action.
Result of Image-Verb Matching Task
Test Statisticsa,b
RT
Chi-Square 7.939
df 2
Asymp. Sig. .019
a. Kruskal Wallis Test
b. Grouping Variable: Condition
Ranks
Condition N Mean Rank
RT 1.00 48 59.75
2.00 48 83.56
3.00 48 74.19
Total 144
Alternative explanation
• An alternative explanation for the observed effect is that seeing images might have led subjects to activate representations of other actions using the same effector and that the verb presented might be the actions related to those activated actions that might have resulted in delay in rejection
Task 2:Verb-Image Matching Task• Verb in typed format is shown for a
period of 1000 milliseconds
• 500 milliseconds inter stimulus interval
• This is followed by an image
Means Table
Condition
Average RT (in
milliseconds) Standard Deviation
Matching Condition 1020.568 374.1628
Non-Matching Same
Effector Condition1176.36 492.6629
Non-Matching Different
Effector Condition1111.11 529.8168
Condition 1: Matching ConditionCondition 2: Non-Matching Same effector ConditionCondition 3: Non-Matching Different effector Condition
ANOVA
RTSum of Squares df Mean Square F Sig.
Between Groups 612150.829 2 306075.415 1.384 .254
Within Groups 3.251E7 147 221140.118
Total 3.312E7 149
Result of Verb Image Matching Task
Ranks
Condition N Mean Rank
RT 1.00 50 64.60
2.00 50 87.20
3.00 50 74.70
Total 150
Test Statisticsa,b
RT
Chi-Square 6.790
df 2
Asymp. Sig. .034
a. Kruskal Wallis Test
b. Grouping Variable: Condition
Results
• As observed in task, the subjects took longer time to respond when the verb and image share the same effector.
• Thus, to some extent it is evident that the results obtained in the test could not be due to the proposed alternative explanation for test 1 since the same effect is observed even when the order is reversed.
Task 3: Verb-Verb Matching Task• Verb in typed format is shown for a
period of 1000 milliseconds
• 500 milliseconds inter stimulus interval
• This is followed by a verb in typed format
Means Table
ConditionAverage RT (in
milliseconds)Standard Deviation
Matching Condition 1353.48 351.0843
Non-Matching Same
Effector Condition1419.04 396.6966
Non-Matching Different
Effector Condition1247.92 276.2357
ANOVA
RT Sum of
Squaresdf Mean Square F Sig.
Between
Groups795980.991 2 397990.496 3.351 .038
Within Groups 1.889E7 159 118782.609
Total 1.968E7 161
Condition 1: Matching ConditionCondition 2: Non-Matching Same effector ConditionCondition 3: Non-Matching Different effector Condition
Result of Verb-Verb Matching Task
Ranks
Condition N Mean Rank
RT 1.00 54 83.48
2.00 54 91.89
3.00 54 69.13
Total 162
Test Statisticsa,b
RT
Chi-Square 6.500
df 2
Asymp. Sig. .039
a. Kruskal Wallis Test
b. Grouping Variable: Condition
Results
• When the images were replaced by verbs and the task became a lexical-lexical matching task, and still the effect, as hypothesized, was observed
Question
• In all the above tests, the presentation of stimuli was through visual modality. This might raise a question whether the motor area activation is only due to presentation of stimuli in visual modality. To verify this, a study was conducted in which the images were displayed on the screen and the verbs are presented in auditory modality
Task 4: Image-Verb (Auditory) Matching Task• Image is shown for a period of 1000
milliseconds
• 500 milliseconds inter stimulus interval
• This is followed by a verb in auditory mode.
Means Table
Condition Average RT (in
milliseconds)
Standard Deviation
Matching Condition 438.85 138.5288
Non-Matching Same 508.95 219.089
Non-Matching Different 440.08 144.3107
ANOVA
RT Sum of
Squares df Mean Square F Sig.
Between
Groups128683.860 2 64341.930 2.213 .114
Within Groups 3575614.997 123 29070.041
Total 3704298.858 125
Condition 1: Matching ConditionCondition 2: Non-Matching Same effector ConditionCondition 3: Non-Matching Different effector Condition
Result of Image-Verb (auditory) Matching Task
Ranks
Condition N Mean Rank
RT
1.00 42 59.52
2.00 42 70.75
3.00 42 60.23
Total 126
Test Statisticsa,b
RT
Chi-Square 2.491
df 2
Asymp. Sig. .288
a. Kruskal Wallis Test
b. Grouping Variable: Condition
Result
• It is evident from the table that RT in the non-matching same effector condition is greater than those in non-matching different effector condition but it is found to be statistically insignificant.
.
Summary
Condition Task 1 Task 2 Task 3 Task 4
Matching Condition 1281.34 1020.568 1353.48 438.85
Non-Matching same effector 1502.599
1176.36 1419.04 508.95
Non-Matching different effector 1394.467
1111.11 1247.92 440.08
Summary
• We observe that irrespective of statistical significance, the response times to reject pairs in non-matching same effector condition is larger than those in non-matching different effector condition.
SummaryTask TI: Picture-
Lexical Matching Task
T2: Lexical-Picture Matching Task
T3: Lexical-Lexical Matching Task
T4: Picture-Lexical (Auditory) Matching Task
Matching condition
7.45% 9.4% 11% 7.90%
Non matching same effector condition
20.90% 19.55% 18% 20.17%
Non matching different effector condition
6.12% 8.6% 7.33% 6.29%
Average error responses per Subject.
11.49% 9.71% 12.11% 11.45%
Summary
• The higher rate of errors in the non-matching same effector conditions in all the tasks is to be tested and given an explanation.
Conclusion
• The results obtained in the various tasks though may not completely support the claims of embodied semantics, can also not disprove its claims. This study shows that the notion of embodied verbal semantics is to be carefully investigated in other languages to verify the universality of the effect.