2013 aied srl workshop--social deliberatie skills defined_ murray
DESCRIPTION
TRANSCRIPT
![Page 1: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/1.jpg)
Toward Defining, Justifying, Measuring, and Supporting
Social Deliberative Skills
Tom MurrayUMass Amherst
At AIED July 2013, Memphis: Workshop on Self-Regulated Learning in Educational
Technologies: Supporting, modeling, evaluating, and fostering metacognition with computer-based learning environment
![Page 2: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/2.jpg)
Group & CollaborativeWork/Leanin
g
Conflict Resolution
Meaning Negotiation
Problem solving
Planning
Brainstorming & Creativity
Inquiry
Decision making
Knowledge building
Group dynamics
(form, storm, norm)
Peer help/tutoring
![Page 3: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/3.jpg)
3
Social Deliberative Skills:Social/Emotional/Reflective
• 1. Social perspective taking (cognitive empathy, reciprocal role taking...)
• 2. Social perspective seeking (social inquiry, question asking skills...)
• 3. Social perspective monitoring (self-reflection, meta-dialogue...)
• 4. Social perspective weighing (reflective reasoning; comparing and contrasting views...)
![Page 4: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/4.jpg)
Social Deliberative
Skills
Perspective Taking
Social Metacognition
Reflective ReasoningEmpathy
Dialogue and Deliberation
skills
![Page 5: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/5.jpg)
Meaning NegotiationConflict Resolution
Skills needed to bridge different perspectives to build mutual understanding and mutual regard
Social Deliberative Skills: The capacity to deal productively with heterogeneous goals, values, or perspectives in dialogue and deliberation
– Including: collaboration, problem solving, knowledge building, inquiry learning...
![Page 6: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/6.jpg)
Overview
• Background• Supporting them online
– Participants – Facilitators
• Measuring them – Human coding– Machine classification
• Deeper exploration of their meaning– And issues with construct definitions (ontology)
![Page 7: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/7.jpg)
1. Background
![Page 8: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/8.jpg)
8
Text Coding Scheme
![Page 9: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/9.jpg)
Corpora and Rater Agreement
![Page 10: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/10.jpg)
10
Examples of Social Deliberative Skills/Behavior
From authentic dialogues in our online corpora
“ I am probably extremely biased because I am under 21 years old and in college. I wonder if as a 45 year old I will feel differently. ” (self reflection)
“I can’t help but imagine what that is like, for her and for her family.” (perspective taking)
![Page 11: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/11.jpg)
11
Code Frequencies in Several Domains
Exp. Group Total_SD_Skill
Intersubjectivespeech acts
Vanilla (N = 8) 0.29 (0.07) 0.20 (0.09)
Reflective Tools (N = 8) 0.40 (0.08) 0.30 (0.08)
• A significant difference and main effect between Total-SD-Score and grouping, F(1, 14) = 6.89, p = 0.02*, d = 1.46 (a large effect) in favor of the Reflective Tools group
• A significant relationship between Intersub and grouping, F(1, 14) = 4.81, p = 0.05*, d = 1.05 (a large effect) in favor of the Reflective Tools group
![Page 12: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/12.jpg)
Support/Scaffolding (vs. “Education”)
Online Dialogue &DELIBERATION
Outcomes:- Agreements/solutions
- Relationship, Trust (social capital)- SKILL USE (and practice)
Existing
Skills
Adaptive Support(4th party)
Passive Support(interface)
FacilitatorSupport
(Dashboard)
![Page 13: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/13.jpg)
13
![Page 14: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/14.jpg)
14
Mediem
Opinion Sliders
![Page 15: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/15.jpg)
15
Graduate Class 2012—Total Skill
![Page 16: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/16.jpg)
16
![Page 17: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/17.jpg)
17
CohMetrixdiscourse & coherence
LIWClexical categories
![Page 18: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/18.jpg)
18
![Page 19: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/19.jpg)
19
feature comparisons
In-domain Cross-domainTraining corpus
Civic deliberation
Professional community negotiation
Civic deliberation
Professional community negotiation
Testing corpus
Civic deliberation
Professional community negotiation
Professional community negotiation
Civic deliberation
Gender
Accuracy 56.8 52.7 52.7 56.8Precision 56.8 52.7 52.7 56.8
Recall 100.0 100.0 100.0 100.0F2 86.8 84.8 84.8 86.8
LIWC
Accuracy 55.3 55.3 56.2 59.6Precision 58.9 54.6 56.9 59.6
Recall 70.7 90.0 69.7 89.3F2 68.0 79.7 66.7 81.2
Cohmetrix
Accuracy 62.1 54.8 53.4 53.3Precision 61.4 55.3 53.5 58.3
Recall 83.6 74.0 90.0 62.2F2 77.9 69.3 79.2 61.4
LIWC+Gender
Accuracy 52.3 54.6 56.2 60.9Precision 56.8 54.2 56.8 60.6
Recall 67.1 89.6 70.6 88.9F2 64.8 79.3 67.3 81.3
Cohmetrix +Gender
Accuracy 62.4 55.5 53.9 53.8Precision 62.7 55.6 53.9 58.5
Recall 83.6 77.5 87.0 64.0F2 78.3 71.8 77.5 62.8
LIWC +Cohmetrix
Accuracy 62.4 54.5 54.1 55.6Precision 63.3 55.1 54.0 60.0
Recall 84.4 74.5 87.0 65.3F2 79.2 69.6 77.5 64.2
All
Accuracy 62.1 54.3 52.7 54.8Precision 62.3 55.0 53.1 59.7
Recall 84.4 74.0 88.7 63.1F2 78.8 69.2 78.2 62.4
![Page 20: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/20.jpg)
Exploring Social Deliberative Skills
![Page 21: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/21.jpg)
Social Deliberative Skills:Literature BACKGROUND
• Dialogue Acts, Argumentation• Reflective & Critical Reasoning• Meta-skills• Civic Deliberation• Accountable Talk• Research into Perspective taking and Empathy
![Page 22: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/22.jpg)
Dialogue Act Schemes• M. Baker (Rainbow, 2007)
– Social relation – Interaction management – Task management– Opinions – Argumentation – Broaden and deepen
• Dejong (Tr-Diagram, 2005) – Argument– Explanation– Challenge (but)– Evidence– Support/Refute
• Kim (2007)– Acknowledge/
Support/Complement– Inform, Command,
Announce– Answer/Suggestion/
Advice– Correction/Objection– Elaboration/Description– Question
![Page 23: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/23.jpg)
Reflective Reasoning, Critical Thinking, Argumentation
• Reflective Reasoning– Ill-structured problems – about which reasonable people reasonably
disagree• Critical Thinking
– analyze– interpret– integrate
![Page 24: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/24.jpg)
D. Kuhn: Ill-defined constructs in Higher Order Thinking
Argumentation
Inquiry
Critical thinking
Metacognition
Reflective Judgment
Epistemological Knowledge
![Page 25: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/25.jpg)
Metacognition(Winne; Azedevo; Wellman)
• Phases: planning, monitoring, control/regulation, reflection/assessment (repeat)
• Domains: cognitive, motivational/affective, behavioral, contextual; memory, attention; problem solving state
• Self-reflection; self-explanation; help/info-seeking and SR learning
![Page 26: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/26.jpg)
Social Metacognition(White; Scardemalia; Collins)
• Meta-knowledge/skill about:– Self expertise, incl. collab. and communic. skill– Roles & relationships– Social context (and mediation)– Knowledge-building: monitoring; improving; rise-
aboves
![Page 27: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/27.jpg)
Civic Deliberation
– Dahlberg (2001) (& Habermas): – reflexivity, ideal role taking, sincerity, inclusion – exchange of criticizable moral-practical validity
claims
![Page 28: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/28.jpg)
Analytic Aspects of Deliberation:Stromer-Galley
• Introduction - welcoming, context, etc.• Deliberation Process • Technical Process• Meta-Talk (consensus, conflict, clarity)• Summary• Disagree/Agree• Off-topic move• Reasoned Opinion (fact, question, elab. opinion)• Sourcing• Intervention • Invite Others• Social (praise, apology, off topic)
![Page 29: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/29.jpg)
29
Analytic/Cognitive vs. Social/EmotionalGastil & Black: Key features of deliberative dialogue
![Page 30: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/30.jpg)
Accountable Talk
• Social: Listen (pay attention to…), Summarize, Build upon, Refer to
• Knowledge: Verify (check), Unpack (explain), Support (examples, evidence), Link
• Reason: Defend, Challenge, Combine, Predict
![Page 31: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/31.jpg)
Social Deliberative Skills – v1
![Page 32: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/32.jpg)
Primary SD-Skills
• Differentiating facts/opinions (knowing how to reason about each)
• Reflecting on biases and assumptions (mostly one's own but also others)
• Perspective taking (of actual interlocutors; of other groups/identities/cultures, etc.)
• Reflecting on the dialog as a whole (meta-dialogue)
![Page 33: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/33.jpg)
33
Social Deliberative Skills V-2Social/Emotional/Reflective
• 1. Social perspective taking (cognitive empathy, reciprocal role taking...)
• 2. Social perspective seeking (social inquiry, question asking skills...)
• 3. Social perspective monitoring (self-reflection, meta-dialogue...)
• 4. Social perspective weighing (reflective reasoning; comparing and contrasting views...)
![Page 34: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/34.jpg)
34
Text Coding Scheme
![Page 35: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/35.jpg)
Social Deliberative Skill:application of HOSs to me/you/we
Higher Order Skills • argumentation• critical thinking• explanation & clarification• inquiry/curiosity (questioning)• reflective judgment• meta-cognition• epistemic reasoning
Apply these skills, not to EXTERNAL REALITY (“IT”/problem domain) but to theINTERSUBJECTIVE domain
Higher Order Skills applied to:
SELFgoals; level of certainty; feelings, values, assumptions…
YOU goals, assumptions, feelings, values; perspective taking; "believing" & cognitive empathy…
WEagreements, goals; quality of the discourse/collaboration; differences and similarities in values, beliefs, goals, power, roles…
![Page 36: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/36.jpg)
Next: Support & Interventions
• “Teaching:” role-play, simulated dialogues...• Research on Perspective taking & Empathy?• Computer-based
– Scripts– Sentence Openers– Reification (& passive reminders)– Awareness Tools
![Page 37: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/37.jpg)
www.SocialDeliberativeSkills.comProject Collaborators: Bev Woolf, Xiaoxi Xu, Lynn Stephens, Leah Wing, Natasha ShrikantAnd thanks to Art and Nia from CohMetrics lab.
![Page 38: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/38.jpg)
38
Project Overview
![Page 39: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/39.jpg)
A journey through ontological conundrums
• Below is additional material related to the unavoidability of construct overlap and indeterminacy in defining and using abstract concepts such as metacognition, inquiry, etc. (See overlap in Kuhn slide above)
![Page 40: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/40.jpg)
“ ...to Understand, Measure & Support SD-skills”
• Reviewer: “How can study them if you don’t understand them and can’t define them precisely?”
![Page 41: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/41.jpg)
• Scholarly work is "notoriously fraught with definitional disagreement" (Shermer, 2011)
An invitation to co-explore Ontological Indeterminacy
![Page 42: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/42.jpg)
Kurt Fischer’s Dynamic Skill Theory
• Skills develop in response to real tasks• Higher order skills are built doing complex
tasks; no isolated task, therefore no isolated skill (compare “leadership” skill with “territoriality” biological drive)
• ...skills [and knowledge] are not isolated units, but rather
function together in complex structures of inter participation...an ecosystem ...any given skill requires the existence of various others as component [or
interacting] parts...
![Page 43: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/43.jpg)
Therefore....
• Definition of task takes precedence over definition of the skill
![Page 44: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/44.jpg)
Example: “intelligence” and IQ
• Real life target Task• Skills (& knowledge)• Measurement
Caution: the construct definition can be reduced to the measurement definition or enactment
![Page 45: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/45.jpg)
SD-Skills: task-oriented definition
Skills needed to bridge different perspectives to build mutual understanding and mutual regard
Social Deliberative Skills: The capacity to deal productively with heterogeneous goals, values, or perspectives in dialogue and deliberation
– Including: collaboration, problem solving, knowledge building, inquiry learning...
![Page 46: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/46.jpg)
Concept Indeterminacy(Lakoff)
• Abstract Concepts:– Graded (fuzzy)– Have “metaphorical pluralism”– Metaphorical (limited by embodiment)
• More abstract => More indeterminate
Tom Murray | www.perspegrity.com | August 2010 46
![Page 47: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/47.jpg)
Two sources of indeterminacy
• 1. Task structure: Higher order skills are built doing complex tasks; no isolated task therefore no isolated skill
• 2. Nature of categories: Abstract concepts are indeterminate by nature
![Page 48: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/48.jpg)
Real phenomena tend to be chaotic, organic; not categorized
![Page 49: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/49.jpg)
extra slides
![Page 50: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/50.jpg)
Stromer-GalleySD-Skills Strommer-Galley
WELCOMING Introduction - welcoming, context, etc.
PROC-EXPL - process-explanation Deliberation Process
TECH Technical Process
META_SUM Summarize
AGREE Disagree/Agree
DISAGREE Disagree/Agree
OFF_TOPIC Off-topic move
MEDIATE Intervention || Invite Others
SOCIAL Social
![Page 51: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/51.jpg)
Code-descriptions and frequencies
Perspective taking (15%), Big picture thinking (8%), meta-dialogue (5%), self-reflection (3%), Arg_gen (~30%)
![Page 52: 2013 AIED SRL workshop--Social Deliberatie Skills Defined_ Murray](https://reader036.vdocuments.mx/reader036/viewer/2022062613/5453e7b9af79597c338b49d0/html5/thumbnails/52.jpg)
Debate, Dialogue and DeliberationDeliberation: “thoughtful, careful, or lengthy consideration by individuals; and formal discussion and debate in groups” (Davies & Chandler 2011)