a310-g inquire: digging into student data wednesday january 12, 2011
TRANSCRIPT
A310-G
INQUIRE: Digging into Student Data
Wednesday January 12, 2011
Digging out of snow!
Plus/Delta Protocol
Learning Objective
Understand how digging into multiple forms of data informs our understanding of student
learning.
Agenda for Today
Educational Question
Elluminate Intro
Priority Question
Item Analysis
Student Work
Reflection
BreakWe are here!
And now over to you, Anne…
Testing Audio
1. Click on the Tools menu
2. Select Audio
3. Select Audio Set-up Wizard
4. Follow the on-screen directions. Be sure to select the appropriate Audio Output Device
Change to Wide-Layout View
• Click on View
• Select Layouts
• Select Wide Layout
Testing Chat
• Type your favorite animal into the Chat window
• Send to This Room – press return
Emoticons & Raise Hand
• First, raise your (virtual) hand.
• Second, click on the emoticon that represents how are you feeling about Live-Online so far.
Polling
• How would you rate your stress level for this course right now?
A. High
B. Medium
C. Low
Talking on Microphone
• “Whip Around” – Go down the participant list.
• Click mic icon once to talk – THEN CLICK OFF.
• Whip Around
– Your name
Live-Online Norms
• Participate!
– Emoticons
– Polling
– Chat
– Mic
Educational Question
EDUCATIONAL QUESTION
An Educational Question is…
• Designed to narrow AND be generative– By subject area
• Focused on teaching and learning
• Based on analysis of data– State assessment
• Made by the Data Team
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
Emoticons & Chat-In
• Principal Snow has asked you for feedback on her Educational Question.
– How many students are failing?
– How can we improve student engagement?
– What does the performance of our bubble kids look like?
Data Overview: Intentional Collaboration
• Grouping– Small, mixed (grade/subject) table groups– Data team members facilitate table groups
• What do you see? – One data display at a time
• What do you make of it? – Considering all displays
Patterns/TrendsSimilaritiesDifferencesStrengths Struggles
Getting to a Priority Question
PRIORITY QUESTION
EDUCATIONAL QUESTION
Getting to a Priority Question
• Explain Priority Question– What is a Priority Question?– How does a Priority Question sound?
• Frontload Influencing Factors – Data– Resources– Interest/Buy-in
• Introduce Proposal Protocol
Getting to a Priority Question
• Explain Priority Question– What is a Priority Question?– How does a Priority Question sound?
• Frontload Influencing Factors – Data– Resources– Interest/Buy-in
• Introduce Proposal Protocol
A Priority Question is…
• Designed to help us narrow our focus
• Based on collaborative analysis of data presented in the Data Overview
• Focused on teaching and learning
• Narrowed by subgroup or standard
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
0%10%20%30%40%50%60%70%80%90%
100%
Percentage of Students Scoring Proficient/Advanced by Subgroup
subgroups not mutually exclusiveGrade 10 State Comprehensive Assessment: ELA
Franklin High School, 2010
Pe
rce
nta
ge o
f St
ud
en
ts
Getting to a Priority Question• Priority Question
– How are our students with less than 90% attendance performing on the ELA State Assessment?
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
Getting to a Priority Question• Priority Question
– How are our students with less than 90% attendance performing on the ELA State Assessment?
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
– Examine data by standard for select students to maintain focus on teaching and learning.
Fiction
(8 items)
Dramatic Liter-ature (3
items)
Understanding a Text (1
item)
Style and Lan-guage
(4 items)
Nonfiction
(13 items)
Theme
(1 item)
Structures and Origins of
Modern English
(1 item)
Vocabulary and Concept Devel-opment
(5 items)
Genre
(1 item)
Poetry
(3 items)
0%
20%
40%
60%
80%
100%
Average Percentage of Correct Answers for Each Standard Tested
Grade 10 State Comprehensive Assessment: ELAFranklin High School, 2010 (n=699)
ELA State Standard Tested
Ave
rage
Per
cent
age
of C
orre
ct A
nsw
ers
Getting to a Priority Question• Priority Question
– How are our students performing on the Poetry Standard of the ELA State Assessment?
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
Getting to a Priority Question• Priority Question
– How are our students performing on the Poetry Standard of the ELA State Assessment?
Educational Question
Priority Question
Learner Centered Problem
Problem of Practice
Action Plan
– Examine subgroup data for select standard to narrow your focus further.
Getting to a Priority Question
• Explain Priority Question– What is the purpose of a Priority Question?– How does a Priority Question sound?
• Frontload Influencing Factors – Data– Resources– Interest/Buy-in
• Introduce Proposal Protocol
Frontload Influencing Factors• Data
– What did we learn from our analysis of the data presented in the Data Overview?
• Resources– What knowledge and skills do people in our school and district
already possess?– What curriculum resources can we leverage?– What ongoing professional development can we leverage?
• Interest/Buy-in– Where is the interest, excitement, or momentum?
Emoticons & Chat-In
• How do feel about these factors?
– Data
– Resources
– Buy-In
Getting to a Priority Question
• Explain Priority Question– What is the purpose of a Priority Question?– How does a Priority Question sound?
• Frontload Influencing Factors – Data– Resources– Interest/Buy-in
• Introduce Proposal Protocol
Purpose of Proposal Protocol
• Identify a Priority Question– By standard and/or subgroup
• Structure the Conversation– Maintain focus on making a decision– Think deeply about the rationale
Proposal Protocol
• Guidelines for Proposal– “I propose that we …”
– State one specific proposal at a time (one standard)
– Describe your rationale (influencing factors)
• Protocol– Someone states one specific proposal and rationale.
– The facilitator invites others to add additional rationale in support of this proposal or to add concerns about this proposal.
– The facilitator decides when to take a second proposal
Vocabulary and Concept Devel-opment
5
Structure and Origins of Modern English
1Understanding a Text
1
Genre1
Theme1
Fiction8
Nonfiction13
Poetry3
Style and Language4
Dramatic Literature3
Number of Test Items per Standard
State Comprehensive Assessment: ELA2010
Fiction
(8 items)
Dramatic Liter-ature (3
items)
Understanding a Text (1
item)
Style and Lan-guage
(4 items)
Nonfiction
(13 items)
Theme
(1 item)
Structures and Origins of
Modern English
(1 item)
Vocabulary and Concept Devel-opment
(5 items)
Genre
(1 item)
Poetry
(3 items)
0%
20%
40%
60%
80%
100%
Average Percentage of Correct Answers for Each Standard Tested
Grade 10 State Comprehensive Assessment: ELAFranklin High School, 2010 (n=699)
ELA State Standard Tested
Ave
rage
Per
cent
age
of C
orre
ct A
nsw
ers
Proposal Protocol
• Guidelines for Proposal– “I propose that we …”
– State one specific proposal at a time (one standard)
– Describe your rationale (influencing factors)
• Protocol– Someone states one specific proposal and rationale.
– The facilitator invites others to add additional rationale in support of this proposal or to add concerns about this proposal.
– The facilitator decides when to take a second proposal
From Proposal to Priority Question
• We have discussed three proposals in our small group. How do we get to one?
– There seems to be momentum towards one. Does this sound like something we can work on?
– There is no clear favorite. Let’s vote.
• Thoughts on Consensus
From Proposal to Priority Question
• We now have 6 Priority Questions because we did that activity with 6 small groups!
– Data Team collects, reviews, proposes one to faculty, gets feedback from faculty
– Each group follows their own Priority Question
Questions & Answers (or Break)5 Minutes
Educational QuestionPriority Question
Franklin High’s Priority Question
• How are students performing on nonfiction items?– Rationale for Nonfiction: Standard 13• Data
– 1/3 of items assessed nonfiction standard: 13 of 40
– Nonfiction performance in middle; still needed work
• Resources– Nonfiction performance in middle; some success/knowledge
• Interest/Buy-in– Lends itself to “rigor” more easily than vocabulary standard
– All subjects use nonfiction texts in their curriculum
Digging Into Data
PRIORITY QUESTION
EDUCATIONAL QUESTION
Priority Question: How are students performing on nonfiction items?
Digging Into Data: Item Analysis
• Build Shared Meaning of Standard
• Examine Select Item-Level Data
• Examine Select Items
Nonfiction: State Standard 13
• 10th Grade Nonfiction State Standards
– “Analyze the logic and use of evidence in an author’s argument.”
– “Analyze and explain the structure and elements of nonfiction works.”
Shared Meaning for Nonfiction
• What does the term analyze mean?
– “Analyze the logic and use of evidence in an author’s argument.”
– “Analyze and explain the structure and elements of nonfiction works.”
The New Taxonomy of Educational Objectivesby Marzano & Kendall (2007)
• “Systems of Thinking”
– “Level 6: Self-System”
– “Level 5: Metacognitive System”
– “Level 4: Knowledge Utilization”
– “Level 3: Analysis”
– “Level 2: Comprehension”
– “Level 1: Retrieval”
“Cognitive System”
Analyze: Marzano & Kendall (2007)
• Is the information reasonable?
• What generalizations can I make?
• What might happen next?
• How can the information be organized?
• What are the similarities and differences?
Analyze
• Think of and write down one analysis task students could engage in? (1m)
• All-Chat: Share your idea via chat. (3m)
Shared Meaning for Nonfiction
• What does this content mean?
– “Analyze the logic and use of evidence in an author’s argument.”
– “Analyze and explain the structure and elements of nonfiction works.”
Shared Meaning for Nonfiction
• Individually Review General Standard 13 (4m)
– Circle evidence of what 10th grade standards mean
• “Analyze the logic and use of evidence in an author’s argument.”
• “Analyze and explain the structure and elements of nonfiction works.”
Shared Meaning for Nonfiction
• Whole Group (4m)
– What is your evidence? Raise hand to get on mic
• “Analyze the logic and use of evidence in an author’s argument.”
• “Analyze and explain the structure and elements of nonfiction works.”
Data for Nonfiction
• Individually Review Item-Level Data (4m)– Stay at the level of the data
• Whole Group– Chat-In: What do you see? (5m)
– On Mic: How might you group these items? (5m)• We group to focus and save time
Franklin High’s Priority Group
ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL
1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%
29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%
Franklin High’s Priority Group
ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL
1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%
29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%
Franklin High’s Priority Group
ITEM TYPEREPORTING CATEGORY STANDARD SCHOOL
1 MC LT 13 92%2 MC LT 13 90%4 MC LT 13 82%5 MC LT 13 60%6 MC LT 13 72%7 MC LT 13 72%
29 MC LT 13 85%30 MC LT 13 72%31 MC LT 13 65%32 MC LT 13 77%33 MC LT 13 57%
Items for Nonfiction
• Individually: Time to read the passage. (5m)
• Examine Item 5– Answer the question yourself. (15s)– Chat-In: What does a student need to know and
be able to do to answer this question correctly? (5m)
– On Mic: What are you theories about why students choose each incorrect answer. (5m)
– *Repeat for Items 6 & 7
Learning from Item Analysis
• *How can we describe the strengths and struggles of our learners?
– Free write
– Chart paper and markers
– Think, pair, share
Chat-In & Mic
• Imagine yourself “on the balcony…”
– What adaptive challenges might we face when
moving from an Educational Question to a Priority
Question to an Item Analysis?
Digging Into Data
PRIORITY QUESTION
EDUCATIONAL QUESTION
Priority Question: How are students performing on nonfiction items?
Looking at Student Work
• *Individually: Complete the prompt (4m)
• *Triads: Share your approach (6m)
• Individually: Review student work (5m)– Stay at the level of the data
Looking at Student Work Protocol
• On Mic: Whole Group
– What do you see? (10m)
• Starting with Student 1
– What do you make of it? (10m)
• Trends & patterns
• Similarities & differences
• Strengths & struggles
Priority Question: How are students performing on nonfiction items?
Digging Into Data
PRIORITY QUESTION
EDUCATIONAL QUESTION
Priority Question: How are students performing on nonfiction items?
LEARNER CENTERED PROBLEM
A Learner-Centered Problem is…
• … a statement about student learning, not a question.
• … based on evidence we found when digging into data.
• … directly related to your Priority Question.
• … specific and small.
ExamplesPriority Question Learner Centered Problem
Writing: How do students use evidence from text to support their argument?
Students don’t explain how quotes support their argument.
Reading: What strategies do students use for comprehension?
Students don’t look for connections to their own lives as they read.
Math: How do students approach word problems?
Students translate word problems into equations and solve without thinking about what the problem is asking.
Reflection
Taking Stock of your Learning
• Revisit what you wrote yesterday about maintaining a relentless focus on Evidence in all conversations
3 minutes
Plus/Delta Protocol
See you tomorrow…