welcome! making formative and benchmark assessments count wera conference december 2007 1
TRANSCRIPT
![Page 1: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/1.jpg)
Welcome!
Making Formative and Benchmark Assessments Count
WERA Conference
December 2007
1
![Page 2: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/2.jpg)
Goals for the Afternoon
Share lessons learned from the ASK Project (an assessment design project funded by the National Science Foundation). Designing assessment systems for science curriculum.
Understanding best practices.
2
![Page 3: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/3.jpg)
Purposes
Classroom assessment practicesDevelop assessment strategies and practices that lead to better student achievement and enhance instructional practices.
AccountabilityDevelop assessments with the technical quality needed to provide accountability information to districts.
3
![Page 4: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/4.jpg)
External Evaluator
BEAR
Charleston
ME ST LP LF
Fresno
ME ID SE MX
Las Vegas
MS IDLF SE EV
Manhattan
HB ST LF LP
Vista
MS FN
Vancouver
PS HB FN MD
San Diego
PS EV
Formative Evaluator
Advisory Board
Madison
WA EM VB MX
Phoenix
WA EM VB MD
LHS
4
![Page 5: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/5.jpg)
Two Reports from NRC
5
![Page 6: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/6.jpg)
The Assessment Triangle
OBSERVATION INTERPRETATION
COGNITION
6
![Page 7: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/7.jpg)
The Assessment Triangle
OBSERVATION INTERPRETATION
COGNITION
7
![Page 8: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/8.jpg)
Cognition
The Cognitive Perspective Focuses on how people develop
structures of knowledge—content, reasoning, and problem solving.
The Situative Perspective (Sociocultural) Focuses on how people learn to
participate in the practices, goals, and habits of mind of a particular community.
8
![Page 9: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/9.jpg)
Inquiry
Constructs
9
![Page 10: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/10.jpg)
Physics of Sound
Constructs
10
![Page 11: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/11.jpg)
Progress Levels
Strategic
Conceptual
Recognition
Notions
11
![Page 12: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/12.jpg)
The Assessment Triangle
OBSERVATION INTERPRETATION
COGNITION
12
![Page 13: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/13.jpg)
Observation—Item Development
Because you can’t see directly into a child’s mind, you have to find ways to infer evidence of learning. Items are written Panels review items Researchers look at cognitive demands Students and teachers test them out! Empirical data informs further revisions
13
![Page 14: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/14.jpg)
ASK Assessment Items
Embedded Assessment items (diagnostic and part of daily instruction) Notebook entries Notebook sheets Response sheets
Benchmark Assessment items (evaluative, but also formative—given periodically)
Short Answer Open Response Multiple Choice/Multiple Answer
14
![Page 15: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/15.jpg)
Benchmark Assessments
Before starting After Investigation 1 After Investigation 2 After Investigation 3 After Investigation 4 After completing the
module
Survey Inv 1 I-Check Inv 2 I-Check Inv 3 I-Check Inv 4 I-check Posttest
15
![Page 16: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/16.jpg)
The Assessment Triangle
OBSERVATION INTERPRETATION
COGNITION
16
![Page 17: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/17.jpg)
Coding (scoring) guides—
Based on “Progress Levels”
4 Strategic
3 Conceptual
2 Recognition
1 Notions
0 Makes no attempt
(Coding guides are 0-2, 0-3 or 0-4 depending on the level of the question.)
17
![Page 18: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/18.jpg)
Qualitative Methods Teacher feedback, expert panels and
cognitive analysis.
Quantitative Methods Item Response Theory (statistical analysis) ClassMap (computer program/reports)
Interpretation
18
![Page 19: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/19.jpg)
The Assessment Triangle
OBSERVATION
(Item development
)
INTERPRETATION
(Coding guides and measurement
model)
COGNITION
(Construct Maps &
Progress Maps) 19
![Page 20: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/20.jpg)
Focus on Embedded Assessment
How is embedded assessment enacted within a curriculum?
Context: Physics of Sound
20
![Page 21: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/21.jpg)
What students have learned before the lesson we’ll be doing
Sound is caused by vibrations. A sound source is an object that is vibrating.
A sound receiver detects sound vibrations.
Sounds have identifiable properties.
Pitch (a property of sound) is how high or low the sound is (volume = loud or soft).
The faster an object vibrates the higher the pitch (and vice versa).21
![Page 22: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/22.jpg)
Notes to Myself
Review the At-a-Glance chart Read Background for the Teacher and
Teaching Children About… Read through the steps in Guiding the
Investigation
Review the I-Check for Investigation 2
Review the focus questions
Prep for Inv 2
22
![Page 23: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/23.jpg)
Notes to Myself
The three big ideas:
Sounds are caused by vibrations Clarify distinction between pitch and volume
The length of the instrument affects the pitch
Shorter length = higher pitch Longer length = lower pitch
Tension also affects the pitch More tension = higher pitch Less tension = lower pitch
Prep for Inv 2
23
![Page 24: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/24.jpg)
Focus Question
How does length affect the pitch (and therefore the speed of vibrations)?
Record observations:WaterphoneXylophoneKalimbaString Beam
24
![Page 25: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/25.jpg)
Class Discussion
How does length affect the pitch (and therefore the speed of vibrations)?
Record observations:WaterphoneXylophoneKalimbaString Beam
25
![Page 26: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/26.jpg)
Notes to MyselfLength/Pitch
waterphonena √ √ √ √ √ √ na
xylophone √ rev ? ? rev ? ? √
kalimba √ √ rev P/v ? da ? na
String beam √ √ ? √ rev ? ? na
Ada
Badu
Cala
Derek
Edita
Flavo
Gen
Halyn
Key
√ = Got it!
rev = reversed
? = confused
da = didn’t answer the question
na = no answer
26
![Page 27: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/27.jpg)
Notes to MyselfLength/Pitch
27
Can students distinguish high from low pitches?
Do they know the rule?
Why are they inconsistent?
![Page 28: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/28.jpg)
Next Steps
(Can students discriminate pitch?)
Activity:Close your eyes.Listen to the sounds.If the sound has a high pitch, show thumbs up.If the sound has a low pitch, show thumbs down.
28
![Page 29: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/29.jpg)
What are our “rules?”
The longer the length,
the ___________ the pitch,
29
and the ___________ the vibrations.
![Page 30: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/30.jpg)
What are our “rules?”
The shorter the length,
the ___________ the pitch,
30
and the ___________ the vibrations.
![Page 31: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/31.jpg)
Next Steps
Statement disagrees with the rule
Statement needs clarification
Statement agrees with the rule
31
•The longer the length, the lower the pitch, the slower the vibrations.
•The shorter the length, the higher the pitch, the faster the vibrations.
![Page 32: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/32.jpg)
32
![Page 33: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/33.jpg)
33
![Page 34: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/34.jpg)
Next Steps
Statement disagrees with the rule
Statement needs clarification
Statement agrees with the rule
34
•The longer the length, the lower the pitch, the faster the vibrations.
•The shorter the length, the higher the pitch, the slower the vibrations.
![Page 35: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/35.jpg)
35
![Page 36: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/36.jpg)
Next Steps
What do the dots tell you about your use of the “rule?”
Rewrite any sentences you marked with a red or yellow dot.
36
![Page 37: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/37.jpg)
37
![Page 38: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/38.jpg)
Purposes:Summative (measure for giving grades)Formative (continue the learning)
Procedures:Take the testCode, but don’t mark on papersUse self-assessment strategies with students before revealing codes.
38
Benchmark Assessments
![Page 39: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/39.jpg)
Students have Learned… Sound is caused by vibrating objects. Length affects pitch
Longer = lower pitch (slower vibrations) Shorter = higher pitch (faster vibrations)
Tension affects pitch Looser = lower pitch (slower vibrations) Tighter = higher pitch (faster vibrations)
39
![Page 40: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/40.jpg)
Benchmark Assessment Take the I-Check for Investigation 2
40
![Page 41: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/41.jpg)
Behind the scenes
After students take the benchmark, the teacher codes the items to determine what students know and what they still need help with.
The teacher then plans which items to use to help students self-assess their understanding.(we’ll come back to coding later) 41
![Page 42: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/42.jpg)
Self-assessment Strategy Multiple-Choice Corners
Look at item 13
• You will be assigned a letter A, B, or C• (In the classroom students would go to the corner that corresponds with their actual answer.)
• Meet with your group to create an argument to convince the other groups in the room that your answer is the correct one.
• If at some point you “disagree with yourself,” you are allowed to change corners. 42
![Page 43: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/43.jpg)
Self-assessment Strategy Sentence Starters
I used to think….but now I think… I should have gotten this one right, I just…
I know…but I’m still not sure about… The most important thing to remember is…
Can you help me with… Next time I will remember to… I’m confused about… Now I know…
43
![Page 44: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/44.jpg)
Sentence StartersAndy
44
![Page 45: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/45.jpg)
Sentence StartersAndy
45
![Page 46: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/46.jpg)
Sentence StartersRachael
46
![Page 47: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/47.jpg)
Sentence StartersRachael
47
![Page 48: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/48.jpg)
Coding Benchmark ItemsCode If the student…
3
states that the short string will have a higher pitch than the long string (does not need to use the word pitch, but must describe the sound of both strings or make a correct comparison).
2
states that the short string will have a high pitch or the long string will have a low pitch, but does not make a comparison.
1 provides any other answer.
0 makes no attempt.Note: Coding Guides may be 0–2, 0–3 or 0–448
44
![Page 49: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/49.jpg)
Levels of Progress
4 Strategic
3 Conceptual
2 Recognition
1 Notions
0 No attempt
49
![Page 50: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/50.jpg)
Code Inv 2 I-Check
Review the coding guide for first item.
Code responses for all students on that item
Go on to the next item
22 23 24 25 26a
Ada 3
Badu 2
Cala 1
Derek 2
50
![Page 51: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/51.jpg)
51
![Page 52: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/52.jpg)
The Development Process Construct Maps and Frameworks Item Development Field Testing Analysis using item response theory software
(ConstructMap) Item fit Standard setting
Iterative revision of items Progress Levels and Maps
52
![Page 53: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/53.jpg)
Does formative assessment matter?
Benchmark scores were not as high as we thought they should be.
When we interviewed teachers they said they looked over students’ shoulders when they wrote in notebooks, but did not critically look at embedded assessment evidence.
Through FAST Project, worked with a small group of teachers to “force” looking at embedded assessments. 53
![Page 54: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/54.jpg)
1.6 1.2 2.61.615.7 15.9
9.7
22.9
34.9
87.1
60.0
46.7
0%
20%
40%
60%
80%
100%
1 2 3
Series4Series3Series2Series1
Strategic
Conceptual
Recognition
Notions
54
Magnetism and Electricity
Posttest Results
![Page 55: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/55.jpg)
ClassMap
Berkeley Evaluation and Assessment Research Center
55
![Page 56: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/56.jpg)
Beliefs and Actions
The most valuable use of assessment is as a learning
tool.
• Embedded assessment is the major focus.
• Assessment data is used to identify gaps in content understanding for both students and teachers.
56
![Page 57: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/57.jpg)
Beliefs and Actions
Students need to take
responsibility for their own
learning through active investigation and reflection.
• Collaboration rather than competition.
• Students have time to reflect on their learning through class interaction and writing-to-learn activities.
• Students are willing to make their thinking public…
• Students self-assess rather than self-evaluate… 57
![Page 58: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/58.jpg)
Beliefs and Actions
Embedded assessment needs to
reflect student thinking and how it changes and develops.
• Students write initial thinking in their notebooks without teacher intervention.
• Students may add additional information (under line of learning) after feedback, class discussion, or further activities.
58
![Page 59: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/59.jpg)
Beliefs and Actions
Students need immediate and effective feedback to improve learning.
• Teachers work to have thorough understanding of the content…
• Teachers look at notebooks after every lesson to reflect on learning and instruction and to determine next-steps.
• Teachers provide feedback as soon and as often as possible.
• Feedback is based on data and provides information for improved understanding.
59
![Page 60: Welcome! Making Formative and Benchmark Assessments Count WERA Conference December 2007 1](https://reader030.vdocuments.mx/reader030/viewer/2022032722/56649ce15503460f949ab5b5/html5/thumbnails/60.jpg)
Discussion
Questions?
Comments?
60