Data for School Data for School ImprovementImprovement
Training and Networking Training and Networking
Session Session #3#3
Farrington-Kaiser-Kalani Complex AreasNovember 19 / December 8, 2010
Purposes for our Training and Purposes for our Training and Networking SessionsNetworking Sessions
• Learn ways to maximize Hawaii’s Assessment Item Bank: – creating quality assessments– accessing and using reports for data-based decision-
making
• Provide a forum to network with other Complex Area schools to share and gain ideas of how to share this back at your school
Desired OutcomesDesired Outcomes
• understand and apply 5 Key Points to Consider to create a quality assessment
• understand what 5 data reports are generated and determine which best meet your needs
• understand how to analyze reports
• leave with ideas on how you can present this resource to your faculty
Learning Road Trip # 3Learning Road Trip # 3
• The “Road Map”• 5 Key Points to Consider: Creating Quality
Assessments for Quality Data (REVIEW)• Accessing DSI Reports & making meaning of data• Updates
The “Road Map”The “Road Map”
How does the packet training connect to the DSI system?
“Errors using inadequate data are much less than those
using no data at all”-Charles Babbage
“A clever arrangement of bad eggs will never make a
good omelet.” -C.S. Lewis
Choose a quote that resonates within your own personal teaching belief?
5 Key Points to Consider
#1 Check the Alignment or Taxonomic Match of an item
#2 Check the breadth of each item in relation to the benchmark
#3 Think about any prerequisite concepts or skills needed before assessing the selected benchmarks
#4 Understand that some items will show a grade range, i.e. grade 6-8 or grade 9-12.
#5 Maintain the intent of the item bank by how you use the items and what you do with the assessment data
Review
Review ActivityReview Activity
• In your table group, – Sort the descriptors (Yellow cards)– Place them under the appropriate headings
(Purple cards)• Justify the rationale to support your decision
– Discuss why it’s important to consider the 5 Key Points
Key Points Key Points When Using the Item BankWhen Using the Item Bank
#1 Check the Alignment or Taxonomic Match of an item
Above
Perfect
Below
Taxonomic LevelsTaxonomic Levels
Don’t just focus on the verb when thinking about which taxonomic level a benchmark addresses - - need to read and understand what the item is asking students to do.
Taxonomic LevelsTaxonomic Levels
6.10.3 Evaluate algebraic expressions
What is the value of the expression below when x = 4 and y = 2?
x 3 – 3y
Item is just asking students to solve the expression.
(Taxonomic Level 1)
Taxonomic LevelsTaxonomic Levels
7.2.3 Evaluate the adequacy and accuracy of an author’s evidence in support of his or her main points of an argument
Based on Nettie Emory's original diary entry and what you know
about the Oregon Trail, do you think that the story is an accurate portrayal of Nettie's adventure? Explain why you think so.
Item is asking students to make a judgment on whether or not the story gives an accurate portrayal and to
provide supporting reasons why.
(Taxonomic Level 4)
Taxonomic Level = Perfect
Reading ExampleReading Example
Describe an event in your own experience that relates to this passage. Tell how it is similar to or different from what Matt or Eric experienced. Then, explain what effect that experience had on your understanding of the passage and how someone who had not had your experience might have understood the passage differently.
LA.8.3.6 Explain how previous experience can influence one's reading of a text and one's opinion of that text (TL3) Perfect
Reading ExampleReading Example
Math ExampleMath Example
Taxonomic Level = Below
MA.6.11.1 – Analyze how data collection methods and sample size can affect the results of data sets
A poll is being taken at Baker Junior High School to determine whether to change the school mascot. Which of the following would be the best place to find a sample of students to interview that would be most representative of the entire student body?
Math ExampleMath ExampleMA.2.4.5 – Represent a given monetary amount using a combination of coins and bills
Which of the coins from Tony’s bank would he need to give to Marta so that they each have the same amount of money?
Taxonomic Level = Above
Assessment Items Must Match the Instruction
in the Class
• Have students had the opportunity to learn the needed knowledge and skills addressed in the item?
• Is the level of cognitive demand appropriate?• Can the student scaffold to meet the
expectation?
Key Points Key Points When Using the Item BankWhen Using the Item Bank
#2 Check the breadth of each item in relation to the benchmark
Many of our benchmarks have multiple parts
One item will not address all the parts of the benchmark, however, it can provide useful formative information about what a student
knows and can do
MA 4.4.2 Select and apply appropriate customary and metric
units and tools to measure length, perimeter, and area for the degree of accuracy needed
Judgment About ProficiencyJudgment About Proficiency
• must be based on student performance on the whole benchmark
• over time, check that items selected address all parts of the benchmark
Key Points Key Points When Using the Item BankWhen Using the Item Bank
#3 Think about any prerequisite concepts or skills needed before assessing the selected benchmarks
Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)
Organize and represent data in more than one way (e.g., tallies, chart, tables, bar graphs, line plots, line graphs)
(MA 3.11.2 -- TL2)
Compare different representations of the same data and evaluate how well each representation shows important
aspects of the data
(MA 5.12.2 -- TL3)
Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)
What other concepts and/or benchmarks would be prerequisites for this benchmark?
Explain an opinion about an author’s ideas or message by analyzing conventions of fiction (e.g., plot, character,
imagery, motifs, tone, subplots, stock characters)
LA 8.3.5 (TL3)
What message does the author express through the plot of the passage?
(A) Achieving a goal takes practice. (B) Friendships can thrive despite differences. (C) People should feel free to express their feelings. (D) People grow apart as they get older.
What other concepts and/or benchmarks would be prerequisites for this benchmark?
Explain an opinion about an author’s ideas or message by analyzing conventions of fiction (e.g., plot, character,
imagery, motifs, tone, subplots, stock characters)
LA 8.3.5 (TL3)
Grade 4 (Reading: focus non-fiction)• 3.1.6: Locate information in a variety of grade-appropriate sources• 3.2.3: Identify the main idea or problem and solution in a text• 4.1.1: Use new grade-appropriate vocabulary, including
homophones and homographs, learned through reading and word study, including root words, affixes, and word origins
• 4.2.1: Describe how common graphic structures (e.g., typeface, headings, illustrations) organize information in texts
• 4.2.2: Use organizational patterns (e.g., sequential, cause and effect) to access and understand information
• 4.2.4: Distinguish fact from opinion and cause from effect when reading informational texts
• 4.2.5: Summarize main points found in informational texts• 5.2.1: Use organizational patterns (e.g., compare and contrast,
proposition and support to access information)• 5.2.2 Make inferences and draw conclusions about grade-
appropriate text
• 6.2.3 Draw and support conclusions about information or ideas in a text
* TIP: students should always give supporting evidence even though a benchmark doesn’t ask for it
Expanding the Search
Key Points Key Points When Using the Item BankWhen Using the Item Bank
#4 Grade Range:Understand that some items willshow a grade range, i.e. grade 6-8 orgrade 9-12.
Reading Passages:
• Follow the NAEP model of using the same passage for different grade levels, keeping in mind what is age appropriate. Questions will differ depending on the grade level benchmark
• May address multiple and different grade level benchmarks
• Plans to have Lexile measures for each passage
Grade Span
Reading Passages & ItemsReading Passages & Items
1. Describe your reaction to reading the story, and what was it about the story that made you react that way?
LA.12.3.1 Evaluate how an author's purpose and message are
supported by his/her construction of text and use of
literary devices (TL3) - Above
2. Explain the narrator's feelings about the grandmother.
LA.9.3.1 Use textual evidence (e.g., knowledge of rhetorical and
literary devices) to interpret and draw conclusions about
literature (TL3) - Perfect
3. Soon after the grandmother arrived in London, her uncle persuaded her to
(A) emigrate to the United States
(B) marry someone he had chosen
(C) become a professional singer
(D) work as an artist's model
LA.8.2.4 Make inferences based on explicit and implied
information (TL3) - Perfect
4. Explain what you think the grandmother was trying to communicate
to the narrator by the gift of the photo album.
LA.11.3.1 Explain how the theme or meaning of a literary text can
represent a view or comment on life (TL3) - Above
High School Math
Math courses, such as Algebra I & II, Geometry, Calculus and Trigonometry may be offered at different grade levels
depending on the school’s master schedule. Items aligned to benchmarks in these courses will reflect a grade 9-
12 range.
Key Points Key Points When Using the Item BankWhen Using the Item Bank
#5 Maintain the intent of the item bank by how you use the items and what you do with the assessment data
Formative assessment to meet student needs• NOT a practice items for HSA• NOT a replacement for “Quarterly Assessments”
“Allow Student Comments”
Use Item Bank Features to Involve StudentsUse Item Bank Features to Involve Students
Involve Students in the Assessment Involve Students in the Assessment ProcessProcess
• Using the comment feature, students can: explain their answer share thinking reflect on how they think they did give feedback to the teacher on items
Differentiated AssessmentDifferentiated Assessment
Selecting different items:
• based on lower/higher taxonomic levels and/or grade level benchmarks
• selecting reading items that address the same grade level benchmarks, but have easier or more difficult reading passages
Use the Data
• For feedback
• To inform instruction
• To promote collaboration
NOT for punitive reasons
NOT for punitive reasons
Inputting Constructed Response Scores
Accessing DSI ReportsAccessing DSI Reports
Individual Assessment Data (whole class and individual students)
5 Types of Score Reports5 Types of Score Reports
• Total Score (by %)
• Proficiency
• Item Analysis
• Item Response
• Performance Summary (individual students)
Making Meaningful Use of Making Meaningful Use of DataData
Example Timeline for Example Timeline for Second & Third QuarterSecond & Third Quarter
Teachers given time to “try out” using the bank
at their comfort level
Understanding the Bank: Beyond Navigation
(5 Key Points)
Using Bank to Enhance Instruction
Creating additional assessments
Accessing Reports & Making Meaningful Use
of Data
Administer an assessment
Accessing Reports & Making Meaningful Use of
Data; Taking Action
Share your thoughts with each other…..
• connections to what you’re doing
• something that you’ve tried and it worked
• something you might like to try
What’s Next. . .
Upcoming Session #4Upcoming Session #4
• End of January – registration memo will be coming out
• Focus: – updates– Taking targeted action after data analysis