Data for School Data for School ImprovementImprovement

Training and Networking Training and Networking

Session Session #3#3

Farrington-Kaiser-Kalani Complex AreasNovember 19 / December 8, 2010

Purposes for our Training and Purposes for our Training and Networking SessionsNetworking Sessions

• Learn ways to maximize Hawaii’s Assessment Item Bank: – creating quality assessments– accessing and using reports for data-based decision-

making

• Provide a forum to network with other Complex Area schools to share and gain ideas of how to share this back at your school

Desired OutcomesDesired Outcomes

• understand and apply 5 Key Points to Consider to create a quality assessment

• understand what 5 data reports are generated and determine which best meet your needs

• understand how to analyze reports

• leave with ideas on how you can present this resource to your faculty

Learning Road Trip # 3Learning Road Trip # 3

• The “Road Map”• 5 Key Points to Consider: Creating Quality

Assessments for Quality Data (REVIEW)• Accessing DSI Reports & making meaning of data• Updates

The “Road Map”The “Road Map”

How does the packet training connect to the DSI system?

“Errors using inadequate data are much less than those

using no data at all”-Charles Babbage

“A clever arrangement of bad eggs will never make a

good omelet.” -C.S. Lewis

Choose a quote that resonates within your own personal teaching belief?

5 Key Points to Consider

#1 Check the Alignment or Taxonomic Match of an item

#2 Check the breadth of each item in relation to the benchmark

#3 Think about any prerequisite concepts or skills needed before assessing the selected benchmarks

#4 Understand that some items will show a grade range, i.e. grade 6-8 or grade 9-12.

#5 Maintain the intent of the item bank by how you use the items and what you do with the assessment data

Review

Review ActivityReview Activity

• In your table group, – Sort the descriptors (Yellow cards)– Place them under the appropriate headings

(Purple cards)• Justify the rationale to support your decision

– Discuss why it’s important to consider the 5 Key Points

Key Points Key Points When Using the Item BankWhen Using the Item Bank

#1 Check the Alignment or Taxonomic Match of an item

Above

Perfect

Below

Taxonomic LevelsTaxonomic Levels

Don’t just focus on the verb when thinking about which taxonomic level a benchmark addresses - - need to read and understand what the item is asking students to do.

Taxonomic LevelsTaxonomic Levels

6.10.3 Evaluate algebraic expressions

What is the value of the expression below when x = 4 and y = 2?

x 3 – 3y

Item is just asking students to solve the expression.

(Taxonomic Level 1)

Taxonomic LevelsTaxonomic Levels

7.2.3 Evaluate the adequacy and accuracy of an author’s evidence in support of his or her main points of an argument

Based on Nettie Emory's original diary entry and what you know

about the Oregon Trail, do you think that the story is an accurate portrayal of Nettie's adventure? Explain why you think so.

Item is asking students to make a judgment on whether or not the story gives an accurate portrayal and to

provide supporting reasons why.

(Taxonomic Level 4)

Taxonomic Level = Perfect

Reading ExampleReading Example

Describe an event in your own experience that relates to this passage. Tell how it is similar to or different from what Matt or Eric experienced. Then, explain what effect that experience had on your understanding of the passage and how someone who had not had your experience might have understood the passage differently.

LA.8.3.6 Explain how previous experience can influence one's reading of a text and one's opinion of that text (TL3) Perfect

Reading ExampleReading Example

Math ExampleMath Example

Taxonomic Level = Below

MA.6.11.1 – Analyze how data collection methods and sample size can affect the results of data sets

A poll is being taken at Baker Junior High School to determine whether to change the school mascot. Which of the following would be the best place to find a sample of students to interview that would be most representative of the entire student body?

Math ExampleMath ExampleMA.2.4.5 – Represent a given monetary amount using a combination of coins and bills

Which of the coins from Tony’s bank would he need to give to Marta so that they each have the same amount of money?

Taxonomic Level = Above

Assessment Items Must Match the Instruction

in the Class

• Have students had the opportunity to learn the needed knowledge and skills addressed in the item?

• Is the level of cognitive demand appropriate?• Can the student scaffold to meet the

expectation?

Key Points Key Points When Using the Item BankWhen Using the Item Bank

#2 Check the breadth of each item in relation to the benchmark

Many of our benchmarks have multiple parts

One item will not address all the parts of the benchmark, however, it can provide useful formative information about what a student

knows and can do

MA 4.4.2 Select and apply appropriate customary and metric

units and tools to measure length, perimeter, and area for the degree of accuracy needed

Judgment About ProficiencyJudgment About Proficiency

• must be based on student performance on the whole benchmark

• over time, check that items selected address all parts of the benchmark

Key Points Key Points When Using the Item BankWhen Using the Item Bank

#3 Think about any prerequisite concepts or skills needed before assessing the selected benchmarks

Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)

Organize and represent data in more than one way (e.g., tallies, chart, tables, bar graphs, line plots, line graphs)

(MA 3.11.2 -- TL2)

Compare different representations of the same data and evaluate how well each representation shows important

aspects of the data

(MA 5.12.2 -- TL3)

Some concepts and benchmarks (at a lower grade or on grade level) may be prerequisites for other benchmarks (based on content and/or taxonomic level)

What other concepts and/or benchmarks would be prerequisites for this benchmark?

Explain an opinion about an author’s ideas or message by analyzing conventions of fiction (e.g., plot, character,

imagery, motifs, tone, subplots, stock characters)

LA 8.3.5 (TL3)

What message does the author express through the plot of the passage?

(A) Achieving a goal takes practice. (B) Friendships can thrive despite differences. (C) People should feel free to express their feelings. (D) People grow apart as they get older.

What other concepts and/or benchmarks would be prerequisites for this benchmark?

Explain an opinion about an author’s ideas or message by analyzing conventions of fiction (e.g., plot, character,

imagery, motifs, tone, subplots, stock characters)

LA 8.3.5 (TL3)

Grade 4 (Reading: focus non-fiction)• 3.1.6: Locate information in a variety of grade-appropriate sources• 3.2.3: Identify the main idea or problem and solution in a text• 4.1.1: Use new grade-appropriate vocabulary, including

homophones and homographs, learned through reading and word study, including root words, affixes, and word origins

• 4.2.1: Describe how common graphic structures (e.g., typeface, headings, illustrations) organize information in texts

• 4.2.2: Use organizational patterns (e.g., sequential, cause and effect) to access and understand information

• 4.2.4: Distinguish fact from opinion and cause from effect when reading informational texts

• 4.2.5: Summarize main points found in informational texts• 5.2.1: Use organizational patterns (e.g., compare and contrast,

proposition and support to access information)• 5.2.2 Make inferences and draw conclusions about grade-

appropriate text

• 6.2.3 Draw and support conclusions about information or ideas in a text

* TIP: students should always give supporting evidence even though a benchmark doesn’t ask for it

Expanding the Search

Key Points Key Points When Using the Item BankWhen Using the Item Bank

#4 Grade Range:Understand that some items willshow a grade range, i.e. grade 6-8 orgrade 9-12.

Reading Passages:

• Follow the NAEP model of using the same passage for different grade levels, keeping in mind what is age appropriate. Questions will differ depending on the grade level benchmark

• May address multiple and different grade level benchmarks

• Plans to have Lexile measures for each passage

Grade Span

Reading Passages & ItemsReading Passages & Items

1. Describe your reaction to reading the story, and what was it about the story that made you react that way?

LA.12.3.1 Evaluate how an author's purpose and message are

supported by his/her construction of text and use of

literary devices (TL3) - Above

2. Explain the narrator's feelings about the grandmother.

LA.9.3.1 Use textual evidence (e.g., knowledge of rhetorical and

literary devices) to interpret and draw conclusions about

literature (TL3) - Perfect

3. Soon after the grandmother arrived in London, her uncle persuaded her to

(A) emigrate to the United States

(B) marry someone he had chosen

(C) become a professional singer

(D) work as an artist's model

LA.8.2.4 Make inferences based on explicit and implied

information (TL3) - Perfect

4. Explain what you think the grandmother was trying to communicate

to the narrator by the gift of the photo album.

LA.11.3.1 Explain how the theme or meaning of a literary text can

represent a view or comment on life (TL3) - Above

High School Math

Math courses, such as Algebra I & II, Geometry, Calculus and Trigonometry may be offered at different grade levels

depending on the school’s master schedule. Items aligned to benchmarks in these courses will reflect a grade 9-

12 range.

Key Points Key Points When Using the Item BankWhen Using the Item Bank

#5 Maintain the intent of the item bank by how you use the items and what you do with the assessment data

Formative assessment to meet student needs• NOT a practice items for HSA• NOT a replacement for “Quarterly Assessments”

“Allow Student Comments”

Use Item Bank Features to Involve StudentsUse Item Bank Features to Involve Students

Involve Students in the Assessment Involve Students in the Assessment ProcessProcess

• Using the comment feature, students can: explain their answer share thinking reflect on how they think they did give feedback to the teacher on items

Differentiated AssessmentDifferentiated Assessment

Selecting different items:

• based on lower/higher taxonomic levels and/or grade level benchmarks

• selecting reading items that address the same grade level benchmarks, but have easier or more difficult reading passages

Use the Data

• For feedback

• To inform instruction

• To promote collaboration

NOT for punitive reasons

NOT for punitive reasons

Inputting Constructed Response Scores

Accessing DSI ReportsAccessing DSI Reports

Individual Assessment Data (whole class and individual students)

5 Types of Score Reports5 Types of Score Reports

• Total Score (by %)

• Proficiency

• Item Analysis

• Item Response

• Performance Summary (individual students)

Making Meaningful Use of Making Meaningful Use of DataData

Example Timeline for Example Timeline for Second & Third QuarterSecond & Third Quarter

Teachers given time to “try out” using the bank

at their comfort level

Understanding the Bank: Beyond Navigation

(5 Key Points)

Using Bank to Enhance Instruction

Creating additional assessments

Accessing Reports & Making Meaningful Use

of Data

Administer an assessment

Accessing Reports & Making Meaningful Use of

Data; Taking Action

Share your thoughts with each other…..

• connections to what you’re doing

• something that you’ve tried and it worked

• something you might like to try

What’s Next. . .

Upcoming Session #4Upcoming Session #4

• End of January – registration memo will be coming out

• Focus: – updates– Taking targeted action after data analysis