southern regional education board 1 preparing for the 2014 hstw assessment mike ross consultant,...
TRANSCRIPT
SouthernRegionalEducationBoard
1
Preparing for the
2014HSTW Assessment
Mike Ross Consultant, HSTW / MMGW SW Ohio
Susanne DavisHSTW Assessment Coordinator, Ohio
SouthernRegionalEducationBoard
2
Topics Covered
What is the HSTW Assessment?
Administration Nuts & Bolts
Best Practices for Successful Assessments
SouthernRegionalEducationBoard
3
What is the HSTW Assessment?
Required in even-numbered years Subject tests in Reading,
Mathematics and Science based on NAEP frameworks
Student SurveyCourse-taking patternsSchool and classroom experiences
Teacher Survey Provides schools with
comprehensive school-level data breaking out student achievement by experiences
SouthernRegionalEducationBoard
Examples of How Schools Use the Results
Identify improvement needs Serves as basis for Site Review
Process (SRP) Gain support for school
improvement Engage the entire faculty in school
improvement Set priorities and goals for school
improvement Improve classroom practices
4
SouthernRegionalEducationBoard
6
Five Components
Reading testMathematics testScience testStudent survey
Course-taking patternsSchool and classroom
experiencesTeacher survey (online)
SouthernRegionalEducationBoard
Testing Times
Reading 75 min. Mathematics 80 min. Science 55 min.
Total 3.5 hours
Student Survey Section 1 - staff person completes
Sections 2 & 3 - untimed (about 35-45 min.)
7
SouthernRegionalEducationBoard
8
Test Content
Used 2009 NAEP frameworks as a starting pointModified to reflect college and career
readiness goals
Reading – focus on informational texts
Mathematics – topics through Algebra II and some pre-calculus
Science – life, physical and earth science
SouthernRegionalEducationBoard
9
Reading Test
75 minutes80% informational texts; 20%
literary nonfictionIncludes meaning vocabularyIncludes paired texts (to assess
critical analysis)Does not include literary fiction
or poetry
SouthernRegionalEducationBoard
10
Reading Test
Passage length: 500-1500 words
Cognitive targets15% locate/recall55% integrate/interpret30% critique/evaluate
SouthernRegionalEducationBoard
11
Mathematics Test
80 minutes
Scientific or graphing calculators permitted for one sectionType of calculator will neither advantage
nor disadvantage student
TopicsPre-algebra through Pre-calculus,
including Algebra II
SouthernRegionalEducationBoard
12
Mathematics Test
Content11% - Number Properties and
Operations29% - Measurement/Geometry24% - Data Analysis, Statistics
and Probability36% - Algebra
SouthernRegionalEducationBoard
13
Mathematics Test
Cognitive Targets50% - Low Complexity
recall routine procedures
45% - Moderate Complexitymake connections within
mathematics or between mathematics and real world
5% - High Complexity
analyze a problem mathematically
SouthernRegionalEducationBoard
14
Science Test
55 minutesContent
43% - Life science 42% - Physical science15% - Earth and space science
SouthernRegionalEducationBoard
15
Science Test
Science Practices70%
• Identifying science principles• Using science principles
30%• Using scientific inquiry• Using technological design
SouthernRegionalEducationBoard
16
Student Survey Section 1 – Course Experience
Staff member should complete this section for each student
Based on Course Matching and Transcript Analysis
Section 2 – School ExperienceStudents complete this sectionSchool and classroom experiencesBased on HSTW key practices
Section 3 – General InformationStudents complete this section
SouthernRegionalEducationBoard
17
Teacher Survey
Online survey! School and classroom experiences
School missionHigh expectations and extra helpGuiding and supporting studentsCurriculum content and engaging
students in learningTransitionsLeadership and professional growth
SouthernRegionalEducationBoard
18
Results Reporting
1. School Report (June, 2014) PDF file on CD
2. HSTW Award of Educational Achievement (by May 2, 2014)
3. Student Reports - Discontinued
SouthernRegionalEducationBoard
19
HSTW Assessment Report
Mailed to schools June, 2014 Executive Summary -“3-minute
report”One page summary of
performance and implementation Overview -“30-minute report”
Slightly longer summary Full report -“3-hour report”
Tables breaking out achievement by experiences
Teacher survey results
SouthernRegionalEducationBoard
20
School Report
Executive SummaryOne-page snapshotGraphs and chartsIncludes:
• Implementation summary• Percentage of students
meeting readiness goals: 250, 257, 258
• Percentage of students at each proficiency level
• Blank quadrant for state data
SouthernRegionalEducationBoard
22
School Report
The OverviewApproximately 15 pagesGraphs and chartsIncludes:
• Mean scores• Percent meeting readiness goals• Percent at each proficiency level• Performance on HSTW indices• More detailed implementation
summary
SouthernRegionalEducationBoard
24
School Reports
Full report
All those tables!
Student achievement broken out by survey questions
Teacher survey results
SouthernRegionalEducationBoard
26
HSTW Award of Educational Achievement Award certificates mailed to
schoolsby May 2, 2014
Awarded to students who:Meet all three readiness goals
(reading 250, math 257, science 258)
Complete at least 2 of 3 HSTW recommended curriculum goals• English/language arts,
mathematics, scienceComplete a concentration
• Career/technical, mathematics/science, humanities
SouthernRegionalEducationBoard
28
Award of Educational Achievement
Opportunities for High Schools
Local community colleges
Local employersAdditional incentives for
students
SouthernRegionalEducationBoard
30
2014 Student Assessment and Teacher Survey
Window
January 24– February 14(3 weeks)
Plan ahead to get the best results!
SouthernRegionalEducationBoard
31
Conducting the Assessment Process
OrderingPreparingAdministeringReturning materials
SouthernRegionalEducationBoard
32
Submit orders today if possible, but no later than Friday, Sept. 27.
Do NOT order online! Do NOT send order to SREB!
The good news :
There is no cost to your school! HSTW SWO pays for the entire
Assessment!
Ordering the Assessment
SouthernRegionalEducationBoard
33
Unless there are special reasons, we encourage you to test no more than 60 students.
You will receive extra test sets. However, when submitting scan sheets to be scored, do not exceed the number ordered!
Schools testing more than 60 must pay for each additional student: $44 per student.
A cancellation fee may be charged if you cancel testing or miss the return deadline.
Other Ordering Information
SouthernRegionalEducationBoard
34
Test Administration Guide Sampling Instructions Student Roster Form Checklist Test Description and Sample
Question Booklet Tasks and Timelines Course Description Guide Course Matching Chart
Administration Materials Online www.sreb.org
SouthernRegionalEducationBoard
Preparing for the Assessment
Administration materials are now available online at: www.sreb.org
Booklets & scan sheets will arrive by January 17, 2014
35
SouthernRegionalEducationBoard
37
Identify Students for the Assessment Select one of two options:
1. RANDOM SAMPLE OF 60 SENIORS
- OR -
2. ALL SENIORS (ENTIRE CLASS)
Select a true random sample by using the instructions provided. You must also select five alternates at random.
Preparing for the Assessment
SouthernRegionalEducationBoard
38
Other Tips for Selecting Students
Only include currently enrolled seniors.
Only exclude students if their IEP exempts them from state assessments.
Same accommodations provided for state assessments should be provided for this assessment.
SouthernRegionalEducationBoard
39
Other Tips for Selecting Students
Administrator Guide will contain suggested accommodations for English-language learners.
Contact ETS by October 25 to request large-print or audio CD versions .
Be reasonably assured that all students can complete ALL 4 components – reading, math, science, AND student survey.
SouthernRegionalEducationBoard
40
Random Sampling Options
What is a “random sample”?
What is the easiest way to draw a random sample?
How to use random number tables
SouthernRegionalEducationBoard
Random Sampling
Simple random sample - every time a selection is made, every student on the list has an equal chance of being the one selected.
Selection Bias – anything that contaminates or distorts the process.
SouthernRegionalEducationBoard
What’s the easiest way to select a random
sample?
Draw names from a hat, box, bowl, or drum!
Draw until you have selected 60 + 5 names!
42
SouthernRegionalEducationBoard
43
Group ActivityRandom Sampling
Procedures: Random Number Table
1. Develop a list of all eligible students (delete names of students who moved or have IEP’s stating they may not participate).
2. Number the remaining names of students on the list
3. Use the random number sheet to select students, starting with column one and working down.
4. The table provided works for classes of up to 350. For larger classes, obtain a larger random number table from SREB.
5. Add student names to the roster until you have the appropriate number.
SouthernRegionalEducationBoard
CAREER CENTERS and HOME SCHOOLS:
How to handle “double-selected” students
44
Career Center selects random sample.Home School selects random sample.Center and home school should compare lists
to see if any students were “double-selected” (i.e., randomly selected by BOTH schools).
If a student is double-selected, test the student at the site where he/she takes senior academic courses.
Contact ETS (Lisa Rion – [email protected] 609-734-5646) to obtain a special student code.
Enter that special code on the double-selected student’s answer document. That student’s results will go into both reports.
SouthernRegionalEducationBoard
45
Preparing for the Assessment
Complete the Student Roster - VERY IMPORTANT!!
Fully complete form including assessment answer document ID numbers.
File with school testing records for five years! Also give one copy to principal.
• You will need this fully completed roster form in Spring 2015 to conduct the follow-up survey of graduates
• Also needed to identify students who fail to include their names (for student reports and awards)
SouthernRegionalEducationBoard
47
Preparing for the Assessment
NotificationNotify students, parents and teachersProvide information on assessment
and/or schedule informational meetingRefer to your Board of Education policy
where applicable
SchedulingSchedule time and place (SREB
recommends administering over 2-3 days)
Include make-up sessionsIdentify assistants (follow recommended
ratios)
SouthernRegionalEducationBoard
48
Preparing for the Assessment
Course-Taking Preparation
Work with department chairs to complete course-matching chart (may update previous work)
Prepare student transcripts (including Fall grades and Spring schedule)
SouthernRegionalEducationBoard
50
Other Tips for Course-Matching1. Get each department head for English,
math and science to help with this process.
2. Include any courses the seniors have taken from grades 9-12 within English, math, science or career-technical areas.
3. Using the course description guide, match each course from your school’s master schedule to the SREB course title on the course-matching chart. Example from the course-matching guide
Regular, Advanced, or College Prep Algebra I:“Algebra I courses include the study of properties and
operations of the real number system, evaluating rational algebraic expressions …”
SouthernRegionalEducationBoard
51
Preparing for the Assessment
Receive Assessment Materials
Booklets & scan sheets arrive at site- by January 17, 2014
Verify contents according to instructions
Keep materials secure (locked)
SouthernRegionalEducationBoard
52
Administering the Assessment
Prepare Assessment MaterialsAssign a numbered answer
document to each student• Record the i.d. numbers on the student
roster
A staff person should complete Student Survey Section One – “Course Experience” for each senior
– Reduces testing time– More accurate
SouthernRegionalEducationBoard
Administering the Teacher Survey
Detailed instructions will be provided, including:
• Sample letter to teachers• Survey and log-on instructions
Teachers must enter the correct HSTW site code. Ohio site codes are 5-digit numbers beginning with “3 6”
53
SouthernRegionalEducationBoard
54
Administering the Assessment
Review Test Arrangements
Confirm dates, times, locations and assistants
Remind students, teachers and parents
Prepare Students
Practice items and use of calculators
SouthernRegionalEducationBoard
Idea Exchange #1
What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to prepare for the Assessment?
55
SouthernRegionalEducationBoard
56
Administering the Assessment
January 24 – February 14, 2014
Schedule make-up sessions for absent students.
It is crucial that students complete ALL FOUR components!!
SouthernRegionalEducationBoard
57
Administration Procedures
Assemble materials and set up the room.
Supervise student check-in. Supervise the session using the
administration scripts. You may answer questions about the
content of the student survey, but not the three subject tests.
Keep materials secure at all times. Schedule makeup sessions. Check i.d. information on scan sheets.
SouthernRegionalEducationBoard
59
Administering the Teacher Survey
January 24– February 14, 2014Schedule early in the window!Include all full- and part-time classroom
teachers 9th-12th
RECOMMENDED – Administer during a staff meeting or other function to guarantee completion. Use computer lab.
Plan on one hour (although most teachers will finish in about 45 minutes).
Surveys are anonymous.Teachers shared by MMGW & HSTW
should take survey version for site at which they are most involved.
SouthernRegionalEducationBoard
Idea Exchange #2
What are some of the most useful or effective things your site has done in previous years (or plans to do this year) to administer the Assessment?
60
SouthernRegionalEducationBoard
Educational Testing Service Test Processing Center
Ewing, New Jersey
• ETS processes thousands of tests every day!• Misplaced materials can be very hard to locate.• Use the pre-paid, bar-coded shipping labels to ensure
that your materials are delivered to the right location!
Returning Materials
SouthernRegionalEducationBoard
62
Returning MaterialsPack according to instructions!
Use the original ETS boxes. Use the ETS pre-paid mailing labels.Return items in their designated box
or envelope:
White Box Brown Box “To Be Scored” “NOT To Be
Scored” 60 answer sheets All test booklets plus header sheet
SouthernRegionalEducationBoard
Returning Materials
VERY IMPORTANTDo not return more student tests
than you ordered!
For example, if you ordered 60, do not return more than 60 in the
white box.
Your school will be responsible for paying $44 for each additional
answer document sent to ETS for scoring.
63
SouthernRegionalEducationBoard
Returning Materials
Include control forms
Use the pre-paid shipping labels
Keep receipts and UPS tracking numbers
64
SouthernRegionalEducationBoard
Returning Student Materials
1. Postmark deadline to mail materials to ETS: February 14, 2014
2. Use the prepaid labels, original boxes and the envelope provided. Include the completed survey control form.
3. If you misplace mailing materials, contact your Regional Office as soon as possible.
4. Return everything! Pack all test books AND extra scan sheets (used or unused) in original boxes and return as instructed.
5. Make sure your materials are picked up.
**KEEP YOUR UPS RECEIPT- this is the only method of tracking your materials.**
65
SouthernRegionalEducationBoard
66
Teacher Surveys
All Teacher Surveys must be completed and submitted online by February 14,
2014.
SouthernRegionalEducationBoard
67
Dates to Remember Orders to Regional Office
TODAY! (no later than Friday, Sept. 27) Administration materials available online
Now! Requests for large print or CD versions to
Educational Testing Serviceby October 25, 2014
Testing materials arrive at sites by January 17, 2014
Student & Teacher administration window January 24 – February 14, 2014
SouthernRegionalEducationBoard
68
Dates to Remember
Student materials postmark deadlineFebruary 14, 2014
Award certificates and student reports mailed to sitesby May 2, 2014
School reports mailed to sitesJune, 2014
SouthernRegionalEducationBoard
69
Top Ten Mistakes (in no particular order)
1. Returning extra student scan sheets (i.e., more than ordered)
2. Missed deadlines• holding make-up tests late in window• shipping too close to deadline• forgetting to have materials picked up
3. Site code errors • Site code zip code !• All Ohio site codes are 5-digit numbers
beginning with “3 6”
4. Incorrect shipping • shipping to wrong address• shipping in wrong box or envelope• damaging scan sheets by using tape,
rubber bands, or paper clips
SouthernRegionalEducationBoard
70
Top Ten Mistakes (continued)
5. Failure to administer the Student Survey• No subject tests can be scored without the
Student Survey!
6. Incorrectly marking the “Assessment Status” box on student answer sheets
• Student tests marked “absent” will not be read by the scanner
7. Random sampling errors -“Frankensamples”
• Adjusting the sample to match site demographics, programs, feeders, etc.
• Picking every 3rd or 4th student• Asking for volunteers• Drawing sample from selected classes only
SouthernRegionalEducationBoard
71
Top Ten Mistakes (continued)
8. Using incorrect test materials • using Xeroxed copies of scan sheets• using old scan sheets
9. Lost items • student roster• pre-paid mailing labels• original shipping cartons• UPS tracking numbers
10. Incomplete student information• names left blank on scan sheets• incorrect demographic information
SouthernRegionalEducationBoard
72
Make the 2014Assessment a
Success! Communication
Letters to students and their parentsAnnouncements to faculty and staff
CooperationImportance of the assessment to the
school’s improvement goalsImportance of students, administration
and faculty taking it seriouslyHandling make-up workCooperation from teachers whose
classes might be interrupted by the assessment
SouthernRegionalEducationBoard
73
Make the 2014 Assessment a
Success! Encourage all students to do their best.
Build enthusiasmBe honest – explain how this helps
the school and future studentsMake students feel special for
participating in the assessment and thank them for their participation
SouthernRegionalEducationBoard
Recommended Next Steps
to Get Started1. Place order.
2. Download and review materials from www.sreb.org .
3. Assemble a small planning team.
4. Schedule dates, times, locations.
5. Announce preliminary plans to faculty and staff.
74
SouthernRegionalEducationBoard
75
Previous participation in an assessment Name, phone and email for the assessment
contact person (will be receiving materials for a year!)
Complete shipping address (no P.O. boxes) Principal’s name, phone and email Number of students to be tested and
sampling method SWO Region pays for 60 seniors at your
site. You must pay for any additional students beyond 60 or special composites.
Week you plan to administer the assessment
Number of teachers to be surveyed
Ordering Information Needed
from You
SouthernRegionalEducationBoard
76
Contact Information Susanne Davis, HSTW SW Ohio
[email protected](513) 677-5641
Missy Kramer, HSTW SW [email protected](513) 619-2330
Fran Cowart, [email protected](404) 879-5611
Lisa Rion, Educational Testing [email protected](609) 734-5646