Understanding Quick Scores & This Year’s Change

Dr. Nakia TownsAssistant Commissioner for Data and Research

June 1, 2015

2

Essential Questions

What is a quick score? What changed about calculating quick scores this

year in grades 3-8 and why? What is the impact of the change in methodology

for calculating quick scores this year in grades 3-8? How is the department communicating directly

and how is it supporting districts to communicate with parents and teachers regarding the change in quick score methodology?

What is the plan regarding quick scores for 2015-16, with the transition to TNReady, and beyond?

3

Key Terms

Raw Score: the number of questions a student answered correctly.

Scale Score: generated from the raw score; this allows a comparison of student performance from year to year. Because the questions on the test change each year, scale scores give us a metric to compare apples to apples.

Cut Score: generated from the scale score; determines a student’s performance level (i.e., below basic, basic, proficient, advanced).

Quick Score: generated from the raw score; only to be factored into a student’s end of year grade, as required by law. This score is on a 100-point scale because student grades are on a 100-point scale.

Performance Levels: generated from the scale score to represent student’s level of proficiency. Performance levels are categorized as below basic, basic, proficient, or advanced.

4

TCAP Scoring Flow Chart

5

What is a Quick Score?

3rd Grade RLA – What is proficient?

Though 72 percent correct means a student is proficient, this would be a low pass to failing grade on most A-F grading scales for 3-8.

Quick scores convert the number of questions students answered correctly (raw score) on TCAP into a measure on a 100-point grading scale as typically used by teachers.

A quick score is not the percent correct on TCAP. A quick score is not a percentile rank.

72% correct

760 Scale Score

43 Raw Score

43 correct answers out of 60 questions

Quick Score?

6

Quick Score Methodology Cube Root Scale

In grades 9-12, the state has a uniform grading policy. The quick score calculation for EOCs has used the cube root scaling method since 2011.

7

Quick Score MethodologyInterval Scale

Grades 3-8 used the interval scaling method to calculate quick scores prior to this year, 2014-15. This method fixed quick scores to TCAP performance levels for grades and subject.

(1) Number Correct = 0, then Numeric Score (or Quick Score)=0,(2) If Raw Score Basic Cut = 29 (2011 RLA, Grade=5), then 70 (Quick Score),(3) If Raw Score Proficient Cut = 48 (2011 RLA, Grade=5), then 85 (Quick Score),

8

Quick Scores 2014-15

This past fall, prior to leadership transition, the office of assessment logistics made the decision to stop including TCAP performance levels with quick scores.

The department also decided to use the cube root scaling method for both high school EOCs and 3-8 Achievement.

These changes were made to streamline the calculations, providing a consistent methodology in high school EOCs and 3-8 Achievement.

It’s important to note, the interval scaling method cannot be used during assessment transitions before teachers complete standards setting to determine scale score cuts.

As the cube root methodology was used for EOCs in prior years, there was no change this year for students, teachers, and parents in grades 9-12.

9

Quick Scores 2014-15

The change in quick score methodology for grades 3-8 was not adequately or widely communicated in fall 2014, resulting in confusion for students, teachers, and parents in grades 3-8 with regard to interpreting quick scores.

Though no performance levels were published with quick scores, many relied on the prior interval scaling method and believed that a quick score of 85 was equivalent to proficient.

Possibly, some teachers may have communicated to parents that students were proficient who may not have been.

Possibly, some schools may have prematurely estimated their TCAP proficiency rates in advance of official accountability releases in June and July.

10

Quick Score MethodologyImplications

The bar for proficiency has not changed. Scale score cut points are the same in 2015 as they were in 2014.

The quick score methodology does not impact TCAP performance levels.

The reality of student performance with regard to scale score cuts does not depend on the quick score.

Subject Grade Proficient Cube 2014 Subject Grade Proficient CorrespondingRaw Cut Root QS Method QS Raw Cut Quick Score

RLA 3 760 43 89 85 RLA 3 760 42 854 760 44 90 85 4 760 44 855 755 43 89 85 5 755 42 856 752 43 89 85 6 752 42 857 760 44 89 85 7 760 43 858 760 46 91 85 8 760 45 85

2015 Proficient Raw Score Cuts 2014 Proficient Raw Score Cuts Proficient Scale Cut

Proficient Scale Cut

11

Quick Score MethodologyImplications

How do quick scores impact teachers? Quick scores do not impact evaluation or TVAAS, and quick

scores have no bearing on personnel decisions tied to performance.

Quick scores have no impact on teacher, school, or district accountability metrics.

How do quick scores impact students? For grades 3-8, there is no statewide uniform grading policy.

This means that a quick score of 84 in one district may be considered a B, while in another district it may be considered a C.

For grades 9-12, there is a statewide uniform grading policy, so that the quick score range for an A is consistent in all districts.

Quick scores count for 15-25% of the student’s semester grade or 7.5% to 12.5% of the end of year grade in 3-8.

12

Additional Information and Support

Monday, May 18 – Quick scores using cube root scaling methodology released to districts for grades 3-8.

Tuesday, May 19 – Commissioner learns of fall 2014 decision to change 3-8 quick score calculation and determines this change has not been previously communicated.

Wednesday, May 20 – Commissioner emails directors indicating quick score methodology change and timeline for additional information. Also included in Director Update.

Friday, May 22 – Detailed information released to districts regarding the quick score methodology for grades 3-8 achievement and high school EOCs, as well as tables showing quick score that is equivalent to scale score proficiency cuts.

Wednesday, May 27 – Conference call with directors and district data and accountability personnel to answer questions. May 22 and conference call information included in Director Update.

Thursday, May 28 – Emailed detailed quick score information to Tennessee Education Association, which was posted to TEA website.

13

Additional Information and Support

Thursday, May 28 – Released raw scores for spring 2015 achievement 3-8 and high school EOCs to all districts. Second conference call with district data and accountability personnel.

Friday, May 29 – Launched Understanding Quick Scores website. • www.tn.gov/education/data/quick_scores.shtml

Friday, May 29 – Released Educator Update and Principal Update with message from Commissioner and additional quick score information.

Friday, May 29 – Released Classroom Chronicles blog post from Commissioner with apology for department communication failure and additional quick score information for all educators.

Saturday, May 30 – Released raw scores and corresponding quick score using interval scaling methodology for spring 2015 achievement 3-8.

June 1-15 – Regional face-to-face meetings scheduled in East, Central and West Tennessee to answer additional questions for district personnel regarding quick scores and TCAP.

June 1-15 – Additional meetings with various stakeholder groups.

14

Quick Scores in 2015-16 and Beyond

New TNReady assessments for math and English language arts, as well as new social studies and US History assessment will be introduced next year.

Teachers will not complete standards setting to define scale scores cuts for any of these subjects until summer 2016.

Though quick scores can be calculated in advanced of standards setting, the department will seek additional stakeholder feedback to determine what is most appropriate.

Over the next few months, the department will engage TOSS (superintendents) Working Group in discussion about quick scores for 3-8 achievement and high school EOCs for 2015-16.

In August 2015, the department will communicate initial decision regarding quick scores and follow-up in September with more details during regional assessment meetings, as well via department updates and on public websites.

15

Resources

Learn more about Understanding Quick Scores: www.tn.gov/education/data/quick_scores.shtml

Learn more about TNReady: www.tn.gov/TNReady Preview questions for TNReady via MICA Item Sampler:

www.micatime.com• All teachers have logins – Commissioner email on 5/19.

Contact district testing coordinator or support@micatime.com. • Public access via “About” menu.

Learn more about all TCAP Assessments: www.tn.gov/education/assessment/

QUESTIONS?