Texas English Language Proficiency Assessment System (TELPAS) Holistic Rating Training

Audience: Teachers who will serve as TELPAS raters

Fulfills federal requirements for assessing English Language Learners (ELLs), K-12

Aligned with the Texas English Language Proficiency Standards (ELPS)

Results used in state and federal accountability and performance-based monitoring system

2

General Information about TELPAS (p. 3)

Grade Levels

Language Domains

K–1 Observational assessments in listening, speaking, reading, and writing

2–12* Observational assessments in listening and speaking; writing collections for writing

3

Holistically Rated Components of TELPAS (p. 3)

*Online multiple-choice reading tests are used to assess the reading domain for grades 2–12.

Georgetown ISD Calendar of Events for TELPAS (p. 4 – Scheduling)

4

Events Dates

Administration procedures training for holistic component By February 1

New raters complete online basic course (K-1 or Gr. 2-12) By February 4

Returning raters review TELPAS basics (see Slide 26 for resources)

Before attempting calibration activities

All raters complete online calibration activities January 31 – Feb. 11

Collect samples for Gr. 2-12 writing collections February 1 – March 25

Training for Gr. 2-12 TELPAS Online Reading Test By March 4

TELPAS Assessment Window (holistically rating all ELLs & completing online reading test for Gr. 2-12)

March 7 – April 1

Campus Verifiers ensure that Gr. 2-12 writing collections meet all assembly criteria

March 25 - 31

Extended Collection Window (if needed to collect more writing samples)

March 28 – April 4

Extended Assessment Window (to rate/test ELLs in extenuating circumstances)

April 4 – 8

Deadline for CTCs to enter all holistic ratings in the Texas Assessment Management System

April 8

Test Security & Confidentiality Requirements (p. 5)

TELPAS – secure testing program (just like TAKS) Only trained, certified staff and/or their trained

designees who have signed oaths have access to test materials

Must receive training & sign oath before handling test materials, confidential information, or administering a test

Must administer tests in strict accordance with instructions in manuals

No duplication of secure materials without TEA approval

Test Security & Confidentiality Requirements (p. 5)

No viewing, revealing, or discussing test content nor answering questions about test content before, during, or after a test administration unless specifically authorized to do so by test administration procedures

No reviewing or discussing student responses during or after a test unless authorized to do so by test administration procedures

No changing or amending any student response or instructing a student to do so

Security Oaths (p. 6)

Oaths for TELPAS raters, writing collection verifiers, and test administrators found in Appendix A (p. 107)

Each oath must be read and completed AFTER training and BEFORE handling or viewing secure test materials or confidential information.

Bottom of oath (in the box) must be signed AFTER completion of test administration.

Testing Irregularities (p. 6)

Definition: Incidents resulting in a deviation from documented testing procedures

Each person is responsible for reporting immediately to the campus or district testing coordinator any violation or suspected violation of test security or confidentiality.

Failure to report is in violation of 19 TAC, Chapter 101 and could results in sanctions.

Penalities for Prohibited Conduct (p. 7)

Placement of restrictions on the issuance, renewal, or holding of a Texas educator certificate, either indefinitely or for a set term.

Issuance of an inscribed or non-inscribed reprimand. Suspension of a Texas educator certificate for a set

term. Revocation of a Texas educator certificate without

opportunity for reapplication for a set term or permanently.

Additionally, student test results may be invalidated.

Examples of Testing Irregularities(p. 7-8)

Eligibility Error IEP Implementation Issue Improper Accounting for Confidential or

Secure Materials Monitoring Error Procedural Error

Potential Referral to the Educator Certification and Standards Division (p. 9)

Viewing a test (unless authorized to do so) Scoring a test Discussing secure test content, student

responses, or student performance Duplicating, printing, capturing, or photographing

a secure test Assisting students with responses Tampering with student responses or holistic

ratings

Reporting of Disciplinary Actions Against Students for Cheating on State Assessments (p. 9)

A district may invalidate a student’s test if the district determines that the student cheated.

Additional disciplinary action may be taken at the local level.

Any local actions must be submitted to TEA using a special online reporting form.

A separate online incident report form and documentation are necessary ONLY IF the district determines that testing personnel contributed to, caused, or did not detect the cheating due to inadequate monitoring or another error.

RaterResponsibilities

(p. 11 - 49)

Districts identify ELLs in grades K–12 and designate raters to assess students

Training on test administration procedures and holistic rating is conducted to prepare grades K–12 raters to assess English language proficiency consistent with the holistic scoring rubrics, the Proficiency Level Descriptors (PLDs)

Raters gather writing samples for ELLs in grades 2–12 to include in each student’s writing collection

Raters use PLDs to assign students a rating in each language domain assessed

Overview of the Rating & Reporting Process (p. 13)

14

Rater Credentials (p. 13-14)

Have the student in class during spring semester Be knowledgeable about the student’s ability to use

English Hold valid teacher certificate or permit Successfully complete holistic training & calibration

activities Rate the student in all eligible language domains

Each TELPAS Rater must:

A TELPAS Rater is a teacher designated to be the official rater of an ELL’s English language proficiency.

Eligibility Requirements (p. 14)

ARD Decisions: only in rare cases and made on a domain-by-domain basis as documented in the student’s IEP

Other Rare Exceptions: such as for new enrollees

All ELLs in Gr. K-12, including ELLs with parent denials, are required to be assessed in all language domains, with the following exceptions:

Accommodations (p. 15)

Holistic ratings are based on student classwork and observations of student in daily instruction.

For ELLs receiving SPED services, raters take into account the ability of students to use English to access the general curriculum at their enrolled grade level in accordance with accommodations called for in their IEP.

Accommodations generally should not interfere with the holistic rating process.

Talk to your Campus Testing Coordinator if you have any concerns in this area.

Rater Checklist (p. 16)

Will help you keep track of holistic rating activities

We will go through each activity in more detail in the next set of slides.

Activity 1: Complete Training on Administration Procedures (p. 17)

Today’s training is a part of your training on TELPAS holistic rating procedures.

Read and study the TELPAS Manual for Raters and Test Administrators.

Sign your Oath of Test Security and Confidentiality for TELPAS Rater on p. 109 after this training session and return it to the Campus Testing Coordinator (CTC). If you need more time to read the manual before you feel comfortable signing the oath, then return it by the date designated by your CTC.

Activity 2: Complete Holistic Rating Training Requirements (p. 18-23)

An online training process is used. New raters complete an online basic training

course. (The course is optional, but highly recommended, for returning raters.)

Both new and returning raters complete calibration activities.

All eligible language domains are included in the calibration activities, rather than just the writing domain.

New Raters

Online Basic Training Course

If not calibrated: Supplemental Holistic

Rating Training

Final OnlineCalibration Activity

Set 3

OnlineCalibration Activities

Sets 1 and 2:Set 2 required if not successful

on Set 1

Returning Raters

OnlineCalibration Activities

Sets 1 and 2:Set 2 required if not successful

on Set 1

Spring TELPAS Administration Procedures Training

Covers holistic rating training requirements and procedures for conducting TELPAS

(Basic Training Course is optional for Returning Raters.)

Activity 2: Page 19

Online Basic Training Course (Required for New Raters & Highly Recommended for Returning Raters) One for raters of K–1 and one for raters of Gr. 2–12

(Approximate Completion Time: 4-5 hours) Online Calibration (Required for New and Returning

Raters) Separate calibration activities for grades K–1, 2, 3–5, 6–8, and

9–12 (Approximate Completion Time: 1-2 hours)

22

Spring 2011 Online Training Components (p. 19)

Online courses are designed to prepare raters to assess the English language proficiency of students in a uniform manner statewide.

New Raters: Complete Online Basic Training Course by February 4 (Deadline for GISD New Raters)

Training courses and calibration activities are grade cluster specific (K-1, 2, 3-5, 6-8, 9-12), so confirm your cluster BEFORE starting your online training.

If you will rate students from more than one grade cluster, consult your Campus Testing Coordinator for instructions.

23

Spring 2011 Online Training Components (p. 20)

Complete course in a distraction-free setting. You may complete the course in more than one sitting. The rating practice activities must be completed

independently. (online oath included) Five practice activities are included for each language

domain. There is no required number of students to rate correctly

since the activities are for practice. It is a violation of state assessment procedures to share or

discuss answers to the rating practice activities with individuals who have not yet completed the course.

You will be issued a course certificate upon completion.

24

Online Basic Training Course (p. 21)

Each calibration set contains 10 students to rate, and all holistically assessed language domains are represented in each set. For K–1: listening, speaking, reading, and writing For 2–12: listening, speaking, and writing

A rater needs to rate at least 70 percent of the students correctly to calibrate.

If a Returning Rater is unsuccessful on the first set, you must complete the Online Basic Training Course (if not already completed) and provide your CTC with your training certificate (GISD requirement).

25

Online Calibration Activities (p. 21-22)

Resources for New & Returning TELPAS Raters (p. 21)

Educator Guide to TELPAS Gr. K-12 (web-based guide to familiarize educators with TELPAS; includes student video segments, authentic student writing, & sample test questions)http://ritter.tea.state.tx.us/student.assessment/ELL/telpas/K-12InfoGuide.pdf

PowerPoint presentations are available on TEA’s Student Assessment Division website in the Rater Training for 2010-2011 section at:

http://www.tea.state.tx.us/student.assessment/ell/#telpas Making the ELPS-TELPAS Connection Grades K12 Introduction Introductory Training on the PLDs Gr. K-1 Introductory Training on the PLDs Gr. 2-12

26

(See bottom on p. 21 in TELPAS Rater Manual.)

Raters complete only as many sets as it takes to calibrate. Many raters will be able to calibrate on the first set of

activities, and many others will be able to calibrate by the end of the second set.

Calibration sets are available on January 31 and must be completed by February 11 (in GISD).

Individuals who do not calibrate by the end of the second set will be provided with supplemental training prior to the third and final set.

27

Online Calibration Activities (p. 22)

Raters not calibrated after Set 2 to get supplemental support and then complete Set 3 before start of testing window (before March 7)

Raters need access code to complete Set 3

3rd Calibration Set: Available February 16

28

Supplemental Support After Second Calibration Set for Raters (p. 22)

Calibration – Key Points (p. 23) 3 opportunities to calibrate Resources available as warm-ups (see p. 21 & slide 26) Online oath Self-paced Students presented in random fashion but domain order fixed (listening,

speaking, reading (K-1 only), & writing Score & rating annotations available right away Although the 2nd set is available right away, DO NOT ATTEMPT until you

check in with your CTC. If you did not complete the Online Basic Training Course, you MUST complete it before attempting the 2nd set. Review other support materials as well.

Training is complete after successful calibration on Set 1 or Set 2. Print Calibration certificate and give it to your CTC.

Raters not successful after 2 attempts will receive supplemental training before Set 3 can be accessed.

Accessing the TrainingCenter Website (p. 23)

See Appendix D starting on p. 123 for specific information about accessing the TrainingCenter website.

Site code for new TELPAS users: telpas2011 New raters must register using the directions on p. 128-

131 Returning raters already have an account and should

review p. 132-134. Returning raters should review the “My Info” link and

update email, phone #, region, district, campus, etc. as needed. Name changes must be handled through Pearson at 800-627-0225.

30

Updates to the Texas TrainingCenter

31

TELPAS TrainingCenter Updates

Improvements made for easier navigation to the correct training component

32

TELPAS TrainingCenter Updates

33

TELPAS TrainingCenter Updates

34

TELPAS TrainingCenter Updates

35

TELPAS TrainingCenter Updates

36

Activity 3: Assemble & Verify Gr. 2-12 Student Writing Collections (p. 24-25)

Writing collections must contain at least 5 samples (1 narrative about a past event; 2 academic from math, science, or social studies; 2 other samples).

Writing assigned on or after February 1 may be considered. More than 5 samples should be collected. The more recent the writing samples, the more accurate the

collection will be. Include student name and date on all samples. Staple the cover sheet and verification checklist found in

Appendix C (p. 121-122) to the front of the writing collection. Photocopies are okay if clear and legible. Collect samples from February 1 through March 25. Campuses

may set earlier deadlines or interim deadlines, so check with your CTC.

37

Writing Examples (p. 25-27)

Type 1: Basic descriptive writing on a personal/ familiar topic

Type 2: Writing about a familiar process Type 3: Narrative writing about a past event

(REQUIRED) Type 4: Reflective writing Type 5: Extended writing on a topic from language

arts Type 6: Expository or procedural writing from

science, math, or social studies (2 samples REQUIRED)

38

Tips for Good Writing Collections (p. 27)

Encourage students to take their time and write in as much detail as possible.

Include writing tasks that allow students to truly showcase the English they know while writing on topics that are comfortable and familiar.

Also include writing tasks that stretch and push students’ limits so that you can see the full extent of their language abilities and any elements of second language acquisition they still need to develop.

39

What Not to Include in a Collection (p. 28)

Papers containing language directly copied from a textbook, lesson, or other written source

Papers with heavy reliance on a dictionary or thesaurus Papers that show a teacher’s corrections Papers that have been polished through editing by

peers, parents, or teachers Papers written primarily in a student’s native language Worksheets or question-answer assignments Photocopies of TAKS written compositions or responses

to TAKS open-ended questions Papers that are brief, incomplete, or obviously reflect

writing that was rushed

40

Verifying the Writing Collection Components (p. 28-29)

TELPAS Raters and paraprofessionals are NOT permitted to serve as a verifier.

All raters complete the verification checklist (see p. 122) for their assigned ELL students as well as the front of the cover sheet (see p. 121).

Campus verifiers (usually the CTC) must complete the online training course, Assembling and Verifying Gr. 2-12 Writing Collections, on the Texas TrainingCenter website by February 28 (GISD requirement).

Collections that do not meet the requirements will be returned to the rater so that additional samples may be collected.

The verifier will initial in each box and sign the verification checklist at the bottom.

41

Activity 4: Prepare to Rate Your Students(p. 30-36)

Language Domain Definitions Proficiency Level Descriptors and Global Definitions Progress Through the Proficiency Levels Social and Academic Language Proficiency Academic Language Proficiency Versus Academic Achievement Observing Students in Each Language Domain Collaboration with Others (GISD requires that you collaborate

with other teachers and school personnel on your holistic ratings.)

42

Prior to rating your students (& after all online activities have been completed), review p. 30-36 on the following topics:

Activity 5: Rate Your Students (p. 37)

TELPAS Rater Manual, which includes the PLDs The GISD Version of the TELPAS Rater Roster (Do not

use the one in Appendix B. The GISD version has a column to indicate the initials of the collaborator for each student.)

The TELPAS Writing Collection Cover Sheet (p. 121-122) run front/back for Gr. 2-12 students

43

You will need the following:

Keep student writing collections & student rating rosters in locked storage when not in use.

The primary rating window in GISD is March 7 through April 1.

Procedures for Rating Students (p. 38)

Use the PLDs on p. 39-43 to rate your students. Always refer to the PLDs You may copy the PLDs and make notes on individual students if helpful. Start with students whose English proficiency levels are clearest to you. Ratings should be based on a variety of observations. Evaluate the most current level of English proficiency & consider at which

level the student performs most consistently. Collaborate with others. Complete all ratings for one student before proceeding to the next student. Compare each student to the PLDs and do not let proficiency levels of

other ELLs influence your ratings. Except for Gr. 2-12 writing, your determination is based on reflections of

observations, not formal language observation tasks. Remember that academic language proficiency is not the same as

academic achievement. A student does not have to be a high achiever in order to have an advanced high level of English language proficiency.

44

Activity 6 & 7 (p. 44-48)

You will not be required to create and manage rating entry groups, so disregard Activity 6.

You will not be required to enter rating information online, so disregard Activity 7.

Your Campus Testing Coordinator or the District Testing Coordinator will complete these tasks.

45

Activity 8: Prepare Materials for Return (p. 49)

Verify holistic ratings and rater information you supplied on the Georgetown ISD TELPAS Student Rating Roster are accurate and complete.

Review the accuracy of any other student data you have been asked to verify.

Make sure you have completed, signed, and dated the GISD TELPAS Student Rating Roster and obtained the initials of a Collaborator for each student.

Gr. 2-12: Make sure that you have completed the TELPAS Writing Collection Cover Sheet (front & back) and stapled it to the front of each writing collection.

46

Activity 8: Prepare Materials for Return (p. 49)

Be sure that you have signed the last box of the TELPAS Rater Oath and returned it to your CTC

Return your GISD TELPAS Student Rating Roster Turn in your student writing collections Return your TELPAS Manual after all TELPAS activities

have been completed.

47

TELPAS Calendar of Events in GISD: Let’s Review

48

Events Dates

Administration procedures training for holistic component By February 1

New raters complete online basic course (K-1 or Gr. 2-12) By February 4

Returning raters review TELPAS basics (see Slide 26 for resources)

Before attempting calibration activities

All raters complete online calibration activities January 31 – Feb. 11

Collect samples for Gr. 2-12 writing collections February 1 – March 25

Training for Gr. 2-12 TELPAS Online Reading Test By March 4

TELPAS Assessment Window (holistically rating all ELLs & completing online reading test for Gr. 2-12)

March 7 – April 1

Campus Verifiers ensure that Gr. 2-12 writing collections meet all assembly criteria

March 25 - 31

Extended Collection Window (if needed to collect more writing samples)

March 28 – April 4

Extended Assessment Window (to rate/test ELLs in extenuating circumstances)

April 4 – 8

Deadline for CTCs to enter all holistic ratings in the Texas Assessment Management System

April 8