TELPAS 2012

Texas English Language Proficiency Assessment System

Agenda

• Test Security• Who is a Rater? • What Training is Required• Assemble and Verify Writing Collections• Creating Rating Sessions and Entering Data• Creating and Conduction Reading Sessions• Appendices• Webinars

Test Security and Confidentiality RequirementsConfidentiality Requirements•Maintaining the confidentiality of the TELPAS program involves protecting the contents of all online assessments and student performance documentation. This requires compliance with, but is not limited to, the following guidelines:•Before handling secure test materials, all testing personnel who meet the requirements to participate in state testing must undergo training and must sign the appropriate security oath affirming that they understand what is considered confidential.•Before accessing the secure online administrative features of the Assessment Management System, trained and qualified testing personnel who will be administering online assessments must first read and accept a statement of confidentiality.•All tests must be administered in strict accordance with the instructions contained in the test administration materials.•No person may view, reveal, or discuss the contents of an online assessment before, during, or after a test administration unless specifically authorized to do so by the procedures outlined in the test administration materials. If circumstances necessitate that an online assessment be examined, permission MUST first be obtained from TEA•No person may duplicate, print, record, write notes about, or capture by any electronic means any portion of a secure assessment instrument without prior approval from TEA.•No individuals other than students during an assessment may respond to test items.•No person may review or discuss student responses or student performance data during or after a test administration unless specifically authorized to do so by the procedures outlined in the test administration materials.

Penalties for Prohibited ConductIn accordance with 19 TAC §101.65, any person who violates, assists in the violation of, or solicits another to violate or assist in the violation of test security or confidentiality, and any person who fails to report such a violation, may be subject to the following penalties:

•placement of restrictions on the issuance, renewal, or holding of a Texas educator certificate, either indefinitely or for a set term;•issuance of an inscribed or non-inscribed reprimand;•suspension of a Texas educator certificate for a set term; or•revocation or cancellation of a Texas educator certificate without opportunity for reapplication for a set term or permanently.

Release or disclosure of confidential test content could result in criminal prosecutionunder TEC §39.0303, Section 552.352 of the Texas Government Code, and Section 37.10of the Texas Penal Code. Further, 19 TAC §249.15 stipulates that the State Board forEducator Certification may take any of the above actions based on satisfactory evidencethat an educator has failed to cooperate with TEA in an investigation.

Additionally, irregularities resulting in a breach of test security or confidentiality may result in the invalidation of students’ assessments.

Security OathsAll district and campus personnel who participate in state-mandated testing and/orhandle or view secure test materials or confidential information must meet the eligibilityrequirements and be trained and sign a security oath. Any person who has more thanone testing role (e.g., a TELPAS rater who also serves as a reading test administrator)must receive appropriate training and sign a security oath for each role. Security oathsfor TELPAS raters, writing collection verifiers, and test administrators can be found inAppendix A.

NOTE: Any person who serves as a test administrator, even as a relief for a regular testadministrator, must receive appropriate training and sign the test administrator oath.

Each oath for TELPAS raters, writing collection verifiers, and test administrators must beread and completed after training and before handling or viewing any secure testmaterials or confidential information. Oaths must be returned to the campus coordinatorand will be kept on file in the district for at least five years.

Testing IrregularitiesManual p. 6-8•Eligibility Error•Individualized Education Program (IEP) Implementation Issue•Improper Accounting for Confidential or Secure Materials•Monitoring Error•Procedural Error•Reporting of Testing Irregularities•Reporting of Disciplinary Actions Taken Against Students for Cheating on State Assessments

Rater CredentialsA TELPAS rater is a teacher designated to be the official rater of an ELL’s English language proficiency. Each teacher (including a substitute teacher) selected to rate a student must

■ have the student in class at the time of the spring assessment window; ■ be knowledgeable about the student’s ability to use English in instructional and informal

settings; ■ hold valid education credentials such as a teacher certificate or permit; ■ be appropriately trained in the holistic rating process as required by the TELPAS

administration materials in accordance with 19 TAC, Chapter 101; and ■ rate the student in all eligible language domains.

Raters may include bilingual teachers, ESL teachers, general education teachers, specialeducation teachers, gifted and talented teachers, and teachers of enrichment subjects.Paraprofessionals may NOT serve as raters. The teaching relationship of the rater to thestudent rated is submitted with the student’s holistic ratings as part of the assessmentprocess.

New Raters

Online Basic Training Course

If not calibrated: Supplemental Holistic

Rating Training

Final OnlineCalibration Activity

Set 3

OnlineCalibration Activities

Sets 1 and 2:Set 2 required if not successful

on Set 1

Fall ELPS-TELPAS Professional Development(For new teachers and new prospective raters --

recommended)

Returning Raters

OnlineCalibration Activities

Sets 1 and 2:Set 2 required if not successful

on Set 1

Spring TELPAS Administration Procedures Training

Covers holistic rating training requirements and procedures for conducting TELPAS

• TEA recommends completion of training requirements by the first day of the TELPAS assessment window (March 19)

• Basic training courses and calibration activities will be open throughout TELPAS assessment window to allow districts to handle extenuating circumstances allow raters to refer back to course information and practice activities as

they rate their students

End Dates for Holistic Rating Training

Training Certificates

• Training certificates are generated for raters who complete training courses and calibration activities. A certificate of completion is issued when a rater completes the

online basic training course. A certificate of successful completion is issued when a rater

demonstrates sufficient calibration (70 percent or better) on a calibration set.

• A district may, at its discretion, require a returning rater to repeat new rater training. This may be advisable if testing personnel believe that a

returning rater who was unable to meet rating accuracy requirements in the past would benefit from retaking the online basic training course prior to calibration.

Additionally, TEA recommends that districts consider requiring this if they have not completed TELPAS training since the 2008–2009 school year.

General TEA Definitions of New and Returning Raters

General TEA Definitions of New and Returning Raters

Raters who have completed K–1 training but not 2–12 training will be new raters if they switch to 2–12 training, and vice versa.

• Online Basic Training Course (Required for New Raters) One for raters of K–1 and one for raters of grades 2 and above Includes rating activities to give raters opportunities to practice applying

the rubrics (no state-required level of accuracy established)• Online Calibration (Required for New and Returning Raters)

Separate calibration activities for grades K–1, 2, 3–5, 6–8, and 9–12• Assembling and Verifying Grades 2–12 Writing Collections

Optional course that may be used to supplement test administrator procedures training

Online Training Components

Grade Clusters of Training• K–1 training components are separate from those of grades 2 and

above• It is important that raters know which grade cluster of training to

complete• For rater with students in multiple clusters in grades 2–12

Rater required to train in the cluster with most ELLs As a best practice, rater should review some online basic training

course practice activities in additional cluster(s)• For rater with students in multiple clusters including K–1

Rater required to train in K–1 and one other cluster in grades 2–12 Rater of K–1 who will rate more than one additional cluster within

grades 2–12 should train in the cluster (2, 3–5, 6–8, 9–12) with the most ELLs

As a best practice, rater should review some online basic training course practice activities in other cluster(s)

• Raters complete only as many sets as it takes to calibrate. Many raters will be able to calibrate on the first set of activities,

and many others will be able to calibrate by the end of the second set.

• With supplemental support, very few raters should have difficulty calibrating by the end of the third and final set.

Online Calibration Activities

Important Note: Individuals are not authorized by TEA to serve as raters unless they complete the state-required training and calibration activities. Raters who complete all requirements but do not successfully calibrate may be authorized to serve as raters at the discretion of the district, but districts are required to provide rating support in all domains for these raters.

• There are three sets of online calibration activities.• Each calibration set contains 10 students to rate, and all holistically

assessed language domains are represented in each set. For K–1: listening, speaking, reading, and writing For 2–12: listening, speaking, and writing

• A rater needs to rate at least 70 percent of the students correctly to demonstrate sufficient calibration.

Online Calibration Activities (p. 16, 18)

• Raters not successful after calibration set 2 to get supplemental support and then complete set 3 before the start of testing window

• Raters need access code to complete set 3• Supplemental training resources to be posted in

coordinator Resources section of TrainingCenter website

Districts, not ESCs, have primary responsibility for supplemental training of raters.

Supplemental Support After Second Calibration Set for Raters (p. 17)

Criteria for Supplemental Support Providers

• A supplemental support provider must be a 2010–2011 fully trained and calibrated rater in the applicable K–1

or 2–12 span complete the recorded Web-based training calibrate on either set 1 or set 2

• Training delivered through a Web-based recorded presentation that is accessible from the coordinator resources section of the Training Center Separate training for K–1, 2–5, and 6–12 Training will be available beginning January 30 Both new and returning supplemental support providers need to

complete this training• Districts should designate enough supplemental support

providers to cover the needs of the district Encouraged to train at least one supplemental support provider for

K–1, 2–5, and 6–12 Reviewing the number of raters unable to calibrate by end of

calibration set 2 last year may help anticipate this year’s needs Districts with large ELL populations may find it beneficial to have a

supplemental support provider on each campus

Training of Supplemental Support Providers

• Individual or group sessions Raters attend a one-on-one or group session with a supplemental

support provider before completing their third and final calibration set.

• Brief meeting followed by additional consultation at rater’s request Rater meets briefly with the supplemental support provider for

instructions on resources to review. Rater schedules time with the support provider to get clarification,

if needed, before completing the third and final calibration set.

Note: The models are described in detail in the recorded Web-based training.

Two Basic Models for Providing Supplemental Support

Responsibilities of Testing Coordinators for Holistic Rating Training

• Ensuring raters understand the importance of being properly trained on the holistic rating process

• Ensuring new and returning raters understand which training to take and for which grade clusters

• Monitoring that all raters complete their training requirements and receive supplemental training support if needed

• Ensuring that any individual who serves as a rater but is not successful on the calibration portion of training is provided rater support during the TELPAS administration

Monitoring Course Completions and Performance on Calibration

Activities

Available Reports for Online Training and Calibration

• TELPAS Returning Rater Planning Roster Uses records from the 2008–2009, 2009–2010, and 2010–2011 school

years to generate a list of previously trained raters• TELPAS Confidential Course Completion Roster

Lists online basic training course completion and in-progress status and performance by user for the current year

• TELPAS At-a-Glance Training & Calibration Report Provides a comprehensive list showing basic training course and

calibration completions for all registered users for the current year• TELPAS Confidential Calibration Summary Report

Provides calibration summary information by grade cluster (K–1, 2, 3–5, 6–8, 9–12) and for grade clusters combined

Important Notes about Reports

• Personnel with administrative authorization are able to access reports on demand to monitor TELPAS online training and calibration.

• The reports are updated nightly. • In order for users to appear in the correct reports, their

location (region, district, campus) must be up to date in the My Info section of the TrainingCenter.

• Users who have registered in the TrainingCenter and have started but not completed a calibration set are not included in the summary reports. Only completed sets are presented in the reports.

• However, users who have started a training course will appear in the course roster as being “in progress.”

Assemble and verify grades 2–12student writing collections (p. 19-23)

Assembling Grades 2-12 Writing Collections•Type 1: Basic descriptive writing on a personal/familiar topic•Type 2: Writing about a familiar process•Type 3: Narrative writing about a past event•Type 4: Personal narratives and reflective pieces•Type 5: Expository and other extended writing on a topic from language artsWhat Not to Include in a CollectionVerifying the Writing Collection Components (p. 24)

Prepare to rate your students (p.25 – 31)ListeningThe ability to understand spoken language, comprehend and extractinformation, and follow social and instructional discourse through which information is providedSpeaking The ability to use spoken language appropriately and effectively in learning activities and social interactionsReading The ability to comprehend and interpret written text at the grade appropriate levelWriting The ability to produce written text with content and format to fulfill grade appropriate classroom assignments

Procedures for Rating Students (p. 33)

Create and manage rating entry groups (p.39 – 40)• Creating a Rating Entry Group • Adding Students to an Existing Rating Entry Group• Removing Students From a Rating Entry Group

Enter rating information online (p. 41 – 42)

• Enter proficiency ratings and rater information• Extenuating Circumstances• X – ARD Decision• Rater Information

Roles That Can be Used for TELPAS Rating Entry Functions

• Online session administrator Role for tech staff or other personnel who will help CTC create and

manage online test sessions and/or rating entry groups

• Rating entry assistant Role for personnel who will just enter holistic ratings and other rater

information (this role cannot set up rating entry groups)

Rating Entry Groups• Ratings and other rater information cannot be

entered until the student is placed in a rating entry group

• These groups are not dependent on reading test sessions or reporting groups

• Students may be arranged according to their TELPAS rater or in any other campus grouping The enrolled grade level in PEIMS does NOT restrict the

organization of students in rating entry groups• Testing coordinators or online session administrators

have access to create these groups It is recommended that districts create rating entry groups

1 to 2 weeks before testing

Creating TELPAS Rating Entry Groups

For more information about how to use this function, refer to section 10 of the User’s Guide for the Assessment Management System.

Creating TELPAS Rating Entry Groups

Individual students or whole groups can be added to a rating entry group.

Manage TELPAS Rating Entry Groups

This view shows the list of rating entry groups that have been created for a specific campus. Personnel assigned to enter ratings will need to know which group(s) they are responsible for. To view the students in an existing Rating Entry Group, click the Rating Entry Group name.

Enter Holistic Rating Information Screen

• Holistic ratings and rater information entered from the TELPAS Student Rating Rosters for grades K–12 ELLs

• Information can be entered/updated at any time during the TELPAS administration window

TELPAS Student Rating Roster

Person(s) assigned the role of rating entry assistant will use the information supplied by raters on this roster to enter the rating information in the online system.

Enter TELPAS Holistic Rating Information screen

Enter TELPAS Holistic Rating Information screen• To enter holistic rating information for a student, click the "+"

next to the rating entry group name to expand the group, and then click the student’s name.

• Using the information provided on the TELPAS Student Rating Roster, enter the student’s proficiency ratings for the language domains. Transfer the rater information from the roster by selecting a response to the two questions in the Enter Rater Information section.

• If a student meets state-established criteria for not being assessed, select Extenuating Circumstances or ARD Decision for the applicable language domain(s) in accordance with instructions in the test administration manuals. If Extenuating Circumstances or ARD Decision is selected for every domain, select No Domains Rated in the Enter Rater Information section.

Create test sessions (p. 52 – 56)• Creating a New Test Session• Adding/Removing Students• Modifying a Test Session• Moving Students to Another Session• Printing Authorizations

Browser-based TELPAS Student Tutorials

• Updated student tutorials made available January 2012 Available to assist students in becoming familiar with the

browser-based testing interface used for TELPAS reading Prior versions of the tutorials should be removed The browser-based version of the tutorials can be accessed via a

browser; no files need be downloaded or installed Access the tutorials at

http://www.TexasAssessment.com/TELPAS-tutorials

Roles That Can be Used for TELPAS Reading

• Online session administrator Role for tech staff or other personnel who will help CTC set up and

manage online test sessions and/or rating entry groups

• Online test administrator Role for reading test administrators who just need limited

permissions such as starting/stopping sessions and resuming tests

Ensure that TestNav has been configured BEFORE Creating Sessions

• Proctor caching servers should be designated under Test Setup > Configure TestNav BEFORE test sessions are created

• After inputting your server information and saving, you must also verify the Configuration Details and Save

Creating TELPAS Reading Test Sessions• When creating the session, ensure that a server name

appears in the Proctor Caching Server field CTCs and Online Session Administrators can edit this field if necessary

For instructions about how to create online test sessions, refer to page 52 in the TELPAS Manual and to 8.2 in the User’s Guide for the Texas Assessment Management System.

Test Session Details Functions • Resume Test• Mark Test Complete• Add Students• Remove• Move

For more information about how to use these functions, refer to page 53 in the TELPAS Manual and in 8.2 in the User’s Guide for the Texas Assessment Management System.

Mark Test Complete• Reading tests should only be marked complete for the

following reasons: “Do not score” situations

• Absence for entire testing window• ARD Decision• TEA-approved paper administration for a student already in an

online test session• Other (illness or testing irregularity)

Student forgot to click Submit before exiting out of the test

District coordinators, campus coordinators, and online session administrators have access to complete this task.

Score Codes• Score Codes

Score Absent ARD Decision TEA-Approved Paper Administration Other Student Not to Be Scored

“Score” selection should be chosen only if you are marking a test complete because a student did not submit on his own. A student’s test is automatically scored when the student clicks the Final Submit button at the end of the test.

Student Test Details• Accommodations Information

Accommodations are now separated into Type 1, 2, and 3. Refer to the Accommodation Resources page at http://www.tea.state.tx.us/student.assessment/accommodations for detailed information about accommodations.

Type 1—available for students who have a specific need and who routinely, independently, and effectively use the accommodation during classroom instruction and testingType 2—includes the requirements of Type 1, along with additional specific eligibility criteriaType 3—requires the submission and approval of an Accommodation Request Form to TEA

If the student doesn’t receive reading test accommodations, the fields should be left in the “<select>” position.

Conduct AdministrationTesting Environment

■ There must be at least one test administrator for every 30 students. Smallersessions may be advisable in some situations.

■ To support test security during online testing, testing staff should make adetermined effort to prevent students from seeing one another’s monitor. Thismay include providing an obstruction such as a manila folder taped to eachside of the monitor or a cardboard carrel placed around each workstation.

■ Bulletin boards and instructional displays that might aid students duringtesting must be covered or removed. Any rooms to which students may berelocated must also be prepared.

■ No element of the testing room’s environment should hinder any student’sperformance. The testing room should be quiet, well lighted, well ventilated,and comfortable. Each student should have enough space in which to work.

■ A “Testing—Do Not Disturb” sign should be posted outside the testing room.

Conduct AdministrationSchedulingDistricts may establish starting times for testing. A student who arrives aftertesting has begun may be tested if sufficient time remains in the day and ifthe student has not had contact with students who have already completedtesting.

■ Every effort should be made to administer the TELPAS reading test tostudents who are absent on a given day of testing within the TELPASwindow. If originally scheduled, they should be moved to another test session and assessed later in the testing window. students are not present for the test session for which they were

Testing Procedures ■ Ensure that the seating chart has been completed, including the names of students testing and the students’

locations in the room. Also include the names of all test administrators/monitors involved in a session. If a seating chart has not been provided for you, contact your campus coordinator. If students are moved to another room to finish testing, an additional seating chart must be completed.

■ Make sure that all student workstations are cleared of books and other materials not required for the test. ■ Students must remain seated during testing and are not allowed to converse while testing. ■ Each student must be allowed to work at his or her individual speed. Since the reading test is untimed, each

student must be allowed to have as much time as necessary to respond to every test item. Districts are not required to test beyond the regular school hours, but they are free to do so if they choose. Districts should exercise judgment about starting a test session after lunch, as some students may not have enough time to work at their individual speed to respond to the test items before the end of the school day.

■ Students must not use reference materials. ■ Students are allowed to use scratch paper since they do not have test booklets in which to write notes. All scratch

paper must be turned in to the campus coordinator after testing ■ Test administrators must actively monitor the testing room while students are working. Test administrators should

confirm that students do not have access to nonallowed materials. ■ Some students may finish a test earlier than others. After students have finished testing, students may be allowed

to quietly read books or leave the testing room. ■ Before a student who has finished testing leaves the room, the test administrator must ensure that the student’s

test is in “Submitted” status.

Testing Procedures

• Answering Questions• Computer Monitor Glare• Breaks• Lunch• Cell Phone Use• Emergencies• School Emergencies• Changing Testing Rooms

Starting and Monitoring a Test Session (p. 62 – 65)• Starting a Test Session• Monitoring Student Status During a Test Session

Ready Active Exited Resumed Resumed-Upload Completed Marked Complete

• Exiting a Student’s Test• Resuming a Student’s Test• Stopping a Test Session

Appendices (p. 83)• Appendix A

Oaths of Test Security and Confidentiality

• Appendix B Student Rating Roster

• Appendix C Writing Collection Cover Sheet Writing Collection Verification Checklist

• Appendix D TELPAS Rater’s User Guide for Online Holistic Rating

Training

Date Activity

Jan 13 TELPAS Manual due in districts

Jan 16 Assembling and Verifying Grades 2–12 Writing Collections course available

Jan 27 End date for district coordinator training—all TELPAS components

Jan 30 Online basic training courses for new K–1 and 2–12 raters available

Jan 30 Supplemental support provider recorded Web-based training available

Feb 10 End date for campus coordinator training—holistically rated components

Feb 20 Calibration window opens for new and returning raters—first 2 sets

Feb 20 End date for training raters on administration procedures

Feb 20 Earliest eligibility date for TELPAS writing samples

Feb 29 3rd and final calibration set available

Mar 19–Apr 11 TELPAS assessment window

Mar 19 – 23 TELPAS Reading 2nd – 12th grade. All LEP and PD’s

Mar 19 – Apr 5 Enter Holistic Ratings

Key Dates for Holistically Rated Components

Policy ChangeSecond Semester Immigrant

Non-English Readers

ELLs in grades 2–12 who first enter U.S. schools in second semester are no longer permitted to receive automatic score of beginning without taking TELPAS reading test Now required to take test Will receive scores based on test performance

Policy ChangeNewly Enrolled ELLs

• “20-day rule” gone • Not necessary with new shorter testing window • New policy: No holistically rating ELLs from other Texas

districts, states, or countries who enroll on or after first day of testing window (March 19)

• Unchanged: 2–12 reading test still required

Guiding Principle Students who Move

• If new ELL enrolls before March 19 (start of testing window), receiving campus responsible for all assessments

• If new ELL enrolls on or after March 19, receiving campus only responsible for grade 2–12 reading test (if not already administered)

• Applies whether new student is from another school district, state, or country

Reading Test Session Monitoring• TELPAS Summary Report (District Level)

Displays by campus the number of students with each status (Not Started, In Progress, Completed, and Resolution Required) for the holistic rating and reading portions

• TELPAS Combined Status Report (District and Campus Level) Displays the overall status of each student within the district or campus

and enables coordinators to know whether rating information has been entered, reading tests have been completed, and Years in U.S. Schools information has been entered

TELPAS Holistic Rating Student Status Report – District & Campus Level

TELPAS Holistic Rating Snapshot – District & Campus Level

Monitoring and Finalizing TELPAS Student Data Submissions

• TELPAS Combined Status Report (District & Campus Level) Shows the overall status of each student within the district

or campus Certain key elements must be completed before the

record is acknowledged as complete• Holistic ratings & rater information entered• Grades 2–12 reading tests submitted through TestNav• Years in U.S. Schools information entered

TELPAS Combined Status Report

Questions & Answers

Assessment/Research/Evaluation Dept.February 2012