TELPAS Rater Training

TELPAS

Holistically Rated Assessments

Online Holistic Rating Training

Online holistic rating training prepares teachers to rate English language proficiency consistent with the holistic rating rubrics—the proficiency level descriptors (PLDs) from the English Language Proficiency Standards (ELPS). Raters are required to complete holistic rating training annually. The online basic training courses and calibration activities are grade-cluster specific. The training is available through the Texas English Language Proficiency Assessment System (TELPAS) Online Training Center website, which requires a username and password for access. Raters who created an account in the previous year should use those same credentials for the current year.

District and campus testing coordinators are responsible for ensuring that all individuals designated to serve as TELPAS raters fulfill their holistic rating training requirements. Online basic training courses may be completed independently. Calibration activities must be completed in a monitored setting. District coordinators receive daily passcodes for each calibration set, which are used to control access to the sets. A rater should not be given a passcode to revisit a specific calibration set after the day it was completed.

The chart outlines the steps and components of the holistic rating training process.

Grades K–12 Holistic Rating Training Flowchart

Teachers must be assigned to complete holistic rating training in the grade cluster that corresponds to the grade levels of the students they will rate. Note the guidelines below for teachers who are assigned to rate students in more than one grade cluster (e.g., students in grades 5 and 6 or in grades 1 and 2).

  • Multiple clusters within grades 2–12. These raters are required to complete training in the cluster in which they have the most English learners (ELs). As a best practice, they should also review online basic training course practice activities in the additional cluster(s) to check their readiness to apply the PLD rubrics appropriately.
  • Multiple clusters that include K–1. These raters must complete training for K–1 and at least one other cluster (within grades 2–12). Raters with more than one additional cluster (within grades 2–12) should train in the cluster in which they have the most ELs. As a best practice, they should review online basic training course practice activities in the other cluster(s) to check their readiness to apply the PLD rubrics appropriately.

NOTE: Individuals are not authorized by TEA to serve as TELPAS raters unless they complete the state-required holistic rating training requirements, including calibration activities. Raters who complete all requirements but do not successfully calibrate by the end of set 2 may be authorized to serve as raters at the discretion of the district. However, districts are required to provide rating support for these teachers so that their assigned students are assessed consistent with the PLDs.

Rater Credentials

K–1 Rater Credentials

A teacher (including a substitute teacher) selected to rate an EL must

  • have the student in class at the time of the spring assessment window
  • be knowledgeable about the student’s ability to use English in instructional and informal settings
  • hold valid Texas education credentials, such as a teacher certificate or permit
  • be appropriately trained in the holistic rating process
  • rate the student in all eligible domains

For K–1, teachers must rate each student in all domains for which the student is eligible. A campus is not permitted to use different raters for different domains.

Raters may include bilingual teachers, ESL teachers, general education teachers, special education teachers, gifted and talented teachers, and teachers of enrichment subjects. Paraprofessionals may not serve as raters.

2–12 Rater Credentials (writing only)

Raters do not need to have the students in class at the time of the spring assessment window. However, teachers are still required to assemble student writing collections. If using the option of centralized raters, districts will need to develop their own process to gather the writing collections districtwide, campuswide, or by grade level(s). Electronic assembly of writing collections is allowable, however districts must ensure that the process is valid and FERPA regulations are maintained.

Please note that districts, at their discretion, can implement additional requirements for 2–12 raters of writing collections.  

2–12 Rater Credentials (listening, speaking, and writing)

In rare circumstances, an EL may be approved for a special administration for the listening and speaking domains. In this situation, the teacher must rate the student in all the eligible domains. A campus is not permitted to use different raters for different domains.

A teacher (including a substitute teacher) selected to rate an EL must

  • have the student in class at the time of the spring assessment window
  • be knowledgeable about the student’s ability to use English in instructional and informal settings
  • hold valid Texas education credentials, such as a teacher certificate or permit
  • be appropriately trained in the holistic rating process
  • rate the student in all eligible domains

Raters may include bilingual teachers, ESL teachers, general education teachers, special education teachers, gifted and talented teachers, and teachers of enrichment subjects. Paraprofessionals may not serve as raters.

For more information on special administration for an online test, refer to the Special Administration of an Online Assessment section in these Coordinator Resources.

Raters are not permitted to serve as verifiers. Their sole responsibility is to rate students in the applicable domains and assemble writing collections.

Assembling and Verifying Writing Collections

The TELPAS Rater Manual outlines specific requirements for assembling and verifying the contents of the writing collections. These requirements support the validity and reliability of the writing assessment by ensuring that students are assessed on the types of writing that best depict their ability to write in English during content-area instruction. For districts that choose to designate centralized raters, a process to gather and verify the writing collections must be established. 

Testing coordinators establish and oversee processes to ensure that writing collections are assembled and verified as required. An optional TELPAS Online Training Center course titled Assembling and Verifying Grades 2–12 Writing Collections may be used to supplement the test administration procedures training of testing coordinators, campus administrators, raters, and writing collection assemblers, and verifiers.

Key Holistic Rating Training Dates

Date

Activity

October 19

TELPAS Rater Manual available online

January 11

TELPAS Online Training Center opens; Assembling and Verifying Grades 2–12 Writing Collections course available

January 22End date for district coordinator training—all TELPAS components

January 25

Online basic training courses for new K–1 and 2–12 raters available

February 5

End date for campus coordinator training—all TELPAS components

February 8

Calibration window opens for new and returning raters

February 8

Earliest eligibility date for TELPAS writing samples

February 19

End date for training raters and verifiers on administration procedures

February 22–April 2

TELPAS assessment window, grades K–12 listening, speaking, and writing and grades K–1 reading

February 22–April 9TELPAS Assessment window, grades 2–12 reading


The online courses and calibration activities remain available until the end of the TELPAS assessment window. However, it is recommended that districts require raters to complete their training requirements by the first day of the TELPAS assessment window. If there are extenuating circumstances and a district determines that an additional rater needs to be trained after the assessment window has begun, the rater must complete all training requirements before rating students for the assessment.

Assessment Validity and Reliability

Districts are required to implement procedures to support the validity and reliability of the TELPAS rating process. The procedures may be districtwide or vary by campus. The requirements are described in the Online Administrations, Prepare for Online Administrations, and TELPAS Writing Collections sections in these Coordinator Resources.

TEA conducts periodic validity and reliability audits of the TELPAS holistic assessment process to examine the extent to which test administration procedures are followed and the holistic ratings are valid and reliable.

Accommodations Information

Information regarding designated supports available during TELPAS can be found on the Accommodation Resources webpage.

In rare cases, a student may need a special administration of an online test, which will require TEA approval. Refer to the Special Administration of an Online Assessment page of these Coordinator Resources.

Icon with the letters C and C inside, used to represent Campus Coordinator.Plan for and conduct administration procedures training for raters, writing collection assemblers, and verifiers.

❏ Designate Raters and Verifiers

  • Select raters and writing collection verifiers in cooperation with your district coordinator and other designated leadership staff. Districts are encouraged to train as many new teachers as possible each year because rater training provides teachers with professional development to improve instruction for ELs.
  • As a best practice, individuals who lack familiarity with the ELPS and TELPAS and who will be trained as new raters in the spring should be identified and provided with staff development in the fall.
  • Raters for K–1 students and for 2–12 students approved for a special administration for listening and speaking assessment must be the teachers of the ELs they rate and have the students in class at the time of the spring assessment window.
  • While ESL teachers who have ELs in pull-out programs may be designated as raters if necessary, keep in mind the importance of training content-area teachers as raters because TELPAS ratings must reflect the ability of students to understand and use English during content-area instruction.
  • Writing collection verifiers may be LPAC administrators, school counselors, or other suitable staff but must not be paraprofessionals or teachers serving as TELPAS raters this year.

❏ Schedule Test Administration Procedures Training for Raters, Writing Collection Assemblers, and Verifiers

  • This training is required annually for staff assigned to these roles.
  • Training for test administration procedures for raters and verifiers should take place by the test administrator training date specified on the Calendar of Events. If necessary, additional individuals may be trained after this date to address extenuating circumstances.

❏ Provide Access to the TELPAS Rater Manual to Raters, Writing Collection Assemblers and Verifiers Before Training Session

  • Direct raters, writing collection assemblers, and verifiers to read the manual.

❏ Conduct Administration Procedures Training for Raters, Writing Collection Assemblers, and Verifiers

  • Some key topics to cover during these sessions include:
    • TELPAS test administration procedures and timelines, including requirements and timelines for assembling and verifying grades 2–12 writing collections
    • the ways in which test security, confidentiality, and integrity apply to the holistically rated assessments
    • fundamentals of the holistic rating process for raters who do not yet have this background
    • a review of the holistic rating online training requirements for raters, including grade cluster assignments, training deadlines, training components for new and returning raters, information about location of monitored sessions for calibration, rating support policy for any raters who do not successfully calibrate, and the process for logging into the TELPAS Online Training Center
    • consequences of cheating
  • Make sure new and returning raters have a signed oath on file before they create a new account or log in to the TELPAS Online Training Center with their existing account.
  • Make sure new raters understand which training to take and for which grade clusters.
  • Raters must be well trained to use the PLD rubrics by reviewing the TELPAS Rater Manual.
  • You may wish to assign raters and verifiers to complete or review the TELPAS Online Training Center course titled Assembling and Verifying Grades 2–12 Writing Collections. A TEA-provided presentation titled Grades 2–12 Writing Collection Overview is also available on the TELPAS Resources webpage.
  • Raters, writing collection assemblers, and verifiers must sign the appropriate security oath after training and prior to handling secure materials.

Two icons with the letters D and C inside unsafe-one, and the letters C and C inside the other, used to represent District Coordinator and Campus Coordinator.Prepare for and conduct monitored calibration sessions.

❏ Designate Staff to Conduct Calibration Sessions

  • A testing coordinator or other designated personnel must serve as a proctor for monitored calibration sessions. The number of proctors needed depends on the number of raters being trained.
  • Proctors may be LPAC administrators, school counselors, or other suitable staff but must not be teachers serving as TELPAS raters or personnel serving as TELPAS centralized raters this year.
  • Proctors must have received administration procedures training and sign the appropriate security oath.

❏ Schedule Monitored Calibration Sessions

  • Monitored calibration sessions must be completed before raters holistically rate their assigned students for TELPAS.
  • Districts must establish local procedures for verifying the completion of the online basic training course before new raters can access calibration activities.
  • The number of calibration sessions needed depends on the number of raters who must be trained to rate the ELs on each campus.
  • Most raters will need 1–2 hours to complete a calibration set, and each calibration set must be completed in one session.
  • Local procedures must be established for completion of a second calibration set. Calibration sets 1 and 2 may be completed in one session or in separate sessions, as determined by local procedures. Once a rater completes a set, he or she cannot visit that set on another day.
  • The use of headphones is required for raters to complete the calibration activities that include the listening and speaking domains.
  • A printer should be available for printing certificates.

❏ Conduct Monitored Calibration Sessions

  • Ensure that a trained proctor is available to actively monitor the calibration session.
  • Campuses must maintain a roster of participants for each calibration session. A sample roster is available for use from the Coordinator Resources for TELPAS Holistic Rating Training document, accessible to the district coordinator, in the STAAR Alternate 2, TELPAS, and TELPAS Alternate Assessment Management System.
  • Ensure proctors have the correct calibration passcodes for the day of the session. Daily calibration passcodes are available from the Coordinator Resources for TELPAS Holistic Rating Training document. Passcodes are updated weekly throughout the TELPAS testing window.
  • Raters must be given the correct passcode for the day once they have logged in for their calibration session. Make sure raters understand in which grade cluster they should calibrate.
  • Raters should not be allowed back into a calibration set after the passcode used to access the set has expired, including going back to review completed calibration sets.
  • Raters may use a copy of the PLDs during their calibration activities. Raters can be provided with scratch paper, if needed. Any notes taken during calibration should be done on the scratch paper or on a separate copy of the PLDs provided. All notes taken by raters during the session must be turned in to the proctor and destroyed.
  • Remind raters that calibration must be completed individually and that collaborating or discussing responses to calibration activities with others is not allowed.
  • Raters must begin with calibration set 1. If raters are successful on set 1, a certificate will be available from the scoring summary tab to print and turn in to the proctor. Raters may print an extra copy of their certificate for their own records.
  • If raters are unsuccessful on set 1, notify raters of local procedures on how to proceed to set 2.
  • Collect calibration session roster and calibration certificates for documentation. Collect copies of PLDs, scratch paper, and any other materials with notes used during the calibration session to be destroyed.

Two icons with the letters D and C inside unsafe-one, and the letters C and C inside the other, used to represent District Coordinator and Campus Coordinator.Monitor holistic rating training.

❏ Monitor Holistic Rating Training

  • New Raters. New raters may complete the online basic training course independently. Rating practice for grades K–1 and 2–12 is separate from the course and can be completed in the TELPAS Online Training Center after completing all modules of the course. Online calibration must be completed in a monitored setting. Before new raters attend a monitored calibration session, campus coordinators must ensure that new raters complete the appropriate online basic training course.
  • Returning Raters. Returning raters may review rating practice activities from the online basic training course before attending a monitored calibration session. Returning raters who have not completed calibration activities within the last three school years are required to complete the online basic training course before calibrating. Online calibration must be completed in a monitored setting.
  • Training certificates and professional development hours. A certificate of completion for the K–1 and 2–12 online basic training courses will be issued within the TELPAS Online Training Center upon completion of the modules and practice activities. There is no state-required level of rating accuracy established for the online basic training course practice activities. A certificate of successful completion is issued within the TELPAS Online Training Center when a rater demonstrates sufficient calibration (70 or better for grades 2–12; 70 or better for grades K–1) on a calibration set.
    • Five hours are awarded for the K–1 online basic course.
    • Three hours are awarded for the 2–12 online basic training course.
    • One hour is awarded for the optional assembling and verifying course.
    • Two hours are awarded for calibration.

Certificates will be available to print immediately after completion of a practice set or successful completion of a calibration set, but it may take up to 24 hours for emailed certificates to arrive in a user’s inbox. Certificates will be saved in the TELPAS Online Training Center year to year, but as a best practice, it is still recommended that educators print and keep a copy of their certificates. Training certificates and session rosters from monitored calibration sessions will serve as documentation for demonstrating raters’ completion of holistic rating training requirements.

On this page