TELPAS Rater Training Activities
- Bruce Malone
- Julie Cole
- Shaina Seltzer
TELPAS raters are required to be trained on test security, test administration procedures, and holistic ratings. Training should be provided annually to ensure a standardized test administration and the best testing experience for Texas students. Annual training is especially important for understanding and implementing new or updated policies and procedures. TELPAS raters must successfully complete monitored calibration activities.
❏ Understand Rater Responsibilities
Campus coordinators must ensure that raters, proctors, and verifiers understand their responsibilities, which include, but are not limited to, the following:
using the PLDs to holistically rate students
participating in test administration training
participating in online holistic rating training
completing monitored calibration activities
implementing the test administration processes and procedures stated in the applicable manuals
maintaining the security of test materials until they are returned to the campus coordinator
reporting any suspected violation of test security to the campus coordinator
❏ Review the Applicable Test Administration Materials
Campus coordinators should direct raters, proctors, and verifiers to review and become familiar with the following resources:
rater manual
test administration instructions, as applicable
training modules in LMS
❏ Attend Test Administration Training
Campus coordinators schedule and conduct test administration training sessions.
All raters, proctors, and verifiers are required to receive training on test security and administration procedures, as well as training in procedures unique to specific testing programs and circumstances. Training sessions should be completed by the recommended dates on the Calendar of Events. If necessary, additional individuals may be trained after these dates.
The applicable test administration materials should be available during the training session.
Recommended training modules for raters, proctors, and verifiers in LMS include:
Test Security for the Texas Assessment Program
Managing Secure Materials for the Texas Assessment Program
Required topics for rater, proctor, and verifier training include:
Test security
test security procedures and test security oaths
reporting testing incidents
handling testing incidents
consequences for adult cheating, specifically collaborating or discussing calibration set responses or falsifying holistic ratings
Establishing timelines for
completing online holistic rating training
scheduling and completing holistic ratings
completing holistic rating rosters
verifying grades 2–12 writing collections for holistic administrations, as applicable
uploading holistic ratings in TIDE, as applicable
Preparing for test administrations
identifying eligible EB students
understanding the fundamentals of the holistic rating process
using the PLDs for rating
online holistic rating training components, including grade cluster assignments
location and date of monitored calibration sessions
process for calibrating
support for raters who do not successfully calibrate
understanding procedures for special holistic administration of writing, as applicable
During test administrations
proper testing procedures
completing ratings
completing and storing secure test materials such as rating rosters and writing collections
availability of campus coordinator and other testing supervisors
Entering and verifying student data in TIDE, as applicable
score codes
program information
rater information
holistic rating upload files
Returning materials to the campus coordinator
❏ Participate in Online Holistic Rating Training
Campus coordinators are responsible for ensuring that all individuals designated to serve as TELPAS raters complete their holistic rating training requirements. The online holistic rating training prepares teachers to rate English language proficiency consistently with the holistic rating rubrics—the PLDs from the ELPS.
The online holistic rating trainings are available in LMS until the end of the TELPAS assessment window. However, campus coordinators should require raters to complete their training requirements before the first day of the TELPAS testing window.
The trainings are specific to grade clusters, and raters must be assigned the holistic rating training in the grade cluster that corresponds to the grade levels of the students they will rate.
Raters who are assigned multiple clusters within grades 2–12 for special holistic administrations are required to complete the trainings and separate practice activities in the cluster in which they have the most EB students. Raters should also review the separate practice activities in the other clusters to ensure their readiness to apply the PLD rubrics appropriately.
Raters who are assigned multiple clusters including kindergarten and grade 1 are required to complete the trainings and separate practice activities for kindergarten and grade 1 and for at least one other cluster within grades 2–12. Raters with more than one additional cluster within grades 2–12 should complete the training and practice activities for the cluster in which they have the most EB students. Raters should also review the separate practice activities in the other clusters to ensure their readiness to apply the PLD rubrics appropriately.
New raters are required to complete the online holistic rating trainings and separate practice activities for the grade cluster they are assigned. These online trainings may be completed independently or in a group. New raters must successfully complete the assigned online trainings and separate practice activities before they access rater calibration activities.
Returning raters should take the online holistic rating trainings each year that correspond to the grade levels of the students they will rate.
Optional trainings and practice are available in LMS for educators who are required to assemble and verify special holistic administrations of TELPAS grades 2–12 writing.
After completing a calibration set, a rater's summary page will indicate a Pass or Fail and how many responses were correct out of the total number. Only raters who have passed the calibration will receive a certificate of completion in LMS.
Raters will be awarded professional development hours upon completion of the online trainings and practice activities.
Five hours are awarded for the kindergarten and grade 1 training.
Three hours are awarded for the grades 2–12 training.
One hour is awarded for the optional assembling and verifying trainings.
Kindergarten–Grade 12 Holistic Rating Training Flowchart
❏ Attend Monitored Calibration Sessions
Campus coordinators schedule monitored calibration sessions.
District personnel must establish local procedures for verifying the completion of the online holistic rating trainings and practice activities before new raters can access calibration activities. New raters are required to successfully complete rater calibration activities.
Returning raters are not required to repeat calibration activities if they have successfully completed the activities, but they are strongly encouraged to do so.
Calibration activities must be completed in a monitored setting. District coordinators receive passcodes for each calibration set through the Secure File Center.
The number of calibration sessions needed depends on the number of raters who must be trained to rate the EB students on each campus.
Most raters will need 1–2 hours to complete a calibration set. Calibration sessions are not required to be completed in one session; however, a campus will need to determine a process to ensure that security and confidentiality measures are still in place if calibration is allowed to span more than one session.
Local procedures must be established for completion of a second calibration set. Calibration sets 1 and 2 may be completed in one session or in separate sessions. Once a rater completes a set, he or she cannot revisit that set on another day.
The use of headphones is required for raters to complete the calibration activities for the listening and speaking domains.
Campus coordinators and proctors conduct monitored calibration sessions.
Monitored calibration sessions must be completed before raters holistically rate their assigned students for TELPAS. The calibration activities are available until the end of the TELPAS testing window. However, campus coordinators should require raters to complete their training requirements before the first day of the TELPAS testing window.
A trained proctor must be available to actively monitor the calibration session.
A roster of participants for each calibration session must be maintained. A sample calibration session roster is available in TIDE and accessible to the district coordinator.
Proctors must have the correct calibration passcodes. The passcodes are used to control access to the sets.
Raters must be given the correct passcode once they have logged in for their calibration session. Campus coordinators or proctors should ensure that raters understand which grade cluster they should calibrate for.
Raters are not allowed to revisit calibration sets after the day they are completed.
Raters may use a copy of the PLDs during their calibration activities. Raters may also be provided with scratch paper, as requested. Copies of the PLDs and any scratch paper used during the calibration session must be destroyed.
Calibration activities must be completed individually. Collaborating or discussing responses to calibration activities is not allowed.
Raters must begin with calibration set 1. If raters are successful on set 1, they should provide a copy of their certificate to the proctor.
If raters are unsuccessful on set 1, they should be notified as to how to proceed to set 2.
Educators are not authorized by TEA to serve as TELPAS raters unless they have successfully completed calibration activities at least once. If a rater does not successfully calibrate by the end of set 2, the rater may still be authorized to serve at the discretion of the district. District testing personnel are required to provide rating support for these teachers so that their assigned students are assessed consistently with the PLDs.
A certificate of successful completion is available in LMS when a rater demonstrates sufficient calibration (70 or better) on a calibration set.
Certificates will be available to print immediately after successful completion of a calibration set.
Raters will be awarded professional development hours upon successful completion of the calibration activities. Two hours are awarded for successful calibration.
Calibration session rosters and calibration certificates must be maintained at the local level.
❏ Review and Sign a Test Security Oath
Raters, proctors, verifiers, and other campus personnel participating in the Texas Assessment Program must complete an Oath of Test Security and Confidentiality after training and before handling secure test materials and content.
Raters, proctors, verifiers, and other campus personnel must initial each item on the test security oath, as applicable.
A copy of the completed test security oath (electronic or paper) must be submitted to the campus coordinator.