Inter-rater agreement is an important aspect of any evaluation system. It ensures that evaluators agree that a particular teacher’s instruction on a given day meets the high expectations and rigor described in the state standards. As a result, these consistent and dependable ratings lead to fairness and credibility in the evaluation system.
CDE has developed tools and resources to support school districts and BOCES in the implementation of inter-rater agreement. The graphic below outlines key components to consider and the IRA Guidance document provides and in depth discussion of each.
Another tool is Elevate Colorado, an online inter-rater agreement training system, developed in partnership with My Learning Plan. This online system helps evaluators develop a deeper understanding of the professional practices embodied the Colorado State Model Evaluation System's teacher rubric. Using the system, evaluators can view short videos of practicing teachers, determine evidence of high quality teaching practices in the videos according to the Colorado State Model Evaluation System rubric and then receive feedback showing how closely their evidence matched evidence identified by master scorers.
An additional resource to support fair and consistent evaluations is The Resource Guide for Deepening the Understanding of Teacher's Professional Practices. The resource guide is meant to support Colorado school district employees, administrators and teachers in developing an understanding of the Colorado State Model Evaluation System Teacher Rubric. As individuals apply the rubric to their own teaching and to that of others, the resource guide can support observers and coaches in accurately identifying evidence for the professional practices. The guide also supports classroom teachers to accurately reflect on their teaching and plan for implementation of specific practices in their instruction. The research, examples provided and the glossary can be used to support the development of a common language for Colorado school district employees to analyze, reflect and plan instruction.