Inter-rater agreement is an important aspect of any evaluation system. It ensures that evaluators agree that a particular teacher’s instruction on a given day meets the high expectations and rigor described in the state standards. As a result, these consistent and dependable ratings lead to fairness and credibility in the evaluation system.
CDE has developed tools and resources to support school districts and BOCES in the implementation of inter-rater agreement. The graphic below outlines key components to consider and the IRA Guidance document provides and in depth discussion of each.
Another tool is Elevate Colorado, an online inter-rater agreement training system, developed in partnership with My Learning Plan. This online system helps evaluators develop a deeper understanding of the professional practices embodied the Colorado State Model Evaluation System's teacher rubric. Using the system, evaluators can view short videos of practicing teachers, rate the videos according to the Colorado State Model Evaluation System rubric and then receive feedback showing how closely they rated the videos relative to ratings from master scorers.
An additional resource to support fair and consistent evaluations is The Resource Guide for Deepening the Understanding of Teacher's Professional Practices. The resource guide is meant to support Colorado school district employees, administrators and teachers in developing an understanding of the Colorado State Model Evaluation System Teacher Rubric. As individuals apply the rubric to their own teaching and to that of others, the resource guide can support observers and coaches in accurately identifying evidence for the professional practices. The guide also supports classroom teachers to accurately reflect on their teaching and plan for implementation of specific practices in their instruction. The research, examples provided and the glossary can be used to support the development of a common language for Colorado school district employees to analyze, reflect and plan instruction.
Please contact Dawn Paré at firstname.lastname@example.org with your feedback, questions or additional resources.
Join an Inter-rater Agreement Professional Learning Community
The CDE Educator Effectiveness team has created a professional learning community for districts interested in working together as a community of educators to increase understanding of the teacher quality standards. The Inter-rater Agreement Professional Learning Community (IRA PLC) for 2015-16 will be a true professional learning environment where members support each other based on individual needs. Discussion, dialogue and feedback is at the heart of the work whether it’s face-to-face or via webinar. Educators can receive CDE professional development credits for this work. Individuals or teams can participate.
There are multiple ways to engage in this work depending on your readiness and interest. One way is through the monthly newsletter which provides information on new videos and other engagement opportunities. Another way is through the IRA PLC discussion board which will focus on shared resources and shared learning in a members only environment.
New videos, training opportunities and introduction to the discussion board *To receive the newsletters directly, please contact Dawn Pare' October 2015 Resource: Improving Observer Training: The Trends and Challenges *If you would like to join the discussion board, please contact Tricia Majors
IRA PLC Monthly Newsletter
IRA PLC Discussion Board
Carnegie Foundation for the Advancement of Teaching
New videos, training opportunities and introduction to the discussion board
*To receive the newsletters directly, please contact Dawn Pare'
Resource: Improving Observer Training: The Trends and Challenges
*If you would like to join the discussion board, please contact Tricia Majors