Inter-rater agreement is an important aspect of any evaluation system. It ensures that evaluators agree that a particular teacher’s instruction on a given day meets the high expectations and rigor described in the state standards. As a result, these consistent and dependable ratings lead to fairness and credibility in the evaluation system.
CDE has developed tools and resources to support school districts and BOCES in the implementation of inter-rater agreement. The graphic below outlines key components to consider and the IRA Guidance document provides and in depth discussion of each.
Another tool is Elevate Colorado, an online inter-rater agreement training system, developed in partnership with My Learning Plan. This online system helps evaluators develop a deeper understanding of the professional practices embodied the Colorado State Model Evaluation System's teacher rubric. Using the system, evaluators can view short videos of practicing teachers, rate the videos according to the Colorado State Model Evaluation System rubric and then receive feedback showing how closely they rated the videos relative to ratings from master scorers.
An additional resource to support fair and consistent evaluations is The Resource Guide for Deepening the Understanding of Teacher's Professional Practices.
The resource guide is meant to support Colorado school district employees, administrators and teachers in developing an understanding of the Colorado State Model Evaluation System Teacher Rubric. As individuals apply the rubric to their own teaching and to that of others, the resource guide can support observers and coaches in accurately identifying evidence for the professional practices.
The guide also supports classroom teachers to accurately reflect on their teaching and plan for implementation of specific practices in their instruction. The research, examples provided and the glossary can be used to support the development of a common language for Colorado school district employees to analyze, reflect and plan instruction.
Please contact Dawn Paré at email@example.com with your feedback, questions or additional resources.
Join an Inter-rater Agreement Professional Learning Community
The CDE Educator Effectiveness team is creating a professional learning community for districts interested in working together as a community of educators to increase understanding of the teacher quality standards. The Inter-rater Agreement Professional Learning Community (IRA PLC) for 2015-16 will be a true professional learning environment where members support each other based on individual needs. Discussion, dialogue and feedback is at the heart of the work whether it’s face-to-face or via webinar. Educators can receive CDE professional development credits for this work. Individuals or teams can participate. Contact Dawn Paré at firstname.lastname@example.org or 303-866-5926 to get started.
Share your skills and expertise with educators across Colorado to support inter-rater agreement
The Educator Effectiveness Unit is partnering with Colorado educators to create a library of professional practice that will be made available in Elevate Colorado. The library will deepen Colorado educators’ understanding of the Teacher Quality Standards and will support coaches and evaluators in identifying quality teaching practices.
To make this a robust system with the goals and values of Colorado educators, CDE would like to film interested teachers’ lessons and reflections, and collect supporting artifacts that will then be aligned to the professional practices by experienced Master Scorers.
If you are interested in being filmed or learning more, please contact Dawn Paré at email@example.com.