Presented at the American Educational Research Association 2009 Annual Meeting, Classroom Observation Special Interest Group, on Thursday, April 16, 2009.
Presenters
- Ning Rui and Jill Feldman, Research for Better Schools
Summary
In this study, the authors closely examined the item and domain-level inter-rater reliability of the Memphis Striving Reader Project Classroom Observation Protocol (MSRP-COP). The study relies on data collected by ten pairs of raters who observed classroom implementation related to the use of MCLA strategies in Spring 2008. A combination of reliability measures (e.g., joint probability of agreement, Cohen's kappa, polychoric correlation, and ICC) were selected dependent upon which were appropriate given the scale of each item set. Results indicate that most items in physical environment, cognitive demand, and student class engagement can be assessed with moderate reliability. Items in classroom climate and instructional modes yielded mixed estimates. Recommendations are provided for possible improvement of similar instruments.
Download