EPAA/AAPE is a peer-reviewed, open-access, international, multilingual, and multidisciplinary journal designed for researchers, practitioners, policy makers, and development analysts concerned with education policies. EPAA/AAPE accepts unpublished original manuscripts in English, Spanish and Portuguese without restriction as to conceptual and methodological perspectives, time or place. EPAA/AAPE publishes issues comprised of empirical articles, commentaries, and special issues at roughly weekly intervals, all of which pertain to educational policy, with direct implications for educational policy.
Sean Kelly is a Professor in the Department of Administrative and Policy Studies at the University of Pittsburgh. He studies the social organization of schools, student engagement, and teacher effectiveness.
Robert Bringe University of North Carolina-Chapel Hill United States
Robert Bringe is a graduate student in the department of Economics as the University of North Carolina Chapel Hill. A former high school teacher, he is interested in studying policies related to teacher pay, teacher accountability and heterogeneous teacher effects
Esteban Aucejo Arizona State University United States
Esteban Aucejo is an Associate Professor in the W.P. Carey School of Business at Arizona State University, and a research associate at the Centre of Economic Performance. He studies economics of education, labor economics and human capital development.
Jane Cooley Fruehwirth University of North Carolina-Chapel Hill United States
Jane Cooley Fruehwirth is an Associate Professor of Economics at the University of North Carolina-Chapel Hill and a fellow of UNC’s Carolina Population Center. She studies teacher effectiveness, social context, educational inequality and mental health.
Using global observation protocols to inform research on teaching effectiveness and school improvement: Strengths and emerging limitations
Sean Kelly, Robert Bringe, Esteban Aucejo, Jane Cooley Fruehwirth
Abstract
An essential feature of many modern teacher observation protocols is their “global” approach to measuring instruction. Global protocols provide a summary evaluation of multiple domains of instruction from observers’ overall review of classroom processes. Although these protocols have demonstrated strengths, including their comprehensiveness and advanced state of development, in this analysis we argue that global protocols also have inherent limitations affecting both research use and applied school improvement efforts. Analyzing the Measures of Effective Teaching study data, we interrogate a set of five potential limitations of global protocols. We conclude by discussing fine-grained measures of instruction, including tools that rely on automated methods of observation, as an alternative with the potential to overcome many of the fundamental limitations of global protocols.
Bell, C. A., Qi, Y., Croft, A. J., Leusner, D., McCaffrey, D. F., Gitomer, D. H., & Pianta, R. C. (2014). Improving observational score quality. In Kane, T., Kerr, K., Pianta, R. (Eds.), Designing Teacher Evaluation Systems: New Guidance from the Measures of Effective Teaching Project (pp. 50–97). San Francisco, CA: Jossey-Bass.
Campbell, S., L. & Ronfeldt, M. (2018). Observational evaluation of teachers: Measuring more than we bargained for? American Educational Research Journal, 55, 1233–1267.
Liu, S., Bell, C. A., Jones, N. D., McCaffrey, D. F. (2019). Classroom observation systems in context: A case for the validation of observation systems. Educational Assessment, Evaluation, and Accountability, 31, 61–95.
Reddy, L. A., & Dudek, C. M. (2014). Teacher progress monitoring of instructional and behavioral management practices: An evidence-based approach to improving classroom practices. International Journal of School & Educational Psychology, 2, 71–84.
White, M. C. (2018). Rater performance standards for classroom observation measures. Educational Researcher, 47, 492–501.
education policy analysis archives
