Linking Teacher Education to Redesigned Systems of Accountability : A Call for Multiple Measures in Pre-Service Teacher Effectiveness

In this written commentary for the special issue of Education Policy Analysis Archives focused on “Redesigning Assessment and Accountability,” we call for teacher preparation to embrace a multiple measures philosophy by providing teacher candidates Education Policy Analysis Archives Vol. 26 No. 12 SPECIAL ISSUE 2 with rich opportunities to engage with data from a variety of sources, beyond teacher test scores and principal evaluations. We apply and extend Bae’s (2018) argument to teacher preparation policies, urging teacher educators to develop programs that promote continuous improvement. We argue that teacher education can and should prepare candidates to engage in multiple measure systems, critically evaluate data and sense make to construct meaning, reflect on and improve their practice to meet the needs of all students, and ultimately advocate for next-generation accountability systems that authentically foreground and prioritize continuous improvement.


Introduction
The recent history of public schools has been one of increasing homogenization and test-based accountability.Typically, the departure point is linked to the No Child Left Behind Act (NCLB).NCLB was the eighth reauthorization and largest expansion of ESEA, and it required annual assessments for students in grades 3-8 and once during high school.Schools and districts from every state were now accountable to federal student performance mandates that aligned assessments to state content standards.States were required to report the level of student proficiency using Adequate Yearly Progress (AYP) for all relevant student subgroups, including groups defined by race and ethnicity, poverty, disability, and English language status (Haertel & Herman, 2005).Under NCLB, states monitored school and district performance and penalized schools failing to meet AYP for three or more consecutive years with an increasing series of sanctions (Fritzberg, 2004;Haertel & Herman, 2005).
Not surprisingly, schools responded with a sharpened focus on student test performance in English and mathematics.Schools aligned curriculum and formative assessments to NCLB metrics, resulting in tens of millions of student assessments administered annually.The accompanying assessment data windfall logically extended reform efforts to the link between teacher and student performance via value added models (VAM), which Race to the Top grants later incentivized.Researchers and organizations such as the American Educational Research Association (AERA) and the American Psychological Association (APA) called into question the technical merit and appropriateness of these models, particularly for high-stakes use (Goldhaber, Goldschmidt, & Tseng, 2013).Nonetheless, as California State Representative George Miller stated, "[t]he mission became about the test" (Baron, 2014, para. 3).
The backlash to these events is well documented in political, research, and mass media circles (Ravitch, 2010).Participation in the Common Core and aligned assessments has declined and Trump has repeatedly questioned their future (Trump, 2017).Parents have increasingly opted students out of testing programs (Bennett, 2016;Pizmnoy-Levy & Saraisky, 2016).The singular focus on tests has led many to advocate for change; recently, in this special issue, Bae (2018) called for a multiple measures approach to K-12 accountability that promotes "continuous support and improvement rather than mere compliance and efforts to avoid punishment" (p.2).Doing so has the potential to improve the validity of inferences about students and teachers (Baker & Linn, 2002;Betebenner & Linn, 2009).
In this commentary, we suggest teacher preparation embrace the same multiple measures philosophy articulated by Bae (2018) by providing pre-service teachers rich opportunities to engage with data from a variety of sources, beyond state assessments and principal evaluations.For inservice teachers to meet the demands of evolving accountability and evaluation systems, teacher education must prepare candidates to reflect meaningfully on their craft, solve problems and think critically, and collaborate with coaches, teacher teams, and other stakeholders.

Embedding Multiple Measures in Teacher Education
Research on the use of multiple measures within teacher evaluation suggest that teachers receive generally similar signals, but "may well receive different indicators of their effectiveness from each" (Strunk, Weinstein, & Makkonen, 2014, p. 21).This suggests that teachers need explicit training on how to reconcile inconsistent or even contradictory data.Bae (2018) calls for professional development on data literacy to help teachers understand data and make well-informed professional decisions about their practice.We agree and extend Bae's (2018) argument to teacher preparation, arguing that teacher education programs must be at the forefront of this work.
We advocate that teacher educators promote continuous improvement using multiple measures, in ways that meaningfully prepare candidates for the K-12 reform climate and also align with institutional values and best practices.We ground our argument in recommendations around high-quality professional learning systems, and believe teacher education is particularly well poised to produce teachers with a disposition toward lifelong learning within and beyond their programs.In fact, we believe the work of teacher education embodies three of the seven features of effective professional development as identified by Darling-Hammond, Hyler, and Gardner (2017): It can support collaboration, provide coaching and expert feedback, and provide opportunities for reflection.Teacher education should embed these experiences early in a teacher's career-in the context of multiple measures of teacher performance-to build a workforce that can tackle 21 st century challenges by analyzing a host of data beyond student test scores.
Teacher education can embrace this multiple measures philosophy by providing teachers with applied experiences analyzing data from a variety of sources.In particular, we believe teacher education has the unique opportunity to prepare pre-service educators to engage with teacher effectiveness measures already in use in K-12 districts.By exposing candidates to real-world data early, we can create a new generation of teacher candidates armed for the reform climate within K-12 education and prepared to become "critical consumers" of data-or educators who can examine data critically and use it to reflect meaningfully on their practice.
Our argument for multiple measures within teacher education is motivated by research from a pilot program within three major public universities exploring the use of multiple measures, including K-12 student feedback, during student teaching experiences (Farley, Clayton, & Kaka, 2018).In this pilot, mentor teams and pre-service teachers were able to leverage cutting-edge technology, such as video observations shared over the web, in conjunction with the mix of state and local indicators prescribed by Bae (2018).This helped triangulate key areas for professional growth that might otherwise have been overlooked, such as cultural proficiency or classroom climate.While understudied in teacher education, the use of data dashboards to present multiple facets of a teacher's emerging practice represents an opportunity to more holistically guide and evaluate professional growth.
Results from this pilot were promising: Pre-service teachers who piloted a multiple measures approach were able to co-construct knowledge about their practice instead of passively receiving feedback from coaches.In this way, candidates developed critical sense-making skills that enabled them to navigate sometimes competing messages.While we do not yet know how those skills will translate to the holistic evaluation of student performance and school accountability, it is an encouraging proposition.
Our results also reinforce Bae's (2018) proposition that next-generation accountability systems embrace a mixture of state-and locally developed indicators.Pre-service teachers in our pilot consistently rated local measures more favorably, reporting they were more informative and more likely to shape practice.However, the inclusion of the state teachers' rubric was also foundationally important because it is the rubric on which teachers are evaluated upon program completion.By preparing candidates to engage with data already in use within K-12 systems, new teachers may be better prepared to meet the demands of evolving accountability and evaluation policies and more likely to possess a holistic view of teacher learning and development, beyond narrow conceptions of teacher and student performance.This will prepare teachers who can leverage student performance and instructional data to create high-quality learning opportunities for all students -those Bae (2018) defines as supporting "deep content learning, critical thinking and problem-solving, communication, and collaboration abilities" (p.3).It may also prepare teachers who are able to adeptly navigate uncertain policy climates and advocate for sensible and meaningful reform.
Even if ESSA does result in diversified measures that support "continuous improvement rather than …compliance through external mandates" (Bae, 2018, p. 19), candidates must be prepared to enter a system that largely perceives of teachers and schools as the primary agents of change.Teacher education can serve an important role by preparing and training candidates to engage in multiple measure systems, critically evaluate data and sense make to construct meaning, reflect on and improve their practice to meet the needs of all learners, and ultimately champion nextgeneration accountability systems that authentically foreground and prioritize continuous improvement.

Conclusion
The focus on teacher education advocated here is timely and necessary for three reasons: First, educator evaluation reform has been rampant (Rowan & Raudenbush, 2016), mirroring the proliferation of school-based accountability reform described by Bae (2018).While ESSA may not focus explicitly on evaluation reform, most states have adopted legislation that make it difficult to remove or alter.Second, teacher educators have a responsibility to prepare teachers to engage with revised accountability and evaluation systems upon program completion.Finally, ongoing professional learning for in-service teachers is threatened under the Trump/Devos administration: the 2018 education budget will likely eliminate or significantly reduce Title II funding (Ujifusa, 2017), the primary federal mechanism for teacher professional development.Without this investment, teacher educators must be doubly prepared to provide candidates with rich, jobembedded experiences that can support lifelong learning.
Ultimately, a philosophical shift from compliance toward continuous improvement in K-12 policy necessitates a renewed focus on the pre-service teachers who will one day work in those settings.We advocate for a next phase of evaluation within teacher education-one that supports a learning orientation and prepares teachers to integrate information from multiple measures to reflect meaningfully on their practice and better meet the needs of all students.This system can ensure candidates are prepared to enter the rapidly changing K-12 reform climate and work within 21 st century accountability systems.