An integrative approach to professional development to support college- and career- readiness standards

Though scholars agree that professional development (PD) is a key mechanism for implementing education policies that call for teacher change, and that PD generally needs to be content-focused, active, collaborative, coherent, and sustained, the application of this framework has yielded mixed results. In this qualitative study, we employed structured interviewing methods to explore how district leaders across five states are implementing collegeand careerreadiness (CCR) standards across the United States by creatively adapting and integrating the features of this PD framework in order to meet the demands of this mandated educational policy. We illustrate a revised model for how 70 district officials are conceptualizing these features of PD to support CCR standards-based learning.


Introduction
Policymakers in the United States have long held teachers accountable to the goals of the standards-based reform movement, which argues that rigorous academic standards, high quality curriculum and instruction aligned to these standards, and accountability systems that track district and school performance all interdependently boost student achievement (Cohen & Mehta, 2017). The most recent iteration of the standards-based reform movement has produced college-and career-readiness (CCR) standards in English language arts (ELA) and mathematics. CCR standards, currently present in every state plus the District of Columbia, provide guidelines for challenging K-12 learning that is intended to prepare students for colleges and 21st-century careers.
Yet CCR standards alone cannot accelerate student outcomes. Teachers must receive professional development (PD) that supports their adaptations to the rigorous teaching and learning expectations embedded in the standards: a wide range of disciplinary content expertise , deep student understanding of key concepts, and critical student engagement in the core subject areas (Floden et al., 2017). This PD is especially critical given that many of these teachers' preservice programs, K-12 schools, and/or college programs were not designed with these current CCR standards in mind, suggesting that teachers are now expected to implement rigorous pedagogy that they themselves might not have experienced.
The importance of PD is further bolstered in the Every Student Succeeds Act (ESSA), passed by the United States Congress in 2015. ESSA considers PD to be high quality when it is sustained, intensive, collaborative, inclusive of other teachers, job-embedded, data-driven, and grounded in evidence-based practices. ESSA also raises the bar for the general education of students with disabilities (SWDs) and English learners (ELs), as the legislation limits the percentage of SWDs eligible for the alternate assessment to 1% of the total student population, and it moves accountability provisions for ELs from Title III to Title I. These shifts indicate that all teachers are held increasingly accountable to the performance of SWDs and ELs on state assessments aligned to the CCR standards, suggesting a greater need for PD that addresses these demands. ESSA additionally emphasizes the need to professionally develop school principals as instructional leaders, a focus that had not previously been articulated in federal educational legislation (Young et al., 2017).
PD is often relied on as a critical lever for implementing educational policies that call for changes to professional practice, as well as for teachers' commitment to these changes (Carney et al., 2016;Lobman & Ryan, 2008;Smith & Rowley, 2005;Youngs, 2001). Many educational scholars have coalesced around five features of PD-content focus, active learning, collective participation, duration, and coherence (Desimone, 2009)-as the conditions that tend to foster the growth of teacher knowledge, skills, and beliefs about their practice. But the actual effects of these features on student learning have yielded mixed results (see Garet et al., 2011;Fischer et al., 2018;Yoon et al., 2007). In response, researchers have refined the features of PD in different ways-by adding in a focus on curriculum (e.g., Penuel et al. 2011); by including modeling, coaching, and feedback (e.g., Darling-Hammond et al., 2017); or by exploring the contexts in which certain features are more or less effective (e.g., Kennedy, 2016). This work has contributed to a more robust conception of teacher PD, though enactment of these f eatures still has not consistently produced positive teacher and student outcomes.
These findings speak to the overall challenge of translating theory into practice when implementing educational policies. It is therefore incumbent upon researchers to cont inuously adapt theoretical frameworks to reflect emerging insights about the complexities of implementing policies through PD mechanisms. We argue that the process for this theoretical adaptation involves the following phases: (1) engaging with educational leaders to understand how they are identifying and implementing effective features of PD in service of a specific policy reform; (2) examining if and how organizational knowledge, skills, and beliefs (i.e., organizational capacity) improved in the intended direction; and (3) exploring the impact on teacher change and student learning. Our study addresses the first phase, where we analyze how educational leaders perceive their implementation of teacher PD in order to enact standardsbased policies.
More specifically, we examine the perceptions of school district leaders (e.g., superintendents, assistant superintendents, directors of curriculum, directors of PD, directors of student support), as they shoulder the daunting responsibility of orchestrating syst em-wide strategies for implementing state policy mandates (Durand et al., 2016;Johnson et al., 2015). The new rigors of CCR standards-based policy expectations and pressures push district leaders to think creatively about the features of their PD and methods for shifting teacher practice. ESSA's focus on leadership development and the increasing levels of accountability in educating SWDs and ELs in CCR standards-aligned classrooms also require district leaders to implement PD as a mechanism for building organizational capacity throughout a district system rather than just focusing on the instructional practices of general education teachers.
Despite these policy demands on district leaders, most PD research examines interventions as initiatives or programs developed outside the central office (Crowley, 2017). Recent research documents the tendency for district PD providers to focus more on the logistics of PD than its substance (Morel & Coburn, 2018), concluding that district leaders are less likely to influence what information teachers receive about the standards. Yet these logistical decisions directly impact whether teachers have access to content-focused, collaborative, active, sustained, and coherent PD, and there is little documentation of how and why district leaders make these logistical decisions in light of the field's mixed evidence on the effectiveness of certain PD features. Additionally, scholarship on PD for general education teachers, special education teachers, teachers of English learners, and principals tends to be conducted in silos, leading to a lack of research on how district leaders provide PD across this interdependent system of education providers. There is therefore a need for literature that showcases (a) how district leaders are strategically leveraging their PD efforts for general education teachers, teachers of SWDs and ELs, and principals to support classroom -level implementation of the CCR standards; (b) how these district decisions reveal new and potentially influential interactions among the features of high-quality PD in the context of the CCR standards; and (c) how they respond to the contextual challenges that often permeate PD environments.
We use qualitative data from 2016 to 2017 to examine how district leaders report their PD systems and strategies for different teacher types and school leaders in order to meet the needs of the CCR standards policy, and to what extent these district decisions build on our understanding of the empirically supported features of effective PD identified in the past several decades. Our findings help inform the conceptual model we present in this paper, which illustrates how district leaders in our study are perceiving their st rategic integration of different PD features to cultivate a system that supports teacher learning in the context of the CCR standards. We hope that the PD model we present in this study aids practitioners and researchers in better understanding the ways that some educational leaders in the United States approach PD in support of a system-wide instructional shift anchored in federal and state CCR policies. Our intention is not to purport a singular conceptual model generalizable across district contexts. Instead, we demonstrate one model that emerged from our interviews of district administrators in 24 districts across five states, which we believe can be adapted to other local contexts.

Conceptual Framework
There is considerable evidence that for PD to be effective in improving teacher practice and student learning, at least five features need to be in place: (a) content focus: activities that are focused on subject matter content and how students learn that content; (b) active learning: opportunities for teachers to observe, receive feedback, analyze student work, or make presentations; (c) coherence: content, goals, and activities that are consistent with the school curriculum and goals, teacher knowledge and beliefs, the needs of students, and school, di strict, and state reforms and policies; (d) sustained duration: PD activities that are ongoing throughout the school year and include 20 hours or more of contact time; and (e) collective participation: groups of teachers from the same grade, subject, or school participate in PD activities together to build a learning community.
Evidence supporting these aspects of professional development comes from crosssectional studies (Garet et al., 2001), longitudinal studies (Desimone et al., 2002;Desimone, et al., 2013), and literature reviews of qualitative and quasi-experimental studies (Fischer et al., 2018;Kennedy, 2016;Kraft et al., 2016). Studies using this framework have shown some positive effects on teachers and students (e.g., Gersten et al., 2010;Penuel et al., 2011), but also documented limited or no effects (e.g., Garet et al., 2008;Garet et al., 2011), or effects that depend on how features interacted with other elements of the PD program (Kennedy, 2016).

The Complexity of Translating a PD Conceptual Framework to Practice
Recent work has demonstrated how the dynamics of everyday realities in schools necessitate more nuanced understanding of the characteristics of PD and how they can be leveraged to support teachers. Thus, refinements of this PD framework are offered below, where we illustrate how scholars observe these features interacting in real-world settings. Collective participation is effective when teachers engage in productive discourse and activities that allow for teacher reflection and resource sharing, which can result in higher levels of teacher efficacy and adaptive expertise (Carney et al., 2016;DeLuca et al., 2017;Durksen et al., 2017;Ronfeldt et al., 2015;Smith & Rowley, 2005;Von Esch & Kavanagh, 2018). The emphasis on productive discourse is important given that not all collaboration yields substantive conversations about teacher practice (Horn et al., 2017) or explicit connections to a school's instructional agenda (Stosich et al., 2018). Further, scholars have found that collaborative inquiries are most effective when they are grounded in opportunities to interact with the actual curriculum or assessments being used in classroom settings (Akiba & Wilkinson, 2016;Green et al. 2013;Hodges & Jong, 2014;Stosich, 2016), or when they include teachers from the same schools building their capacities to implement instructional changes together (Youngs, 2001).
Collective participation in the form of collaboration between general education teachers and "intervention" teachers (e.g., special teachers, EL teachers) and school leaders is also proving to be a fruitful endeavor, though this literature is more limited in scope. Griffin and colleagues (2018) suggested that collaborative inquiry between general and special education math teachers in online forums increased their beliefs about student learning and enhanced their understanding of mathematics content. Babinski and colleagues (2018) observed how the school teams of classroom and English as a Second Language teachers worked together to align content and instructional strategies for their ELs, leading to more impactful pedagogy for ELs compared to teachers in the control group. Penner-Williams et al. (2017) also pointed to PLCs as a key PD mechanism for learning strategies specific to ELs. Stosich et al. (2018) raised the potential benefits of principals and teachers engaging in joint-inquiry around the implementation of CCR math standards, and the challenges of doing so in environments that expect principals to demonstrate expertise rather than a learning stance.
The paucity of the literature focused on collaborative PD for intervention teachers may very well be due to the lack of PD opportunities that show these teachers how to scaffold their instruction in alignment with the content of the standards (Murphy & Haller, 2015;Von Esch & Kavanagh, 2018). There is also limited literature on the inclusion of principals in PD for teachers, despite the value of including principals to help clarify the goals of PD and to encourage teachers to implement the learnings from the PD (Binkorst et al., 2018;Buttram & Farley-Ripple, 2016;Floden et al., 2017). To address these gaps, we examine how districts in our study approach the inclusion of intervention teachers and principals in collective PD experiences.
Another common feature of PD, content focus, is often discussed in tandem with coherence, active learning, and collective participation. Content-focused PD alone does not necessarily affect student learning; instead, it proves to be effective when it coherently adapts to local needs and connects to teachers' daily activities (Allen & Penuel, 2015;Kennedy, 2016;Koellner, & Jacobs, 2015), when it invites teachers to actively unpack assessment systems aligned to the standards (Popp & Goldman, 2016) or make decisions around curriculum design (Penuel et al., 2011), and when it includes opportunities for collaborative knowledge sharing (Griffin et al., 2018). The observation that math teachers who exhibit less anxiety around the content tend to influence higher gains in student achievement (see Kukata et al., 2017) suggests the importance of content-focused PD in deepening teachers' confidence in their subject areas, as long as the PD contains elements of these other features. These studies do not distinguish, however, whether specific groups of teachers (e.g., general education teachers, intervention teachers) receive a higher volume of content PD compared to other groups in the same district, and if so, why these imbalances exist. In our study, we investigate why and how these content PD decisions are made in the context of diverse school-based stakeholders' needs.
Duration, another prominent feature of PD given the widely accepted notion that oneoff PD workshops are minimally effective, is insufficient on its own. However, when active learning is coherently integrated into this extended learning time, the PD is found to be more effective. Ongoing opportunities for teachers to shape their own professional learning goals, learn and practice classroom strategies, engage with instructional tools, interact with coaches who model and then provide feedback on pedagogical moves (Binkhorst et al., 2018;Kennedy, 2016;Kraft et al., 2016;Sailors & Price, 2015;Supovitz & Mayer, 2000) all help to maximize the utility of PD contact hours. Furthermore, in a study of PD spillover effects, Sun and colleagues (2013) found that teachers were more likely to effectively assist their colleagues when they participated in PD of longer duration with active learning strategies related to writing instruction.
Duration is a challenge when teachers have limited time built into their schedules for PD, which in turn gives teachers little time to process the coherence of newly introduced standards, aligned curriculum, or other instructional initiatives (Allen & Penuel, 2015). Coherence is also a challenge when district leaders do not play a large role in designing and/ or funding PD to ensure alignment with the overall instructional direction of the school system and with the individual goals of school-based staff (Durand et al., 2016;Johnson et al., 2015).
District leaders have the ability to shape the conditions for PD and determine the types of professional learning that teachers and principals experience (Akiba & Wilkinson, 2016;Desimone & Garet, 2015;Morel & Coburn, 2018), yet their decision-making is rarely included in PD literature (Whitworth & Chiu, 2015). We therefore pay special attention to providing insights about district leaders' strategies for addressing duration and coherence of their teachers' PD experiences.
Our brief review of the literature highlights the complexity of translating a conceptual framework into effective PD in practice, given the myriad groups of teachers that influence the uptake of standards-based reform, and the ways these features often overlap with each other. By grounding our study in this conceptual framework, we are able to draw insights about how schools are enacting, combining, and organizing features of PD in ways intended to facilitate professional learning efforts, which in turn leads us to offer a more complex, interactive conceptual model to illustrate how features of PD might interact on the ground to leverage teacher learning in today's standards-based environment.

Methods
Our analysis draws on data from case studies of districts in five states-California (CA), Texas (TX), Kentucky (KY), Ohio (OH), and Massachusetts (MA). We collected the data through structured interviews with district officials in each of these five states. In each state, we requested interviews with four to six district officials with knowledge of the district's approach to standards-based reform through the lenses of curriculum, PD, assessment and accountability, and specialized instruction for SWDs and ELs. We initiated the request with either the Superintendent or the Superintendent's designee (such as the Director of Teaching and Learning), who then connected us with other district officials to interview. We ultimately recruited 70 district officials in 24 districts across the five states. The interviews were conducted in the spring, summer, and fall of 2016 and 2017.
The case study districts were part of a larger study of the implementation of CCR standards; they were selected from a state-representative sample of districts that participated in the survey portion of the larger study. For the purpose of this study of the implementation of PD in service of standards-based reform, we chose districts for variation on urbanicity (we include a rural, suburban, and urban district in each state), choosing districts with a sufficient number of SWDs and ELs, given the focus of the work on understanding standards implementation and supports for all teachers and students, including special populations.
The structured interview protocol we developed covered questions related to the adoption and implementation of CCR standards, including questions about PD, curriculum, assessments, district context, and overall successes and challenges of standards-based reform. For each question, we included prompts related to PD supports for SWD and EL teachers, as well as for school leaders. After completion, the interviews were transcribed and then coded deductively using the five features of effective PD. In addition, other relevant codes related to key reform areas (e.g., SWDs, ELs, curriculum, leadership, assessment) were included both as they related to the study's focus on standards implementation and as they emerged in the data. In order to develop inter-rater reliability, we engaged in paired coding and a process of dialogic engagement, which Ravitch and Carl (2016) defined as an ongoing and collaborative process of dialogue among a research team. After each cycle of coding, the research team discussed emergent themes pertaining to how the five features were manifesting in current models of district PD, and whether these themes were linked to other potentially influential characteristics such as district urbanicity. This iterative coding and dialogic engagement process resulted in multiple re-readings of the data and, finally, a convergence on three major findings, which we present below.

Findings
Even though the movement to adopt CCR standards across the United States can be traced back to 2009, when the Common Core State Standards were first released to the public, district leaders in 2016 and 2017 were still reporting how they needed to help their principals and teachers understand fundamental content and instructional demands of their state's ELA and math standards. Table 1 shows more specifically when each of the states in this study adopted their CCR standards in ELA and math in order to situate district leade rs' contexts for interacting with state standards policies. As stated earlier, the role of district leaders is to help translate policy to practice by facilitating the conditions for teachers to learn how best to implement mandated state standards in ELA and math. District leaders agreed that the state's new CCR standards called for significant shifts in teacher practice, as the ELA standards "ask kids to read at high levels, higher than what we have previously asked kids to read at before" while the math standards ask them to "think like mathematicians" and engage in more authentic and conceptual problem -solving tasks than ever before (Interview 24, KY). Thus, in order to effectively implement their state's policy requiring the integration of these new ELA and math expectations into classroom instruction, district leaders are continuously relying on PD to help teachers understand and apply the instructional shifts.
Across all of the states in our study, district leaders describe how they incorporate the features of high-quality PD in integrative ways to create stronger learning experiences for teachers implementing these standards. Our interviews unveiled three major themes related to the ways that some district leaders are starting to reconceptualize PD to serve this purpose. First, district leaders are broadening the traditional notion of collective partic ipation by including intervention teachers in the same PD experience as general teachers, and by providing PD opportunities on the same topics for school leaders, which serves to improve coherence and build content knowledge across a policy system. This suggests a more comprehensive landscape of professional learning than has been reflected in previous literature. Second, we are seeing active learning being used as a mechanism for creating collaborative PD opportunities grounded in the state's content standards. Lastly, our data showed an intentional interaction between duration and coherence-what we call sustained coherence-which indicates that district administrators are attempting to make teacher PD learning consistently oriented towards the policy by relying on instructional coaches and district-facilitated professional learning communities (PLCs).

A New Kind of Collective Participation
In an effort to expand the traditional notion of collective PD, districts are designing opportunities not only for general education teachers to learn together but also for principals, special education teachers, and teachers of English language learners to partake in the same experiences. District administrators describe the inclusion of school leaders and interventio n teachers in both formal PD workshop settings and in school-based PLCs. This finding demonstrates a promising, inclusionary trend by indicating that standards-based policy implementation is not the responsibility of only one stakeholder group nor the privilege of only teachers of certain children.
By creating opportunities for this inclusionary learning, administrators are reporting how they are encouraging all teachers to take more ownership over the performance of SWDs and ELs so that they too can access high quality instruction that comes with implementing CCR standards. In at least 17 districts, administrators promote this message by requiring everyone to receive joint PD on instructional methods specific to these special populations, such as Sheltered Instruction Observation Protocol (SIOP) or co-teaching to support SWDs. By including general teachers in district-wide PD sessions for SWDs and ELs, district leaders described "trying to get all of our teachers better informed" (Interview 13, TX) and encou raging the integration of these strategies into the general education classroom. One example of this collaboration is found in a district in California, where a cohort of special education and general education teachers gather in PD to "study and create Common Core aligned lessons together" (Interview 9, CA) to collectively deepen their understanding of differentiation in a CCR standards-based classroom.
Similarly, of the 14 districts that describe PLCs, eight spoke about the intentional inclusion of intervention teachers in conversations about student mastery of the standards. These collaborative interactions among teachers with multiple areas of expertise allow for more robust conversations around scaffolding instructional support. It also shifts the disco urse towards the CCR standards as applicable to all students. As one Kentucky district official said: I think in the past there was an exclusionary mindset of separatism from Special Ed. and General Ed. and I think with the professional learning communities that they're coming together, looking at student work, student data, and using one another as springboards, and sharing conversations around how to support gaps in knowledge. (Interview 28,KY) This administrator speaks to the value of bringing together general education and special education voices in a single PLC in order to promote inclusion rather than exclusion. An ESL coordinator in another district also testified to the need to subvert exclusionary practices towards teachers of ELs, as they are often seen as the "stepchild" that does not usually get a seat at the PLC. Thus, he and his "content counterparts" work together with all teachers so the teachers "are hearing one voice and not two" (Interview 23, TX). Sentiments such as these point to the way in which these collaborative opportunities can engender coherence, another feature of effective PD. District administrators are increasingly speaking of PD as a vehicle through which to achieve greater "instructional coherence and organizational coherence" (Interview 28, KY) among teachers of general education students, SWDs, and ELs so that the policies of standards-based reform can seem to connect with each other, instead of operating in silos. In previous waves of standards-based reform, state standards were seen as policies for general education populations, separate from policies for SWDs that mandate individualized instruction, and separate from policies that require English language proficiency testing. The collective practices referenced in this study instead encourage more information sharing, stronger networks, and calibration of instructional expectations, helping to combat potential feelings of isolation for SWD or EL teachers who may be alone at their school sites.
Including school leaders in PD further expands this sense of coherence and also lends legitimacy to the message of shared ownership. PD is used to train school leaders in instructional strategies that their teachers are being asked to implement and to gain these leaders' commitment to integrating PD for general and special teachers. In five districts, school leaders are either required or encouraged to partake in new PD that their teachers receive. The value in this practice, according to one administrator, is that "we want the administrator to support the idea. Cause once you send a teacher [to PD] who's excited and ignited and you send them back and the principals says we're not doing that… it doesn't work. So I need [principals] to get excited too" by also attending the PD (Interview 12, CA). In two of these districts, building leaders are receiving PD on EL strategies so they can serve as EL champions within their own buildings, which entails advocating for their teachers to att end the district's EL PD or providing feedback to teachers to improve instructional practices for ELs. These practices speak not just to the importance of inclusive PD in creating coherent mindsets, but also to how new notions of collective participation in PD can have impacts on students' educational experiences.
Collective participation can also be a way to address content gaps that exist due to the traditional practice of excluding intervention teachers and principals from content PD. Administrators in three districts specifically direct their content-focused PD towards special education teachers, while four other districts make sure to target principals and assistant principals in their content-focused PD. Because special education teachers "get a little information about being an elementary school teacher and a little about middle school and a little about high school and a little bit about disabilities and a little bit about behavior" without getting "intensive training around how to be a reading interventionist" (Interview 19, TX), some districts are switching gears to invite more special teachers to content PD and focus less on compliance-based PD. One example is from an urban district in California. This district reported that "our Special Ed. teachers are not that fluent in math, you know, the mathematics itself, and so, one of our goals is to close that gap, and we have done that by partnering with our General Ed. experts" (Interview 9, CA).
A similar move is also being undertaken to include school leaders in content-focused PD. Four districts reported offering content-focused leadership PD to groups of principals and highlighted the importance in doing so because principals needed to offer instructional feedback that is aligned to the standards. A Texas administrator summarizes this sentiment: The instructional leader who conducts observations and provides feedback needs to be knowledgeable on what is being delivered in the classroom. Because if you're not involved and don't understand the standards all you're doing is reading pedagogy. Reading pedagogy is great, but that's not going to get you there if the pedagogy is not addressing the standard or what the standard is calling for. (Interview 11,TX) Here, the district official describes the need for principals to develop content expertise aligned to the teaching and learning expectations of the standards in order to provide meaningful feedback. These efforts to widen the reach of content-focused PD to also envelop principals, assistant principals, and special teachers suggest promising signs of building the whole system's capacity to implement the CCR standards.

Collaboration through Active Learning
A second major theme relates to pairing collective participation with active learning to target teachers' content knowledge development. Districts tended to describe three different forms of active learning with regards to building standards-based content knowledge: teachers developing curricular products aligned to the standards as a form of learning ( n = 5), coaches and other district personnel modeling standards-based instruction while teachers act as students (n = 6), and teachers, as well as principals, observing each other's lessons and debriefing the observations (n = 7). While there are some instances in which district officials described lecturestyle PD, the topics of these sessions were less related to the district's focus on the CCR standards and were more one-off opportunities to hear from researchers on growth mindset and other educational "fads." While the idea of incorporating active learning into PD experiences is not necessarily new, we are now seeing it being paired with collective participation for both teachers and leaders. With districts merging active learning and collective participation, these features are working as connected mechanisms for elevating PD experiences to better meet the goals of state standards policy. For example, in an urban district in Texas, principals visit each other's campuses on a monthly basis in a system called instructional rounds. They reflect on instructional strengths and areas of growth, and then they are able to compare "who made more progress" using the "quantitative and qualitative data from the rounds process" (Interview 9, TX). This competitive spirit, coupled with the learning that stems from peer observation experiences, motivates teachers to continuously improve their implementation of the standards, according to officials in this district. Another urban district in Kentucky brings together elementary school teachers, in what they call a "large group PLC," to collaboratively design lessons. The group observes one of their peers delivering the lesson they designed together, and then they participate in a debrief where they discuss what they noticed, "so it 's a collective experience in planning, delivery, and then, observation, and then, all that conversation that happens along with it has been very powerful" (Interview 27, KY). This speaks to the power of convening teachers across a district who are able to combine their respective "wisdom of practice" (Shulman & Wilson, 2004) to design, deliver, and debrief the same, standards-aligned lesson.
A rural district administrator in Massachusetts provided one example of teachers actively role-playing students in a collaborative setting. In a PD session at the beginning of the year, she asked teachers to predict which PD goals were immediate, mid-term, and long-term. After putting each goal on an individual sticky note, the facilitator asked each teacher to create three columns and sort the sticky notes into the immediate, mid-term, and long-term categories by passing the sticky notes around the table. They then had a discussion about the teachers' choices and whether they align with the district's intentions. She later explained to them that this sticky note strategy was one that she used with her students: "when they're reading something, you take the vocab words out before hand and you have them sort them and predict whether the word connects with the setting, the characters, or the conflict of the story, and then afterwards… they see did they have them in the correct categories and if not where should it have gone" (Interview 9, MA). In this example, the PD facilitator is not only asking teachers to reflect on the instructional shifts that they think should be long-term or short-term goals, she also is explicitly demonstrating for them active learning processes that they can apply to their classrooms. As this interview participant later said, her desire is to create "intriguing learning opportunities for the adults" so that they are fully engrossed in the PD and then internalize what they learn.
District officials note a variety of benefits to these active learning approaches. As one administrator in a suburban Ohio district shared, teachers learn the content standards best when they "have a product that they're working on, otherwise [the learning] doesn't happen" (Interview 26, OH). The process of developing useful, standards-based resources together is more engaging for teachers than just handing them the list of standards and asking them to analyze them, according to this administrator. For another administrator in a rural California district, her integration of practice in PD sessions gives teachers opportunities to "fail in a nonthreatening environment" so that teachers can "learn from it and do something different next time" (Interview 11, CA). This represents another district's commitment to active learning through role-playing, a highly impactful mechanism for adult learning (Kilgour et al., 2015).

Sustained Coherence
The third theme involves the reliance on duration to provide coherence. Both features are contingent on governance structures and union contracts, such as wheth er central offices have the authority to streamline PD to be consistent with system-wide reforms, or whether schools have the authority to determine PD that is most aligned to their unique needs. In addition, the frequency and length of PD often depends on union contract rules in states that have unions. Therefore, it holds that district administrators tend to speak about coherent PD mechanisms while also referring to the duration of these efforts, leading to the concept of sustained coherence. In fact, we find that districts that describe extended PD opportunities for their school staff also frequently comment on how these PD sessions connect to their curriculum and instructional goals, all of which help translate the policies of standards-based reform into practice (Chingos & Whitehurst, 2012) . The most common mechanisms districts are using are providing ongoing district coaching opportunities to schools ( n = 12) and facilitating school-based PLCs (n = 7), both of which provide regular checkpoints to ensure that teachers are meeting the district's instructional expectations.
District leaders describe implementing a range of creative solutions for extending the time teachers spend in PD in order to foster coherent implementation of the CCR standards. In one suburban Texas district, teachers convene before the start of each quarter, so that district administrators can provide PD on the upcoming curriculum units that teachers are expected to follow. This decision is an example of the marriage of coherence and duration: by previewing the units that teachers will be teaching on a quarterly basis, the district is prioritizing PD that develops teachers' ongoing, consistent understanding of the curriculum. Nine other districts require monthly check-ins with principals, teachers, or both, and this time is used to follow up on the district's instructional goals. In an urban district in California, central office administrators provide guidance to schools that outline what building leaders should cover during mandated PD days, which are once or twice a month throughout the school year. One urban district administrator in Ohio explains shifting from the disconnected, half-day PD days to a series of two-day PD sessions with greater district oversight. PD used to be "here's your book, you know, good luck, go home and figure it out" to an ongoing set of PD trainings that teachers are required to attend and implement in their classrooms (Interview 15, OH). These strategies not only increase the time teachers spend in PD, but also work to ensure that time is meaningful and beneficial.
Instructional coaches represent one common mechanism for ongoing professional learning in schools. Twelve districts mention some form of instructional coaching, six of which describe their model as a sustainable means of establishing district wide instructional coherence. These coaches serve two purposes. First, they meet regularly with the central office to understand the district's instructional direction, and then support teachers in aligning with this direction. Second, while they are dispatched by the district to promote the central office's instructional goals inside the schools, they are also expected to personalize their supports to meet individual teacher needs. An example of these dual purposes is found in a Kentucky district, where coaches gather every other week to read books about instructional shifts embedded in the Common Core State Standards and discuss how they might push this thinking into their respective school sites. Additionally, they differentially work with teachers (e.g., modeling a lesson, co-teaching) based on individual needs so that it's "never a 'gotcha' visit" (Interview 29, KY). These activities show how coaches help "support everyone consistently and uniformly… because we don't want some people to believe certain things only work for some kids" (Interview 17, OH), while also being flexible enough to provide ad-hoc PD when necessary. The coaches therefore serve as human resources who offer support that is both sustained and coherent, either because they align with district vision or because they meet individual needs.
One district in rural Ohio is taking the idea of sustained coherence even further and has been working with the same external PD coaches for over three years. In this district, the same two university professors have been working with teachers continually on ELA, math, special education, and Universal Design for Learning instruction for the past three years. They work with teachers every other month and "they'll stay the whole day" and teachers will invite them into their classrooms, and then district staff "meet with them later and then… they'll make suggestions" (Interview 14, OH). The prolonged nature of this relationship is conducive to the professors building trust with teachers and even being asked in their classrooms to observe their instruction and provide feedback. Furthermore, the consistent nature of their visits is a shift in PD strategy for the district, as this interviewee made clear: [Our PD] used to be like you know someone would come in and talk to us and blah blah blah, and then you'd leave and you'd forget it. And now with these guys, I mean continual, you don't forget about it because they don't let you forget about it. (Interview 14, OH) Thus, the second benefit of having regular check-ins with the PD providers is the built-in accountability to improving one's instructional practice. These sustained interactions undoubtedly contribute to a sense of stability within the seemingly unstable policy environment that is frequently associated with national standards-based reform movements.
Finally, district participation in school-based PLCs represents another example of how coherence and duration can also occur through collective participation and active learning, whose pairing we highlighted earlier. While PLCs are ordinarily positioned as school-based initiatives (DuFour et al., 2006), district leaders are increasingly involving themselves in training and supporting PLCs to follow district wide protocols for engaging with the CCR standards. At least seven districts in our study describe having central office staff, including instructional coaches, walk teacher teams through data-based inquiries of their students' strengths and needs, with one district sharing that they actually created a new job description for an administrator to support PLCs. District participation in school PLCs is particularly prevalent in Ohio, as most of the Ohio districts in our sample engage in state-mandated collaborative team meetings where they analyze student data in 90-day continuous improvement cycles. An urban Ohio administrator shared how this PLC structure helps teachers see how "everything comes together," and that the district's standards and curriculum "have a connection between them" (Interview 15, OH), a clear endorsement of the utility of PLCs as channels for building coherence.

Limitations
Our findings should be interpreted with several caveats in mind. One, we do not probe into PD activities that are interdisciplinary, such as cultural competency, social justice, ethical practice, and other important influences on the teaching profession (Sinnema et al., 2017). While this PD is necessary, it is outside the scope of our study. Two, the data for this study are based on the perspectives of district administrators, who may portray more positive than negative examples of PD. This analysis does not include principal and teacher voice to triangulate the themes we identify, nor do we analyze how these PD experiences influence teacher change or student learning. Since this is a study on district leaders, as policy implementers, designing and justifying strategic decisions around PD to support specific policy goals, we rely hea vily on the district perspectives collected through this research.
What our study does offer is the identification of PD trends based on district leaders' descriptions of their work; given the significant influence that district leaders have on the professional learning environments experienced by their principals and teachers, our data from 24 districts in five states offer a unique opportunity to gain insights into how districts across the country are attempting to design PD initiatives to respond to the policy demands of the CCR standards.

Discussion
In this paper, we refine our understanding of the features of effective PD according to the ways in which district leaders report the design of their PD models. Our first contribution to the literature is our analysis of how district leaders believe they are integrating PD features to enhance the goals of standards policies in schools. As we suggest throughout the analysis, these features do not manifest in separate and neat categories: instead, they overlap with one another to form an interdependent approach to PD. Second, we highlight how and why district leaders strategically integrate the PD features for their general education teachers, intervention teachers, and principals, whereas previous scholarship on PD has traditionally focused on one or two of these role groups at a time. This inclusionary approach generates specific themes around the affordances or challenges of PD methods that would not have otherwise surfaced if only one or two of the role groups had been the center of analysis. Ultimately, th is study sheds light on district leaders' understandings of PD and how they implement PD in service of broader policy goals. The focus on district perspectives addresses the limited exploration of the role of district leaders in creating a system of support surrounding PD (Whitworth & Chiu, 2015) despite the large influence they have over the successful or unsuccessful leveraging of PD in support of instructional reform (Akiba & Wilkinson, 2016;Durand et al., 2016;Morel & Coburn, 2018). Figure 1 illustrates the relationships between the three main findings and how they operate together to support the broader goals around instructional ownership over the learning of all students and eventually, organizational capacity to undertake CCR standards-based reform. In this model, the PD features are not merely "characteristics" of PD. Instead, collective participation is a structural precondition, active learning is a mechanism that generates sustained coherence and content/content standards understanding, and these interactions ideally work to produce the long-term outcomes of ownership and capacity.

Conceptual Model for PD
Foregrounding most of the approaches that our respondents describe is the collective participation of groups of teachers in joint PD ventures, paralleling a similar focus on teacher collaboration in the literature (e.g., DeLuca et al., 2017;Horn et al., 2017;Ronfeldt et al., 2015) and in national standards for professional learning. A promising trend found in 17 of the 24 districts in this study is the inclusion of SWD teachers, EL teachers, and principals in the PD opportunities typically reserved for general education teachers, and vice versa, in an attempt to hold all teachers responsible for the learning of all students. Such practices bode well, especially given the important role principals play in mobilizing resources and staff around PD priorities (Buttram & Farley-Ripple, 2016) and the value of sharing strategies and expertise among general teachers and special teachers in PLC contexts (Griffin et al., 2018;Li & Peters, 2016). This level of collective participation then yields two outcomes: coherence and content focus. The common exposure to the same instructional vision that principals, general education teachers, and teacher specialists receive through this PD engenders instructional coherence, while the extra effort to include principals and non-general education teachers in PD that builds their content expertise means that more stakeholders are equipped to champion the implementation of content standards for all student groups. However, collective participation for the sake of collaboration does not necessarily lead to meaningful professional learning (Kennedy, 2016;Stosich, 2016). Another promising trend is, therefore, the use of active learning mechanisms to make this collective participation more valuable for teachers and principals. District leaders tend to describe three types of active learning in these collaborative settings: teacher development of curricular and instructional tools, teachers acting as students while central office staff model CCR standards-based pedagogy, and teachers and principals engaging in peer observations and debriefs. The benefit of these approaches, according to district leaders, is that when teachers are more engaged in interactive learning environments, they encounter more "ah ha" moments when it comes to understanding their content, and the content of the standards. Furthermore, when teachers are embedded in the sense-making processes themselves, they are more likely to form coherent connections between the standards, their curricular materials, and the policy goals of their school and district leaders (Allen & Penuel, 2015).
This discussion of coherence is linked to the idea of duration, as coherence forms and builds when there is extended engagement in learning overtime (Chrispeels et al., Daly 2008;Johnson et al., 2014). We see the two features as symbiotic because district leaders' abilities to promote coherence and duration both tend to be contingent on the political culture of the district. In other words, governance structures (e.g., site-based decision making) and teacher contracts frequently dictate the extent to which district leaders can choose how, and how often, teachers spend their PD hours. "Sustained coherence" may therefore be more achievable in districts that have centralized processes for requiring teachers to engage in prolonged, common instructional growth initiatives. We do not have data that indicates which of our districts are more centralized, how many hours of PD teachers are required to have, and how they spend these hours, though we do have district narratives that showcase the intentionality with which they attempt to establish sustained coherence. Two commonly referenced mechanisms for forging this sustained coherence are district-trained instructional coaches, who promote both central office goals and individual teacher goals in schools throughout the academic year, and district facilitation of ongoing PLCs.
Though district leaders hope that the impact of these PD initiatives will be shared ownership over CCR-aligned instruction for all student subgroups and organizational capacity (i.e., knowledge, skills, and beliefs) to undertake these policy reforms, undergirding these efforts are the enduring challenges of (a) human capital turnover and capacity constraints, (b) scale in terms of both population density in urban areas and geographic spread in rural areas, and (c) developing shared understanding of the meaning of college-and career-readiness, especially for those adults who have not directly experienced 21st-century CCR expectations. The first challenge is partially due to administrative and teacher turnover in the district, especially in rural districts where teachers "use the small town as a lily pad to jump to something bigger... and so we're constantly in the mindset of starting over" (Interview 25, TX). For administrators, the turnover leads to a rotating door of PD plans, while for teachers, the cost of orienting, inducting, and/or mentoring new teachers is a financial burden that then takes away from other PD opportunities. The second challenge of scale leads to a "fragmented delivery system" (Interview 9, CA) because PD does not end up reaching everyone in equal doses. Finally, the movement to implement CCR standards for students necessitates that all adults in the system must also have an understanding of what rigorous college instruction looks like, and what competitive industries are looking for in their future employees. This leads to the third challenge of changing organizational culture to embrace CCR expectations when not everyone in the system experienced those expectations themselves.
District leaders attempt to mitigate these challenges by fostering teacher networks as a form of stable PD support in the midst of central office turbulence, exploring digital learning opportunities through online platforms that more people on a larger scale can access, and consistently modeling CCR shifts in PD environments. In one district, to address loss of expertise when administrative and teacher turnover occurs, officials created a network of EL teachers across the district so they can support and train each other. The presence of teacher networks indicates promising new avenues of research on the diffusion of instructional expertise from veteran teachers who do receive focused, content-specific PD to teachers new to the district who missed those opportunities (Sun et al., 2013). To address the second challenge, three districts utilize online learning platforms to provide everyone the same access to PD. Virtual learning may indeed be a viable option for professional learning and resource sharing in the absence of opportunities to meet face-to-face (McConnell et al., 2013;Tseng & Kuo, 2014). To address the third challenge, at least three districts engage in internal PD so they themselves can constantly embody, model, and message the expectations of the CCR standards to their schools. These consistent demonstrations of the district's instructional values, beliefs, and expectations through their modeling of the standards may also yield the shifts in organizational culture that this standards-based movement calls for (Schein, 2010).
The conceptual model for PD presented here demonstrates trends in how district leaders describe their strategic usage of the PD features to facilitate instructional improvement in service of policy reform. We are not claiming that this model should be implemented in all district-wide endeavors that depend on PD as a driving force; rather, we hope that this model is instructive for both researchers and practitioners. Practitioners can adapt the model's theory of change based on their local contexts. It is typical for a district reform's theory of change to be underspecified, or to be more based on wishful thinking than intentional planning (Bryk et al., 2015). It would benefit districts to use a conceptual model, such as the one that emerged from this study, as a starting point for their development of a PD strategy that is feasible given the organizational structure of the district system, and given the policy pressures that they face.
The model is also instructive to researchers who seek conceptual frameworks for studying the effectiveness of teacher-learning initiatives. The presence of the five PD features is necessary but not sufficient for ensuring the improvement of teacher practice and student learning; instead, researchers should look for how practitioners make sense of this constellation of features given their local contexts and use practitioner-developed models as the basis of evaluation studies. Though practitioners are learning every day about the PD approaches that work or do not work for them, as a field, we "fail to organize, refine, and build on these lessons" in a timely and accessible manner (Bryk et al., 2015, p. 11). Because our conceptual model is both grounded in the literature and derived from district administrators' direct accounts of their PD practices, we present an opportunity for scholars to learn from the frontline professionals and apply these learnings on a more systematic scale.
Overall, the goal of this study was to examine how theoretical frameworks, when applied to the messiness of day-to-day practice, warrant continual adaptation and reorganization in order to align with evolving expectations of teachers (e.g., instruction should be rigorously aligned to CCR standards) and structures (e.g., collaborative PLCs with SWD and EL teachers) within the field of education. From this approach, we contribute to the literature b y identifying three main themes that build our understanding of how teacher PD strategically integrates and leverages specific features to support instructional change.

Conclusion
As the past several decades have taught us, system-wide change will not occur without district and school leaders infusing a PD mission into their everyday roles and responsibilities (Elmore & Burney, 1997;Fink & Resnick, 2001;Spillane & Thompson, 1997;Togneri & Anderson, 2003). The integration of intensive capacity-building efforts into local infrastructures for change is necessary given the ambitious targets set by CCR policy (Cohen & Mehta, 2017). For complex organizations to adapt to changing educational contexts, system leaders need to move beyond tinkering with technical changes to teaching (Richmond et al., 2016) and instead foster a culture of deep learning that affects all members of the organization (Edmondson, 2008). Our work here provides insights from a set of districts across the country on how they are applying features of teacher PD to contribute to this culture of learning.
Given that this study is not an evaluation of PD effectiveness, we encourage other researchers to explore whether the trends we discuss here are not just changes to the sy stem but improvements that positively influence teacher and student outcomes (see Bryk et al., 2015 , for this dichotomy). Does the inclusionary approach to collective participation lead to a sense of shared ownership over the education of SWDs and ELLs, and is this shared ownership associated with student learning gains for these populations? Do active learning approaches improve teachers' content knowledge, pedagogical content knowledge, and alignment of their instruction with the content standards? Do districts with more centralized governance structures and teacher contracts that allot enough time and district control over PD exhibit more successful implementation of sustained coherence? What are the affordances and challenges of other theories of change that districts develop using different configurations of the features of PD? We hope that our work contributes to future studies that answer these important questions. education policy analysis archives Volume 28 Number 111 August 3, 2020ISSN 1068-2341 Readers are free to copy, display, distribute, and adapt this article, as long as the work is attributed to the author(s) and Education Policy Analysis Archives, the changes are identified, and the same license applies to the derivative work. More details of this Creative Commons license are available at https://creativecommons.