Contexts for Teacher Practice : ( Re ) Considering the Role of Context in Interventions in Early Childhood Teacher Engagement with New Approaches to Shared Book Reading

Although the randomized controlled trial has been regarded as the sine qua non in recent years, we argue that understanding contexts for teacher practice is a critical factor in the implementation of intervention, but has often been overlooked in educational research design. In this paper, we argue that randomized trials may not be ideal for educational contexts that are multifaceted, complex and often part of other community-based initiatives. To make this argument, we describe a study from “the context out,” a different lens than is typical in education research. In doing so, we consider how varied contexts create both constraints and affordances for teachers to engage in an intervention. Examining how teachers engage with new curricular practices, we focus Education Policy Analysis Archives Vol. 24 No. 89 2 closely on five early childhood centers in high poverty neighborhoods, using mixed methods to understand both the influence of the intervention as well as the conditions that either supported or thwarted changes in practice. Our findings suggest that context plays a major role in educational practice, which has important implications for approaches to research in the future. Given this reality, we argue that an understanding of the context in which an intervention takes place must be considered in any calculation of “what works” in classrooms.


Introduction
Since No Child Left Behind, evidence-based practice has emerged as the sine qua non to improve how the educational community approaches research, and how this research is translated into meaningful intervention within classrooms.Within this model, randomized controlled trials (RCTs) of specific interventions, from literacy to mathematics to behavior modification, have become the means for determining "what works" in education.As any visit to the What Works Clearinghouse highlights, the list of "what works" has become long over the last decades and far more comprehensive.
Nevertheless, it is questionable whether the implementation of "what works" has led to instructional improvements.In fact, one could argue that we are still no closer to closing the achievement gap than we were some 15 years ago (Reardon, Greenberg, Kalogrides, Shores, & Valentino, 2013).To some educational researchers, the answer to this conundrum is that interventions are not being implemented with fidelity.To others, however, the lack of substantial improvements is not entirely surprising.Practice-based researchers have argued that the dynamic, real-time, real-world life of unique classrooms present a challenge to randomized controlled trials and the implementation of any intervention with so-called fidelity (Erickson, 2014;Schorr, 1997).Rather, classrooms require teachers who are prepared to be nimble, to respond to contextual circumstances and immediate student needs, activities not typically associated with a controlled experimental design where such factors are viewed as extraneous, and subsequently partialed out.
Although the randomized controlled trial is a powerful research design, especially for interventions that are conceptually neat, they may not be ideal for educational contexts that are multi-faceted, complex and are part of other community-based initiatives.Knowing that something can work (e.g.RCT), may be fundamentally different from knowing how to make it work in diverse contexts.Both understanding the variation in contexts and educational outcomes, and responding effectively to those elements, is at the heart of the goal of improvement science (Bryk, 2016).Given this reality, it may be necessary to broaden our lens to determine what constitutes credible evidence to assess and understand whether, how and why an intervention may or may not achieve its desired results.In short, understanding the context in which an intervention takes place must be considered in any equation of "what works" in classrooms.
In this paper we describe an intervention study from a different lens than is typically reported.Specifically, we propose to look from the classroom out, rather than the intervention in, considering how context creates both constraints and affordances for teachers to engage in practice.We theorize context as the social, institutional, political and personal factors that influence teaching and learning.This includes the geographical setting (neighborhood, street, school, and classroom), the institutional norms and district policies which may frame the expectations for teaching and learning, as well as the dynamic interactions within classrooms which help us to understand "the connected whole" in which teachers engage in professional practice (Cole, 1996, p. 135).From this perspective, contexts for teacher practice are constituted by dynamic, process-oriented interactions and factors that create constraints and affordances for how teacher teach.Constraints and affordances are attributes, or elements, of each unique context that create and/or limit opportunities for action and activity (Hammond, 2009;Kennewell, 2001).
Within this framework, we approach an understanding of the work of teachers as they engage with new professional practices within their unique contexts.As Erickson (2014) encourages, this knowledge plays an important role in designing curriculum and professional development that recognizes the complexities of teaching that exist beyond single interactions between teacher and student.Rather than "intervention in" approach, where our interest would be in seeing whether or not teachers implement the interventions with fidelity, our "context out" approach places primary importance on understanding how specific contexts (e.g.physical, social, institutional, political) for teacher practice are comprised and influence the potential uses of evidence-based practices.It may also suggest additional factors to consider in designing and implementing interventions under varying contextual conditions.

Background and Policy Context
Our work began with a rather typical scenario in educational research.Responding to a request for proposals, the second author received a grant in 2013 targeted to improving early literacy development for preschoolers in the highest poverty neighborhoods in New York City (Neuman, 2013).The project was designed to implement the Books Aloud! program, a well-researched early literacy initiative between libraries and early childhood centers (Neuman, 1999) by creating a more integral connection between the school library in five child care centers (described below in greater detail).
We invited four child-care centers, and one Head Start to participate in the project (see Tables 1 and 2).Each of these centers satisfied the criteria stipulated by the grant: they resided in the most severely economically depressed area of New York City; they included a small, but generally under-utilized library; and they served children, ages 3 through 5 who were eligible for free and reduced lunch.Because we were providing resources including additions to the library, we were able to quickly recruit administrators to agree to our intervention and data collection efforts across the five centers.

The Centers
The participating centers came from different small-sized communities in the county and represented Community-based Organizations (CBO), organizations funded through the Administration of Children's Services.Typical of early childhood programs, centers also received a patchwork of funding from other sources, including "universal prekindergarten" school-district funding and federal dollars (Early Head Start; Head Start).
Starting in 2012, however, as part of a city-wide quality improvement system, then Mayor Bloomberg created EarlyLearnNYC, a $486-million dollar initiative designed to merge these funding streams and raise the quality of child care services.EarlyLearnNYC also attempted to improve the coverage of care across the city by shifting the supply of contracted slots to neighborhoods that needed them the most (Hurley, Kramer, Rosenbaum, & Miller, 2014).Nevertheless, in the course of trying to cut bureaucratic red tape and reduce barriers, these policy reforms created new ones.For example, in bringing about sweeping changes to the contracted childcare systems, many longstanding childcare programs lost contracts, while other received funding to service neighborhoods where they had little or no history (Hurley et al., 2014).Dozens of programs were eliminated, and thousands of publicly funded childcare seats went unused.
Our childcare sites existed in the midst of this policy shift.Three of our centers, Mitchell, Grove Hill, and Shadyside, were nestled within large high-rise public housing complexes and went from small to smaller during the project, teetering on the edge of financial collapse and no longer able to even adequately fill two classrooms of preschoolers.Mapledale, on the other hand, aided by a savvy administrator and grant-writer, was able to survive through outside funds.Only Guardian Head Start, located between two public housing towers, remained unscathed through the transition, although it was re-competing for its contract under Head Start reauthorization.
In addition to these policy changes each site had multiple and varied accountability expectations.For example, two of the centers were accredited by the National Association for the Education of Young Children (NAEYC), and subject to their standards, teacher-child ratios, and developmentally appropriate practices.Head Start, on the other hand, has its own Head Start Outcomes Framework, recently revised (2012) that included descriptions of three new domains for learning and guidelines for bring additional attention to school readiness.The funds devoted to the Universal PreK program were accompanied by new guidelines for meeting the Common Core State Standards for 4-year olds.
All of these changes occurred to a workforce that was minimally paid (average salary $35,000), and in many cases, overworked (8-hour days, with 2 weeks of vacation yearly).Several teachers had their masters' degrees in early childhood (23%); 46% had their bachelor's degrees, and 31% had high school diplomas and above (see Table 2).According to the teachers, none had received professional development related to any of these changes in guidelines and standards for EarlyLearnNYC, NAEYC accreditation, Universal Pre-K, or Head Start re-competition.These were the settings for our intervention.

The Critical Nature of Context
Although many researchers have begun to turn their attention to the role of understanding local contexts, it rarely has made its way into our research designs.Nevertheless, there is growing evidence that the contexts in which teachers teach including the policy context, the provisions made for conveying changes in policy through professional development, and support from leadership has a great deal to do with their ability to create high quality environments for their students (Dennis & O'Connor, 2012).
Too often, unfortunately, conceptions of context are limited to specific physical or geographic settings, such as a neighborhood or a specific school.However, the physical site and geographical location of a classroom are but two elements of the context in which teachers engage with their professional work.Instead of asking "what" is the context?, Erickson & Schultz (1977) argue it is better to ask, "when is a context?"This approach pushes towards a dynamic understanding of context that considers how the interaction between sites of action, people and processes create spaces that influence how teachers practice.From this perspective, context is constituted by the interactions within (and across) physical spaces, rather than by that space or setting alone (Rogoff, 1990).
As such, the contexts in which teachers practice must be taken into account when we consider new approaches to teaching.For early childhood settings, in particular, where policies such as financing, licensure, and eligibility of services are tightly bounded to the preschool experiences of young children, it seems especially imperative to understand how context interacts with the complex realities of changes in practice (Graue & Ryan, 2009).To better understand this "context out" approach, we examined our own intervention from within the classroom to ask:  How do contexts provide for and/or limit opportunities for teachers to engage with new approaches to pedagogy? In what ways do changes in policy and institutional contexts affect teacher practices? Given the influence of various context-related factors, what implications might this have for designing intervention research?

Method Research Design
Educational programs are implemented in real world contexts, with characteristics that are both unique to a particular context, as well as those that are shared across contexts (Klingner & Boardman, 2011).To provide for the most robust analysis, therefore, we addressed these important questions using a mixed methods design.Given the uniqueness of each center, we theorized that a single-case design approach might provide us with formative information for examining the potential changes in classroom practice.Single-case research typically has an applied focus, often used in novel approaches in unique settings (Neuman & McCormick, 1995).In this case, we used an A-B-A-B time-series withdrawal design, where A is baseline (or business-as-usual), and B is the intervention (Kratochwill et al, 2010).Single-case design can bypass an error often found in group comparison studies because each case serves as its own control.The underlying principle is that if an intervention is effective, it should be possible to visually see a change in the dependent variable from the period prior to intervention (i.e., baseline) to the period during and after the intervention.In our case, we quantified the number of utterance level exchanges as our dependent variable between participating children and teachers in each class during shared book reading.Prior to starting the intervention and during the intervention itself, we continued to measure the number of exchanges at regular time intervals.Repeated measurement taken at baseline and intervention reduces several threats to internal validity, such as maturation and instrumentation, and statistical regression because patterns illustrative of these problems would have been recognized at baseline when stabilization was established.
We began our study by walking the neighborhoods surrounding our centers to get a sense of the community and the local resources that might be available to children and their families.For example, we identified the proximity to the public library, and whether or not one could walk easily to access its literacy resources, the availability of cable or broadband access for the local community, of bookstores, newsstands, and so forth.Our goal was to gather various forms of qualitative information that would allow us to better understand how these factors might create affordances and constraints for literacy development.
We then looked within each center site to better understand how the library within the center was used (or not); the teachers' use of these and other literacy resources within their classroom environments, and their curriculum planning and development.We visited classrooms to examine resources and to observe classroom routines.Given that all the centers reported using the same curriculum (e.g.Creative Curriculum), we conducted informal classroom observations to get a sense of how, and in what ways the existing curriculum was used.
Finally, to examine the social validity of the intervention (Wolf, 1978), we conducted interviews with teachers.Together, these mixed methods provided us with multiple approaches and lenses to address our research questions.

The Intervention
The Books Aloud intervention was an adapted version of a highly successful early literacy intervention to promote shared book reading in preschools (Neuman, 1999).Books Aloud was designed to better integrate library services by: i) creating a lending library of high quality books; (ii) providing training to teachers, school leaders, and paraprofessionals on research-based early literacy activities through 10 text-sets using narrative nonfiction and information text; (iii) engaging parents in reading to their child through a build-a-home library program; and (iv) improving children's oral language skills and academic vocabulary (e.g.critical to understanding concepts and content in subject areas) as measured by curriculum-based assessments and standardized assessments in receptive and expressive language.
In particular, the text sets were targeted to topics identified in the school district's effort to align all curricular materials with the Common Core in preschool.Such topics included the human body, healthy foods, and other topics related to living things.Each text set included five books, carefully organized to introduce children to multiple genre: rhyming books, narrative nonfiction, and information books.Accompanying each text set were lesson plans, designed to scaffold and support teachers as they engaged the children in conversation during each read aloud.Lessons included the 'gradual release of control' starting with call and response interactions that then lead to more openended exchanges.Each lesson plan included educative materials for the teachers, such as a definition of keywords and concepts, language tips, and strategies for 'what to look for' in gauging the children's responses (Neuman, Pinkham, & Kaefer, 2015).
Teachers were encouraged to use the materials during whole group, shared-book reading during morning meeting time, two to three times weekly for approximately 15-minutes.The goal was to enhance the frequency and quality of teacher-child interactions in texts that represented multiple genre and increasing levels of textual complexity.

Participants
With the permission of the administrator at each site, our next step was to select a volunteer teacher to pilot our instructional materials.Our purpose was to observe the potential of our intervention for enhancing the number of quality language interactions in classrooms during shared book reading events.Furthermore, we wanted to engage each teacher as an active participant in the research, having him or her give us feedback on the materials' usefulness for their children's needs.These teachers received a 2-hour, one-on-one training session prior to enacting the program, and ongoing support throughout the project based on their requests for support.

Procedures
We devised an observational tool (known as PRINT!) to examine how our text set lessons and materials might change the quality of the interactions within classrooms.Based on our previous research, our presumption was that lesson plans with suggestions for engaging children in conversations around books would promote more talk than we had observed in classrooms.The measure was designed to capture the number of cognitively challenging, interaction-based utterances during the shared book reading time based on previous research (Bova, 2011;Pollard-Durodola et al., 2011).These types of utterances included information-seeking and open-ended questions, defining word meanings, connecting words and concepts and labeling.
Nevertheless, in our initial piloting of the measure, we observed only a limited number of interactions that could be easily identified as cognitively challenging.Consequently, we decided to focus on the frequency of conversational turns (e.g.teacher query-child response, as well as the other way around), assuming that increases in the number of turns might provide an initial indicator of an enhanced language environment in the classroom.
We then held a day-long training session with research assistants to develop consistency in our coding of interactions.Following training, research assistants were given five videos to code independently.Inter-rater reliability determined by Cronbach's alpha was .90.
After training observers and reaching reliability, we began the first phase of the A-B-A-B design.In this design type, a non-treatment phase is initiated until the behavior in question demonstrates stability.During this phase we established baseline by recording the number of teacher-child interactions during the business-as-usual storybook reading using the classroom discourse as our single unit (e.g.instead of the individual).Once this baseline (A) was established, we began our first text set.During this time, observers visited each classroom twice a week and recorded the number of interactions for the text set book reading (B) over a 2-week period.Following this period, teachers then returned to their traditional storybook reading activity, using books of their own choosing (A).In this manner, we withdrew the intervention with the assumption that the interactional behaviors would return to baseline.Therefore, our approach was to vary the experimental condition every two weeks, with teachers engaged in storybook reading in a businessas usual manner (A) for 2-weeks, followed by our Books Aloud text set approach (B) over the next 2weeks.Throughout the experimental period, research assistants conducted observations twice per week.We randomly videotaped sessions at each site, and had other qualified observers code them to determine reliability.Reliability was high, estimated over .95.
At the mid-point of the experiment, we conducted a fidelity checklist.Using an adapted form from our previous project (Neuman, Pinkham, & Kaefer, 2015), we examined the fidelity to the lesson plan (e.g.setting a purpose for reading; introducing vocabulary; engaging children in open-ended responses; discussion) using a six-item, four-point Likert-type scale ranging from 1 (low-fidelity) to 4 (high-fidelity).Teachers varied in their enactment of the intervention (as we will describe below), from a low of 33% adherence to the lesson to a high of 90%.
Towards the end of the year, we engaged in hour-long conversational interviews with teachers to examine the social validity of our intervention (Wolf, 1978).Specifically, we asked them to discuss the intervention, to describe some of the challenges that might have influenced their levels of engagement in the project, and to inform us about the kinds of policy shifts that might affect their day-to-day activities.We also conducted final interviews with each site director to better understand their insights on our project and their teachers' engagement with our intervention.

Analytic Strategy
Our context-out strategy began with the classroom.Using each classroom as a single-case, we graphed the total number of conversational turns at baseline then during the intervention, followed by a return to baseline once it was absent.These graphs provided an important lens to understand which teachers appeared to take up and engage with the intervention, as well as which did not.We then used visual inspection to examine the effectiveness of our intervention.With an A-B-A-B design, we would expect increases in teacher-child interactions during the intervention phases of the experiment (B) compared to the business-as-usual instruction (A).Next, to better understand how and why certain teachers were able to engage with our intervention, we created qualitative case summaries (Stake, 1995) of each classroom as a nested site of teacher practice within a childcare center, a broader neighborhood community and a larger policy and accountability framework.In these case summaries, we aimed to capture the dynamic factors of each site that constituted contexts for teachers' practice from the interactions between teachers, directors, sites, and policies.
Using Dedoose as an analytical tool, we joined our interviews, photographs of the neighborhoods, centers and sites, and video observations to the case summaries to create sets of data for each case.Coding within and between these sets, we worked to understand and identify both case specific and cross-case themes that influenced how our teachers engaged with the shared book reading curriculum.From this process, we wrote a case study memo for each site that reflected the context dynamics at play within each classroom, as well as larger themes at play across these sites (Saldana, 2009(Saldana, , 2012)).These memos became sites of further meaning-making, and allowed us to begin to see connections across the sites in terms of constraints and affordances for teacher engagement, the role of policy shifts on daily practice, as well as how varying dynamics within the contexts of their practice influenced the ways in which teachers used the intervention.

The Effectiveness of the Intervention
As shown in Figure 1, two of the classrooms showed some evidence of an increase in conversational turns resulting from the intervention.The teachers at Mapledale, and to a lesser extent, Mitchell both engaged in more teacher-child interactions during shared book reading when using the text-sets than not.During the final text set, however, there seemed to be some intervention fatigue with conversational turns essentially flat compared to business-as-usual.B1" B2" B3" B4" B5" B6" B7" B8" I1" I2" I3" I4" W1"W2"W3"W4" I5" I6" I7" I8" W5"W6"W7"W8" I9" I10"I11"W12" B1" B2" B3" B4" B5" B6" B7" B8" 5" 6" 7" 8" 9" 10" 11" 12" 13" 14" 15" 16" 25" 26" 27" 28" 29" 30" 31" 32"   text set at Grove Hill, there were few increases in the number of interactions between the children and teachers during the intervention period.These results were consistent with our analysis of the fidelity to implementation checklist.For example, none of these teachers were inclined to use or refer to the lesson plans provided, or to ask open-end questions.In these cases, the intervention did not appear to influence how teachers interacted with children and vice versa.Neither the multiple genre nor the suggested prompts to engage children more fully in the texts seemed to affect how teachers and children interacted in these shared book readings.

Number of Verbalizations
These results highlight the potential dissimilarities among participants within intervention research, but once again, do not provide insights into why such differences occur.Within our intervention study, we began to look more closely at the dynamic contexts for teachers' practice through the analysis of our qualitative data.In other words, instead of merely noting a failure to implement with fidelity, we sought to understand the reasons for the lack of implementation.Using evidence from these analyses, we next examined the potential constraints and affordances that may have influenced teacher engagement in our intervention.

Constraints and Affordances
Our qualitative analysis provided an important lens for understanding the contextual factors that might have influenced teacher engagement.Each center was grappling with policy and institutional changes emerging from EarlyLearnNYC, although in different ways and with different degrees of success.As shown in Tables 3 and 4, all of our sites had different combinations of constraints and affordances that reflect different leadership styles, the influence of different policies, community dynamics and varied resources.
Our observations revealed a number of constraints in the enactment of the intervention.
Shifting Policy Contexts.The paramount constraint was the shifting policy context, specifically, the city-wide EarlyLearnNYC.Along with new funding requirements, the policy also implemented new accountability measures aimed at improving the quality of care and learning of children in these sites.Consequently, the directors and teachers were focused on understanding and implementing these new requirements.Similarly, the policy context for Head Start was shifting, framed by the new model instituted under the Head Start Act of 2007 that required Head Start programs to re-compete every five years for continued funding.In addition, all five sites were newly funded under the new Universal Pre-K (UPK) program, and those classrooms had additional requirements and supervision.Together, these policies (some of which were still in formulation) created confusion and uncertainty among directors and teachers regarding standards, requirements and programs.
Rising Accountability Requirements and Reformulated Reimbursements.Relatedly, under the new EarlyLearnNYC policy, there were new requirements for paperwork, and missed deadlines with consequences that varied by site (see Table 3).Throughout our project, teachers and site leaders repeatedly noted how the increasing accountability requirements influenced their days, shifting their work from teaching to paperwork.According to Richard, the director at Mapledale, "Our teachers have so many responsibilities beyond teaching.Like the assessments, and showing how they are meeting standards for the Department of Education, or EarlyLearnNYC standards.It's a lot of work to protect their time to actually let them teach." Four of our centers had multiple layers of accountability.In addition to the NYC requirements, two of our sites were accredited by the National Association of Early Childhood (NAEYC) with its accompanying requirements and standards.At these sites, researchers observed children's writing and/or drawing work posted on bulletin boards with distinct sets of standards labeled on the object, each attesting to the multiple standards teachers were working under.As one teacher told us, "It wasn't easy to link your intervention to all the different standards I'm required to teach.Sometimes all I dream about are standards."Or in the case of Ms. Fordham at Guardian Head Start "My lessons are due on Thursdays, and I have to tell Anita (site director) what I am teaching and why from the standards, three kinds of standards, and then be able to show her when she comes to watch me." Under the new reformulation of reimbursements, monies now were to follow the child.Two of our centers (Shadyside and Grove Hill) were especially affected; when children left the program classrooms, so did their funds, with the result that there were fewer funds to support a full-time teacher.As of the end of 2014, EarlyLearnNYC led to the closure of nearly 50 centers due to inability to support enough classrooms to make the site financially viable (Hurley et al., 2014).
Site Instability.Changes in enrollment had visible ripple effects on teacher assignment and the stability of instruction.Classrooms were closed; teaching assistants and teachers were required to move into different classrooms to deal with fluctuating populations of children.In two sites, participating teachers were new to teaching preschoolers, having been Infant/Toddler teachers the previous year.Staffing instability from declining enrollment also affected the families in the neighborhood.Directors had to turn away interested families because there was neither enough children to fill a classroom nor available space to add new children to existing classrooms.
As student enrollment declined, staff were laid off and overall working conditions worsened.At Grove Hill, assistant teachers threatened to quit.At Shadyside, new assistants came and went, leaving our participating teacher, Maria, constantly negotiating for more support in her classroom.Planned projects and activities (even field trips) were cancelled or delayed due to a lack of support staff.The staff turnover left little time to prepare for our intervention.
Shifting Community Demographics.At Grove Hill and Shadyside, shifting community demographics also affected enrollment.Gentrification across Brooklyn had greatly altered the fabric of the neighborhood, displacing working-class African American families who had lived there for generations.As Lisa, our teacher at Grove Hill noted, "These families were like a backbone here -working-class families who had resources, who had been here for a long time.Now they are leaving and the families that remain have a lot more needs." On the other hand, Shadyside's enrollment was experiencing a shift from serving predominantly African American families in a historically African American neighborhood, to serving a growing population of Latino children and families.This naturally affected enrollment, but it also affected programming and staffing.Teachers needed to adapt to the language needs of emergent bilinguals who had previously represented a minor constituency in their program.For both sites, shifting demographics paired with the new policies under EarlyLearnNYC created serious constraints for enacting a new intervention.
Affordances.Nevertheless, at three of the sites, certain affordances served as protective measures against the uncertainties that these new policy initiatives created.These affordances seemed to influence the extent to which our participating teachers were able to engage with and take up the intervention.
Protected Teacher Preparation Time.At both Mapledale and Mitchell, teacher planning and preparation time was heavily protected, giving both Maya and Damien at least one hour each day to prepare for their lessons.In contrast, teachers at Grove Hill and Shadyside did not have any preparation time.Lisa, at Grove Hill, told us, "If one child doesn't sleep, which almost always happens, or there is a staff meeting, then there is no time, and there isn't any time for me during the rest of the day, so sometimes I could be ready, but mostly I had to just do my best." Community/Site Resources.Three of our sites, Mitchell, Mapledale and Guardian Head Start were each part of larger community-based initiatives.These organizations provided additional financial and material resources, such as safety-nets for funding gaps and teacher trainings across sites.In addition, two of our sites, Mitchell and Mapledale, were situated within large buildings that offered a multitude of community services such as a senior center, an afterschool program and a community health center and gym.Both sites benefited from the cross-connection of programs.Bulletin boards were filled with children's artwork about their families and their communities, and were reflected in conversations between children and teachers and in the daily curriculum.
Supportive Leadership.Some directors expressed strong support for the abilities of the participating teachers.Damien's site leader at Mitchell, indicated delight that "Damien is getting this confidence.We know he is a great teacher, and he is starting to know that too."At Mitchell and Mapledale, this respect was reflected in our interviews with the teachers.Maya and Damien both felt that they were being "invested in" as teachers.Their directors often expressed confidence and support in the teachers' abilities to make decisions, use good professional judgment, and provided an environment in which the teachers felt comfortable in taking risks.In these settings, the teachers seemed to willing and even eager to take on new challenges, as evidenced in their higher fidelity to the intervention lessons.
Early Childhood Teacher Identity.Relatedly, this affordance reflects the professional identities of participating teachers in the project.During observations, both Maya and Ms. Fordham would tell us explicitly why they made certain pedagogical or curricular choices within their classrooms.Their confidence and strong identities were reflected in their work as they engaged with the text sets.They brought their own ideas and approaches into their teaching, and integrated the themes throughout their daily practice.These choices reflected the teachers' rich professional knowledge of their students and what they felt their students needed, sometimes despite accountability constraints (Woodrow, 2007).
These constraints and affordances represent some of the contextual factors that influenced the enactment of our intervention.They reflect the political, institutional, specific site and classroom realities that our teachers were busy negotiating throughout our project.For some of our teachers, like Maya, the affordances overrode the constraints.But for others, the constraints were too overwhelming and did not allow them to concentrate on teaching.For example, teaching at Guardian Head Start had become a grinding daily act of compliance.Ms. Fordham's professional knowledge and decision-making power had long since been taken over by detailed lesson plans and high levels of accountability.The site director at Guardian was so focused on accountability that Ms. Fordham was fearful to use our text sets and accompanying lessons as anything more than a script, the exact opposite of our goal.
For Maria and Lisa, our teachers at Shadyside and Grove Hills, respectively, their role was more of a childcare provider than a teacher.At these two sites, the constraints of shifting community demographics, and the rising accountability and overall lack of resources under the new EarlyLearnNYC policies created powerful roadblocks to teaching.As a result, their focus was mostly on making it through the day, hoping to stop the inevitable closing of their sites.

Conclusion
Our central finding in this study is that contextual factors outside of schools affect what goes on inside them.Such outside factors including the shifting policies of funding and accountability for early childhood centers seemed to influence teachers' involvement and engagement with our intervention.Analyzing the individual variation through a single-subject withdrawal design, it was evident that some contexts provided more support for engagement than others.Subsequent analyses of the constraints and affordances in these contexts through our qualitative analyses helped our researchers to better understand the complicated issues faced by these centers.
According to Klingner and Boardman (2011), at least some of the challenges in educational research such as the persistent achievement gap between students of color and white students, can be explained by a research gap, a failure to conduct different types of research best suited for addressing complex issues in our schools.Randomized controlled trials have been highly useful for developing knowledge that something can work.However, how to make an intervention actually work reliably for different subgroups of students and teachers working under very different conditions requires designs that are more sensitive to diverse contexts and populations (Cobb, McClain, Lamberg, & Dean, 2003).Traditionally, educational researchers have undervalued the importance of these contextual conditions, preferring to control for these extraneous variables rather than to understand them.
In contrast, in this study we sought to better understand these factors.Examining the social validity of our program (Wolf, 1978), our efforts were to understand how the intervention was actually being experienced by the teachers it was supposed to help.The intervention was clearly aligned with the Common Core, a focus of preschool efforts in New York City; it emphasized asking higher-order questioning techniques, a skill that was often targeted in quality reviews in centers; and it provided a library of informational texts to both teachers and the families in these classrooms.
Nevertheless, our interviews indicated that it did not address those issues that were more central to these teachers.The political and policy context had changed dramatically, with new and additional standards, requirements, accountability and funding priorities.Four of our five centers were in jeopardy of being closed due to new and sudden declines in enrollment; our fifth center, Guardian Head Start, was required to re-compete for funding for the first time in over 30 years.Needless to say, our intervention could not address the issues that mattered most and that created the greatest challenges to their daily practice.
Often intervention study results like ours are simply dismissed out of hand as a failure of teachers to implement interventions with fidelity (Erickson, 2014).Certainly, the teachers in our study struggled to follow our intervention in the ways we envisioned in design sessions at the university.But merely describing the failure to reach fidelity does not explain why this is the case.It is not that fidelity doesn't matter; it is that why an intervention is not working, or doesn't matter to a teacher, needs to be better understood.Had our research stopped at merely evaluating the intervention, rather than scrutinizing the myriad of factors influencing their work, we would not have understood they ways in which policy shifts and other contextual factors were influencing teacher engagement (Dennis & O'Connor, 2012).
From this "context out" perspective, researchers can support teachers' instructional work with a better knowledge of the constraints and affordances at play, building on the knowledge of unique communities and sites of learning.This mixed method approach to understanding the role of context allowed us to better understand the ways in which policy shifts, such as EarlyLearnNYC were leading to a host of constraints on our teachers.While EarlyLearnNYC was implemented to both save money and improve care, as with so many policies, it "creat[ed]… a lifeworld… [with] consequences for all those involved in early childhood" (Ryan & Graue, 2009, p. 191).These consequences included a destabilization of learning environments for young children while the policy was transforming the ways in which provisions for care were made in New York City's low income communities (Hurley et al., 2014).
For our continuing work on Books Aloud, this research has powerful implications.First, it suggests that the meaningfulness of an intervention, even one with the best intentions, must be based on the participants' point of view.We cannot merely assume that a new program regarded as useful to a school administrator or Center Director will have meaning to those who are charged with enacting it.Rather, we must begin to engage practitioners more fully with researchers and others in development, testing, and enhancing the clinical work of schooling.Second, it suggests that a better understanding of the context would have more likely led to an intervention that was better designed to address difficult problems that the schools and centers were facing.Too often, solutions come from outside in, rather than inside out.And third, schools and centers are complex organizations, with many different actors assuming many different roles.Understanding this context would have helped us to take advantage of the enormous talents of individuals within the organization in ways that could have more effectively contributed to children's learning and development.
As Bryk (2015) has insightfully noted, accelerating how we learn to improve in schools will require us to view improvement as a science, and as a science, consider more thoughtfully how these contextual factors may contribute to outcomes.From a policy standpoint, this means a greater focus on developing systematic and organized methods with the explicit goals of improving outcomes.Too often, we have overlooked the necessary steps of learning from context and from the participants themselves.We certainly need to know what mechanisms work to help children and teachers create meaningful knowledge and learning experiences.However, as Jones, Pickard and Stronach (2008) note, educational research should be linked to "deep issues concerning learning and motivation, rather than tied to the evaluation of ephemeral initiatives in a naïve kind of 'what works' rationale in order to improve instructional outcomes in meaningful ways" (p.27).These deep issues are often hidden from view when we do not know the contexts for teacher practice.Valuing what both "intervention in" and "classroom out" studies tell us about when, why, and how teachers take up evidence-based practices may inform "what works" approaches to improving educational outcomes.
While randomized controlled trials may be considered the gold standard for policymakers, mixed-method approaches to understanding how and why interventions may or may not work within real classrooms may be a better way to make sure that meaningful instructional improvements are actually taking place (Cobb et al., 2003).Although the drive for "what works" is laudable, we must not marginalize the existing wisdom, operational know-how, and local knowledge that may allow us to create and implement effective, local, and sustainable interventions within the unique contexts of teacher practice.

Figure 1 .
Figure 1.PRINT!Data from Mapledale and Mitchell

Figure 2 .
Figure 2. PRINT!Data from Guardian Head Start, Shadyside and Grove Hill

Table 1
Site and Neighborhood Demographics Note: All names for sites and persons are pseudonyms.Neighborhood level data obtained from 2013 Census.

Table 2
Note: All names for sites and persons are pseudonyms.