Scaling Down: A Modest Proposal for Practice-based Policy Research in Teaching

This proposal addresses the assumptions underlying the “Science Wars” about the purpose of educational research. The author proposes a more modest “Lake Woebegon” approach to school reform that supports long-term professional education, a shift away from “Best Practices” to “Pretty Good Practices,” low fidelity implementation of interventions, and the importance of local context. Action research is presented as one possible qualitative approach for conducting “practice-based research.” Keywords: educational policy; educational research; school reform; qualitative methods; Science Wars Bajando las expectativas: Una propuesta modesta para una investigación de las políticas de enseñanza basada en las practicas. Resumen: Esta propuesta trata de los supuestos que subyacen a las "guerras científicas" para determinar el propósito de la investigación educativa. El autor propone un enfoque más modesto epaa aape Education Policy Analysis Archives Vol. 22 No. 9 SPECIAL ISSUE 2 basado en el modelo del "Lago Woebegon" para reformas escolares apoyando la educación profesional a largo plazo, y alejándose del modelo de las "mejores prácticas" y acercándose a uno que de "practicas bastante buenas” , intervenciones de baja fidelidad, que den importancia al contexto local. La investigación-acción se presenta como un posible enfoque cualitativo para una investigación de las políticas de enseñanza basada en las practicas. Palabras clave: política educativa; investigación educativa; reforma escolar; métodos cualitativos; Guerras Científicas. Abaixando expectativas: Uma proposta modesta para uma pesquisa das política de ensino baseadas nas práticas Resumo: Esta proposta aborda os pressupostos subjacentes às "guerras da ciência" para determinar a finalidade da pesquisa educacional. O autor propõe uma abordagem mais modesta com base no modelo de "Lago Woebegon" de reformas escolares para apoiar a formação profissional de longo prazo, e longe do modelo de "melhores práticas" e aproximando-se a um modelo das " práticas bastante boas" usando intervenções de “baixa fidelidade” dando importância para o contexto local. A pesquisa-ação é apresentada como uma possível abordagem qualitativo das pesquisa de política de ensino baseadas nas práticas. Palavras-chave: política educacional; pesquisa educacional; reforma escolar; métodos qualitativos; guerras da ciência.


Introduction
I am sorry to say this, but I do not think the science wars are over.They are not about methods-quantitative versus qualitative.They are about substance-about basic assumptions concerning the purposes of educational research.And I am being ingenuous in the title-this modest proposal is not modest at all, it is a strong critique, a challenge to conventional wisdom.I am invoking Jonathan Swift's ironically modest proposal to address the problem of hunger in Irelandhe suggested that the poor should be required to cook and eat their own babies.It was a policy recommendation that had no uptake by stakeholders.
Here is the problem in a nutshell: Mark Twain is said to have said, "History doesn't repeat itself; at best it sometimes rhymes."Our current educational reform policy paradigms presume the opposite-that history does repeat itself quite closely indeed, and thus "what works" in one time and place will work in lots of other places at other times.We find out "what works" in a few settings, using the Gold Standard of a randomized field trial of a model of educational practice, identifying and tracking measurable outcomes of the practice.This is a research model [and as a paradigm, it is also an ideology, a body of conventional wisdom, what Foucault (1979) called a "discourse formation"]. 1 It is a research paradigm borrowed from the "hard sciences" of physics and chemistry and also used in medicine.The presumption is that once we had certain evidence of the outcomes of a set of practices we could then replicate that model of practice in many other places."Fidelity of implementation" is essential in this process of "scaling up" what works in school reform-exact copying of "best practices" that are easily exportable on a large scale, if teachers and building principals will only do what they are told to do-adopt the "research-based practices" whose efficacy someplace else was proved by "scientific" research.
A fundamental problem with this is that rational planning in social engineering requires of the future that it not be original; that it holds still.Such planning requires, as does physics and chemistry, a "uniformity of nature" that many in the social sciences have argued just does not exist in the lability and variability of human social life.2Even in medicine, the attempt to discover general laws that enable prediction and control over the environment that are comparable to those in physics and chemistry-certain knowledge of cause that will inform detailed prescriptions for clinical practice-does not work nearly as well as the early proponents of "research-based practice" in medicine had hoped.This is because an actual physician does not treat a disease entity in general-a physician treats a particular patient who manifests that disease entity in particular, situated ways.Likewise, a teacher does not teach children in general, but particular children in particular circumstances of learning and teaching in classrooms and in community life.
Admittedly, about 20 years ago the Blue Meanies [the proponents of audit-culture-justified "high stakes accountability" approaches to educational reform-see Barzelay (2001) on this worldwide movement in public policy thinking] took over the Yellow Submarine (public schools and other agencies of human service provision).But it is becoming increasingly clear that the Blue Meanies do not know how to run the Submarine.What they promised to deliver in increased efficiency and effectiveness is not happening.The future continues to be original, the local refuses to hold still.General prescriptions for practice do not fit the circumstances of specific situations.Twelve years ago many of my colleagues and I went around saying that "No Child Left Behind" was a train wreck waiting to happen.We can now see that it is happening, albeit in slow motion.The high stakes accountability reformers have driven themselves into a blind alley.Their days are numbered.They have made systems of educational provision less loosely coupled than before.But schools and school systems will never be entirely tightly coupled.Teachers, as frontline service providers in a human service institution will continue to function as "street level policy makers," complying with some mandates from above in the organization but not complying with all of the mandates (see Lipsky, 1980;Schwille, Porter, & Gant, 1980).Yet there is good news-qualitative approaches to policy inquiry can help centrally located policy makers back out of the blind alley they are in.
To do this, however, we all must give up on the grandiosity of the current worldview-the paradigm of "scaling up" and "best practices" and "high fidelity implementation."A much more modest approach is necessary-scaling down expectations for what research can do, and for how school improvement can take place.In thinking about this I draw on my experience growing up in a real town version of Garrison Keillor's "Lake Wobegon," the little town on the Minnesota prairie.As Keillor says, in his fictional ethnographic reporting, when people in Lake Wobegon were asked how things were going we would say, "It could be worse."If things were going really well we would say, "Not too bad."So the first thing we can learn from Lake Wobegon is to scale down the expectations for reform-promise less-abandon sweeping claims like those in the title for the program for inner city schools called "Success for All." Recognize that real school improvement will take years upon years, and it will cost money, because good teaching is a labor-intensive enterprise, and there is no end run around that.Two generations of teachers have been de-skilled by highly scripted teaching materials in an attempt at installing "teacher-proof" curriculum.Instead what is needed is serious investment in long term continuing professional education-support for the clinical judgment of teachers rather than vain attempts to replace the complexity of that judgment with simple compliance to administrative fiat.
A second thing we can learn from Lake Wobegon is the example of its grocery store.Keillor calls it "Ralph's Pretty Good Grocery."He says, "If Ralph's hasn't got it you probably don't need it."For God's sake, let's stop talking about "Best Practices."They do not travel well, even from one classroom to the next, let alone across school districts, states, and regions of the country.Yet in places where good teaching takes place and good outcomes result, recurringly, there are "Pretty Good Practices" that can be identified, and described in specific enough detail by qualitative researchers, including teachers themselves, so that other teachers could understand what those practices looked and felt like, and could try them out in their own classrooms.Practices likely to be good are good enough for one's own local field trial-trying them out as a way to start growing new practices for yourself.
A third thing we can learn from Lake Wobegon is to be satisfied, perhaps more than satisfied, with "low fidelity implementation."When I grew up we did not have high fidelity at allthe radio had static on it and phonograph records were scratchy.But we could still listen with enjoyment to music in low fidelity, Jack Benny was still very funny in low fidelity, and we learned from the words of Edward R. Murrow in low fidelity.A Lake Wobegon approach to school improvement would involve "low fidelity implementation" of "pretty good practices."That means the celebration of local adaptation rather than an attempt to stamp it out by centralized planning and monitoring-policy that provides wiggle room-provides for custom tailoring of practices to fit the particularity of local circumstances.Educational "treatments" as enacted are not unitary phenomena-a small cluster of "best practices" to be imitated slavishly from one setting to another.Educational treatments as instantiated on the ground are the enactment of daily life in classroomsways of life lived in real time and place, not static models of life in a social world of nowhere and nowhen.In fact, good practices are inimitable.Like history, they do not repeat themselves, exactly, even though sometimes they rhyme.
We need educational research that on the one hand is able to take account of the uniqueness of particular historical circumstances-the particulars of practices at a certain time and in a certain place-and that on the other hand is also able to take account of history's rhyming and indeed is able to help us in the recognition of rhyming.We need research that provides that help gently, encouragingly.
This can be done in action research-call it design research if you will-that respects the sense-making of local social actors who want to make schools better places for teaching and learning.It can be done by academics and by staff of action research organizations in collaboration with teachers and school building principals, and it can be done by school practitioners themselves.Examples of the former approach are: The Institute for Community Research in Hartford, Connecticut; Research for Action in Philadelphia, Pennsylvania; Designs for Change in Chicago, Illinois; CREATE at the University of California, San Diego; and CONNECT at the University Lab School at the University of California, Los Angeles (see also Erickson, 2006;Erickson & Christman, 1996; among many others).Examples of the latter approach are: The Teachers' Learning Collaborative in Philadelphia, Pennsylvania, many other teacher inquiry groups, and the overall approach of the National Writing Project, headquartered in Berkeley, California (see also Ballenger 1998;Cochran-Smith & Lytle, 1993, 2009;among many others).This is work addressed to (and partly or wholly conducted by) a local audience of practitioners who (usually) are trying out something new in order to attempt to improve their practice.It is research ("re-search"-paying closer than usual attention to what is going on locally, and doing so recursively) but it is not primarily addressed to an audience of central policy decisionmakers, although they can be informed by case study reports of the local attempts at change.This kind of research does not attempt to discover general laws.Rather it is an inquiry into "what is working" locally, with detailed attention and reporting devoted to inquiry into the "what and how" of local practices, in order to determine specific local mechanisms of cause-why what is working does so, why it sometimes works better, why it sometimes falters, see the discussion in Maxwell (2004) (and this is continuing inquiry because history does not repeat itself exactly, from year to year, even in the same setting.) Aristotle, in the Nichomachean Ethics, Book 6 (1934) called such inquiry "phronesis"-the prudential knowledge of local circumstances gathered by wise governing officials in order to make local policy decisions-in contrast to "episteme" (knowledge of things in general such as that found in mathematics; akin to modern day "hard science" knowledge) or to "techne"-the procedural knowledge of the skilled craftsman.Current policy discourse presumes that what is needed for policy decision-making is episteme but what is actually needed is phronesis. 3Phronetic case study inquiry can include quantitative as well as qualitative data-when things are counted phronetically this is usually reported as descriptive statistics, less usually analyzed and reported by means of inferential statistics.But phronetic inquiry uses close attention and description to develop a grounded theory of local process and local cause, with iterative revisions of procedures of quantification.
In other words, in this work there is a strong qualitative warrant for deciding which things to count-which are the entities ("qualities") that need to be counted in order to get smarter about what is going on.That qualitative warrant precedes attempts to try to count things and identify ranges of occurrence and relative frequency of occurrence.(Thus this is quantitative analysis that is Bayesian in spirit.)Often this kind of design research can be considered as a time-over-time quasiexperiment (see as a very recent example Mehan, 2012).It is like gardening as contrasted to high tech industrial agriculture; the way my grandmother grew African violets, with repeated effects of lush blooming.(Exported from her house in Minneapolis only seventy five miles west to Lake Wobegon, where we lived, Grandma's violet plants stopped blooming under my mother's care and usually died within two months.Grandma's more than "pretty good practices"-her specific, phronetically informed ways of growing flowers-did not travel at all well.And I know my mother did not have a good case study description of what her mother actually did with her violets.Grandma did not show us that, in close detail.Hmm.That is interesting.)

Conclusion
Educational policy discourse and educational policy research conduct have things to learn from Lake Wobegon, Minnesota, my hometown.First, scale down the expectations for change-not for its magnitude, but for its pace.Give up the grandiose social engineering vision of school reform as the construction of an interstate highway system.Settle for a more realistic vision of school change as slogging step by step through a swamp, one swamp at a time, each swamp a bit different from the others.Second, aim for "pretty good practices;" things to try out locally to see how they work out locally-pretty good, not so good, awful.Third, recognize that "low fidelity implementation" is not a problem, but an opportunity.Reread "low fidelity" as the possibility that local adaptation is being done ingeniously.Respect and appreciate the intentions and capacities of local teachers and students.(Garrison Keillor says appreciatively that in Lake Wobegon all the women are strong, all the men are good looking, and all the children are above average.)Above all do not try to remove the clinical judgment of teachers-rather, strengthen it by policies that provide for teachers' judgmental "wiggle room" and for the provision of deep, continuing professional education, so as to develop further teachers' and principals' capacities for "proximal formative assessment" of student learning, including students' learning to affiliate or disaffiliate with the academic tasks and identities that everyday life in classrooms presents (see Erickson 2007aErickson , 2007b)).Remember that you can never force learning on anyone-students, teachers, administrators, parents.In Lake Wobegon we used to say, "You can bring a horse to water but you can't make it drink."Do not expect history to repeat itself.Rather, try to learn how to recognize how and why it sometimes rhymes.Scale down rather than up.
In other words, if you want genuine and enduring school reform, prepare to replace "research-based practice" with "practice-based research."