The failure of the U.S. education research establishment to identify effective practices: Beware effective practices policies

Authors

  • Stanley Pogrow San Francisco State University

DOI:

https://doi.org/10.14507/epaa.25.2517

Keywords:

Equal Educational Opportunity, Urban Education, Compensatory Education, Literacy, Success for All, Educational Evaluation/Assessment, School reform, and Evidence Based decision-Making

Abstract

One of the major successes of advanced quantitative methods has been its seeming ability to provide unbiased determinations of which education practices are effective for education in general and for improving the educational achievement and opportunity of the neediest students.  The power of this methodology as applied in the top education research journals has led to periodic implementation of federal and state effective practices policies. In such policies the government or its proxy determines which programs are effective and then requires or encourages schools to spend its funds exclusively on those proven programs. For example, the federal Investing in Innovation (i3) initiative requires those applying for its largest dissemination grants to have had their intervention validated as effective by the What Works Clearinghouse (WWC). Some are now even advocating that all expenditures of federal funds by school districts be restricted to purchasing research validated programs. While this seems like rational policy, between 1998 and 2002 I produced a series of articles that showed that the most research validated program to develop the reading skills of students born into poverty was not actually effective in practice. Was this dichotomy between published research proving something to be effective and what was happening in the real world an anomaly or a more widespread problem? This article (a) analyzes the validity of the gold standard scientific methodology used by the top research journals and WWC to determine whether practices are effective, and (b) examines the history of effective practices policies and their actual effectiveness. I conclude that the increasingly sophisticated methods used to assess the effectiveness of practices (a) are flawed and exaggerate actual effectiveness, and (b) do not provide the type of information practitioners need. As a result, research on effective practices tends to mislead rather than inform practice and are a major reason why efforts to reform high-poverty schools have had limited success. I therefore conclude that effective practices policies should not be implemented. I then suggest ideas for reforming the scientific process used to assess the effectiveness of education interventions.

Downloads

Download data is not yet available.

Author Biography

Stanley Pogrow, San Francisco State University

Stanley Pogrow (PhD, Stanford University) is currently Professor of Educational Leadership and Equity at San Francisco State University, and Professor Emeritus at the University of Arizona. His research and teaching interests are policies and practices for reducing the achievement gap and accelerating the learning of students born into poverty and the use of research to improve leadership decision-making. His most recent book is Authentic Quantitative Analysis for Education Leadership Decision-Making and EdD Dissertations.

Downloads

Published

2017-01-23

How to Cite

Pogrow, S. (2017). The failure of the U.S. education research establishment to identify effective practices: Beware effective practices policies. Education Policy Analysis Archives, 25, 5. https://doi.org/10.14507/epaa.25.2517

Issue

Section

Articles