This article has been retrieved   times since May 2, 2006

Gathering Evidence on an After-School Supplemental Instruction Program: Design Challenges and Early Findings in Light of NCLB Madhabi Chatterji Teachers College, Columbia University Young Ae Kwon Kwon Learning Center Clarice Sng Teachers College, Columbia University Citation: Chatterji, M., Kwon, Y.A., & Sng, C. (2006). Gathering evidence on an after-school supplemental instruction program: Design challenges and early findings in light of NCLB. Education Policy Analysis Archives, 14(12). Retrieved [date] from http://epaa.asu.edu/epaa/v14n12/. Abstract The No Child Left Behind (NCLB) Act of 2001 requires that public schools adopt research-supported programs and practices, with a strong recommendation for randomized controlled trials (RCTs) as the “gold standard” for scientific rigor in empirical research. Within that policy framework, this paper compares the relative utility of federally-recommended RCT versus the demonstrated extended term mixed-method (ETMM) designs as options for monitoring effects of novel programs in real-time field settings. Guided by the program’s theory of action, a year-long, two-phase study was conducted to monitor the context, processes and early outcomes of an after-school supplemental program in a New York elementary school. In both phases, the design combined a matched-groups, quasi-experiment with qualitative classroom observations and descriptive surveys. Early findings showed some positive, albeit “gross” program effects. Although findings are tentative, the ETMM approach enhanced interpretations by shedding light on relevant environmental variables, causes for program instabilities and sample attrition, and factors affecting treatment fidelity and scaling-up of the program beyond the pilot year. Keywords: research evidence; supplemental instructional programs; rigorous evaluation methods.   Access this article in PDF format.

The World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu Editor: Sherman Dorn, University of South Florida Production Assistant: Chris Murrell, Arizona State University General questions about appropriateness of topics or particular articles may be addressed to the Editor, Sherman Dorn, epaa-editor@shermandorn.com. The Commentary Editor is Casey D. Cobb: casey.cobb@uconn.edu. EPAA Editorial Board Michael W. Apple University of Wisconsin David C. Berliner Arizona State University Robert Bickel Marshall University Greg Camilli Rutgers University Casey Cobb University of Connecticut Linda Darling-Hammond Stanford University Mark E. Fetler California Commission on Teacher Credentialing Gustavo E. Fischman Arizona State University Richard Garlikov Birmingham, Alabama Gene V Glass Arizona State University Thomas F. Green Syracuse University Aimee Howley Ohio University Craig B. Howley Appalachia Educational Laboratory Patricia Fey Jarvis Seattle, Washington Daniel Kallós Umeå University Benjamin Levin University of Manitoba Thomas Mauhs-Pugh Green Mountain College Les McLean University of Toronto Michele Moses Arizona State University Heinrich Mintrop University of California, Los Angeles Anthony G. Rud Jr. Purdue University Michael Scriven University of Auckland Lorrie A. Shepard University of Colorado, Boulder Kevin Welner University of Colorado, Boulder Terrence G. Wiley Arizona State University John Willinsky University of British Columbia     EPAA English Language Graduate-Student Editorial Board Noga Admon New York University Jessica Allen University of Colorado Cheryl Aman University of British Columbia Anne Black University of Connecticut Marisa Cannata Michigan State University Chad d’Entremont Teachers College Columbia University Carol Da Silva Harvard University Tara Donahue Michigan State University Camille Farrington University of Illinois Chicago Chris Frey Indiana University Amy Garrett Dikkers University of Minnesota Misty Ginicola Yale University Jake Gross Indiana University Hee Kyung Hong Loyola University Chicago Jennifer Lloyd University of British Columbia Heather Lord Yale University Shereeza Mohammed Florida Atlantic University Ben Superfine University of Michigan John Weathers University of Pennsylvania Kyo Yamashiro University of California Los Angeles EPAA Spanish & Portuguese Language Editorial Board Associate Editors Gustavo E. Fischman Arizona State University & Pablo Gentili Laboratório de Políticas Públicas Universidade do Estado do Rio de Janeiro Founding Associate Editor for Spanish Language (1998—2003) Roberto Rodríguez Gómez Universidad Nacional Autónoma de México Argentina Alejandra Birgin Ministerio de Educación, Argentina       Email: abirgin@me.gov.ar Mónica Pini Universidad Nacional de San Martin, Argentina       Email: mopinos@hotmail.com, Mariano Narodowski Universidad Torcuato Di Tella, Argentina       Email: Daniel Suarez Laboratorio de Politicas Publicas-Universidad de Buenos Aires, Argentina       Email: daniel@lpp-buenosaires.net Brasil Gaudêncio Frigotto Professor da Faculdade de Educação e do Programa de Pós-Graduação em Educação da Universidade Federal Fluminense, Brasil       Email: gfrigotto@globo.com Vanilda Paiva       Email:vppaiva@terra.com.br Lilian do Valle Universidade Estadual do Rio de Janeiro, Brasil       Email: lvalle@infolink.com.br Romualdo Portella do Oliveira Universidade de São Paulo, Brasil       Email: romualdo@usp.br Roberto Leher Universidade Estadual do Rio de Janeiro, Brasil       Email: rleher@uol.com.br Dalila Andrade de Oliveira Universidade Federal de Minas Gerais, Belo Horizonte, Brasil       Email: dalila@fae.ufmg.br Nilma Limo Gomes Universidade Federal de Minas Gerais, Belo Horizonte       Email: nilmagomes@uol.com.br Iolanda de Oliveira Faculdade de Educação da Universidade Federal Fluminense, Brasil       Email: iolanda.eustaquio@globo.com Walter Kohan Universidade Estadual do Rio de Janeiro, Brasil       Email: walterko@uol.com.br Canadá Daniel Schugurensky Ontario Institute for Studies in Education, University of Toronto, Canada       Email: dschugurensky@oise.utoronto.ca Chile Claudio Almonacid Avila Universidad Metropolitana de Ciencias de la Educación, Chile       Email: caa@rdc.cl María Loreto Egaña Programa Interdisciplinario de Investigación en Educación (PIIE), Chile       Email: legana@academia.cl España José Gimeno Sacristán Catedratico en el Departamento de Didáctica y Organización Escolar de la Universidad de Valencia, España       Email: Jose.Gimeno@uv.es Mariano Fernández Enguita Catedrático de Sociología en la Universidad de Salamanca. España       Email: enguita@usal.es Miguel Pereira Catedratico Universidad de Granada, España       Email: mpereyra@aulae.es Jurjo Torres Santomé Universidad de A Coruña       Email: jurjo@udc.es Angel Ignacio Pérez Gómez Universidad de Málaga       Email: aiperez@uma.es México Hugo Aboites Universidad Autónoma Metropolitana-Xochimilco, México       Email: aavh4435@cueyatl.uam.mx Susan Street Centro de Investigaciones y Estudios Superiores en Antropologia Social Occidente, Guadalajara, México       Email: slsn@mail.udg.mx Adrián Acosta Universidad de Guadalajara       Email: adrianacosta@compuserve.com Teresa Bracho Centro de Investigación y Docencia Económica-CIDE       Email: bracho dis1.cide.mx Alejandro Canales Universidad Nacional Autónoma de México       Email: canalesa@servidor.unam.mx Rollin Kent Universidad Autónoma de Puebla. Puebla, México       Email: rkent@puebla.megared.net.mx Perú Sigfredo Chiroque Instituto de Pedagogía Popular, Perú       Email: pedagogia@chavin.rcp.net.pe Grover Pango Coordinador General del Foro Latinoamericano de Políticas Educativas, Perú       Email: grover-eduforo@terra.com.pe Portugal Antonio Teodoro Director da Licenciatura de Ciências da Educação e do Mestrado Universidade Lusófona de Humanidades e Tecnologias, Lisboa, Portugal       Email: a.teodoro@netvisao.pt USA Pia Lindquist Wong California State University, Sacramento, California       Email: wongp@csus.edu Nelly P. Stromquist University of Southern California, Los Angeles, California       Email: nellystromquist@juno.com Diana Rhoten Social Science Research Council, New York, New York       Email: rhoten@ssrc.org Daniel C. Levy University at Albany, SUNY, Albany, New York       Email: Dlevy@uamail.albany.edu Ursula Casanova Arizona State University, Tempe, Arizona       Email: casanova@asu.edu Erwin Epstein Loyola University, Chicago, Illinois       Email: eepstei@wpo.it.luc.edu Carlos A. Torres University of California, Los Angeles       Email: torres@gseisucla.edu