Gathering evidence on an after-school supplemental instruction program: Design challenges and early findings in light of NCLB.

Authors

  • Madhabi Chatterji Teachers College, Columbia University
  • Young Ae Kwon Kwon Learning Center
  • Clarice Sng Teachers College, Columbia University

DOI:

https://doi.org/10.14507/epaa.v14n12.2006

Keywords:

research evidence, supplemental instructional programs, rigorous evaluation methods.

Abstract

The No Child Left Behind (NCLB) Act of 2001 requires that public schools adopt research-supported programs and practices, with a strong recommendation for randomized controlled trials (RCTs) as the “gold standard” for scientific rigor in empirical research. Within that policy framework, this paper compares the relative utility of federally-recommended RCT versus the demonstrated extended term mixed-method (ETMM) designs as options for monitoring effects of novel programs in real-time field settings. Guided by the program’s theory of action, a year-long, two-phase study was conducted to monitor the context, processes and early outcomes of an after-school supplemental program in a New York elementary school. In both phases, the design combined a matched-groups, quasi-experiment with qualitative classroom observations and descriptive surveys. Early findings showed some positive, albeit “gross” program effects. Although findings are tentative, the ETMM approach enhanced interpretations by shedding light on relevant environmental variables, causes for program instabilities and sample attrition, and factors affecting treatment fidelity and scaling-up of the program beyond the pilot year.

Downloads

Download data is not yet available.

Author Biographies

Madhabi Chatterji, Teachers College, Columbia University

Madhabi Chatterji, Ph.D., is Associate Professor of Measurement, Evaluation and Education at Teachers College, Columbia University. Her research interests are in evidence-gathering and evaluation of field interventions with systemic designs, designing classroom and institutional assessment systems, and in the design and validation of construct measures with classical and Rasch measurement methods.

Young Ae Kwon, Kwon Learning Center

Young Ae Kwon, Ed.D., is the owner and director of the Kwon Learning Center at Charlotte, North Carolina. Her research interests are in effective after-school programs for minority children and advanced statistical models.

Clarice Sng, Teachers College, Columbia University

Clarice Sng, Ed.D., is Associate Director, Office of Accreditation and Assessment, Teachers College, Columbia University. Her research interests are in instrument design and construct validation, mixed-method and longitudinal research.

Downloads

Published

2006-05-02

How to Cite

Chatterji, M., Kwon, Y. A. ., & Sng, C. (2006). Gathering evidence on an after-school supplemental instruction program: Design challenges and early findings in light of NCLB. Education Policy Analysis Archives, 14, 12. https://doi.org/10.14507/epaa.v14n12.2006

Issue

Section

Articles