Focusing on Short-term Achievement Gains Fails to Produce Long-term Gains

Abstract

The short-term emphasis engendered by No Child Left Behind (NCLB) has focused research predominantly on unraveling the complexities and uncertainties in assessing short-term results, rather than developing methods and assessing results over the longer term. In this paper we focus on estimating long-term gains and address questions important to evaluating schools and identifying educational policies and practices that produce long-term sustained gains. Estimates are made of annual pass rates on state exams using fixed effect models for six years of pass rates at grades 3, 6, 8 and 10; the percentages of schools making statistically significant gains, gains, losses, and statistically significant losses in pass rates are determined. Estimates are contrasted using models that include and exclude demographic characteristics. The percentages of schools with statistically significant gains varied markedly from 38 to 6 at grades 6 and 10, respectively; the percentage of schools with statistically significant declines ranged from less than 8 percent at grades 3, 6, and 8, to 23 percent at grade 10. Including demographics increased the percentages of schools with statistically significant gains and lowered the percentages with statistically significant declines. The results suggest that schools with higher proportions of free-reduced lunch and minority students are more likely to have statistically significant gains with demographic controls. Estimates of pass rate trends are made using Monte Carlo simulations; from these simulations the percentages of schools that may be mislabeled as having statistically significant gains and losses are determined. Even with six years of trend data, results suggest that chance can still play a significant role in mislabeling school performance, especially in grades having weak overall trends.