Haney (2000) examined a number of aspects of the Texas record of educational progress. This brief response concerns one particular indicator: the 1992-1996 gain in mathematics scores from 4th to 8th grades as measured by the National Assessment of Educational Progress (NAEP). In terms of the NAEP scales scores—not the achievement level percentages—the Texas gain from 1992-1996 was about 49 points. In any metric, this represents a sizable gain. In order to give some perspective to this accomplishment, it is customary to compare states. Implicitly, the rationale for doing so is that some states do better than others, and through a process of competition and selection the level educational level of students can be bootstrapped. Since the Texas gain was the largest of any state, it could be argued that there is much merit in its methods and efficiencies.
          However, Haney raised a number of questions about whether this was a gain in achievement or whether it could be attributed to a large degree to changed in grade retention and dropout rates. There is a study on the 4th-8th grade mathematics gains that Haney did not consider which is relevant to this point. The Math cohort study by Barton et al (1998) estimated gains in math for a cohort of students in 4th grade who attended 8th grade four years later. To those who look to statistics to support the educational record of Texas (and to those who would take credit for the miracle), there is good news and bad news in this study.
          First, the good news. In the cohort study, Texas students gained about 52 points from 4th to 8th grade. Thus, unless students are retained in the 4th and 5th-8th grades disproportionately, there can be little question that the NAEP scores have gone up substantially. (Haney shows that for grades 2-8, the transition ratios are uniform. Questions arise in the 9th-10th grade transition.) But in regard to a comparison among states, the miracle in Texas looks much like the median elsewhere. Of 35 states and two districts (Guam and D.C.), the 52-point gain of Texas was good enough to earn Texas a rank of 17th or about the 46th percentile. Though Texas outranked four other states by less than one point, it should also be mentioned that six states outranked Texas by less than one point.
          This latter finding brings up a central point in the NAEP mathematics results for 1992 and 1996. In fact, the states are pretty well bunched up in the middle. In terms of statistical significance, Texas is different only from Guam (with a 40-point gain), and is not significant from Nebraska (ranked 1st with a 57-point gain). Was there a miracle in NAEP gains from 1992 to 1996 in Texas? The answer very clearly is no. Texas was average.
          One more simple representation helps to illustrate this latter point. In Figure 1, the state cohort gains are plotted against median state income (average across 1995-1997). Though a slight linear trend is evident (with Arizona and Hawaii being negative outliers), the story is relatively clear once more. With respect to wealth, which is one of the most reliable predictors of achievement, Texas stands in the middle of the pack—that is, no worse than most other states in delivering educational services to all students. Certainly, there is no criticism that can be leveled against Texas that cannot also be leveled against others states. However, within a paradigm that promotes healthy competition among states as a means of developing effective education policy, the points of light in Texas are not beacons.

Figure 1. 1992-1996 NAEP cohort gains in mathematics plotted against median family income.

About the Author

References

Barton, P. and Coley, R. (1998). Growth in School: Achievement Gains From the Fourth to the Eighth Grade. Princeton, NJ: Educational Testing Service.

General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .