Texas Gains on NAEP : Points of Light ?

The 1992-1996 gain in mathematics scores on NAEP from 4th to 8th grades in Texas is placed in perspective. The "miracle" in Texas looks much like the median elsewhere. Of 35 states and two districts (Guam and D.C.), the 52-point gain of Texas was good enough to earn Texas a rank of 17th or about the 46th percentile. Taking into consideration the wealth of states, Texas stands in the middle of the pack—no worse than most other states in delivering educational services to students. Haney (2000) examined a number of aspects of the Texas record of educational progress. This brief response concerns one particular indicator: the 1992-1996 gain in mathematics scores from 4th to 8th grades as measured by the National Assessment of Educational Progress (NAEP). In terms of the NAEP scales scores—not the achievement level percentages—the Texas gain from 1992-1996 was about 49 points. In any metric, this represents a sizable gain. In order to give some perspective to this accomplishment, it is customary to compare states. Implicitly, the rationale for doing so

is that some states do better than others, and through a process of competition and selection the level educational level of students can be bootstrapped.Since the Texas gain was the largest of any state, it could be argued that there is much merit in its methods and efficiencies.
However, Haney raised a number of questions about whether this was a gain in achievement or whether it could be attributed to a large degree to changed in grade retention and dropout rates.There is a study on the 4th-8th grade mathematics gains that Haney did not consider which is relevant to this point.The Math cohort study by Barton et al (1998) estimated gains in math for a cohort of students in 4th grade who attended 8th grade four years later.To those who look to statistics to support the educational record of Texas (and to those who would take credit for the miracle), there is good news and bad news in this study.
First, the good news.In the cohort study, Texas students gained about 52 points from 4th to 8th grade.Thus, unless students are retained in the 4th and 5th-8th grades disproportionately, there can be little question that the NAEP scores have gone up substantially.(Haney shows that for grades 2-8, the transition ratios are uniform.Questions arise in the 9th-10th grade transition.)But in regard to a comparison among states, the miracle in Texas looks much like the median elsewhere.Of 35 states and two districts (Guam and D.C.), the 52-point gain of Texas was good enough to earn Texas a rank of 17th or about the 46th percentile.Though Texas outranked four other states by less than one point, it should also be mentioned that six states outranked Texas by less than one point.This latter finding brings up a central point in the NAEP mathematics results for 1992 and 1996.In fact, the states are pretty well bunched up in the middle.In terms of statistical significance, Texas is different only from Guam (with a 40-point gain), and is not significant from Nebraska (ranked 1st with a 57-point gain).Was there a miracle in NAEP gains from 1992 to 1996 in Texas?The answer very clearly is no.Texas was average.
One more simple representation helps to illustrate this latter point.In Figure 1, the state cohort gains are plotted against median state income (average across [1995][1996][1997].Though a slight linear trend is evident (with Arizona and Hawaii being negative outliers), the story is relatively clear once more.With respect to wealth, which is one of the most reliable predictors of achievement, Texas stands in the middle of the pack-that is, no worse than most other states in delivering educational services to all students.Certainly, there is no criticism that can be leveled against Texas that cannot also be leveled against others states.However, within a paradigm that promotes healthy competition among states as a means of developing effective education policy, the points of light in Texas are not beacons.

Gregory Camilli
Email: camilli@rci.rutgers.eduGregory Camilli is a professor in the Rutgers Graduate School of Education, and former Chair of the Department of Educational Psychology.His interests include measurement, program evaluation, and policy issues regarding student assessment.Dr. Camilli teaches courses in statistics and psychometrics, structural equation modeling, and meta-analysis.His research interets include efficacy studies of Head Start, implementation variability in cooperative learning and technology, and factors related to differential item functioning.

Figure 1 .
Figure 1.1992-1996 NAEP cohort gains in mathematics plotted against median family income.