7. Other Evidence on Education in Texas

7.1 Dropout Data on Texas Revisited

How many students drop out?

• In 1997-98, a total of 27,550 students in Grades 7-12 dropped out of
  Texas public schools.
• Statewide, the annual dropout rate was 1.6 percent, unchanged from 1996-97.
• The 1997-98 actual longitudinal dropout rate, calculated for a cohort of students tracked from 7th to 12th grade, was 14.7 percent.

Who drops out and why?

• About 77 percent of dropouts were overage for grade, down from over 80 percent in 1996-97.
• On average, males continued to drop out at a slightly higher rate than females.
• Hispanic students had the highest average dropout rate, at 2.3 percent, followed by African American students (2.1%).
• Reasons cited for dropping out of school included poor attendance, entering non-state-approved General Educational Development (GED) programs, and pursuing a job.

Are they leaving certain districts?

• School districts with the largest enrollments (50,000 or more students) had the highest average dropout rate, at 2.1 percent.
• Generally, districts with lower student passing rates on the Texas Assessment of Academic Skills (TAAS) had higher dropout rates.

How do we compare nationally?

• Based on the Current Population Survey, an estimated 4.6 percent of students in Grades 10-12 dropped out of school across the nation.
• Texas had one of the lower dropout rates out of 32 states that met required Common Core of Data collection standards for school year 1996-97. (TEA, 1999, 1997-98 Report on Texas Public School Dropouts, p. iii)
  

Source:1997-98 Report on Texas Public School Dropouts Texas Education Agency. Austin, Texas, September 1999 (Revised December 1999), p. 15 (p. 22 of pdf version)
          These data obviously indicate that the annual dropout rate in Texas (that is the numbers of dropouts reported in grades 7-12 divided by the grade 7-12 enrollment) has fallen dramatically in the last decade. I refrain from commenting further on these results until after summarizing other evidence on dropouts in Texas.
          IDRA Attrition Studies. In the mid-1980s, under a contract with the Texas Department of Community Affairs, the Intercultural Development Research Association (IDRA), undertook a series of studies, one aim of which was to estimate "the magnitude of the dropout problem in the State of Texas" (IDRA, 1986, p. i). After describing the paucity of previous reliable research on dropouts in Texas, the IDRA researchers developed an index of attrition to estimate dropout rates not just statewide, but also at the level of school districts in Texas.

The index developed and used by IDRA consists of taking grade level enrollments for a base year and comparing them to enrollments in subsequent years. Since school and district enrollments are not constant, with changes in size due to increasing or declining enrollments, it is necessary to take the growth trend into account in computing attrition rate. The size change ratio was calculated by dividing the total district enrollment for the longitudinal study end year, by the total district enrollment for the base study year. Multiplying the base year enrollment by the district change ratio produces an estimate of the number of students expected to be enrolled at the end year. (IDRA, 1986, p. 9).

          Comparison of the TEA and IDRA data reveals two broad findings. First, for the academic year 1988-89, their estimates of dropouts are somewhat comparable. For that year, the IDRA reported attrition rates of 37%, 20% and 48% for Black, White and Hispanic students respectively. And if we multiply the TEA-reported annual dropout rates for grades 7-12 by six to approximate a longitudinal dropout rate across this grade span, we get 45.1%, 27.3% and 48.6% for Black, White and Hispanic students respectively. These estimates are not terribly close, but at least they are in the same ballpark. And the differences are in the directions one would expect. The TEA reported data yield slightly higher percentages since they cover grades 7-12, while the IDRA attrition percentages cover just grades 9-12.
          Second, after 1988-89, the IDRA and TEA results diverge dramatically. The IDRA data show attrition increasing between 1988-89 and 1998-99, while the TEA data show dropouts to be decreasing sharply over the same period. The divergence is so dramatic as to make one wonder whether the two organizations are referring to the same state—or even living on the same planet. The IDRA results show increases in attrition such that by 1997-98, 49% of Black, 31% of White and 53% of Hispanic students dropped out between grades 9 and 12. In contrast, the TEA reported data suggested longitudinal dropout rates for grade 7-12 of 12.6%, 5.4% and 13.8% for Black, White and Hispanic students respectively. In other words, the IDRA results indicate that the dropout problem in Texas in the late 1990s was four to six times worse than the TEA was reporting.
          Whose estimates are to be trusted;those of the IDRA or of the TEA? Before giving my answer to this question, let me summarize results of one more organization, this one from outside Texas.
          NCES Dropout Studies. Over the last decade the National Center for Education Statistics (NCES) has issued a series of reports on dropouts in the United States. The eleventh report in the series presents data on high school dropout and completion rates in 1998, and includes time series data on high school dropout and completion rates for the period 1972 through 1998. The high school completion rates are based on results of the Census Bureau's Current Population Surveys (CPS) of random U.S. households conducted in October of each year. The CPS surveys have not been designed with the specific intent of deriving state level high school completion rates and so in order to help derive reliable estimates, the NCES analysts who prepared the dropout reports have calculated averages across three years of CPS surveys. Also, it should be explained that the CPS data are based on self-reports of high school completion whether it be via normal high school completion or via alternative high school completion such as the GED testing. (Note 19)
          Table 7.2 reproduces a table from the latest NCES dropout report, showing high school completion rates of 18 through 24 year olds, not currently enrolled in high school or below, by state: October 1990-92, 1993-95 and 1996-98. As can be seen for all three time periods, these data show Texas to have among the lowest rates of high school completion among the 50 states. In each time period, the median high school completion rate across the states was about 88%, while the completion rate for Texas was about 80%. This pattern indicates that the median noncompletion rate across the states is about 12% while that of Texas is about 20% (about 66% worse than the median of the other states).

Comparing evidence on dropouts in Texas. We have now described and summarized five different sources of evidence on dropout rates in Texas: 1) dropout data reported by the TEA; 2) IDRA attrition analysis results; 3) the most recent NCES report on high school completion, based on CPS surveys; 4) cohort progression analyses from grade 9 to high school graduation and from 6 to high school graduation discussed in Part 5 above; and 5) estimated dropouts for 1996-97 based on 1995-96 grade enrollments and 1996-97 retention rates (reported in Section 5.5 above). How can we make sense of these vastly different estimates of the extent of the dropout problem in Texas, with dropout rate estimates for the late 1990s ranging from a low of 14.7% reported by the TEA as the "1997-98 actual longitudinal dropout rate" for grades 7 through 12, to a high of the 42% attrition rate reported by IRDA, also for 1997-98, but only for grades 9 through 12?
          First, it seems clear that the TEA-reported dropout rates can be largely discounted, as inaccurate and misleading. A November 1999 report from the Texas House Research Organization, The Dropout Data Debate, recounts that "In 1996, the State Auditor's Office estimated that the 1994 dropout numbers reported by the Texas Education Agency (TEA) likely covered only half of the actual number of dropouts" (p. 1). The report goes on to recount numerous problems in TEA's approach to calculating dropout rates including changing rules over time in how to define dropouts, relying on district reports of dropouts, while at the same time, beginning in 1992-93 using dropout rate as a key factor in TEA's accountability ratings of districts, and apparent fraud in district reporting. The TEA has developed a system for classifying school leavers in dozens of different ways and many types of "leavers" are not counted as dropouts. Indeed, in 1994, the TEA started classifying students who "met all graduation requirements but failed to pass TAAS" as non-dropout "leavers."
          Second, based on a comparison of the cohort progression analyses from grade 9 to high school graduation with those from 6 to high school graduation, it seems clear that the IRDA attrition analyses may represent somewhat inflated estimates of the extent of dropouts because of the increased rate of retention of students in grade 9 (see Figure 5.3). Still the IDRA approach does have one virtue as compared with cohort progression analyses; namely, it attempts to adjust for net immigration of students into Texas schools. I return to this point later. But first let us compare the other three sources of evidence.
          The estimates of dropouts for 1996-97 based on 1995-96 grade enrollments and 1996-97 retention rates indicated that about 68,000 high school students dropped out of school between 1995-96 and 1996-97. Adding the missing students across the three grades to estimate longitudinal dropout rates suggests overall dropout rates of 27% across grades 10-12 (22.5% for White, 33.7% for Black and 38.5% for Hispanic students). These estimates correspond relatively well with the grade 6 to high school graduation cohort analyses (results of which were graphed in Figures 5.5 and 5.6). These results showed that of grade 6 students in the cohort class of 1997, 75.8% of White students and 61.1% of Black and Hispanic students graduated in 1997, implying that 24.2% of White and 38.9% of minority students did not graduate and may have dropped out. Overall for the cohort class of 1997, 31% of the students in grade 6 in 1990-91 did not graduate in 1997. (The 27% figure just cited is slightly lower, presumably because it does note take into account students who drop out between fall of grade 12 and high school graduation the following spring).
          Can these results be reconciled with the most recent NCES report on high school completion, based on CPS surveys? Recall that this report indicated that for 18 through 24 year-olds in Texas (not currently enrolled in high school or below) surveyed in October 1996-98, 80.2% reported completing high school. This implies a non-completion or dropout rate of 19.8%. (The CPS survey samples on which this estimate is based are not large enough to derive separate estimates by ethnic group.) It should be noted first that the CPS surveys of 18-24 year-olds in 1996-1998, do not correspond very precisely with the cohort of students in the Texas class of 1997. Nonetheless, two other factors may explain why the CPS derived non-completion (or dropout) estimate of 19.8% is so much lower than 31% estimate derived above for the class of 1997.
          One possibility suggested by a previous National Research Council report is that the CPS household surveys tend to under-represent minority youth generally and to underestimate high school dropout rates specifically. In discussing evidence on educational attainment of Black youth, Jaynes and Williams (1989, p. 338) comment that "after age 16, there are very serious, and perhaps growing, problems of surveying the black population, especially black men," and go on to discount a dropout estimate from CPS data from the 1980s for Blacks as simply not credible. If the CPS surveys do in fact under-represent minority youth, this would deflate the overall dropout estimates for Texas derived from this source, since all indications (even those from the TEA) are that dropout rates in Texas are higher for Black and Hispanic youth than for White youth. (Note 20)
          The other possibility, alluded to previously, is that the CPS surveys are based on self-reports of high school completion whether it be via normal high school completion or via alternative high school completion such as the GED testing. To explore this possibility, I consulted annual Statistical Reports from the GED Testing Service (1990-1998). Before presenting results from this source, it may be useful to explain the GED Testing program briefly.
          The Tests of General Educational Development were developed during World War II to provide adults who did not complete high school with an opportunity to earn a high school equivalency diploma. There are five GED tests: Writing Skills, Social Studies, Science, Interpreting Literature and the Arts, and Mathematics. States and other jurisdictions that contract to use the GED tests establish their own minimum scores for award of the high school equivalency diploma, with the condition that state minimum requirements cannot be below a floor approved by the Commission on Educational Credit and Credentials (an agency of the American Council on Education). For most of the past 10 years, the approved minimum was that examinees had to attain standard scores of at least 40 on each of the five GED tests or an average standard score of at least 45. "In the United States, this minimum standard of 'Minimum 40 or Mean 45' was met by an estimated 75% of the 1987 high school norm group." (GED Testing Service, 1995, GED 1994 Statistical Report, p. 31). In the early 1990s, four states were using this Commission-approved minimum passing standard on the GED tests for award of the high school equivalency degree: Louisiana, Mississippi, Nebraska, and Texas. An additional 27 states were using a similarly low "Minimum 35 and Mean 45" standard. The GED has been widely used in Texas; and in 1996, Texas became the first state in the nation to issue more than 1,000,000 GED credentials since 1971, when the GED started tracking this statistic" (GED Testing Service, 1997, GED 1996 Statistical Report, p. 27).
          About this time, in keeping with the national movement to raise educational standards, the GED Testing Service decided to raise the minimum passing score on the GED:

In concert with the secondary schools movement to raise standards, in January 1997 the GED Testing Service raised the minimum score required for passing the tests. The new standard is one that only 67 percent of graduating seniors can meet. (GED Testing Service, 1998, GED 1997 Statistical Report, p. ii). (Note 21) (Source: GED Testing Service, 1990-1999, Statistical Reports, 1989, 1990, 1991, 1992, 1994, 1996, 1997, 1998. Washington, D.C.: American Council on Education.)

GED policies which make it easier to get a GED are designed primarily to help high school dropouts. By doing so, however, they may have the perverse effect of encouraging additional students to drop out. This is because by making it easier to get a GED the policies may increase the expected earnings of high school dropouts and, therefore, increase dropout rates. . . .   In general less strict GED policies probably increase dropout rates. (Chaplin, 1999, p. 6).

7.2 Patterns of Grade Retention in the States

          Note that from the first source I took the grade 9 retention rate for 1995-96 or 1996-97, whichever was latest. Note also that the high school completion rates suffer from the problems discussed earlier regarding CPS data as a source of evidence on high school graduation and dropouts. Nonetheless even a casual inspection of these data reveals a clear pattern. States with the higher rates of grade 9 retention tend to have lower rates of high school completion. This pattern can be seen more clearly in Figure 7.3. (Note 27)
          Interestingly, Texas with a grade 9 retention rate of 17.8% has a slightly lower high school completion rate (80.2%) than we would expect given the overall pattern among the states shown in Figure 7.3--even though, as previously discussed this rate for Texas may well be inflated relative to other states because of the high rate of GED taking in Texas. Obviously, such a correlation between two variables, in this case, higher rates of grade 9 retention associated with lower rates of high school completion, does not prove causation, but such a relationship certainly tends to confirm the finding from previous research that grade retention in secondary school leads to higher rates of students dropping out of school before high school graduation

7.3 SAT Scores

          One possible explanation for why gains on TAAS do not show up in gains on the SAT is that increasing numbers of students in Texas have been taking the SAT over the last three decades. Not surprisingly, state officials in Texas have advanced this idea to explain the obvious discrepancy between dramatic gains on TAAS in the 1990s and the relatively flat SAT scores for students in Texas. To evaluate this possibility, we can look at numbers of students taking the SAT annually from 1972 to the present. It is indeed true that the numbers of students taking the SAT in Texas have increased faster (from around 50,00 annually through most of the 1970s to 100,000 in 1998) than nationally (from about 1 million annually to 1.2 million recently). However, it is also true that over this period the population of Texas has been increasing far faster than the U.S. population. Murdock et al. (1997, p. 12) report for instance that the population of Texas grew from 11.2 million in 1970 to 18.7 million in 1995 (a 67% increase) compared to a national population increase from 203 million to 263 million (a 29% increase). They also point out that the youth population of Texas in particular has been growing faster than the national youth group.
          A better way to evaluate the hypothesis that increases in SAT-taking in Texas explain the flat pattern in SAT scores is to compare the numbers of SAT-takers to the high school population. One such statistic is reported by the College Board, namely the percent of high school graduates taking the SAT. Figure 7.6 shows relevant data for the 50 states for 1999. Specifically, this figure shows state average SAT-M scores for 1999 compared with the percentage of high school graduates in 1999 taking the SAT.

          As can be seen in Figure 7.6, there is a clear relationship between these two variables. States with smaller percentages of high school graduates taking the SAT tend to have higher average SAT-M scores. States with larger percentages of high school graduates taking the SAT tend to have lower average SAT-M scores.
          What about Texas? According to College Board data, in 1999 Texas had 50% of high school graduates taking the SAT, scoring on average 499 on the SAT-M. This means that Texas, according to the pattern shown in Figure 7.6, has a slightly lower SAT-M average than states with comparable percentages of high school graduates taking the SAT. For example, according to the 1999 College Board data, there were seven states that had between 49% and 53% of high school graduates taking the SAT (Alaska, California, Florida, Hawaii, Oregon, Texas and Washington). Among these states Texas had the lowest SAT-M average in 1999 (499), except for Florida (498). Leaving aside Florida, Texas had an SAT-M average 15-25 points below states with comparable percentages of high school graduates taking the SAT. These results clearly indicate that the relatively poor standing of Texas among the states on SAT scores cannot be attributed to the proportion of secondary school students in Texas taking the SAT.
          Moreover, the College Board data may actually understate the relatively poor performance of Texas students on the SAT. This is because Texas has such a poor record regarding student progress to grade 12 and graduation. Even if we use the very conservative estimates of high school completion derived from CPS data (and reproduced in Table 7.2 above) we see that Texas has a rate of non-completion of high school among young adults of about 20%—more than 5 percentage points above the national rate (and as the discussion in Section 7.1 indicated, this figure surely underestimates the extent of the high school dropout problem in Texas).

7.4 NAEP Scores Revisited

• 1990 Mathematics, grade 8
  
• 1992 Mathematics, grades 4 and 8
  
• 1992 Reading, grade 4
  
• 1994 Reading, grade 4
  
• 1996 Mathematics, grades 4 and 8
  
• 1996 Science, grade 8
  
• 1998 Reading, grades 4 and 8
  
• 1998 Writing, grade 8

The Developmental Status of Achievement Levels

The 1994 NAEP reauthorization law requires that the achievement levels be used on a developmental basis until the Commissioner of Education Statistics determines that the achievement levels are "reasonable, valid, and informative to the public." Until that determination is made, the law requires the Commissioner and the Board to make clear the developmental status of the achievement levels in all NAEP reports.

In 1993, the first of several congressionally mandated evaluations of the achievement-level-setting process concluded that the procedures used to set the achievement levels were flawed and that the percentage of students at or above any particular achievement level cut point may be underestimated. Others have critiqued these evaluations, asserting that the weight of the empirical evidence does not support such conclusions.

In response to the evaluations and critiques, NAGB conducted an additional study of the 1992 achievement levels in reading before deciding to use those levels for reporting 1994 NAEP results. When reviewing the findings of this study, the National Academy of Education (NAE) Panel expressed concern about what it saw as a "confirmatory bias" in the study and about the inability of this study to "address the panel's perception that the levels had been set too high."

In 1997, the NAE Panel summarized its concerns with interpreting NAEP results based on the achievement levels as follows: "First, the potential instability of the levels may interfere with the accurate portrayal of trends. Second, the perception that few American students are attaining the higher standards we have set for them may deflect attention to the wrong aspects of education reform. The public has indicated its interest in benchmarking against international standards, yet it is noteworthy that when American students performed very well on a 1991 international reading assessment, these results were discounted because they were contradicted by poor performance against the possibly flawed NAEP reading achievement levels in the following year."

The NAE Panel report recommended "that the current achievement levels be abandoned by the end of the century and replaced by new standards . . . ." The National Center for Education Statistics and the National Assessment Governing Board have sought and continue to seek new and better ways to set performance standards on NAEP. For example, NCES and NAGB jointly sponsored a national conference on standard setting in large-scale assessments, which explored many issues related to standard setting. Although new directions were presented and discussed, a proven alternative to the current process has not yet been identified. The Acting Commissioner of Education Statistics and NAGB continue to call on the research community to assist in finding ways to improve standard setting for reporting NAEP results. The most recent congressionally mandated evaluation conducted by the National Academy of Sciences (NAS) relied on prior studies of achievement levels, rather than carrying out new evaluations, on the grounds that the process has not changed substantially since the initial problems were identified. Instead, the NAS Panel studied the development of the 1996 science achievement levels. The NAS Panel basically concurred with earlier congressionally mandated studies. The Panel concluded that "NAEP's current achievement-level-setting procedures remain fundamentally flawed. The judgment tasks are difficult and confusing; raters' judgments of different item types are internally inconsistent; appropriate validity evidence for the cut scores is lacking; and the process has produced unreasonable results."

The NAS Panel accepted the continuing use of achievement levels in reporting NAEP results only on a developmental basis, until such time as better procedures can be developed. Specifically, the NAS Panel concluded that ". . . tracking changes in the percentages of students performing at or above those cut scores (or in fact, any selected cut scores) can be of use in describing changes in student performance over time." In a recent study, eleven testing experts who provided technical advice for the achievement-level-setting process provided a critical response to the NAS report.

The National Assessment Governing Board urges all who are concerned about student performance levels to recognize that the use of these achievement levels is a developing process and is subject to various interpretations. The Board and the Acting Commissioner of Education Statistics believe that the achievement levels are useful for reporting on trends in the educational achievement of students in the United States. In fact, achievement level results have been used in reports by the President of the United States, the Secretary of Education, state governors, legislators, and members of Congress. The National Education Goals Panel and government leaders in the nation and in more than 40 states use these results in their annual reports.

However, based on the congressionally mandated evaluations so far, the Acting Commissioner agrees with the recommendation of the National Academy of Sciences that caution needs to be exercised in the use of the current achievement levels. Therefore, the Acting Commissioner concludes that these achievement levels should continue to be considered developmental and should continue to be interpreted and used with caution. (Greenwald et al., 1999, pp. 14-16)

The TLI is very much like the T-score previously described. Unlike the T-score, however, the TLI is anchored at the exit level passing standard, rather than at the mean of the distribution. To distinguish between the [Rasch] scale score system and the TLI, TEA chose a two-digit metric for the TLI so that it anchored at the exit level passing standard with a value of 70 and a standard deviation of 15. (TEA, the 1996-97 Texas Student Assessment Program Technical Digest, p. 34)

          The last column in Table 7.4 shows the 1994 to 1999 gains on TAAS divided by the relevant TAAS test standard deviation (15 for the reading and math TAAS tests and 200 for the TAAS exit level writing test). These results, average test score changes divided by the relevant standard deviations, may be interpreted as effect sizes.
          Before discussing the meaning of the results shown in the last column of Table 7.4, a brief summary of the idea of effect size may be helpful. (Yes, dear reader, yet another digression. But if you know about meta-analysis and effect size, just skip ahead.) The concept of effect size has come to be widely recognized in educational research in the last two decades because of the increasing prominence of meta-analysis. Meta-analysis refers to the statistical analysis of the findings of previous empirical studies. With the proliferation of research studies on particular issues, statistical analysis and summary of patterns across many studies on the same issue have proven to be a useful tool in understanding patterns of findings on a research issue (Glass, 1976; Cohen, 1977; Glass, McGaw & Smith, 1981; Wolf, 1986; Hunter & Schmidt 1990, and Cooper & Hedges, 1994 are some of the basic reference works on meta-analysis). In meta-analysis, effect size is defined as the difference between two group mean scores expressed in standard score form, or—since the technique is generally applied to experimental or quasi- experimental studies—the difference between the mean of the treatment group and the mean of the control group, divided by the standard deviation of the control group (Glass, McGaw & Smith, 1981, p. 29). Mathematically this is generally expressed as:

          As can be seen here, the gap between the average NAEP scores of White students in Texas and those of Black and Hispanic students is fairly consistently in the range of 25 to 35 points. There is a tendency for Hispanic students in Texas to score slightly better on NAEP tests than Black students; but overall, Hispanic and Black students in Texas score on average between two-thirds and a full standard deviation below the mean of White students. Moreover, at grade 4, there is an increase in the White-minority gap in NAEP reading scores between 1992 and 1998. In 1992, the NAEP grade 4 reading average was 224 for White students, 200 for Black students and 201 for Hispanics. By 1998, the corresponding averages were 233, 197 and 204.
          At this point, the reader may begin to doubt the consistency of my approach to data analysis. In Section 4.1, when discussing the issue of adverse impact, I applied three tests of adverse impact: the 80% rule, tests of statistical significance, and evaluation of practical significance of differences. The critical reader may well wonder whether, if I applied these same standards to the NAEP results for Texas, and in particular the 1996 NAEP math results for math, I might so easily dismiss the significance of the gains apparent for Texas.
          Apparent gains for Texas in NAEP math scores between 1992 and 1996 were indeed statistically significant. And in terms of practical significance, critical readers may well be asking themselves, even if the gains were not large in terms of the standard deviation units perspective suggested in the meta-analysis literature, gains on the order of a third of standard deviation, when apparent for a population of a quarter million students (roughly the number of fourth graders in Texas in 1996), are surely are of practical significance. Also, it may be recalled from Section 3.4 above that the NAEP math gains for Texas fourth graders between 1992 and 1996 were greater than the corresponding gains for any other state participating in these two NAEP state assessments. So any reasonable person must concede that the apparent improvement of Texas grade 4 NAEP math average from 217.9 in 1992 to 228.7 in 1996 (a gain of about one-third of a standard deviation), if real, is indeed a noteworthy and educationally significant accomplishment.
          But there is that "if." The other perspective not yet brought to bear in considering changes in NAEP test score averages is advice offered in Part 1. When considering average test scores, it is always helpful to pay attention to who is and is not tested.
          NAEP seeks to estimate the level of learning of students in the states not by testing all students in the states in a particular grade, but through use of systematic and representative sample of schools and students. Without getting into details of NAEP sampling, let us focus here on the fact that not all students sampled are actually tested. Some students selected for NAEP testing are excluded because they are limited English proficient (LEP) or because of their status as special education students, whose individualized education plans (IEPs) may call for them to be excluded from standardized testing.
          NAEP researchers have long recognized that exclusion of sampled students from NAEP testing has the potential to create bias in NAEP results. Here is how one NAEP report discussed the issue:

The interpretation of comparisons of achievement between two or more assessments depends on the comparability of the populations assessed at each point in time. For example, even if the proficiency distribution of the entire population at time 2 was unchanged from that at time 1, an increase in the rate of exclusion would produce an apparent gain in the reported proficiencies between the two time points if the excluded students tend to be lower performers. (Mullis et al., 1993, p. 353).

          Grade 4:           0.95(228.7) + 0.05(190.4) = 226.9

          Grade 8:           0.97(270.2) + 0.03(225.5) = 268.9

When it comes to educational achievement, by nearly any measure except TAAS, Texas looks a lot like America. Texas was near the national average on many measures of educational performance when TAAS was introduced—and remains there. (Rodamar, 2000, p. 27).

7.5 Other Evidence

          What have been the results of this "college readiness" testing program? I found the graph reproduced in Figure7.7 in a report available on the same website.

Reviewing these results from TASP testing, and comparing them with results of TAAS testing (see Figure 3.1 for example), the conclusion seems inescapable that something is seriously amiss in the Texas system of education, the TAAS testing program or the TASP testing program—or perhaps all three. Between 1994 and 1997, TAAS results showed a 20% increase in the percentage of students passing all three exit level TAAS tests (reading, writing and math). But during the same interval, TASP results showed a sharp decrease (from 65.2% to 43.3%) in the percentage of students passing all three parts (reading, math, and writing) of the TASP college readiness test.