The Achievement Gap : Should We Rely on SAT Scores to Tell Us Anything About It ?

Increasing numbers of students taking the SAT have declined to identify their race/ethnicity. I examined the impact of nonrespondents on the validity of reported racial/ethnic differences and year-to-year changes in test performance. Using an analysis reported by Wainer (1988) and SAT data from 1996 to 2003, I confirmed Wainer’s findings that non-respondents prevent accurate estimations of group differences based on SAT data. I then explored the impact of College Board press release information on news reports about the achievement gap. I found frequent reports of racial/ethnic differences in SAT scores and year-to-year changes in scores but negligible consideration of non-respondents. Press releases and media reports should


Introduction
The term "achievement gap" has taken on particular and important meanings in the past decade."The achievement gap" has become a shorthand way to refer to differences in academic achievement between European Americans and members of minority groups who historically have been disenfranchised.For some, the gap refers exclusively to differences between African Americans and European Americans.For others, it refers to a broader group of students: those who aren't facile in English, the poor, or members of other disadvantaged ethnic groups.
Regardless of who is included in one's definition, the literature abounds with descriptions of gaps in student performance on test scores, which are probably the most commonly used indicators of student achievement (Jencks & Phillips, 1998;Herrnstein & Murray, 1994;Lee, 2000;Kober, 2001).Additional evidence of a gap has been found in data related to other indicators of achievement, such as grades, dropout rates, college attendance or earnings.(Portes, 1996;Ferguson, 2001;Kane, 1998;Vars & Bowen, 1998;Johnson & Neal, 1998;Roderick & Cambrun, 1999) National organizations of schools dedicated to closing the gap, such as the Minority Student Achievement Network, have been formed to address this phenomenon.The benchmark against which minority achievement is measured is White/European-American performance, and closing the gap usually means increasing minority achievement relative to that of White/European Americans.
Reasons for the gap have been explored; the resulting explanations have been legion.Some have focused on the historical legacy of racism, prejudice and segregation (Spring, 2000;House, 1999).Others have examined factors affecting children's readiness for school and their ability to learn in school, such as the role of poverty and other differences associated with SES (Ferguson, 2001;Barton, 2003), cultural differences in language or in adaptation to school (Mercado, 2001, Portes, 1996;Portes, 1999, Ogbu, 1999;Ogbu, 2003), family and parenting (Okagaki & Frensch, 1998;McAdoo, 1978).Another arena of inquiry has focused on how well education serves its varied constituents.Some have looked at inequities in resources and opportunity to learn (Kozol, 1991;Barton, 2003;Mickelson, 2001); others have looked at how schools and educators respond or relate to student diversity (Spring, 2000;Pollock, 2001;Ferguson, 1998;Delpit, 1996;Cohen & Steele, 2002), or how they encourage (or fail to encourage) academic excellence (Ogbu, 2003;Barton, 2003).Finally, many have discussed the qualities of tests or teachers' assessments of students that can contribute to extraneous differences, i.e. error and bias.(Airasian, 2001, Gould, 1981;Valencia & Suzuki, 2001) One common research focus has been change in the gap over time, and change typically is examined using test data.Studies have traced the ups and downs of minority student test performance, usually compared to that of European Americans.These studies employ various types of test scores, most often using NAEP and college admissions scores such as the SAT.(Kober, 2001;Lee, 2000;Miller, 2003;Powell & Steelman, 1996) For several decades, NAEP report cards have been issued that describe student achievement in eight subject areas; these report cards not only provide a current snapshot, but also depict longitudinal change.The accountability movement has triggered issuance of school, district and state report cards, and with the passage of the No Child Left Behind act, these report cards must include disaggregated data and report differences in test performance from one year to the next.Finally, each summer, the College Board and ACT issue reports on student performance on college admissions tests, notably the SAT and ACT assessment.These reports link student achievement to a variety of variables including race/ethnicity.NAEP reports are based on the results of samples that were selected to be representative of the national population of students.Data from state achievement tests are based on the performance of all students at a particular grade level.However, college admissions tests are administered on a self-selected set of students who have interest in attending college.This self-selection means that the data are not representative of students in the nation or a particular state in any year.Furthermore, the percentage of high school students who intend to apply to college fluctuates from year to year.One consequence is that year-to-year differences in student performance can be based on true changes in student knowledge or ability, on changes in who takes the test, or both.These sampling issues present serious problems with using college admissions test scores to make inferences about the academic qualifications of students in the nation, to make comparisons among states, or to track year-to-year differences in test performance.ACT and the College Board are careful to warn against using SAT or ACT test results to make comparisons among states (College Entrance Examination Board, 2002;ACT, 2003).ACT's web site includes information briefs that explain how changes in a school's test performance over time can be influenced by a variety of factors, including who decides to take the test (ACT, 2004).The College Board's guidelines on test use include a warning against using aggregate data to make judgments, not only about states, but also about schools or districts.However, they are less explicit about comparisons among groups of students disaggregated by, among other things, race/ethnicity (College Entrance Examination Board, 2002).
Racial/ethnic comparisons of SAT or ACT data are particularly suspect because each year a substantial portion of students do not report their race/ethnicity on the survey questionnaires that they complete when registering for the tests they are about to take.This is not a recent problem.Fifteen years ago, Howard Wainer (1988) examined the impact of this missing information on the accuracy of racial/ethnic comparisons using data from SAT's administered from 1981 to 1985.Among his findings are the following: The percentage of students not reporting their race/ethnicity (to be called non-respondents here) was substantial each year (12 to 14%)--large enough to be called the "second largest minority group" taking the test.Non-respondents, as a group, were not similar to or representative of the students who did report their race/ethnicity.While there were variations in the performance of non-respondents from year to year and in their performance relative to the nation, their underperformance was somewhat consistent (22 to 26 points below the national SAT-V means and 21 to 28 points below the national SAT-M mean).The error caused by the missing racial/ethnic information for the non-respondents overwhelmed any differences detected each year among disaggregated groups as well as any changes in test performance gaps over time.
He concluded, "…the nonresponse to ethnic identifiers is sufficient to introduce noise of a greater magnitude than the changes being interpreted as real."(Wainer, p. 778).
Wainer's data suggest that the non-respondent group, while not representative of those who did report their race/ethnicity, were nevertheless comparatively stable in terms of their numbers and test performance.Since the mid-1990's, however, this group has changed considerably.
Table 1 reports the numbers of students (in thousands) in each group taking the test each year covered by this study.It also reports the combined number of non-white respondents.Examination of the table reveals that, with the exception of American Indians, the numbers of students in each non-white group increased since 1996.By contrast, the numbers of white students increased, then decreased.Finally, both the proportion and number of non-respondents more than tripled.Figure 1 reports change in the percentage of examinees who were non-respondents on the question of racial/ethnic identity.In the mid-nineties, the percentages resembled percentages in the data reported by Wainer (1988), but this group has steadily increased so that by 2003, they made up one-fourth of all those taking the test.In the process, they became, to use Wainer's parlance, the largest minority group taking the SAT.It should be noted that throughout this time period, non-respondents were the only "racial/ethnic" group that has been majority male.

Figure 1. Percentage of SAT 1 Examinees Not Reporting Their Race/Ethnicity
In sum, then, the recent escalation of interest in the achievement gap, continued use of tests to describe that gap, and the dramatic growth of the non-respondent group raise two questions.First, how have students who declined to indicate their race/ethnicity from 1996 to 2003 resembled those reported by Wainer (1988) and how has this affected the validity of his conclusion regarding the amount of noise that overwhelms the use of SAT scores as an index of the achievement gap and changes in that gap?Second, are there practices in releasing and disseminating information about college admissions test scores and racial/ethnic performance on those tests that seem to foster inappropriate use of these scores when describing the achievement gap to the public in general and the education establishment?
To answer the first question, I used data released on the College Board's web site to track the test performance of non-respondents and to replicate the study conducted by Wainer (1988).
To answer the second question, I examined the College Board's press release information from the summer of 2003, each state's data on non-respondents, press releases about the SAT score results for 2003 issued by state departments of education throughout the United States, and articles from selected local and national newspapers about SAT results.I also revisited articles about the achievement gap that have included SAT information.In all cases, I looked for two kinds of information: 1) data disaggregated by race/ethnicity and discussion of that data and 2) information about non-respondents and their impact on our ability to make inferences about racial/ethic groups or differences between groups.

Question 1
How have students who declined to indicate their race/ethnicity from 1996 to 2003 resembled those reported by Wainer (1988), and how has this affected the validity of his conclusion regarding the amount of noise that overwhelms the use of SAT scores as an index of the achievement gap and changes in that gap?
Method.Using College-Bound Seniors data reported annually by the College Board on its web site, I determined how many students took the test each year and how they performed on the test.This permitted a comparison with those examinees employed in Wainer's analysis.Then I employed Wainer's procedure for estimating the representation of white students in the non-respondent group and determining the degree to which these non-respondents were affecting the validity of examining test performance disaggregated by race/ethnicity.
For this procedure, Wainer used the numbers of students in each racial/ethnic group and the mean SAT-V and SAT-M scores for each group to develop estimates of the percentage of non-respondents who were white.He employed this approach to test the assumption that the mean scores of the non-respondents from each group were the same as those of students who reported their race/ethnicity.Were this to be the case, then the reluctance of some to report their race/ethnicity would have little or no effect on the validity of differences between groups that were found for those based who did report their race/ethnicity.
For each year, he made two estimates, one based on the math scores and the other based on the verbal scores.These estimates were compared for consistency.The degree of difference between them indicated how much the assumption that non-respondents were like those who responded was violated.It should be noted that this estimating procedure yields only rough estimates.
Wainer estimated the proportion of whites in the non-respondent group using the following formula: Where P = the proportion of white non-respondents, (1-P) = the proportion of non-white non-respondents, M SATwhite = SAT mean of self-identified white examinees M SATnonwhite = SAT mean of examinees identifying themselves as members of another group, and M SATnonresponse = SAT mean of non-respondent examinees.
Like Wainer, I used this formula to estimate the percentage of white non-respondents twice, once using SAT-Verbal scores and once using SAT-Math scores.I repeated this analysis for each year from 1996 to 2003.
Findings.Table 2 reports the SAT I mean scores for the verbal portion of the test for each year from 1996 to 2003.In general, the amount of change for any self-identified group over that time period was negligible.The largest change for self-identifying students was an increase in 12 scale-score points in the mean for Asian American students.In contrast, the change for non-respondents was dramatic, an increase of 24 scale-score points.This change in non-respondents' mean scores also contrasts with the means reported by Wainer.The difference between the highest and lowest non-respondent means in that study was 10 points, and there was no discernable pattern of change.Another way to look at non-respondents' SAT performance is to examine how they compared to the national mean.Figure 2 shows that the SAT I verbal performance of non-respondents steadily increased so that by 2003, their mean score slightly exceeded that of the national mean.In other words, not only has this group grown, but it now includes more high-performing examinees.As can be seen in Table 3, the largest increase in mean scores for any identified racial/ethnic group was that for Asian-American scores, an increase of 17 scale-score points.By contrast, the mean for non-respondents increased by 31 scale-score points.The comparison to the change in the national mean depicted in Figure3 shows an increase in scores, with the non-respondents outperforming the national group of examinees in 2003.In other words, the non-respondents became an increasingly able group with respect to math from 1996 to 2003, a pattern not evident in the math scores reported for 1980 to 1985 by Wainer (1988).These changes in the non-respondents' mean test performance on the verbal and math portions of the SAT I , in addition to the dramatic increase in their numbers, suggest that there have been substantial and steady changes in the non-respondent group since the mid-1990's.These changes not only made the group more variable over time, compared to their counterparts in the early 1980's, but also reinforce the need to re-estimate the composition of this group with respect to race/ethnicity and compare it to the composition of those who do report their race/ethnicity.
Using the same procedure employed by Wainer (1988), I estimated the percentage of whites in the non-respondent group twice for each year covered by this study, once based on verbal scores and once based on mathematics scores.Then I used the difference between the verbal and mathematics-based estimates to calculate the difference in the number of estimated white examinees in the non-respondent group for each year.Table 4 reports the results.It should be noted, furthermore, that the implications of this finding cannot be applied in a consistent fashion to each of the states.The map in Figure 4, developed using SYSTAT's boundary map of the continental United States (SYSTAT, 2000), shows the range of percentages of SAT examinees declining to report their race/ethnicity by state.The percentages range from a low of 12.4 percent (North Dakota) to a high of 30.4 percent (Connecticut), and the percentage of non-respondents is moderately correlated (r =.61) with the participation rate reported for each state.Suffice it to say that this analysis would need to be carried out for each state in order to determine the impact non-respondents have on the racial/ethnic findings in that state.

Question 2.
Are there practices in releasing and disseminating information about college admissions test scores and racial/ethnic performance on those tests that seem to foster inappropriate use of these scores when describing the achievement gap to the public in general and the education establishment?
The initial source of information about SAT results for 2003 appears in a College Board press release and associated tables, charts and reports that were issued on August 26, 2003(College Board, 2003).The press release includes text reporting overall changes in performance and participation, differences by gender, changes in performance by members of various ethnic groups, upcoming changes in the SAT Verbal test and a "snapshot" of test takers.
Race/ethnicity is included in statements regarding the diversity of the test-taking population: This year saw the largest increase in the number of SAT takers in more than 15 years.Thirty-eight percent of SAT takers are first-generation college-bound students.The proportion of minority students taking the SAT is at an all-time high of 36 percent, up 1 percentage point from last year and 6 points from 10 years ago.
"Higher SAT scores, a record number of test-takers, and more diversity add up to a brighter picture for American education.While we certainly need to make more progress, the fact remains that we are clearly headed in the right direction," said College Board President Gaston Caperton.
Thirty-six percent of SAT takers in the class of 2003 were minorities.The number of Mexican American SAT takers increased by 56 percent between 1993 and 2003.SAT takers in the Other Hispanic category increased by 50 percent during the same period.
Race/ethnicity also figured into statements about test performance: The overall verbal scores were aided by a strong showing from Asian American SAT takers, whose mean verbal scores were, for the first time, higher than the national mean.Additionally, Mexican American and African American SAT takers improved their average scores by two points and one point, respectively, from a year ago.In fact, virtually all ethnic and racial groups showed stronger performance on their verbal scores compared to a year ago.
Accompanying the text of the press release is a set of 18 graphs and tables depicting, among other things, the diversity of the test-takers (in one table and one graph), the changes in college bound students over time (four graphs/tables), plus racial and ethnic differences in high school preparation, grades and test performance (eight graphs/tables).Nowhere is there a statement about non-respondents, nor are they included in any table or graphic in the press release.This information does appear in Table 4-1 on page 10 of the College Bound Seniors report that can be accessed in a box entitled "Archives" next to the press release documentation on the web site.
In other words, while it is possible to find out that many examinees are declining to report their race/ethnicity, it is not evident in the materials featured as part of the press release.Furthermore, the language of the press release and its featured tables and graphs makes assertions about minority and non-minority SAT test takers that may not be true.The number or percentage of students declining to indicate their race/ethnicity appears in five of these press releases and accompanying materials.In three cases, tables include non-respondent numbers.The Texas press release reports an increase in the percent of non-respondents in Texas and the nation.
The most detailed set of information appears in the Florida press release and accompanying report entitled SAT Trends: Florida and the Nation.The press release itself merely comments on the change in scores compared to 2002: "Florida's average verbal score rose two points, due largely to higher scores among Hispanic, African-American and Asian males."The report's summary includes several bulleted items about the racial/ethnic composition of the test takers.
Example: "Nationwide, the percentage of minority test takers has also been increasing, but at a slower rate than in Florida.In 1988, minorities represented 23% of the test takers nationwide, about the same as in Florida; by 2003 the percentage had increased to 36%, with Asians, whose scores are typically well above average, representing 9.6%, compared to 4.4% for Florida."(p.i) SAT Trends' introductory summary includes a set of bulleted items entitled " SAT Scores by Racial-Ethnic Groups."Not all of these bulleted items discuss test performance; they also cover diversity of the test-taking group, first language, and income.One item provides the most detailed discussion from any press release about non-respondents and how they have changed over time: An increasing percentage of test takers are declining to indicate their race-ethnicity.
In 2002, 19% of test takers in both Florida and the U.S. did not do so.The number of non-respondents in 2003 rose to 24% for Florida and 25% for the U.S….In past years, those who did not provide this information had lower average scores than those who did.In 2003 the trend was reversed…This break in the trend makes any changes in scores by race problematic…." (p.iv).
Despite this warning, however, five of 21 pages of tables included in this report focus on racial/ethnic differences in one way or another.
A content analysis of a representative sample of newspapers or televised news reports is beyond the scope of this research.However, I did gather articles about the 2003 release of SAT scores from newspapers from various cities in the United States; many of them have a national presence in the sense that they are distributed nationally, or other journalists frequently cite them as sources of news information.All 18 of these newspapers published at least one article on the SAT.Twelve of them discussed racial/ethnic performance in one or more of the following ways:

Akron Beacon Journal Atlanta-Journal Constitution Boston Globe
Differences among groups at the national level Differences among groups at the state or local level Changes in group members' performance compared to last year or past years The alleged presence of bias in the test (cited FairTest) Possible causes for differences, such as differences in school funding, SES, or inequalities in education based on race, ethnicity or income Ten of them discussed the participation of students of various racial/ethnic groups in the College Board testing program.Some articles mentioned the level of participation; others discussed increases in participation.A quotation from the president of the College Board, Gaston Caperton, was often included: "Higher SAT scores, a record number of test takers, and more diversity add to a brighter picture for American education." News sources designed to serve a national audience, either a lay audience or one of educators, were mixed in their reporting on SAT scores for the class of 2003.Major national news magazines-Time, Newsweek, and U.S. News and World Report-did not report SAT results.However, USA Today included SAT results in its August 27 issue and of the 15 paragraphs in the article, five discussed the achievement gap.Education Week, in an article dated September 3, 2003, reported on the increase of SAT and ACT examinees and the questionable level of college preparation coursework these students have had.Part of the article focuses on the level of high school courses these students have taken; the last half of the article focuses on racial/ethnic differences in test performance and why such differences exist.Black Issues in Higher Education published an article on September 11 that, like the Education Week article, discusses differences in college preparation for students from various racial/ethnic groups.

Discussion
Taken together, the College Board's press release, those of various states, and news reported from large urban and national news sources suggest that the information the public reads about the SAT is grounded in what the College Board reports in its press release about SAT results.If one were to depend on the information provided in the press release without referring to the College Bound Seniors tables, several incorrect inferences could be made: That the magnitude of "achievement gaps" among racial/ethnic groups can be determined from SAT data That changes in the SAT performance of students from various racial/ethnic groups can be determined That diversity of those taking the SAT can be determined and tracked over time That racial/ethnic differences in courses taken, income, first-generation college enrollment, high school GPA can all be determined from College Board information Wainer's results make it clear that even when the proportions of non-respondents was smaller, there were enough of them to create noise that overwhelms any evidence of change over time in test scores reported for students from various racial/ethnic groups or any evidence of group differences based on SAT results.The results of this study not only confirm Wainer's results but also show that the noise generated by recent groups of non-respondents has increased from overwhelming to deafening.
While Wainer does not discuss inferences about racial/ethnic differences of other kinds, the magnitude of the non-respondent rate and the change in this group's test performance suggest that the other inferences associated with racial/ethnic differences reported in the College Board's press release should be treated as suspect as well.The graphs in Figure 4 illustrate why this should be the case.These results suggest that non-respondents prevent valid use of data from College Bound Seniors to describe majority and minority test-takers with respect to: The size of the achievement gap Changes in the magnitude of the achievement gap The diversity of those taking the SAT Differences in GPA among racial/ethnic groups First-generation college attendance High school preparation

Recommendations
The combined results from this study suggest several further steps.
First, we need to find out more about these non-respondents.Who are they?Why are they declining to indicate their race/ethnicity?Several hypotheses have surfaced.
One idea, derived from the growth in resistance to affirmative action, is that the non-respondents are white males who see no benefit in reporting their race/ethnicity.It is documented that non-respondents are majority male and both sets of estimates appearing in Table 4 suggest a dramatic increase in white non-respondents.
A second suggestion is that students from various groups are resisting reporting their race/ethnicity as a protest against the use of a category they regard as arbitrary and grounded in mistaken notions about race.
A third suggestion is based on findings of Claude Steele and others pertaining to stereotype threat.(Aronson, 2002;Steele & Aronson, 1998) The notion that reporting one's race during a test affects test performance may be filtering its way into the middle-class, professional African-American community.Hence, increasing numbers of high-achieving minority students may not be revealing their race on test applications.
In all probability, there are multiple reasons, as might be inferred from a brief analysis of the 2003 data from my district.It was possible to identify the race/ethnicity of all students, match that to their SAT scores and then compare this racial/ethnic and test score data to that from the College Board's College Bound Seniors 2003 report for Shaker Heights High School The data from the College Bound Seniors report for the high school reveal a 23 percent non-response rate in 2003, which is similar to our state and the national data in terms of percentage.Our non-respondents, unlike their national and state counterparts, are majority female.Compared to those reporting their race/ethnicity, a disproportionate number of our non-respondents are White/European-American.Eighty-two percent of them are White; 18 percent are Black/African-American.By contrast, 36 percent of our high school's SAT test takers in the class of 2003 are Black/African-American; 59 percent of them are White/European-American.
The non-respondent verbal mean was higher than the overall mean, and both groups of non-respondents had higher SAT-V scores than their counterparts.The math scores, on the other hand, present a contradiction.While the White non-respondents had higher SAT-M scores, their Black/African-American counterparts had lower SAT-M scores.
Second, while descriptions of racial/ethnic differences and score changes over time have always been questionable, the sheer size of the non-respondent group makes use of SAT data for such purposes irresponsible.At the very least, test sponsors and state departments of education need to be clear in their press releases about the presence of non-respondents and their impact on one's ability to make inferences about students taking college admissions tests in the same way they warn about other misuses of test results.The College Board and ACT provide very clear cautions against using their data to make inferences about state rankings, for example, and these cautions seems to have taken hold.Education departments' press releases and newspaper articles often make adjustments that seem to heed this caution.For example, they reported data only for other states with similar participation rates, or they warn readers that participation rate and mean scores are strongly related to each other.Furthermore, test companies' press releases themselves need to provide clear information about non-respondents-the number of them, their known characteristics compared to respondents (i.e.test performance, gender).
Finally, those who have used these data in news articles, policy papers, or published research need to reconsider their use.The difference in the opportunities and achievement of children from various racial/ethnic groups is perhaps the most important issue facing education today.We need to examine and describe these differences, but we need to do it with data that we can count on to provide an accurate picture.Alternative data sources such as state data and NAEP results can do the job much more effectively.

Figure 2 .
Figure 2. Comparison of SAT Verbal Scores: National Mean vs. Mean of Those Not Reporting Their Race/Ethnicity Examination of the SAT I results for the mathematics portion of the test yields similar results.As can be seen in Table3, the largest increase in mean scores for any identified racial/ethnic group was that for Asian-American scores, an increase of 17 scale-score points.By contrast, the mean for non-respondents increased by 31 scale-score points.The comparison to the change in the national mean depicted in Figure3 shows an increase in scores, with the non-respondents outperforming the national group of examinees in 2003.In other words, the non-respondents became an increasingly able group with respect to math from 1996 to 2003, a pattern not evident in the math scores reported for 1980 to 1985 by Wainer (1988).

Figure 3 .
Figure 3.Comparison of SAT Math Scores: National Mean vs. Mean of Those Not Reporting Their Race/Ethnicity

Figure 4 .
Figure 4.U.S. Map Showing Percentage of Examinees Declining to Specify their Race/Ethnicity on the Student Descriptive Questionnaire Note 1 Note: Data source: non-respondent data reported by the College Board (2003).

Figure 5 .
Figure 5. Breakdowns of the Race/Ethnicity of SAT-1 Test Takers from the Class of 2003 Based on Different Estimates Note 2

Table 4 . Estimates of Percent of White Non-Respondents based on SAT-V and SAT-M and Difference in the Estimated Number of White Non-Respondents, based on the two estimates
The range of non-respondents who are estimated to be White based on the verbal and math means is much more varied for the verbal portion of the test for this time period (33.6% to 70.0%) than for the time period reported by Wainer (43% to 51%).For the mathematics portion of the test, the estimates based on recent means tend to be larger and more varied (34.4% to 81.7%) compared to those reported by Wainer (1988) (29% to 46%).The differences in the estimated number of white students ranged from very small (632) to more than double the largest estimate reported byWainer (1988) (41,507 vs. 18,000).Wainer develops a hypothetical "extreme case" to illustrate the consequence of these differences on the "real" means of African Americans, compared to Whites.By holding numbers of examinees and means constant, he estimated that the 1980 mean verbal score for African Americans would have to change about 16 points in order for the percentage of non-white respondents based on the SAT-V score mean to equal the estimated percentage of non-white non-respondents based on the SAT-M score mean.Based on 1984 data, the requisite change would have to be 35 points.Following Wainer's lead, I found that the smallest difference, based on 1996 data, would be 1.8 scale-score points, slightly larger than the 1-point change reported for 2002 to 2003.For all other years, the differences are much larger, ranging from 17 points in 1998 to 122 points in 2003.Hence, Wainer's conclusion that the error associated with non-respondents dwarfed the size of any year-to-year gains reported is confirmed.
The article includes two tables, one based on ACT scores, the other based on SAT scores, that report means for disaggregated groups.Of note, the ACT table includes two additional categories: "Prefer not to respond" and "No Resp.".The College Board table does not.Last, but not least, the United States Department of Education, on August 28, 2003, issued a statement about SAT scores as evidence of the achievement gap and disparities in the education system.In his October 8, 2003, prepared remarks for the High School Leadership Summit, Secretary of Education Rod Paige stated that the disparity continues between the education of disadvantaged or low-income students and that of other students.The speech goes on to use SAT performance of African American and Hispanic students as the sole illustration of this point.The only news source that reported on the presence of students who declined to indicate their race/ethnicity was The Miami Herald.Its August 29, 2003, article on the SAT scores of Broward County School District high school seniors states: "Results were broken down by race and ethnicity, but it was difficult to determine how Broward was doing in narrowing the achievement gap between black and Hispanic and white and Asian students.The reason: Almost 1 in 4 students chose not to list their race or ethnicity.The number of students who didn't answer increased by almost 100 percent."Thesource of this insight is unclear.The article refers to Katherine Blasik, Broward's director of research and evaluation.It also reports state results, indicating, perhaps, familiarity with Florida's report on SAT trends.Nevertheless, the readers of the Miami Herald were cautioned not to look at the SAT as a source of information about the achievement gap.

Table 4 .
The first two breakdowns report take-takers based on data reported in the College Board's press release and College Bound Seniors for the class of 2003; the third and fourth breakdowns make use of the estimates of white non-respondents fromThe lower estimate, based on SAT-V 2003 score means, yields a percentage of white test-takers in Breakdown 3 that exceeds by one percent that reported in the College Board's press release with non-respondents removed.Coincidentally, the "increased diversity" from 2002 to 2003 was also one percent.Note that the remaining percentage of non-respondents (those who are estimated to be non-white) is larger than any of the minority groups, except African Americans.The estimate based on math score means yields a percentage of white test-takers in Breakdown 4 that more substantially exceeds that reported in the press release.Indeed it all but negates the increase in minority representation among test-takers reported in the press release(from 30% minority in 1993 to 36% in 2003).The proportion of estimated minority non-respondents is larger than any of the groups of minority respondents except African Americans and Asian Americans.