The Relationship of High School Graduation Exams to Graduation Rates and SAT Scores

The current study examined the effect of high school graduation exams on states’ graduation rates, states’ aggregated SAT scores, and individual students’ SAT scores. Three data sources were used: One source identified states requiring a standardized test for graduation; the NCES provided state aggregated data on graduation rates for the class of 2002; and the College Board provided its 2001 SAT database for all test-takers. After controlling for students’ demographic characteristics (e.g., race, family education and income, GPA and class rank), regression analyses revealed that states requiring graduation exams had lower graduation rates and lower SAT scores. Individually, students from states requiring a graduation exam performed more poorly on the SAT than did students from states not requiring an exam. The impact of high stakes tests’ on students’ motivation to stay in school and on the teaching of critical thinking skills (tested by the SAT) are discussed. Education Policy Analysis Archives Vol. 13 No. 6 2 A major event impacting an adolescent’s passage into adulthood is graduation from high school. The high school diploma is a credential of responsibility, perseverance, and completion of years of education. It is a tool for potential employers, it is a stepping-stone for higher education, and it is a status symbol and certificate of self worth. Current educational policies, including those contained in the No Child Left Behind federal mandate, seek to improve the educational quality for the nation’s schools. However, educational accountability, could threaten the future of many adolescents. High schools requiring standardized examinations for graduation and diplomas could be putting some of their students at a disadvantage by encouraging dropping out of school, or by focusing the curriculum in a way that facilitates performance on the exam at the expense of critical thinking skills, including those assessed by the SAT. On the other hand, the implementation of graduation exams may improve students’ performance on other high stakes tests, such as the SAT, by helping to motivate students to achieve more demanding standards and provide them with valuable test taking experience. Minimal research has looked at the effect of high stakes testing on graduation rates, and only one study has attempted to look at the effect of high school graduation examinations on SAT scores (Amrein & Berliner, 2002). Furthermore, studies that do explore the relations between high stakes tests and students’ achievement outcomes often ignore the confounding effects of demographic factors such as race, family income, and student ability (GPA and class rank). The purpose of the current study was to examine the effect of high school graduation exams on states’ graduation rates, states’ aggregated SAT scores, and on individual students’ SAT scores. In addition, several demographic factors known to impact students’ test results and graduation rates were included.

A major event impacting an adolescent's passage into adulthood is graduation from high school.The high school diploma is a credential of responsibility, perseverance, and completion of years of education.It is a tool for potential employers, it is a stepping-stone for higher education, and it is a status symbol and certificate of self worth.Current educational policies, including those contained in the No Child Left Behind federal mandate, seek to improve the educational quality for the nation's schools.However, educational accountability, could threaten the future of many adolescents.High schools requiring standardized examinations for graduation and diplomas could be putting some of their students at a disadvantage by encouraging dropping out of school, or by focusing the curriculum in a way that facilitates performance on the exam at the expense of critical thinking skills, including those assessed by the SAT.On the other hand, the implementation of graduation exams may improve students' performance on other high stakes tests, such as the SAT, by helping to motivate students to achieve more demanding standards and provide them with valuable test taking experience.
Minimal research has looked at the effect of high stakes testing on graduation rates, and only one study has attempted to look at the effect of high school graduation examinations on SAT scores (Amrein & Berliner, 2002).Furthermore, studies that do explore the relations between high stakes tests and students' achievement outcomes often ignore the confounding effects of demographic factors such as race, family income, and student ability (GPA and class rank).The purpose of the current study was to examine the effect of high school graduation exams on states' graduation rates, states' aggregated SAT scores, and on individual students' SAT scores.In addition, several demographic factors known to impact students' test results and graduation rates were included.

High Stakes Tests
The American Educational Research Association (2000) defined "high stakes" tests as those tests, which "carry serious consequences for students or educators."Examples of high stakes tests for students include those that identify special academic accomplishments, those used for decisions regarding grade retention, and those that determine high school graduation.High school graduation exams were intended to make graduating and receiving a diploma "mean something" in terms of acquired knowledge and skills necessary for employment, college, and life (Center on Educational Policy, 2002).Advocates believe these tests motivate students and help teachers focus on important academic content and skills.The use of high school graduation tests is based on several underlying assumptions (Kane, 2001): (1) A core set of desired outcomes of a high school education can be identified.(2) A high level of student achievement on demanding content is an important goal for high schools.
(3) Student achievement will improve if students are required to pass a high school graduation test based on demanding content.A secondary assumption is that the adoption of the high school graduation test will not have a major negative impact on other indicators of achievement, such as graduation rates, achievement in content areas not on the test, and involvement in extracurricular activities (Kane, 2001).Unfortunately, little research has been done to test these assumptions.
The lack of definitive research regarding graduation exams is particularly unfortunate considering what is at stake for adolescents.Failure to graduate from high school has serious "high stakes" and may produce a lifetime of consequences.With the stakes being this high, the role graduation exams play in adolescents' lives should be fully investigated.Research has revealed problems associated with high stakes tests at other grade levels (primarily elementary).Concern has been expressed regarding the content and validity of the tests, the impact of the tests on teaching, and consequences for students (for a review see Marchant, 2004).The validity of standardized achievement tests not specifically designed to match school curriculum and the use of norm-referenced tests to assess mastery have been challenged.Schools narrowing curriculum and teachers teaching-to-the-test and using inappropriate test preparation approaches have been identified.Similar issues need to be explored at the high school level.

Effects on Graduation and Dropout Rates
One of the most consistent concerns associated with high school graduation exams is that failing the test, or even fear of failing the test, would convince some students to drop out of high school and not graduate.In a longitudinal qualitative investigation of states before and after implementing graduation exams, Amrein & Berliner (2002) found that 62 percent of the states posted increased dropout rates, and 67 percent of the states showed a decrease in graduation rates.Quantitative analyses are needed to confirm these observations; therefore the first purpose of this study was to examine the direct effects of high school graduation exams on graduation rates.It was expected that graduation examinations would have a negative impact on graduation rates.
If graduation exams created any social inequities, the poor and minorities would be the most likely to suffer.Performance on high stakes tests, such as graduation exams, has been found to be directly related to the socioeconomic status of students (Cunningham & Sanzo, 2002); with lower SES students earning lower scores.Furthermore, graduation exams were found to have no effect on the dropout rate of average students, but lower achieving students (more likely low SES students) were 25 percent more likely to drop out of high school than comparable peers in non-test states (Jacob, 2001).In addition, African American and Hispanic students consistently experience significantly higher dropout rates than White students (Rabinowitz et al, 2001).This situation has grown to crisis proportions with dropout rates in some school districts with minority students at 30 or 40 percent (Orfield, Iosen, Wald, & Swanson, 2004).In 2002, it was found that among the states with a higher than national average percentage of African-Americans, 75 percent had high school graduation exams (Amrein & Berliner, 2002).All but one of the ten states with the highest percentage of African-Americans had graduation exams, and none of the ten states with the lowest percentage of African-Americans had graduation exams.These trends point to the importance of including demographic characteristics of students in any analyses on graduation exams and graduation rates.The current study controlled for demographic characteristics of students in its examination of graduation rates and was able to assess the impact of graduation exams above and beyond the confounding effects of demographics.
A caveat regarding the computation of graduation rates or dropout rates is important to consider.The dropout rate for any high school, district, or state typically is not determined simply by looking at the number of students that started school and subtracting the number that graduated four years later.Students may move and transfer to another school, or they may decide to pursue a GED instead.However, some districts and schools have been accused of using mobility and the GED option as a way of "doctoring" dropout rates and increasing graduation rates (Schemo, 2003).Even some ways in which dropouts are reported leave room for misinterpretation.For example, if a state reports an annual dropout rate of 4 percent across secondary grade levels, it is possible for the freshman class to experience a 16 percent dropout by the end of its senior year (Rabinowitz, Zimmerman, & Sherman, 2001).For this study, graduation rates were computed by dividing the number of graduating seniors in each state by the number of freshman four years earlier.Although this method does not consider such confounding factors as mobility or GEDs, it does provide simple consistency across the widely varying techniques of any one school or district in calculating its graduation rates.

Effects on SAT Scores
The SAT I: Reasoning Test is a three-hour exam that assesses verbal reasoning and mathematical problem solving skills (College Entrance Examination Board, 2002).The SAT is a standardized objective test with most items being multiple-choice.The test is not a measure of any set curriculum; instead it is designed to assess skills necessary for success in college.The SAT has even been equated to an intelligence test, measuring students' ability to learn, not mastery of what was learned (Gose, Selingo, & Brownstein, 2001).Although a higher or lower score on the SAT may not have the same devastating impact on students as the denial of a diploma, the SAT is often the objective measuring stick that colleges use for admissions decisions.Therefore, differences in scores can affect students going to their college of choice, or potentially their ability to go to any college at all.Although college success has been predicted better by high school grade point averages than by SAT scores (Bridgeman, McCamley-Jenkins, & Ervin, 2000;Camara & Echternacht, 2000;Hu, 2002), the SAT continues to be regarded as a fair way to assess students from different schools on an objective measure.
The second purpose of this study was to examine the impact of high school graduation exams on students' SAT scores.Given schools' increased emphasis on high stakes tests and students' greater experience with standardized testing in schools with graduation exams, it might be expected that high school graduation examinations would have a positive effect on SAT scores.Furthermore, if graduation exams are related to increased drop out, subsequently decreasing the potential number of students taking the SAT, students' scores should experience an additional boost.The only study to look at the effect of graduation examinations on SAT scores produced no consistent results (Amrein & Berliner, 2002).
Although it has been argued that the SAT is no more an indicator of socioeconomic status than any other standardized test or measure of academic achievement (Zwick, 2002), research has found that SAT scores, like other standardized tests, are indeed influenced by SES (Marchant & Paulson, 2001).Further, it has been suggested that SAT items do not reflect the black experience and overemphasize science (Fleming, 2000).Although the scoring gap between blacks and whites narrowed from 1976 to 1988, since then the racial gap in SAT scores has widened, with a major confounding factor being family income.The effects of these demographic variables are exacerbated when school, district, or state scores are aggregated or averaged to reflect an average of the students' scores.Using aggregated state SAT scores, over 90 percent of the variance among states can be attributed to the factors of family income and parent education (Marchant & Paulson, 2001).Again, the need to consider demographic characteristics of students in any study of high stakes tests cannot be ignored.In this study, demographic factors were controlled in analyzing the effects of graduation exams on SAT scores; but the differing effects of graduation exams on SAT scores by various demographic groups also were examined.

Method
This study used three data sources to investigate the effects of high school graduation examinations on graduation rates and on SAT scores.The Amrein and Berliner study (2002) identified states requiring a standardized test for graduation, and when each state adopted that policy.These data allowed for the identification of which students were required to pass a graduation exam to graduate in the year 2002.From this database, 18 states were identified as having graduation exams and 33 did not (n = 51 states including Washington, DC).The National Center for Education Statistics (2004) provided state aggregated data (n = 51) on enrollment by grade level and graduation numbers.Data from 1999 (freshmen) to 2002 (seniors) were used to compute graduation rates by dividing the number of seniors graduating in 2002 by the freshman enrollment in 1999 for each state (simplification of Green, 2001;Owin, 2002).In addition, this data source provided stateaggregated data on race (percent of minorities), family income (percent eligible for free and reduced lunch), and participation in special education (percent of students with IEPs).The third data source was the College Board's 2001 SAT database for all test-takers (most are juniors) in the country.This database contained over a million test-takers; however, incomplete survey responses reduced the sample (n = 694,900) used in this study.The database included the selected demographic variables of minority status (percent blacks and percent whites), parent education (whether parents have a bachelor's degree or higher), parent income (whether parents earn more or less than $80,000), student grade-point average, and student class rank.

Results
Descriptive analyses were run on the demographic factors to explore the differences between states with a graduation exam to states that do not have such an exam.A review of the state-aggregated demographics (National Center for Education Statistics, 2003) supported previous research indicating that states with graduation exams tended to have more minorities and more students eligible for the reduced lunch program (see Table 1; Amrein & Berliner, 2002).The demographics from the College Board's SAT database showed that with the exception of significantly fewer blacks in states without graduation examinations, the samples of SAT test-takers in each state were not significantly different.

Effects on High School Graduation Rates
A multiple regression analysis was used to examine the impact of states' requirement of a graduation examination on graduation rates.Percent of minorities (race), percent eligible for free and reduced lunch (family income), and percent of students with IEPs (special education) were included in the equation to control for confounding demographic factors.The equation predicted graduation rates (R = .76,p < .001)with all of the variables accounting for a significant amount of unique variance (see Table 2).Percent of minorities (r sp = -.30,p < .01),percent eligible for free or reduced lunches (r sp = -.22,p < .05),and the requirement of a graduation examination (r sp = -.21,p < .05)were all negatively related to the percent of graduates.The percent of students with IEPs (in special education) was positively related to graduation rates (r sp = .25,p < .05).States with a graduation exam requirement averaged a 64 percent graduation rate, 8 percentage points lower than the 72 percent for the states without the requirement (t = 3.19, df = 49, p < .005).Note.Due to missing data sample was reduced to 45 states.

Effects on SAT Scores
The relation between graduation exams and SAT scores were examined on the state aggregated level and on the individual level.First, a multiple regression was used to examine the impact of states' requirement of a graduation exam on state aggregated SAT scores (from the College Board SAT data source).The percentage of minority test-takers, percent of students with parents with bachelors degrees or above, and mean high school grade point average (GPA) for each state were included in the equation.Minority status (r sp = -.12,p < .01),parents' college education (r sp = .40,p < .001),high school GPA (r sp = .24,p < .001),and graduation exam requirement (r sp = -.10,p < .05)were significant predictors for total SAT scores aggregated by state (see Table 3; R = .96,p < .001).A second multiple regression analysis predicting individual student SAT scores used the same variables.Because of the increased size of the sample (from states to individuals), the variables of family income over $80,000 and class rank in the top 10 percent of the high school class were added as predictors.The equation was a significant predictor of individual SAT scores (see Table 4; R = .64,p < .001)with each predictor accounting for a significant amount of unique variance.The requirement of a high school graduation examination had a significant negative impact on individual SAT scores on (r sp = -.04,p < .01).Past evidence suggested that an interaction between race and family income exists in predicting SAT performance (Bolinger, 1992).Using the individual SAT data, further exploration of this effect revealed that all students without a graduation examination requirement significantly outperformed those student with the graduation requirement on the SAT, except those test-takers who were black with family incomes less than $80,000 that were in the top 10 percent of their high school class (see Table 5).They performed better on the SAT if their state required a graduation exam.This group accounted for one percent of the total sample.The effect of aggregating scores was evident in the prediction of SAT scores.With the exception of the percent minority variable, every demographic variable and the requirement of a high school graduation exam accounted for twice as much variance in the state aggregated scores as in individual scores (see Figures 1 and 2).Overall the state aggregated equation predicted over twice the variance in SAT scores than the equation for individual test-takers, with the state aggregated equation leaving only 8 percent of the variance unaccounted for, compared to 59 percent for individuals.

Discussion
The results of this study suggested that both graduation rates and SAT scores may be negatively influenced by the requirement of a high school graduation examination.Even when controlling for substantial demographic variables related to the outcomes, high school graduation examinations contributed to decreased graduations rates and lower SAT scores.These findings for graduation rates were far less surprising than those for SAT scores.Students struggling to succeed in high school might very well find one more hurdle, one hurdle too many.The fact that these adolescents were more likely to be minorities and from lower SES backgrounds was particularly discouraging.For students that have worked, perhaps harder than most, to overcome obstacles, it seems unconscionable to establish a policy that places a potentially insurmountable barrier between them and a diploma.Colleges report their pools of applicants, especially minority applicants, are being reduced by high school graduation exam requirements (Schmidt, 2000).Even if the high school graduation exam were not a barrier to a diploma, it may still be a detriment to higher education.
Finding that graduation examination requirements were negatively related to SAT scores when controlling for demographics was the major result of this study, supporting the instructional concerns expressed by critics of high stakes testing.Research repeatedly yields two findings related to instruction and high stakes testing: teachers tend to narrow the scope of their curriculum to that which is tested, and they tend to abandon more innovative teaching strategies such as cooperative learning and creative projects in favor of more traditional lecture and recitation (e.g., Brown, 1992Brown, , 1993;;Romberg, Zarinnia, & Williams, 1989).The pressure to improve student scores compels some teachers to teach-to-the-test  (Smith, 1991).High stakes testing also seems to encourage the use of instructional approaches and materials that resemble the tests used (Rottenberg & Smith, 1990).Because the nature of items on the SAT, as a "reasoning" test, can look very different than those of a typical achievement test; focus and preparation for the achievement test are unlikely to transfer.Rituals of giving multiple-choice quizzes and providing test preparation often take the place of "normal" instruction when high stakes tests are a factor.Teachers exploring instructional practices informed by current views of learning and supported by cognitive psychology that seek deeper understanding and critical thinking may find those techniques, and even those goals, at odds with the drill and practice suggested by the broad rather superficial coverage typical of schools with graduation exams (Marchant, 2004).High stakes examinations have been found to be a major factor in discouraging teachers from using strategies that promote enquiry and active learning, and this "impoverishment" influences the language of classroom discourse (Wideen, O'Shea, Pye, & Ivany, 1997).Therefore, as more flexible, responsive, innovative student-based instructional approaches are abandoned in favor of achievement test preparation, the ability to reason verbally and mathematically, as reflected by SAT scores, may suffer.Demographic characteristics proved interesting in predicting both graduation rates and SAT scores.There are a couple explanations for the positive relation between the percent of students receiving special education services and high school graduation rates.It is possible that a larger percentage of students receiving support may translate into a larger percentage of students achieving and graduating.It is also possible that students qualifying for special education services receive exemptions from requirements, such as the graduation examination, that might otherwise serve as a deterrent for a diploma.The positive relation of graduation exams to SAT scores for lower-income higher-achieving Black students was an interesting contrast for this small sub-sample.Perhaps for these select students, the negative impact of a decreased focus on reasoning was offset by the increased concentration on test content and structure.This sub-sample represented a very small proportion of the sample (about one percent).This may suggest that, although graduation examinations and high stakes testing may not be in the best interest of most students, some students may benefit from the structure and focus brought to bear by an emphasis on testing, As with any study, this research has limitations.Any means in which graduation rates or dropout rates are calculated are likely to draw some criticism.Graduation rates for this study did not consider mobility of students or students leaving school to pursue a GED.Although these are not irrelevant concerns, confusion regarding how these intentions are recorded and monitored suggests their inclusion may be as much a confound as their exclusion.Another concern with this study is its focus on one graduating class and one year of SAT test-takers.There is, however, no reason to assume that the year chosen was an anomaly.As with many areas in education, the use of high stakes graduation exams merits further research.However, in the absence of substantial benefits from the practice, and with growing evidence of negative consequences, in addition to the expense in time and money, any efforts to increase the use of graduation exams seems ill advised.Justification for the continuation of the practice needs to be clearly established beyond past assumptions.Too much is at stake for too many to base educational policy on assumptions, good intentions, or political interests.Further research evidence is required.

Figure 1 .
Figure 1.Percent of variance accounted for among states calculated from standardized B weights for graduation rates and SAT scores.

Figure 2 .
Figure 2. Percent of variance accounted for among individuals calculated from standardized B weights for SAT scores.