An Analysis of Student Performance in Chicago ’ s Charter Schools

Charter schools have become the cornerstone of school reform in Chicago and in many other large cities. Enrollments in Chicago charters increased by more than ten times between 2000 and 2014 and, with strong support from the current mayor and his administration, the system continues to grow. Indeed, although state law limits charter schools in Chicago to 75 schools, proponents have used a loophole that allows multiple campuses for some charters to bypass the limit and there are now more than 140 individual charter campuses in Chicago. This study uses comprehensive data for the 2012-13 and 2013-14 school years to show that, after controlling for the mix of students and challenges faced by individual schools, Chicago’s charter schools underperform their traditional counterparts in most measurable ways. Reading and math pass rates, reading and math growth rates, graduation rates, and average ACT scores (in one of the two years) are lower in charters all else equal, than in traditional neighborhood schools. The results for the two years also imply that the gap between charters and traditionals widened in the second year for most of the measures. The findings are strengthened by the fact that self-selection by parents and students into the charter system biases the results in favor of charter schools. Education Policy Analysis Archives Vol. 24 No. 111 2


An Analysis of Student Performance in Chicago's Charter Schools
Charter schools have become the cornerstone of school reform in Chicago and nationally.Arne Duncan, who led Chicago schools and was a strong proponent of charters, became Secretary of Education and continued his support of the charter movement. 1Enrollments in Chicago charters increased by more than 10 times between 2000 and 2014, and with strong support from the current administration in Chicago, the system continues to grow.Illinois's original law allowed for 45 charters statewide -15 in Chicago and 30 in the rest of the state.These caps were raised to 30 (Chicago) and 30 (rest of state) in 2003, and to 70 (Chicago) and 50 (rest of state) in 2009.The current law imposes of cap of 75 charters in Chicago, but includes a loop-hole that allows some charters to open multiple campuses.This means that the 75 school limit for Chicago can be bypassed relatively easily and there are now more than 140 individual charter campuses in Chicago (Illinois State Board of Education, 2016). 2 This has happened despite the fact that very little research supports a central tenet of the charter school movement -the claim that charters enhance student performance.Prior work on this question in Chicago is mixed, but most evaluations imply that students in charters do no better than their counterparts in traditional public schools.
This study uses comprehensive data for 2012-13 and 2013-14 and shows that, after controlling for the mix of students and challenges faced by individual schools, Chicago's charter schools underperform their traditional counterparts in most measurable ways.Reading and math pass rates, reading and math growth rates, graduation rates, and, and average ACT scores (in one of the two years) were lower in charters, all else equal, than in traditional neighborhood schools.And the gap in reading and math test scores widened from the first year of data to the second.The findings belie the fact that, because parents and students actively self-select into the charter system, student performance should exceed what one sees in traditional schools, even if charters do no better at teaching their students.
Policy recommendations based on this evidence include potential actions at the local and state levels.At the local level, it is recommended that the Chicago Public School District establish a moratorium on new charter schools and campuses and complete an impact study on how charter school policy has affected the district as a whole.State-level recommendations include returning policy and control powers to local authorities by eliminating the State Charter School Commission, removing the provision that exempts campus expansions from the charter school cap, ensuring that charter schools do not deepen racial segregation, expanding the training required of school board members to include research-based review of the potential effects of charters on segregation and student performance, more detailed information and reporting requirements from charter applicants on their performance in existing schools, stricter reporting standards for charters matching those for traditional schools, more detailed descriptions from existing and proposed charters regarding the 1 The authors acknowledge the helpful comments of two anonymous referees.This work was funded by project support from the Illinois Education Association, the Chicago Teacher's Union, the Illinois Federation of Teachers and by general support from the Ford Foundation, the McKnight Foundation and the Kresge Foundation. 2 Charter schools located in Chicago were limited by a 2003 amendment to the Charter Schools Law to one campus per charter.However, the rule does not apply to charters granted replicating status prior to 2003.As a result, there are 13 charter schools in Chicago that are permitted to create multiple campuses under the same charter (Illinois State Board of Education, 2016).
education practices or innovations they use to improve student performance, and requirements to document continued success to keep their charter.

Chicago School Characteristics, 2000-2014
Charter school enrollments in Chicago increased by more than 10 times between 2000 and 2014, from 5,400 to 54,795 (Figure 1).At the same time traditional school enrollments fell by 20% from 426,700 to 341,465.Enrollments in the other public school alternatives to traditional neighborhood schools -selective, gifted and magnet schools -increased by only about 15%.Charter school growth peaked in 2006, 2007, and 2009.Enrollments grew by nearly 30% in each of those years, but growth was less rapid in more recent years, falling to 8% and 13% in 2013 and 2014.In absolute terms, growth peaked in 2009 and 2011 when enrollments grew by 6,400 and 6,500, but, again, the most recent years showed smaller increases.In 2014, charter schools represented 14% of public school enrollments in the Chicago. 3he demographic profiles of charter and traditional schools are similar and have changed little during the period.Black and Hispanic students were the dominant racial/ethnic groups in both traditional schools and charters.Combined, they represented 96% of enrollments in charters in both 2000 and 2014 and 84 to 86% of enrollments in traditional schools (Figure 2 and Table 1).These overwhelmingly large non-white shares of students mean that very few public schools in Chicago are racially diverse or integrated in the traditional sense.However, traditional schools were still more likely to be racially diverse than charters -in 2014, 41 of 518 traditional schools served between 20% and 60% white students, but no charter schools were in this range.
The overwhelming majority of charters essentially serve a single racial or ethnic group.In 2014, 36 of 50 charters had more than 80% of their students from one group -29 were predominantly black and seven were predominantly Hispanic.Put another way, only about a quarter of charters were not single race schools and all of them were more than 80% non-white.

Comparing Charters to other School Types
The Chicago School District offers several types of schools to students in addition to traditional neighborhood schools ("traditionals") and charters.Table 1 compares charters and traditionals to the three alternative school types serving significant numbers of students -selective schools, schools for "gifted" students and magnet schools. 4The indicators, selected from the data provided by the Illinois and Chicago Boards of Education, include racial mixes and a number of variables often associated with student performance in the empirical studies of school performance.
Traditional neighborhood schools serve the largest numbers of students by far -75% in 2014, with charters coming in a distant second at 14%.Selective, gifted and magnet programs serve a significant number of students, with a combined total of 45,288 students (9,500 fewer than charters) but the gap is widening and, as noted above, charters represent the most rapidly growing part of the system by far.
Traditionals and charters serve a similar mix of students by race.Traditionals serve a higher share of white students, but, as noted above, the most salient racial characteristic of these parts of the system are the large non-white shares -91% for traditionals and 98% for charters.Selective, gifted and magnet schools serve a more mixed, but still highly diverse, group of students.Non-white student shares for these groups of schools range from 70 to 85%.
Traditionals and charters are also comparable on the other selected indicators.Traditionals serve significantly more limited English proficient students and show greater mobility rates (a factor often cited as a negative influence on school achievement rates) than charter schools, but attendance rates, homeless rates, low-income percentages and independent educational program rates are very similar.
Selective, gifted and magnet schools, on the other hand, show significantly more advantageous characteristics for most indicators.Low-income, homeless, independent educational program, limited English, and mobility rates are markedly lower in selective and gifted schools especially.Magnets generally look more similar to traditionals and charters but still differ in significant ways.
The overall message from Table 1 is that simple comparisons among the school types are likely to be misleading.Selective, gifted and magnet schools clearly differ from the other two school types.Although charters and traditionals have similar profiles in many ways, they also differ in significant ways.In particular, they differ in ways with potentially important effects on comparisons of student performance.Table 2 shows another set school of characteristics related to the strategies that schools may use to maintain or enhance student performance.The indicators include some school policies that could potentially affect real performance rates -the number of days in the school year, minutes per day spent on math, science and social studies (SOS), average class sizes by grade -and others that might improve measured student performance by affecting the pool of students actually taking the tests, or remaining in a school -the percentage students taking the tests, suspension rates and expulsion rates (available only for 2012-13).
The most dramatic differences between charters and other school types in the first group of indicators -policies with the potential to directly affect performance -are in the number of school days and class time spent on math.Charters have longer school years on average (by three to five days) and more time spent on math (as high as 25% more).Differences between traditionals and selective, gifted or magnet schools are much narrower.
The second group of indicators -characteristics that might affect measured achievementshow a similar pattern.Charters differ significantly from the other four types.The percentage of students not taking the reading, math and science tests are roughly twice as high in charters.While this does not necessarily mean that more low-performing students are not included in the test results, it does mean that the potential exists.However, the absolute percentages are small enoughin the 1.1 to 1.3% range -to limit any effects on measured performance.
The suspension and expulsion data indicate dramatic differences between charters and the other types of schools as well.Charters use expulsions much more extensively than the other school types.The average expulsion rate is more than 10 times greater in charters than in traditionals.However, charters use suspensions much less extensively than the other school types.The suspension rate for traditionals, for instance, are more than 20 times the average for charters.In sum, the available data for both student and school characteristics show clear potential for actual and measured differences in student performance between charters and other types of schools.The data also make it clear that simple comparisons that do not control for different mixes of students in the different types of schools are unlikely to be accurate.

Student Performance
Charter school advocates often cite improved student performance as the primary rationale for establishing and expanding charter school systems.Charter school parents and students actively choose their school in lieu of their assigned traditional public school and charter schools get no students by default -they must attract them in some way.Both of these factors suggest that superior achievement rates might be expected in charters.
In reality, the findings from an extensive and growing research literature show decidedly mixed results on this issue.This is true for individual city or state studies including those on Chicago charters as well as multi-state studies.One recent review of the literature concluded that "{t}aken in the aggregate, the empirical evidence to date leads one to conclude that we do not have definitive knowledge about the impacts of public charter schools on students and existing schools."(Silvernail  & Johnson, 2014)  Even the studies with the most positive results for charters find mixed results, very small effects for charters, or both.One review of the literature and meta-analysis with relatively positive findings regarding charter school performance concluded "{t}he overall tenor of our results is that charter schools are in some cases outperforming traditional public schools in terms of students' reading and math achievement, and in other cases performing similarly or worse" (Betts & Tang,  2011).In another example, results from two large multi-state studies by the Center for Research and Education Outcomes (CREDO)-the research group that most consistently finds positive performance outcomes for charter schools-are decidedly mixed.CREDO's 2009 study found lower performance by charters (Center for Research and Education Outcomes, 2009) compared to traditional schools.Their 2013 follow-up found positive outcomes, but the measured magnitudes of the effect were very small -on the order of .01 standard deviations in test scores for growth (Center for Research and Education Outcomes, 2013b).
To some extent, this reflects how challenging it is to isolate the effects of charters (Ni &  Rorrer, 2012).The primary methodological issue affecting charter studies stems from the fact that charter students are self-selected, creating selection bias.The way that parents and students select charters virtually guarantees that, as a group, charter students have greater parental concern for and participation in their education than do students in traditional, assigned schools.By definition, charter parents went to the trouble of enrolling their kids in a school other than the one assigned to them by their school districts.Parents of kids in traditional schools have not universally demonstrated the same degree of participation.This matters because active participation by parents in their children's education is an important contributing factor to student achievement (Bifulco & Ladd,  2006; Goldring & Phillips, 2008; Shute, Hansen, Underwood & Razzouk, 2011; Wilder, 2014). 55 Shute, Hansen, Underwood & Razzouk (2011) found that the most consistent relationships were for parents talking with their child about school, parents holding high expectations for students' academic achievement and parents employing an authoritative parenting style.Wilder's (2014) review of the literature concluded that "the relationship between parental involvement and academic achievement was positive, regardless of a definition of parental involvement or the measure of achievement" (p.377).Bifulco & Ladd (2006) found that that charters tend to be established in areas with above-average proportions of involved parents, and that, within in those areas, more involved parents tend to select into charter schools.Goldring & Phillips  What this means is that we should expect student achievement to be greater, all else equal, in charter schools, even if charters do no better at educating kids.This is true whether charters are selective in other ways ("creaming" the best students) or must accept all applying students as long as space is available (as in Chicago).Charter advocates often imply that charters actually operate at a disadvantage in this way because charter schools in many places are more likely to serve children from low-income or homeless households, or kids with other characteristics associated with lower school achievement.However, factors like these can usually be observed and accounted for statistically while parental values or motivation cannot.

Prior Work on Charter School Performance in Chicago
The largest studies of charter performance in Chicago illustrate the diversity of findings in this field.A study by the Rand Corporation in 2009 (Booker et al., 2009) found no consistent charter school effect on student achievement, either positive or negative.An initial study by CREDO in  2009 (Center for Research and Education Outcomes, 2009) came to similar conclusions but a follow-up in 2013 (Center for Research and Education Outcomes, 2013a) found positive outcomes for charters.
The Rand study used student-level data to evaluate the types of students drawn to charters and whether charter schools were producing achievement gains as measured by test scores, graduation rates, ACT scores and college enrollment rates.Much of the analysis was limited to students who attended both a traditional and a charter school during the years covered by the data.This meant that students could be compared to themselves, with their performance in a traditional school compared to how they did in a charter, to control for selection bias issues.The drawback of this approach is that it greatly limits the group of students included in the comparisons.
The analysis found that charters attracted a group of students that were roughly representative of the traditional schools they left.Achievement levels for charter students prior to entering charters were higher than those in the schools they left but the differences were relatively small.Similarly the racial and ethnic composition of charter schools resembled the traditional schools they left (Booker, Gill, Zimmer & Bass, 2009).
The results for achievement gains measured by test results in grades 3 through 8 were similarly minor.Only small achievement differences were found between charters and traditional schools and the directions of the differences were inconsistent.Achievement in charters lagged behind traditional schools in reading, especially for Hispanic and Asian students and in math for Asian students, while black students did slightly better in charters in math.The analysis also found that charters had negative effects on student performance in their first year of operation but that this deficit disappeared in the second year of operation and beyond (Booker et al., 2009).
Finally, the results for the high school achievement measures -ACT scores, graduation rates and college enrollment -suggested that charter high schools may produce positive outcomes in these measures.However, positive results were limited to students with extended attendance in charters that included both middle and high school grades, a category which included only four charters at the time of the study (Booker et al., 2009).
The two CREDO studies match charter school students to traditional public school students who are as similar as possible and then compare achievement rates for each pair.The results from the 2009 were mixed.In some comparisons, students in Chicago charters under-performed their (2008) concluded that "much of the available research linking parental involvement to school choice indicates that parents who participate in school choice (both public and private school choice) are likely to be more involved in their children's education when compared to parents who do not participate in the choice marketplace" (p.229).traditional school peers; in others there were no discernable differences and in a few comparisons charter school students out-performed their peers.The 2013 results were more positive for charters, with most comparisons showing charter students outperforming their traditional school peers.
However, there are reasons to worry about the CREDO results.The studies match charter students with traditional school students based on race/ethnicity, gender, English proficiency, free/reduced price lunch status, special education status and grade level.The method is designed to control for selection bias by creating a control group like those used in randomized experiments, but the list of matching variables does not include anything that reliably captures parental engagement, a primary source of selection bias in charter studies (Maul, 2013; Miron & Applegate, 2009). 6This compromises the findings that are positive for charters -they may simply be the result of selection bias.It also strengthens the findings that are negative for charters -they are negative in spite of the positive bias.Finally, the method typically does not generate a good match for every charter student, limiting the sample.In Chicago, CREDO found traditional school matches for roughly 90% of charter students.(The reported results provide no way of examining the characteristics of the omitted students to see if they are representative of the overall mix of students or not.) A variety of other smaller scale studies that generally do not control very effectively for differences in student characteristics provide further mixed results.Work by Northwestern's Medill Data Project found that a greater percentage of traditional school students than charter students exceeded the state standards in math and reading testing (Mihalopoulos & Little, 2014).A review of state report cards by the Chicago Tribune found many struggling charters (Hood & Ahmed-Ullah,  2011).And Chicago School Board data show substantially higher expulsion rates for charter schools (Anderson, 2014).An analysis by the Chicago Sun Times also suggests that neighborhood schools have improved greatly in reading, and are now out-performing charters by significant margins (Golab, Schlikerman & Fitzpatrick, 2014).Charter advocates have countered with their own take on the studies that use similarly simple comparisons to conclude that charters out-perform traditionals in various ways (Polaris Charter Academy, 2014; Illinois Network of Charter Schools, 2014;  Ziebarth, 2014).
The work reported here cannot fully settle the debate.The type of data and the resources needed to fully account for selection bias are beyond the scope of this work.Such data could come, for instance, from an experiment with students randomly assigned to charters or traditionals, microdata (student-level data) that would enable comparisons of performance by the same student in different school environments, or data for a comparison group of students who applied to attend charters but were excluded through a lottery system designed to accommodate capacity limitations (Gleason, Clark, Tuttle, Dwoyer & Silverberg, 2010).These alternatives are either impractical or not available.This means that the statistical results reported will be biased in favor of charter schools.In 6 In a review of a CREDO study in Michigan, Maul (2013) notes: "The larger issue with the use of any matching-based technique is that it depends on the premise that the matching variables are sufficient to account for all relevant differences between students; that is, once students are matched on the aforementioned seven variables, there remain no meaningful unobserved differences between students in charter and traditional public schools (other than their school type).School-choice systems always implicate unobservable differences among parents (i.e., parents of charter school students are necessarily sufficiently engaged with their children's education to actively select a charter school).To the extent to which a reader finds it implausible that the seven variables have captured these and all other important differences, she will be unconvinced that these methods can provide true estimates of causal effects."(p.4) In other words, the CREDO methods are likely to bias comparisons of student performance in charter and traditional schools in favor of charters.other words, the results are more likely to show that charter school students out-perform their traditional school counterparts, all else equal.
What the analysis described here can do is provide comparisons using the most recent data available (2012-13 and 2013-14) that include all students in charter and traditional schools in the Chicago public school system and compare the separate results for two years.The available data also make it possible to look at a range of achievement measures, including standardized test pass rates, student academic growth rates, graduation rates and college entrance exam (ACT) scores.

Model and Data
Previous sections make it clear that simple comparisons of student performance in charter and traditional schools are likely to be inadequate.Accurate comparisons must control for differences in types of students served by different schools.However, it is worth summarizing the simple differences as measured by the data use in this analysis to provide a baseline for comparison to other work using simple statistics.
The data for the analysis are publicly available from the Illinois State Board of Education (ISBE) Center for Performance Report Card data site (http://www.isbe.net/assessment/report_card.htm ).ISBE provides extensive annual data for a wide array of variables including student assessment results, school demographics and characteristics, instructional settings, teacher and administrator characteristics, and financial data.The data used for this performance analysis includes student assessment indicators and school demographics.The selected assessment indicators are those most commonly cited in other work comparing charter and traditional schools in Chicago.They include the Illinois Standards Achievement Test (ISAT) for grades 3 through 8, student academic growth, four-and five-year graduation rates, and average ACT scores.The ISAT and student academic growth results provide indicators for elementary and middle schools.Graduation rates and ACT scores provide the measures for high schools.7 Standardized test pass rates by school and grade.The percentage of students meeting or exceeding standards on the ISAT for grades 3 through 8. ISAT results provide the number of students in each of four categories (academic warning, below standard, meet standard, exceed standard) and are reported by school, grade, race and subject, allowing for both school-level and pooled grade-level analysis of student performance.The number of school/grade combinations with data available for this measure varied from 1,155 to 2,593 depending on the year, subject and race.Standardized test pass rates by school.The percentage of students meeting or exceeding standards on the ISAT standardized tests compiled by school, race (all students, black students and Hispanic students) and subject (math and reading).The number of schools with data by this measure varied from 236 to 466 depending on the year, subject and race.(See Table A.11 for the number of schools reporting for each race category.) Student academic growth by school.A measure of students advancing from one performance level on the ISAT to another performance level (or "growing") from one year to the next.The growth measure is expressed as a number between 0 and 200 where a value above 100 represents positive growth and/or consistently high achievement, and a value below 100 represents negative growth and/or consistently low achievement. 8The number of schools with data by this measure varied from 409 to 445 depending on the year and subject.
Four-and five-year graduation rates by school.The percentage of students graduating in four or five years compiled by school (calculated based on federal No Child Left Behind guidelines).The adjusted cohort method is used to generate the estimates.The method divides the number of grade cohort members who graduate in a given year by the number of ninth graders in the cohort four (or five) years earlier, adjusted annually for transfer in, transfers out, emigration, and mortality by in each of the subsequent four (or five) years. 9The number of schools with data by these measures varied from 96 to 107 depending on the year and variable.
Average college entrance exam scores.Average ACT scores compiled by school and subject (math, English, science and comprehensive).The number of schools with data by these measures varied 99.
Tables 3 and 4 provide a baseline summary on the performance measures to be used in the multivariate analysis below for each of the two years.All measures are simple means of school-level measures.The student performance measures include test pass rates in reading and math compiled by race and income, annual growth rates in reading and math, four and five year graduation rates, and ACT scores in English, Math and Science.
In 2012-13, charters exhibit slightly lower average student performance on most measures when compared to all non-charters.The charter indicators are slightly higher for most when compared to neighborhood schools alone (removing selective, gifted and magnets from the noncharter averages).Charters compare well to traditionals in reading and math test scores but lag traditionals on student academic growth and 5-year graduation rates.
The 2013-14 data suggest deterioration (sometimes significant) in the results for charters when compared to other school types.Student performance lags in charters in virtually all of the measured indicators.Math pass rates, academic growth rates and graduation rates show the most significant shortfalls for charters.
As noted above, the charter/traditional comparison is susceptible to several confounding factors.Traditionals must deal with greater challenges than charters in several dimensions -mobility and limited English rates for instance -and lesser ones in others -such as the small difference in poverty rates.In addition, the mix of challenges varies in complicated ways from school to school.
The available school-level demographic measures to be used in the analysis (shown in Table 1) include those most commonly cited as potential determinants of measured student achievement.They include school racial demographics, special student populations (limited English proficient students, students in independent education programs, and homeless students), low-income students, and other student or school characteristics (attendance rate mobility rate and school size). 10he racial mix of a school serves primarily as a proxy for other unmeasured characteristics like family structure, peer group attributes and the effects of long-term discrimination or inequality that are likely to affect achievement and which are correlated with race.
The share of special student populations are expected to negatively affect measured student performance because special needs students often both start with learning deficits and require extra educational resources, drawing resources from other purposes.
Low-income student shares are expected to be negatively associated with school achievement because of a variety of issues in low-income homes that can impact student performance.These include time constraints on parents associated with single-parenting or work, for example that limit parental involvement, greater health problems and/or stress associated with poverty, poor nutrition, and other neighborhood or peer factors.
Attendance rate is included to control for a variety of factors affecting performance, including health factors (also associated with poverty) and parental involvement or commitment to education.It might be argued that attendance rate should not be included in the analysis because it reflects policies pursued by individual schools intended to encourage attendance.However, it was included to control at least partially for parental involvement or commitment.Total school enrollment is included to control for any effects of school size.In particular, it might control for inefficiently sized small charters in their early years of operation.
The estimated models control for these issues by employing multiple regression analyses that include the 12 school-level demographic characteristics shown in Table 1. 11A dummy variable for charter schools is then used to capture any performance differences between charters and traditionals remaining after accounting for school characteristics.
Since standardized test pass rates are reported by school, grade, race and subject, two types of models were estimated.Pooled models with separate observations for each grade were estimated for reading and math pass rates for each of the three race categories and for low-income students.School-level models were also estimated for pass-rates for the same groups.School level models 11 Because of the substantial differences in the populations served and the complicated selection issues associated with selective and gifted schools, selective, gifted and magnet schools are not included in the multivariate analysis.The charter-traditional comparisons change little when these schools are included in the analysis (with appropriate dummy variables added).
alone were estimated for all other performance measures because they are available only at that level of aggregation.
Tables 5 and 6 show how student performance differs between charters and traditionals based on the pooled and school-level multiple regressions for 2012-13 and 2013-14. 12The table shows the measured difference (ζ's) between charters and traditional schools for each of the regression models with notations for the level of statistical significance.Negative numbers indicate that charters under-perform their traditional counterparts after controlling for differences in the characteristics of student populations.For instance, the -4.9 in the top row of the second column of Table 5 indicates that reading pass rates for black students are 4.9 percentage points lower in charters on average, after controlling for the effects of all of the student characteristics shown in Table 1.
The top panel of each table shows the results for reading and math pass rates using pooled data for pass rates for grades 3, 4, 5, 6, 7, and 8. Separate regressions were run for all students, black students, Hispanic students, and low-income students.The second panel shows the results for the same measures using school-wide averages (rather than pooled individual grade results).The remaining panels show results using school-wide data for all racial/ethnic/income groups combined for reading and math academic growth rates, four-and five-year graduation rates, and ACT scores broken out by composite, math, English and science.(The data for these indicators do not include breakouts by race, grade or income.) The results show that charters consistently underperform traditionals across all of the performance measures.Every coefficient in Table 5 (2012-13) is negative and 18 of 24 are statistically different from zero at the 95% confidence level. 13The magnitudes of the coefficients imply that traditionals out-perform charters by substantial margins in some cases.For instance, average pass rates for black students were roughly 40% in both traditionals and charters in 2012-13 (Table 3).The estimated coefficients in the black student regressions range from -4.9 to -7.0, implying shortfalls in charters ranging from 12% to 18%.Similarly, average pass rates for lowincome students in traditionals and charters range from 43% to 46% while the school-level coefficients are -6.1 and -9.3, implying shortfalls of roughly 20%.The graduation rate coefficients imply even larger gaps, in the range from 30% to 40%.The growth rate and ACT coefficients imply smaller shortfalls but are still in the 3% to 6% range.
Only the regressions for Hispanic students imply that charters compare reasonably well with traditionals.None of the coefficients are statistically significant, but, even in this comparison, the coefficients are all negative.
The math and reading test results are even worse for charters in the 2013-14 regressions.As in the 2012-13 models, all of the coefficients comparing charter to traditional performance on the ISAT tests are negative and 12 out of 16 are statistically significant.In addition, 12 of the 16 coefficients are larger in absolute magnitude than their 2012-13 counterparts, indicating widening gaps between charters and traditionals.Since the absolute magnitudes of the independent variables changed little between the two years, the proportional shortfalls for charters deteriorated as well.
The two growth rate and the four-year graduation rate results for 2013-14 are very similar to the 2012-13 indicators.There is some improvement in relative standing of charters in the five-year graduation rate comparison but the results are still negative and statistically significant.The four ACT comparisons, on the other hand, provide the only positive coefficients for charter schools in any of the models.However, they are all very small in magnitude and statistically insignificant. 14n sum, the statistical findings from the models that control for school characteristics likely to affect the measures of student performance do not support the claims made by charter advocates.The findings imply that charters are consistently out-performed by traditional schools across nearly all of the indicators.Based on the results, the best that can be said is that charters do no worse with Hispanic students or in preparing students for the ACT college entrance exams.
Further, all of the reported results are likely to be biased in favor of charter schools because of selection bias, strengthening the implication that charters under-perform their traditional counterparts.Despite the fact that all of the negative coefficients in Tables 5 and 6 are very likely to be biased toward zero, the results support that hypothesis that the charter effect is negative. 13When selective, gifted and magnet schools are included in the analysis with the appropriate dummy variables, they out-perform both traditionals and charters in most cases and the charter-traditional comparisons are substantively unchanged. 14The control variables perform much as expected.Greater shares of special student populations, higher mobility rates and greater low-income shares lower outcomes on average, with low-income student shares and mobility rates showing the most powerful impacts.Higher attendance rates tend to improve outcomes as expected.See Appendix tables for complete results.

Conclusions and Recommendations
Chicago's charter system continues to grow rapidly despite the fact little evidence supports the claim that students perform better in charter schools than in traditional counterparts.This study adds another piece of evidence that implies that students perform at lower levels in charters than traditionals.The clear implication is that it is time to reevaluate where the system is headed and to ensure that all of the information needed to evaluate existing and proposed new charters is available.To allow time for this reevaluation, it is recommended that the Chicago Public School District institute a moratorium on new charter schools and campuses and complete an impact study on how charter school policy has affected the district as a whole.At the state level, the charter school law (105ILCS 5/27A-5) should be changed to remove the provision that exempts campus expansions by charter schools established before 2003 in Chicago for the purposes of maintaining the charter school cap in Illinois because much of the growth that is occurring in the system is in these multicampus schools.
In light of the finding that charter schools are more racially segregated than traditionals and other school types in Chicago, the state should study this effect and design policies that charter schools must follow in order to ensure racial segregation is mitigated.Given that the district is already struggling with racial segregation charters should at least be required to more closely match district-wide racial diversity.Similarly, since school boards authorize and supervise most charter schools in Illinois, the training required of all school board members (105 ILCS 5/10-16a and 105ILCS5/34-3.2) should include a research-based review of the potential effects of charter schools on racial segregation and student performance.
If some charters are allowed to add campuses, they should be required to include as part of the application process a detailed analysis of the applicant's performance in regard to racial diversity in existing schools and student performance in existing schools.The charter school applicant should be required to produce data in accordance with specified methods and data to ensure that information is unbiased.The data and information should be made public with notices of the posting distributed to interested parties, including affected bargaining units and nearby neighborhood organizations.
Given that the Chicago district offers other non-traditional options (selective, gifted and magnet schools in particular), existing and new charter schools should be required to describe the educational practices or experiments that distinguish them from already-available programs in traditional schools.This is particularly true of magnet schools which, like charters, are not permitted to screen students by performance testing.Students are selected for magnets by computerized lottery.Expanding the magnet system is a clear alternative to charters for expanding the availability of non-traditional schools to all students in the city.
Finally, given the high performance levels in the other non-traditional schools, charters should also be required to set goals for improved student performance, demonstrate success in meeting those goals within five years, and continue to demonstrate success every two years in order to retain the charter.Grade dummy variables (Grade3 -Grade8) denote whether a grade is taught in the school or not.Grade dummy variables (Grade3 -Grade8) denote whether a grade is taught in the school or not.

Table 1 .
School Characteristics by Type of School in the City ofChicago in 2012-13 and 2013-2014

Table 2 .
School Characteristics: City of Chicago in 2012-2013

Table 3 .
Traditional and Charter School Performance: City of Chicago in 2012-2013

Table 4 .
Traditional and Charter School Performance: City of Chicago in 2013-2014