Educational Performance and Charter School Authorizers : The Accountability Bind

Charter schools involve a trading of autonomy for accountability. This accountability comes through two forces—markets through the choices of parents and students, and accountability to government through the writing of contracts that must be renewed for schools to continue to operate. Charter schools are supposed to be more accountable for educational performance than traditional public schools because authorizers have the ability to revoke charter contracts. Here, I focus on one central component of accountability to government: performance accountability or accountability for educational outcomes to charter school authorizers through the revocation or non-renewal of charter contracts. In this paper, I suggest that contract-based accountability for educational performance in charter schools may not be working as proponents argued it would. This article explores some explanations for why there are very few examples of charter schools that have been closed primarily because of failure to demonstrate educational performance or improvement. Future work will need to test if these challenges for authorizers hold in a variety of contexts. The conclusion examines the implications of these findings for the future of charter school accountability.


Introduction
Charter schools are premised on the idea that one can "trade" autonomy for accountability-specifically, that if one provides greater autonomy to individual schools, through deregulation and/or school-site control over finances, hiring, curriculum and mission, then one can place greater demands on the educational performance produced by those schools (Kolderie, 1990;Nathan, 1996) According to one analysis of charter school legislation, "[m]ost charter school statutes with sections on legislative intent are quite explicit in their expression of the legislature's demand for accountability for student performance" (Millot, 1996, p. 9). This accountability comes through two forces-markets through the choices of parents and students, and accountability to government through the writing of short-term (generally 3-5 years) contracts that must be renewed in order for schools to continue to operate. (Note 1) While the theory underlying the charter school idea varies somewhat from state to state, a central part of that theory is that charter schools will be more accountable for educational performance than traditional public schools, largely because authorizers have the ability to revoke or not renew charter contracts (Kolderie, 1990;Nathan, 1996). This article focuses on one central component of accountability to government-that of performance accountability or accountability for educational outcomes to charter school authorizers (the public entities that grant operators charter contracts) through the revocation or non-renewal of charter contracts.
The early information -and this is early information, as only 29% of states with charter schools have had schools go through the renewal process -is that "in those states [where schools have come up for renewal], almost all schools seeking renewal have been successful" (SRI International, 2000, p. 56). While a number of schools have closed (39 as of 1999, according to the Center for Education Reform), these closures have mostly been for organizational or financial reasons; as Finn and his colleagues argue, the most common reasons for closure have been, "organizational chaos, management meltdown, and fiscal shenanigans" (Finn, Manno, & Vanourek, 2000, p. 137). One explanation for the continuing operation of the vast majority of charter schools is that they are, indeed, producing improved student achievement. However, research and evaluations suggest a more complicated story, with a mix of success and struggles (Horn & Miron, 1998;Horn & Miron, 1999;Public Sector Consultants & MAXIMUS, 1999;RPP International, 1999).
In this article, I suggest that contract-based accountability for educational performance in charter schools may not be working as proponents argued it would. According to SRI International, "[f]ew charter school authorizers have revoked or not renewed charters because of student performance problems" (SRI International, 2000, p. 57). This article explores some explanations for why there are very few examples of charter schools that have been closed primarily because of failure to demonstrate educational performance or improvement. Possible explanations, including the challenges of determining school quality, the strong and vocal support of charter school communities (relative to the quiet and diffuse public interest), and concerns about damaging the charter school movement, provide strong incentives for authorizers to allow schools to continue to operate. I am not suggesting that no authorizers are taking performance accountability seriously-some clearly are -but that acting as the originators of charter schools intended can be very difficult for authorizers.
Following a brief description of the data sources drawn on, I discuss the accountability ideal for charter schools, and the relationship between this ideal and ideas of a "new accountability" for public education more generally. This is followed by a description of how authorizers are addressing their different roles that relate to accountability -including approving applications, overseeing schools, and granting new contracts. In the next section, I explore some possible explanations for why charters schools are rarely closed and some of the "middle grounds" authorizers have created to provide incentives and sanctions to schools without actually forcing them to close. Future work will need to test if these challenges for authorizers hold in a variety of contexts. The conclusion examines the implications of these findings for the future of charter school accountability.

The Accountability Ideal
Charter school accountability has both unique components, especially the granting of an actual charter contract allowing an entity not governed directly by a school board to operate a public school, and facets that are closely intertwined with broader changes in ideas about public school accountability. In the following two sections, I examine both of these aspects of accountability.

Ideals of accountability for charter schools
One of the challenges of talking about accountability for charter schools-or a host of other issues-is the variation among states as to the interpretation of the charter school idea in legislation (Buechler, 1996 (July) ;Bulkley, 1999c;Mulholland, 1996). Accountability is often separated into two componentsto whom an entity (such as a school) is accountable, and for what they are accountable (Elmore, 1995). As noted earlier, accountability for charter schools has two facets: Market-based accountability, which operates through the choices of parents and students; and Performance-based accountability, which operates through contracts between charter schools and their authorizers specifying the educational and other outcomes the school will produce if it is to continue to operate.
The reliance on both government and the market is a critical aspect of the charter school idea, and a method for ensuring that charter schools serve both parental and broader societal interests. As well, these two forces are intended to combine and create a stronger form of accountability then is found in the traditional public school system, where schools are less likely to face the possibility of closure through either the withdrawal of students or the removal of a contract that allows them to operate (although they are increasingly likely to face sanctions such as closure or reconstitution if they do not meet state-defined performance expectations). The implicit theory of charter schools is that these two forms of accountability will complement and reinforce each other.
According to a study of accountability components in charter school legislation, Lake and Millot find three general responsibilities for charter school authorizers (Lake & Millot, 1998)-(the implementation of each of these areas are discussed below). The first responsibility is in the charter school application itself, and involves the "requirements to become a charter school" (p. 19). Legislation varies, but generally includes some of the pieces that must be included in an application, such as the school's mission, the type of staff who will be hired, the type of educational program that will be offered. Among these requirements are usually the expected outcomes of the educational program and some reference to the methods for measuring those outcomes. The second responsibility is to monitor or oversee the charter school in some way; often, this responsibility rests with the authorizer and with one or more other branches of government (i.e. the state board of education). Within legislation, specific and/or general reports may be required.
Thirdly, at the core of the charter school theory, authorizers must use their authority to choose not to renew a charter for a school that has not met the terms of the contract-including expectations involving educational performance-or to revoke a charter when the operation of a school has clearly strayed from the original intentions. (Note 2) In a book on contracting in education, an idea that has some strong similarities to chartering, Hill and his colleagues describe a contract in this way: A contract is a promise to deliver quality education for children in return for public funds and a warrant to operate a school for some period of time. Some procedure is needed to make sure the school lives up to that promise. Relying solely on parent choice only holds the school responsible for the private benefits of education. (Hill, Pierce, & Guthrie, 1997, p. 67) In the case of charter schools, advocates have focused their rhetoric on renewal as the procedure that will ensure that these schools are meeting publicly desirable educational goals. This is consistent with Lake and Millot's argument that, "[e]ffective accountability requires an efficient means of terminating schools that fail to achieve their contractual requirements, particularly in the area of educational outcomes" (Lake & Millot, 1998, p. 20). The combination of performance expectations and a "contract" creates a theory of charter school performance accountability that rests on two key assumptions: Authorizers can assess the quality of education offered by charter schools, using test scores and, if needed, other methods; and, Authorizers will act on their assessments by revoking or not renewing charters that do not demonstrate that they are providing quality education.

The "New Accountability" in public education
While charter schools are generally considered to be outside the domain of mainstream educational reform efforts such as standards-based reform (cf. Smith & O'Day, 1991), changing ideas about educational accountability influence both. Calls for a "new accountability," according to Elmore, Abelmann, and Fuhrman, have three central components: "a primary emphasis on measured student performance;" "the creation of relatively complex systems of standards" used to make comparisons among schools, districts, etc.; and "the creation of rewards and penalties and intervention strategies to introduce incentives for improvement" (Elmore, Abelmann, & Fuhrman, 1996, p. 65). Within the traditional system, this new accountability has focused on the creation of aligned state standards and assessments that are more challenging and rigorous than in the past, and tools for recognizing and rewarding schools (and sometimes districts) that are rising above expectations and penalizing or offering assistance to schools/districts that are failing to meet them. In theory, "focusing on student performance should move states away from input regulations… and toward a model of steering by results" (Elmore et al., 1996, p. 65). In the theory underlying charter schools, the primary incentives for improving performance are the ability to continue to operate as a charter school by having a contract renewed and to be successful in the market by attracting students and the public resources they bring with them. The reverse of this are the clear sanctions for charters if they fail to improve performance-the removal of their contract, and the loss of students through the market.
Another facet of this new accountability, at least in some cases, is attempts to increase the involvement of parents and communities in school reform. This is done largely through the public reporting of assessment results. The expectation underlying public reporting is that it, "energizes parents and other community members to pressure schools for higher performance, particularly when data show differences in performance among schools that are roughly comparable in the public's eye" (Elmore et al., 1996, p. 67). Public reporting of a range of information, from test scores to attendance and graduation rates, has also been a piece of charter school accountability. However, in the case of charter schools, the purpose of public reporting has primarily been to influence the market; that is, provide information that will better enable parents and students to select among charter schools and between charter schools and other schooling options.
While some progress has been made towards the goals of the "new accountability" (Education Week, 1999), challenges are still abundant; "The reality of educational accountability at the close of the century involves contested standards, a problematic distribution of authority, weak incentives, variable capacity, and rudimentary technology" (Adams & Kirst, 1999, p. 464). Some of these same challenges have emerged for charter school authorizers, particularly regarding contested standards and variable capacity.

Performance accountability and charter school authorizers
While the approach of charter school authorizers to issues of performance accountability is the focus of this article, it is important to embed it within the broader context of the multiple ways in which charter school authorizers address accountability. Following a brief description of the data used here, I quickly describe how charter school applications and oversight by authorizers during the term of a contract are used as tools of accountability issues. This is followed by a more in-depth exploration of the oversight of educational programs and the renewal process.

Data
This research draws on a variety of sources of data. However, as an exploratory piece, it also raises questions and issues that need further consideration, and does not claim to offer a definitive discussion of charter school accountability. I draw on two research studies that examined charter school authorizers, a study of "New Regimes in Educational Governance" conducted for the Consortium for Policy Research in Education (CPRE) (Bulkley, 1999a) and the national study of charter school accountability conducted by the Center on Reinventing Public Education (CRPE) at the University of Washington (Hill et al., 2001). Both of these studies included interviews with charter school authorizers and charter school personnel; the latter involved authorizers and schools in six states (Arizona, Michigan, Georgia, Massachusetts, California and Colorado), and the former with schools and states in Arizona and Michigan. Evidence is also gathered from other recent literature on charter schools (i.e. Arsen, Plank, & Sykes, 1999;Center for Education Reform, 2000;SRI International, 1997;SRI International, 2000;Wells & others, 1998;Wohlstetter & Griffin, 1998).
A report based on the CRPE study was published in 2001 (Hill et al., 2001). This study explored a number of different aspects of the accountability issue, including market, government and internal accountability. That report provides an overview of some of the progress and challenges experienced by charter schools as they have sought to contend with the many forces placing demands upon them. In particular, the report offers a description of the strengthening internal accountability found in many schools (or accountability among immediate stakeholders in a school, including educators, parents, students and community members) and some of the issues faced in addressing external accountability, including accountability to charter school authorizers.

Applications
The first formal stage for any prospective charter school operator is to submit an application to a public organization allowed to authorize charter schools. The expectation of many policy makers and advocates of charter school laws was that these contracts would have very explicit performance objectives (Bulkley, 1999b). However, research suggests this if often not the case. For example, Hannaway's work on educational performance contracting suggests that specificity for performance in educational contracts is often low (Hannaway, 1999), and one study in California found that goals in contracts ranged from concrete and quantitative to informal and process-oriented (SRI International, 1997; see also Wells & others, 1998). In Colorado, charter school applications must explicate student performance standards, measurable objectives for student growth, and assessment and reporting procedures. But, in practice, some plans are very specific while others are "less susceptible to easy measurement" (Clayton Foundation, 1999, p. 51).
When charter school laws were first adopted, authorizers or potential authorizers with little or no experience in granting performance contracts were placed in a position of wanting or needing to evaluate and approve applications almost immediately. In addition, some authorizers were under extreme political pressure to allow some schools to get up and operating quickly. In this unclear and sometimes harried environment, applicants often went through a fairly minimal process (c. f. Bulkley, 1999a). (Note 3) The national situation may be improving, however. For example, every chartering agency in SRI's national survey reported that some or all of its schools had measurable goals in the area of student achievement (SRI International, 2000).
Since those early years, authorizers with more experience have developed clearer guidelines for applicants and, in some cases, clearer guidelines for the evaluation of applications. Many of these authorizers now require information that, they hope, will help to determine if applicants will be equipped to handle the administrative, financial and educational aspects of operating a charter school. (Note 4) For some authorizers working with a large number of schools, the process of granting a charter is seen as a way to influence school quality both by selecting the most promising applications and by having an application process that builds capacity. Thus, these authorizers believe, the process itself can improve the quality of the schools that open by forcing applicants to address some issues involved in operating a charter school that they might not have considered previously. According to a staff person at Central Michigan University, in the application process, "we're going to take you through a structured process, we're going to help you anticipate operational difficulties, we're going to help you prepare your organization so that the first day that you open the doors, you're going [to] be prepared to educate kids." Implicit in this focus on applications as an accountability mechanism is a belief, at least among staff at some of the larger authorizers, that if they make the process rigorous enough at the beginning, then they won't need to "worry" as much about the school in practice. While the theory of charter school accountability has generally focused on some interpretation of student outcomes, staff working for some authorizers expressed a belief that charter schools can be more accountable because of the initial application process. The more rigorous the process, they argue, the more accountable the school.

Oversight
Once an authorizer grants a charter and a new school opens or a pre-existing school begins to operate with a new governance structure, the authorizer is responsible for overseeing the school in a number of different areas. The authorizer must determine if a school is compliant with any applicable laws and regulations, as well as any specific provisions in the charter document. As a part of compliance, the authorizer needs to examine the finances of a school to check if they comply with spending and bookkeeping requirements and determine if schools are "functioning organizations." The following section examines their oversight of the educational programs of charter schools during the contract period.
Authorizers use a variety of tools for oversight, including required reports, site visits, parental complaints and surveys, outcome data, regular meetings and informal contact. There is considerable variation between authorizers as to the types of tools used and the frequency with which they are used, ranging from those who have very minimal contact with schools (generally limited to written reporting unless major problems arise) to others who supplement reporting with regular contact through visits, meetings and phone calls.
The most basic level of oversight for charter school authorizers is ensuring that schools meet legal requirements regarding compliance with state, federal, and local laws and regulations, with acceptable accounting practices, and with reporting requirements. While the theory underlying charter schools in most states revolves heavily around student achievement and school performance, a number of studies suggest that authorizers often focus their oversight on the familiar, such as compliance and financial stability, rather than on performance (Bulkley, 1999a;Garn, 1998;Henig, Moser, Holyoke, & Lacireno-Paquet, 1999;Hill et al., 2001;SRI International, 1997). In one California study, "school district officials note that, given all the ambiguity around student outcomes and what measures are valid, they are holding charter schools accountable more on fiscal, rather than academic, measures" (Wells & others, 1998, p. 19).
Regardless of whether or not compliance requirements directly impinge on the abilities of a school to operate the educational program it desires, there is clearly an opportunity cost for the schools related to compliance; the more time spent on regulatory requirements, the less time and money available for other purposes (Arsen et al., 1999).
Alongside general oversight, there is a subset of charter schools that require additional attention from their authorizers. These "non-functioning organizations" are experiencing major problems such as a substantial loss of students and/or staff, considerable infighting among staff, parents, board members or others in the school community, severe student discipline issues or major financial problems. Authorizers can identify schools that are completely, or in some aspect of their operations, non-functional through a number of sources. These include parental complaints, site visits, financial audits or reporting, and media "exposes." Often, multiple sources inform an authorizer that a school is having serious struggles.
In some cases, authorizers do nothing but simply monitor the situation unless or until it becomes severe. Other times, however, they become more engaged with the troubled school, often "behind-the-scenes." For example, authorizers can work directly with school boards and school leaders to attempt to resolve problems. In Michigan, some universities have taken an active role-sometimes apparent and sometimes behind the scenes-in schools that have been struggling; for example, authorizing staff have helped to identify governance problems in schools and promoted the replacement of board members and school leaders through advice to boards and leaders. Similarly, in Colorado, the school leader in one school visited for the CRPE study left in part because of pressures from the sponsoring district.
Finally, regular monitoring combined with detailed feedback to schools can be useful for authorizers and struggling charter schools. In Massachusetts, in-depth school site visits are one opportunity to assess how well a school is functioning. The two schools in the CRPE study authorized by the Massachusetts Board of Education both had difficulties early on, one where problems where largely governance-oriented, and another where they primarily involved the school's educational program. In each case, the site visit reports provided a form of technical assistance to the schools. By seeing themselves through the eyes of outside experts, school personnel were more able to identify and address their internal problems.

Examining the educational program and performance
In the theory underlying charter schools in most states, the schools are expected to offer an educational program that leads to improved student achievement (Lake & Millot, 1998). Authorizers can look at the outcomes of a school's education program only at the time of renewal or formal review, or they can monitor this program throughout the charter contract period as well.
Monitoring during the Contract. Early on in their work with charter schools, authorizers often focused their energy on ensuring that schools were functional organizations and were in compliance with fiscal and regulatory requirements. As authorizers have gained knowledge and experience with compliance issues, the time required for addressing these issues has been minimized and become more routinized. This has allowed them to turn increased attention to the educational programs offered by schools.
Authorizers use a number of different tools in monitoring academic achievement and a school's educational program. The most common, and certainly the most publicly visible, is student test scores. Some authorizers merely collect test score data, but do little with it during the charter contract period. In other cases, authorizers analyze the scores and provide information to the public and/or school personnel about these outcomes. A recent study by RPP International found that 85% of charter schools reported test scores to their authorizer (RPP International, 2000). However, it is important to note that the fact that student achievement is monitored, or tracked, does not necessarily mean that this data is utilized in decision-making and other actions by the authorizer. For example, in a study of California charter schools, 85% schools said they reported student achievement data to their sponsor, but only 4% said that the sponsor "had ever requested specific actions or imposed sanctions in response . . .." (SRI International, 1997, Part II, p. 16, emphasis in original). This potential lack of use of achievement data is tied closely to a lack of clear standards for schools; as Wohlstetter and Griffin found, "sponsoring agencies, in general, required assessment information on performance from charter schools… but often failed to specify any clear performance standards or consequences" (Wohlstetter & Griffin, 1998, p. 15).
Test scores are not the only information used by authorizers to assess a school's educational program, as the following two examples demonstrate. In Massachusetts, where charter schools negotiate an accountability contract with their authorizer at the beginning of their charter, schools submit an annual report that includes, among other things, a discussion of their progress towards the goals of their accountability contract. Authorizers in several other states also require annual progress reports; these vary from one-page commentaries to more elaborate formal reports that include specific examples and evidence demonstrating educational progress. In Michigan, one university has recently been pushing charter schools to develop goals that are "clear, concise and measurable," in response to concerns that goals in charter contracts have at time been imprecise and progress on them difficult to assess. However, it is unclear what, if any, formal actions are taken if a school's educational program is seen as inadequate by authorizer staff.
Another way of assessing a school's educational offerings and aiding in improvement is through the use of outside organizations. For example, there a number of organizations that either accredit schools or are planning to offer accreditation, including several state charter school organizations. Two school districts in the CRPE study have required that the schools they authorize become accredited by their respective state charter organizations when this becomes possible.
In general, during the period when a charter contract is in effect, authorizers may be more focused on the day-to-day issues of compliance and dealing with non-functioning organizations than they were on broader issues of accountability for the education program schools offer. Even within the domain of "educational accountability," authorizers cite actions such as ensuring that teachers are certified and the school's curriculum is properly aligned with state standards-while these certainly may influence the academic program offered by schools, they are not the kinds of "new accountability" tools that charter advocates have emphasized. Nationally, such monitoring of performance has seldom led to charter revocation (SRI International, 2000).
Renewal. The renewal process for charter schools varies considerably across states and across authorizers. The process and criteria for renewal can be fairly clear and defined, or undefined to the point of great confusion for school personnel. The renewal process has the potential (not necessarily attained) to be a serious undertaking that holds the genuine possibility of a school not receiving a new charter, thus providing schools with a strong external incentive.
The Massachusetts Department of Education is probably the authorizer whose renewal process is most frequently held up as a "model." In Massachusetts, a number of different data sources are used in the renewal process. The most elaborate piece involves an evaluation/renewal inspection by an outside team hired by the authorizer. These teams, which consist of experienced educators and others, conduct 3-4 day visits at each school up for renewal and write a detailed report on their findings for the authorizer. A number of schools have come up for renewal, and all have received new charters along with suggestions for improving their programs. Among other large authorizers, few have developed as clear or rigorous a procedure as has the Massachusetts DOE. For example, in the spring of 1999, one Michigan university's office that works with charter schools recommended that the university board renew all the schools whose contracts were due to expire. The materials provided to the university board included test score data on the MEAP (the Michigan state test) and another national standardized test, alongside other information on school goals. The test scores were mixed, with some schools and grades showing marked improvement and some showing little or negative change; how good is "good enough" was not spelled out either formally or informally to these schools or the university board. Other goals tended to emphasize participation rather than performance, such as attendance in foreign language or computer classes. At another Michigan university that has authorized a significant number of schools, staff said that they expected all their schools to be reauthorized before the renewal process had even begun.
Small authorizers (those that have granted only a few charter contracts) vary considerably in their approaches to renewal and are unlikely to be as clear or thorough as larger authorizers that have more resources and capacity at their disposal. Some have few clear procedures and criteria for renewal. Among the case study schools, those whose charters were granted by small authorizers were generally not very concerned about renewal. For example, personnel at the Georgia Department of Education have emphasized to the two CRPE schools the need for schools to have specific performance outcome goals in their charters, yet there was little focus on performance as an aspect of the renewal process at either school.
Some of the small authorizers have or intend to utilize external information sources in their renewal decisions, including accreditation (discussed above). For example, one district in Colorado hired an independent evaluator to evaluate a school in the year it was being considered for renewal. One California district used an external review of a charter school that was done for other district purposes in its renewal decision alongside an external study completed specifically for the renewal process. While authorizers are using a variety of tools to evaluate a school when its charter is up for renewal, the authorizers in the CRPE study generally expected to renew the charters they had granted when the time came.
Overall, for many authorizers, accountability to government has primarily focused on issues of financial and legal compliance, with some monitoring of educational programs (primarily through test scores). However, it is unclear to what extent government authorizers are using educational monitoring or evaluation to make serious decisions about the operation of individual schools. This challenge to performance accountability is recognized by both advocates and critics of charter schools; as one study noted, in practice, "accountability typically means a half-baked version of the top-down regulation-and-compliance system that the state or community applies to its conventional public schools" (Finn et al., 2000, p. 135).

The Accountability Bind
Charter school authorizers vary in important ways, such as their emphasis on compliance vis a vis oversight of educational programs, their beliefs about the proper role of an authorizer in the day-to-day operations of schools, and their faith in test scores as an appropriate measure of quality. These differences can be explained as a combination of differences in the will and capacity of authorizing agencies. For example, the charter schools office in the Massachusetts Department of Education has operated in a political climate where there is a considerable interest in close monitoring of charter schools, and the office has responded in kind. This contrasts considerably with the two state boards in Arizona, where there is little political will for strong oversight; that political will has been translated into the selection of staff who are supporters of a more "laissez-faire" approach to authorizing, and into minimal funding for charter staff which effectively limits the oversight potential regardless of the inclinations of the staff members.
As the study by SRI International notes, while the vast majority of charter schools have yet to come up for renewal, those that have are generally receiving new contracts (SRI International, 2000). This is consistent with the information provided by the school and authorizing agency staff interviewed for the CRPE study. Interviews with authorizers and a review of the literature on charter schools suggest that they all share a number of significant challenges to closing schools, despite considerable and very important variation across authorizers in their approaches to the renewal issue. These challenges are tied to the some basic assumptions underlying the idea of charter school accountability. Four challenges are: Educational performance is not simple to define or measure, nor is how good is "good enough" in educational quality.

1.
Other aspects of a school's program, often more difficult to measure than test scores, are also important to families and authorizers. In this context, authorizers sometimes turn to "proxies" to assess school quality.

2.
Teachers, parents and students become very invested in particular schools and destroying a community may be more difficult for authorizers than serving a diffuse public interest.

3.
Finally, charter schools have become a highly politicized issue on both sides, and some authorizers are concerned about their decisions reflecting poorly on charter schools as a reform idea.

4.
I call this constellation of challenges the "accountability bind," as authorizers are stuck between wanting to enforce accountability through renewal, but finding doing so fraught with difficulty. I am not suggesting that no authorizers are taking performance accountability seriously-some clearly are -but that acting as the originators of charter schools intended can be very difficult for authorizers.

Challenge 1: Defining educational performance with test scores
One of the greatest challenges for states attempting to develop accountability systems as part of standards-based reform is to create systems that are considered by those within education and the public to be fair and defensible (Elmore et al., 1996). Thus, in order to attach consequences to performance measures, they must be viewed as valid and reliable measures that appropriately distinguish between schools where a desirable level of learning is taking place and others where learning is inadequate. For charter school authorizers, these demands of fairness and defensibility are even greater than for state assessment systems, because the sanction of revoking or not renewing a contract is so high. This challenge sits alongside the challenge of using standardized tests to assess the performance of individual charter schools, which often aim to offer non-traditional educational programs. While authorizers are clearly monitoring the test scores of students in charter schools, it is extremely important to distinguish between gathering information about school quality and using that information for improvement and accountability purposes. Some authorizers have worked with schools to develop expectations that are more aligned to the school's stated purpose than an off-the-shelf standardized assessments, but this makes comparisons with other schools -one way of determining if a school is successful -all the more challenging. As well, charter schools often must adapt their programs to serve the needs of students who enroll at the school, who may or may not match the students expected by the founders; this can lead to a situation where meeting specific contract goals may be an unreasonable expectation. As the Hill et al study notes, "Finding ways to measure not only student achievement on standardized tests but the value-added qualities of charter schools has proven to be a challenge" (Hill et al., 2001, p. vi).

Challenge 2: Assessing quality beyond test scores
Authorizers' uncertainty about closing schools is also tied, at least in some cases, to staff members who feel that the essence of a school can never be captured by quantifiable measures, and who instead rely on a sense of "feel" about the quality of the school. As one staff person said, "[you need to] look into the eyes of the kids and the teachers and if you see the magic… you know you've got something good happening." While this attention to "feel" may well capture aspects of the educational experience offered by a school that would be missed by test scores and other such measures, it is inconsistent with the idea that charter schools should continue to operate based on their ability to demonstrate academic performance.
The challenges of determining "success" have often led authorizers to "proxies" that, they believe, provide indicators about the performance of individual schools. Interestingly, proxies generally seem to be used to justify an authorizer's assessment of "success" in a school, and rarely to argue that a school is not doing well. The most obvious proxy authorizers use is parental choice and satisfaction. While markets and government authority are intended to work separately in creating accountability for charter schools, authorizers-when asked to explain success-frequently point to the market. For example, the existence of waiting lists for slots at a charter school is often given as proof or evidence of the school's worth. The challenge, of course, is that if market accountability is created (obviously) via the market, and government is turning to the market to demonstrate success or performance, than the ideal of joint accountability for performance between markets and government is compromised.
Authorizers also point to the "mark of approval" given by external organizations as evidence of success or an increased likelihood of a successful school. One such mark is accreditation, both from organizations that pre-date charter schools (i.e. accreditation from the North Central Association Commission on Accreditation and School Improvement ) and those that have sprung up specifically to work with these schools (mainly state charter school associations). The approval given by these organizations is a sign of quality in the eyes of many authorizers, although the lack of accreditation is not generally seen as problematic. (Note 5) Finally, educational management organizations (EMOs), sometimes called service providers, have become increasingly common in a number of states (in Michigan, over half of all charter schools now use a service provider). These organizations can offer a variety of services, from simply keeping track of a school's finances to running the entire school, including providing the facility, hiring the staff, and selecting the educational program. Comments by authorizers suggest that, when one school operated by an EMO is seen as successful, than other schools run by the same company are more likely to be seen as successful. As with the proxy of accreditation, this reliance on EMOs may or may not be appropriate, but it is still an indirect mechanism for determining success. These examples suggest that proxies for quality often involve organizations or individuals that are largely outside the government domain, and do not themselves have a direct obligation to serve public purposes.

Challenge 3: The preferences of charter school communities vs. the "public interest"
While authorizers have difficulty with determining what is and is not a successful charter school, they have even more difficulty deciding that a school is unsuccessful enough to justify as high a sanction as closure. Authorizers serve a diffuse public that has general needs for high quality education, but their day-to-day reality centers around working closely with charter school operators and communities of teachers, parents and students who are heavily invested in the continuing existence of a particular school. As one staff member with an authorizer said: The theory sounds great about shutting schools down. The practice is much more difficult. And whether you're revoking a contract or not renewing a contract, they're both gonna be tremendously difficult… the reality is you've got teachers, you've got parents, you've got a community that's now used to the school, depending on the school, loving the school and either way you're gonna have difficulties.
This attitude towards renewal appears to be common among authorizers, despite their recognition of the political rhetoric arguing that charter schools should not be renewed if they can't prove they are improving achievement. The turmoil created for a specific group of educators, parents and children that results from not renewing a charter may outweigh the difficult to define the ideal of performance accountability for authorizers. Their relationships with charter communities bring them into regular contact with parents who are convinced of the value of individual schools and schools that have long waiting lists (with or without "proof" of academic quality), and this positive response of the market can have a profound impact on the thinking of authorizers, despite the ideal separating accountability to the market from that to government.

Challenge 4: Politics and the charter school movement
Finally, a number of authorizers are themselves politically invested in the success of the charter school "movement." The authorizers I have studied, particularly those who are involved with a large number of schools, are generally staffed by people sympathetic to charter schools. Even if they personally support the ideals of performance accountability, they may be hesitant to close schools because of fear that this will be seen as a failure of the general charter school ideal.

Finding the middle ground
Overall, the forces working against the closure of charter schools based on student performance seem likely to result in schools being renewed at a very high rate. What seems to be needed is a less all-or-nothing definition of accountability-more of an ongoing relationship than a single decision point. There are a number of ways authorizers are currently working to establish more productive and educationally substantive accountability relationships with charter schools through "middle grounds" that allow them to take some action short of actually revoking or not-renewing a charter contract. These middle grounds include: focusing on applications as a form of "input" accountability, direct or indirect intervention in schools, and capacity building/technical support.
As described earlier, authorizers (especially large authorizers) have increasingly emphasized the importance of the application process as a quality control mechanism. When asked about how schools are held accountable, several staff members working with authorizing entities responded in part by discussing the rigors of the application process. In light of the difficulty of closing schools, it makes sense that authorizers pour considerable time and energy into doing what is in their power to see that new charter schools have the greatest potential to be successful.
Authorizers can also intervene once schools have begun to operate, especially when problems are present, through direct or indirect methods. Direct intervention involves requiring that schools do something. These interventions often involve compliance issues and schools that are struggling to be functional; for example, a school with financial problems might be told to undergo an external audit. A study by SRI International examined "corrective actions" taken by authorizers, including not only revocation and non-renewal but also probationary measures; they found that such corrective actions were most often related to fiscal or management issues (SRI International, 2000). Direct interventions linked to the educational program appear less common, but do occur. For example, in Arizona, the State Board of Education required all schools it had authorized that had average test scores below the 35th percentile to provide information explaining these scores. (Note 6) While direct interventions happen, indirect interventions seem far more common. These often take the form of discussions about problems between school and authorizer staff, including problems around issues such as staffing, curriculum, test scores, and goals.
Finally, authorizers seek to find a middle ground by providing schools with technical assistance or other capacity-building measures with the hopes that the increased capacity will result in higher achievement and better functioning organizations. Technical assistance can include professional development or, more often, suggestions of where to turn for professional development. Another form of capacity-building is to send external reviewers to schools, with the expectation that they will provide summative evaluations and formative evaluations that offer suggestions and insights that school personnel might find useful. This form of review has been used since the beginning in Massachusetts, and has also appeared in a variety of other states. For example, one California charter school authorized by a district that is very focused on test scores and student achievement experienced declining scores. The district responded by providing an external review designed to offer suggestions for improvement. Finally, authorizers can work to build capacity among parents for making good choices (i.e. through providing information), attempting to improve quality through improving parental decisions in the educational market.
One of the challenge of technical support designed to help schools improve themselves, however, is similar to that faced by states and districts trying to reform "under-performing schools"-it is not always clear to either authorizers or schools what exactly is needed to improve performance. This challenge of an "unclear technology" (Tyack, 1993) is obviously not unique to the charter context, but it may be exacerbated by the separation between charter schools and other entities that may be in a position to support change and provide expertise (i.e. school districts). As Elmore and his colleagues comment, "performance-based accountability systems depend heavily for their success on whether school administrators, teachers, parents, and students know what to do to improve performance" (Elmore et al., 1996, p. 92, emphasis in original).
Attempts by authorizers to find a middle ground, some set of actions that allows them to influence charter schools without taking the undesirable step of closing them, has led to a variety of creative approaches. However, there is a serious challenge for the theory of charter school accountability in an increased focus on middle grounds and a lesser focus on contract renewal.
The way in which charter advocates have presented the idea of a performance contract is, fundamentally, as an "all or nothing" proposition. In arguing for a broad-based system of contracting, Hill and his colleagues make the case that, "Contractors who failed to provide instruction as promised, or whose students' outcomes were low and not improving as anticipated, could be fired or given an ultimatum to improve or be replaced" (Hill et al., 1997, p. 70). In addition, they argue that, "accountability only means something if there are consequences when children do not learn" (Hill et al., 1997, p. 68). The rationale behind the additional autonomy provided to charter schools is that there will be real and significant consequences if performance is not demonstrated.
If, as appears to be the case, renewal is not always (or even often) serving its intended function as a mechanism for performance accountability, the autonomy/accountability bargain on which charter schools are based is out of balance. Given the problems inherent in an all-or-nothing renewal decision, the goal is to make sure that accountability for educational performance is maintained throughout the contract by building in a range of incentives, supports, and consequences less drastic than non-renewal. The challenge is to do this without the authorizer becoming so involved in the day-to-day operations of charter schools that the autonomy that makes them distinctive is threatened.

Conclusion
Charter school authorizers, in the years since states began passing charter school laws, have had to develop a set of procedures and standards to work with public schools governed in a very different way than the hierarchical, bureaucratic method traditionally followed. While their focus has rhetorically been on school performance, they have come up against many of the challenges that the public education system has faced in determining performance and meting out rewards (continued operation in the case of charter schools) and punishments (revocation or non-renewal).
However, internal accountability mechanisms in charter schools may be compensating for the lacking external accountability envisioned by some charter advocates; internal accountability involves, "a set of productive and mutually responsible relationships among teachers, administrators, and parents, united on behalf of effective instruction for children" (Hill et al., 2001, p. iv). The charter school authorizer-charter school relationship has shown itself to be different in most cases than the traditional school-district relationship (with the most common exceptions being districts authorizing public school conversion charters). However, it has often not met the ideals of those charter advocates who envisioned a rigorous authorizer accountability system that only continued to support those schools that had unequivocally improved student achievement. So, where does this leave us? These findings are preliminary, and other states with more explicit laws may have authorizers relying more heavily on renewal and revocation as accountability mechanisms. However, the incentive structure that is operating for authorizers described here seems likely to apply to authorizers more generally. In this environment, one could simply argue that the fundamental theory of charter schools is flawed, and thus this reform effort should be placed in the "loss column." On the other hand, one can turn to other alternatives to support charter school accountability, such as internal accountability and the marketplace and one can focus on middle grounds, including those discussed here and others that will be created over time as authorizers grapple with the challenges of using renewal and revocation or accountability. One possibility, which has its own difficulties, would be to separate authorizers from those who make decisions about renewal. In theory, this would remove the political disincentives of non-renewal and the personal relationships that can convolute renewal decisions. However, a risk in this approach is that authorizers who have developed relationships with schools are likely the entity in the best position to assess the quality of an individual school.
Charter schools are simply one form of contracting in public education. Other forms, such as when a school district contracts directly with a private provider, may encounter similar challenges, i.e. the uncertainty of assessing quality and the reticence to invoke high stakes. However, there are some important differences. The most important such difference is that the stakes in the case of charter schools are so incredibly high. When a district contracts with a private provider, ending the contract generally does not mean ending the existence of the school. In this environment, the theory of high stakes and the actuality of high stakes for private providers may be more closely aligned. (Note 7) In this article, I have questioned one of the fundamental elements of the charter school theory-that accountability to government can be achieved through "all or nothing" contractual arrangements. The accountability bind that this creates for authorizers seems unlikely to change. Thus, charter school advocates may be well advised to reevaluate and adjust their theory-and possibly their rhetoric and legislation-in light of the questionable value of such a heavy reliance on revocation and renewal for ensuring performance accountability. The real risk is that there will be no change in the status quo, and the ideal of charter schools as jointly accountable to the market and government for educational performance and quality will simply be lost, with the market dominating the accountability equation.

Notes
Arizona and the District of Columbia allow charters of up to 15 years. 1.
All states allow charters to be revoked or not renewed for reasons related to educational performance. However, allowing performance to be a criterion and requiring it are different things. Several states (none of which were included in the CRPE study), including Texas, Louisiana and New Hampshire, require in their legislation that schools must demonstrate acceptable levels of performance or improvement, based at least in part on test scores (Lake and Millot, 1998). It is unclear what effect this will have on renewal in those states.

2.
One clear exception to this was applications submitted to the Massachusetts Board of Education, which had a more elaborate application process from the beginning. 3.
The application process can include intensive interviews with prospective authorizers, site visits to potential facilities, and reviews of the intended curriculum and personnel or type of personnel to be hired. 4.
This appears to be changing, at least in some cases, as more authorizers are requesting that their schools go through an accrediting process.

5.
For one school that served a largely at-risk student population, the 35% cut-off point was seen as arbitrary and inappropriate as a tool for judging their success with their particular student body.

6.
While charter school advocates have tended to be skeptical of public school conversions, such schools may in fact be more accountable in certain ways than "new start" schools because there is the real possibility that the charter could be removed since the school itself would continue to operate.