Measuring Competition : Inconsistent Definitions , Inconsistent Results

There is a developing literature examining how charter schools, through the effects of competition, impact performance in public school districts and district-run public schools, also known as the second-level effects of competition. What follows is an examination of how competition is measured in this literature that offers a critique of existing approaches to that measurement. Findings in these studies are problematized by inconsistent findings in other, similar studies; inconsistencies which may be due to inconsistent definitions and metrics of competition. I suggest a more specific definition of competition and suggest that other disciplines may offer guidance in the pursuit of a more consistent measurement of competitive effects.


Introduction
Market-based educational reforms have enjoyed great expansion in recent decades, both in the United States and globally (Lubienski, 2009;Lubienski & Linick, 2011).In light of attention from policy makers, researchers, and the media, there have been many studies of the effects of competition on public school districts and district-run schools; however, these study designs have lacked consistency in how competition is defined and measured.Perhaps, the lack of a consistent definition for competition is partially responsible for the inconclusive evidence supporting or condemning the use of competition as a method of educational reform.While understanding the role of competition in driving improvement is important, without a clear definition of competition and how to measure it, educational researchers will struggle to accurately quantify the ways these reforms impact students.
In the United States, there are many forms of school choice: charter schools, private schools, magnet schools, vouchers, tuition tax credits, homeschooling, and simply moving one's family to a new local school district-typically referred to as Tiebout choice, named for economist Charles Tiebout, the process by which residential choices determine the quality of, and level spending on, local public goods.Essentially, Tiebout choice demonstrates that people will move to localities that tax and spend on local goods at levels that reflect their personal (or familial) priorities-in this case public education (Hoxby, 2001).Families that prioritize public education, and can afford it, will move to higher taxing districts with better public schools, while families that do not prioritize public education, or do not possess the resources to relocate, may live in areas with lower tax rates.Proponents of market-based education reforms like charter schools and vouchers posit that such reforms provide families without the financial capital to move to the school district of their choice with viable alternatives to nearby publicly-run school districts.Indeed, the idea that parents and families should have some control over the education of their children is widely embraced, and reflected in recent federal policies including No Child Left Behind and Race to the Top (Berends, Cannata, & Goldring, 2011).
Many advocates of school choice policies claim that such policies, in addition to offering alternative options, will improve the performance and efficiency of existing public schools by exposing them to competition and forcing schools to compete for students, and ultimately the revenue attached to each student (Chubb & Moe, 1990;Friedman, 1955;Hess, 2001;Hoxby, 2003).Currently, the most popular and quickly growing school choice initiative is the expansion of charter schools (National Alliance for Public Charter Schools, 2012;Teske et al., 2001).The assertion that competition, whether created by charter schools or other market-based reforms, will improve performance in all schools is widely embraced by advocates and the media.For example, Michelle Rhee, former Washington D.C. School Chancellor and now author, pundit, and activist has said, "I think the notion that somehow by introducing competition, whether through charter schools or vouchers, for low income kids that somehow that is going to be a detriment to a system, I actually think that the exact opposite is true" (Jones, 2011).Also, Mary Sanchez, a reporter and editorial columnist for the Kansas City Star wrote, in an article titled Charter schools bring competition to education, "It's part of the competition that is the promise of charter schools.If they don't make the grade, they shutter or reconfigure.If they do well, they raise the bar for all schools" (Sanchez, 2013).
The effects of schools of choice, such as charter schools, voucher accepting private schools, or schools that are part of an inter-district choice scheme, on students attending schools of choice can be referred to as the first-level effects of those schools.The effectiveness of charter schools at improving performance for students in charter schools, a first-level effect, has been studied by many scholars and the results of such studies vary, which is not surprising considering the different methods used to study these schools, the differences between the schools, and the differences in the local and political contexts in which the schools exist (Winters, 2012).There is a growing, yet also conflicted, body of literature that examines the second-level effect of charter schools-the effect a charter school has on the performance of students and schools already in operation (e.g., a local public school).Economists have predicted (Chubb & Moe, 1990;Friedman, 1955;Hoxby, 2003) that the introduction of competition, such as that produced by charter schools, into the educational marketplace will improve educational outcomes for all students.While charter schools have the potential to impact private schools, home schooling, and other educational options currently available to parents, the studies of second-level effects of charter school competition on students attending public school districts and district operated schools are what will be considered here.
There are many well-executed, rigorous studies of the effects of market-based reforms; however, whether these studies capture the true effects of competition and not simply the effects of choice, autonomy, or policy-specific context is not clear.Although inter-district choice, vouchers, and charter schools are all popular reforms designed to inject competition into the educational marketplace and all of these policies can be examined to learn more about the effects of competition on school districts and district-run schools, charter schools have received most of the attention in recent years from educational researchers.Charter school policies have enjoyed vast expansion throughout the United States along with bipartisan support, unlike voucher policies.For the purposes of this examination, I will focus on how researchers have examined the second-level effects of charter schools and how that focus has lacked consistency.Also, while charter schools may impact the entire educational landscape drawing children from private and home schools, here I focus on studies that have examined how charter schools impact public school districts and districtrun public schools.Theoretically, charter schools are expected to drive innovation and school reform in a number of ways: by reducing bureaucracy (Chubb and Moe, 1990), promoting collaborative educational conditions (Fact Sheet: Race to the Top, 2009), and improving efficiency in district-run public schools by generating competition.Competition has been demonstrated to improve efficiency in other markets, such as healthcare and trucking and parcel service (Hoxby, 2003).
Despite the growing focus on the second-level effects of charter schools, one of the central, yet unresolved, issues in the discussion of charter schools is if competition generated by charter schools leads to improved efficiency and performance in district-run public schools (Ni, 2009).The impact of charter schools is most felt through second-level effects, as the vast majority of students still attend district-run public schools, and it is likely to remain that way for some time (Booker, Gilpatric, Grongerg, & Jansen, 2008;Ni, 2009).Even in Ohio, a state with multiple choice programs, including charter schools and vouchers, 79.80% of students attend district-run public schools and only 4.49% of students attend charter schools.Therefore, "a better understanding of the effect of choice (and hence competition) on the behavior of parents and school officials is crucial in assessing current reforms…" (Ghosh, 2010 p. 440).Like charter schools themselves, the studies of second-level effects vary greatly by examining different contexts and measuring different things.
The expansion of school choice reforms such as charter schools, private school vouchers, and tuition tax credits has led to a quasi-market, rather than a true market with perfect competition (with multiple providers, differentiation is product, and easy exit and entrance between providers) in education (Lubienski & Linick, 2011).In educational markets the core of the services provided are somewhat uniform (Brown, 1992), and still largely funded by public monies.Educational quasimarkets exist somewhere between perfect competition and pure monopoly (with a single provider and no exit option), wherein multiple providers provide largely similar products with little price differentiation-this is also known as monopolistic competition (Lubienski, 2003).Despite the fact that educational organizations do not experience pure competition, there is still interest in how the monopolistic competition created by the market-based reforms impact educational outcomes.I argue that to quantitatively validate, or invalidate, the claims made about the effects of competition on educational organizations, an empirically validated measure of competition must be developed so that studies of the effects of competition are reliable and comparable.Further, there must be agreement about the definition of competition.Quality syntheses of the second-level effects of charter schools have been presented, and contribute greatly to this discussion (Ni, 2009;Ni & Arsen, 2010).My purpose here is to build on the previous work of Ni (2009) and Ni and Arsen (2010) and discuss the inconsistent definitions and measurements of competition and how such inconsistencies obscure our understanding of the actual second-level effects of charter schools by blending multiple concepts under the broad definition of "competition."Many studies claim to examine the effects of competition, when, in fact, the study is actually examining the effect of choice.While competition does require choice, in order to examine the effects of competition on an educational organization, the organization must respond to other choices being offered.This distinction between choice, and an organization reacting to the presence or threat of choice, may be the explanation for such variation in the literature of the second-level effects of charter schools.I also begin to explore potential remedies for this problem.Currently, there are many self-described studies of competition, yet this literature lacks the consistency necessary to provide concrete evidence of competition's role in educational outcomes.

Measuring Second-Level Effects of Charter Schools: models and measures
The existing quantitative literature on the second-level effects of charter schools on the performance or efficiency of public school districts, district-run public schools, or students attending such schools encompasses many studies.These studies, using a variety of methods and various measures of competition in many different contexts, have found that charter school competition either improves (Bohte, 2004;Booker et al., 2008;Holmes, DeSimone, & Rupp, 2003;Hoxby, 2003;Sass, 2006;Winters, 2012), impairs (Arsen & Ni, 2012;Bettinger, 2005;Carr & Ritter, 2007;Imberman, 2007;Ni, 2009), or has no effect (Bifulco & Ladd, 2006;Buddin & Zimmer, 2005).The inconsistency of findings within and across empirical models does not indicate that any particular model is superior, but does emphasize the lack of consensus about how to define and measure competition.Even a single study using multiple quasi-experimental models failed to find consensus (Imberman, 2007).

Varying Methods of Examination
When attempting to examine the causal impact of charter school competition on student performance, there are two challenges that must be addressed: charter school locations are endogenously chosen, and students endogenously self-select into charter schools (Ni, 2009).The use of instrument variable estimation (IVE) or fixed effects regression address issue one, and to address issue two researchers should include past performance, school composition, or account for unobserved student heterogeneity (Ni, 2009).Most studies quantitatively examining the second-level effects of charter schools employ either fixed effects regression (Imberman, 2007;Ni, 2009;) or IVE (Bettinger, 2005;Holmes et al., 2003;Imberman, 2007), though difference-in-difference is used as well (Hoxby, 2003).
In an effort to identify the exogenous second-level effects of charter schools, several studies have made use of IVE models.Such models, in an effort to carve out the exogenous impact of the competition rely on an instrument variable (Murnane & Willett, 2011), in this case a variable that is related to the location of the charter but not related to student performance.While this method is helpful for examining causal effects without establishing a random control trial, different data and contexts require some creativity on the creation of the instrument variable.One study used the distance from the district-run public school to nearest charter authorizing university as the instrument variable (Bettinger, 2005); another study used the number of available spaces, or "building stock" for charters to locate as the instrument variable (Imberman, 2007).The use of IVE models has not resulted in consistent findings across studies, some studies have demonstrated that district-run public schools near charter schools perform worse than similar schools not near charter schools (Bettinger, 2005;Imberman, 2007), and another has shown that the second-level effects of charter schools improved district-run public school performance (Holmes, DeSimone, & Rupp, 2003).
Accounting for student and school fixed effects is another popular method scholars employ in an effort to examine the exogenous second-level effects of charter schools.Fixed effects control for the variation within observed units, in this case within student performance across time and within schools.In addition to including student and school fixed effects, some studies have also included "spell effects" which are time invariant student and school factors (Bifulco & Ladd, 2006;Booker et al., 2008;Sass, 2006;Winters, 2012).Scholars argue that by accounting for these factors in the analysis, studies are accounting for any observed variable bias that may endogenously impact the second-level effects of charter schools, and are therefore reporting unbiased results.Like IVE models, fixed effects models have not resulted in consistent findings across studies; some studies have found that the second-level effects of charter schools negatively impacted district-run public schools (Arsen & Ni, 2012;Ni, 2009;), while other studies using fixed effects have found that the competition generated by charter schools has benefitted district-run public schools (Booker et al., 2008;Sass, 2006;Winters, 2012).Some studies have found that charter schools do not impact district-run public schools at all (Bifulco & Ladd, 2006;Buddin & Zimmer, 2005).

Varying Measures of "Competition"
More striking than the variation in methods used to measure the second-level effects of charter schools is the variation in how competition is measured.There is a substantial amount of research examining competition through the lens of charter school presence (Bettinger, 2005;Bifulco & Ladd, 2005;Holmes et al., 2003) and the market share of charter schools (Arsen & Ni, 2012;Hoxby, 2003;Imberman, 2007;Ni, 2009;Winters, 2012).Some studies, in an effort to examine competition as comprehensively as possible, have used measures of presence and market share (Buddin & Zimmer, 2005;Carr & Ritter, 2007) or combined measures of presence and market share (Bohte, 2004;Booker et al., 2008;Sass, 2006).Though there was little consistency in findings derived from these disparate methods of analysis, one would hope that similar measures of competition would result in similar findings.
Studies examining competition solely through the lens of proximity and density of nearby charter schools have found that the presence of charter schools improve (Holmes et al., 2003), impair (Bettinger, 2005), or do not impact district-run public school performance (Bifulco & Ladd, 2005).Likewise, studies examining only the market share of charter schools, as a proxy for competition, have found that district-run public schools benefit (Hoxby, 2003;Winters, 2012) and are harmed (Arsen & Ni, 2012;Imberman, 2007;Ni, 2009) by increased market-share for charter schools.Buddin and Zimmer (2005) found no effect for any measure of competition including measures of proximity, density, and market share; and, Carr and Ritter (2007) found small, negative effects with measures of presence, density, and market share.An aspect of competition that was not examined in many studies was duration of competition.In studies that did employ a measure of duration, Booker et al. (2008) found that sustained charter school presence had positive, significant outcomes for district-run public school performance, Arsen and Ni (2012) found that increased charter school market share over time negatively impacted district-run public schools by generating fiscal stress, and Ni (2009) found that increased charter school market share over time resulted in decreased performance for students attending district-run public schools and lower efficiency for public school districts.
The trend of no consensus ends with studies that examine both the density of charter schools in a given area and the market share enjoyed by those charter schools.In all three studies measuring competition as a function of both density of charter schools and market share of charter schools, charter school competition was found to positively impact the performance of district-run public schools (Bhote, 2004;Booker et al., 2008;Sass, 2006).While it is possible that studies that proxy competition through market share and market density are capturing a different aspect of charter school second-level effects than intended, the findings using this kind of measure suggest more consistency than other measures.

The Challenge of Linking Charter Schools to Competitive Effects
There are many complications that should be accounted for when attempting any measure of the second-level effects of charter schools.For example, the pre-charter educational landscape of private, public, and alternative educational agencies can complicate measures of competitive forces, subjects of study, and outcome measures.Of additional concern, is how charter schools may impact the financial and policy landscape in any given district.Students transferring from public school districts to charter schools can impact the resources available to a public school district.Also, how charter school policy is written matters, as does how, where, and when it is implemented.Charter school advocate Jean Allen said, "If a charter school law isn't strong, school choice options minimal or non-existent, digital learning exists for the few over the many, and teacher quality measures are not assured, students will not have opportunities they need and deserve" (Center for Education Reform Press Release, 2013).How and if a school or district responds to charter school competition may depend greatly on the pressure, or lack of pressure, inherent in charter school policies (Ni & Arsen, 2010).Though quasi-experimental analyses have been used to examine the effects of charter school competition in many states, expecting homogenous results from heterogeneous policies is naïve.
Arguably, there are many ways in which a charter school or charter school policy can induce (or mitigate) change in a public school district, and only one of these avenues is competition (Linick & Lubienski, 2013).If a public school even chooses to respond to a nearby charter school, the possible responses are not limited to competition and may include accommodation, collusion, and cooperation (Ni & Arsen, 2010).Though it is possible to statistically isolate how changes in the educational landscape, such as increasing numbers of charter schools or growing market shares of students leaving district schools for charters, are associated with gains in academic improvement, it is not possible solely through the use of quantitative measures to determine how such changes are due to competitive responses from district leadership, other interactions between the charter and district schools, or other unobserved contextual variables related to the change in the educational landscape.A study of the effects charter schools on academic outcomes for district-run public schools may not be measuring the effect of competition at all, but merely the interaction between different types of schools.For this reason, the term "second-level effect" is a much more accurate label for the outcomes associated with charter schools and district-run public schools than "competitive effect." Current approaches to the study of charter school competition are based on a series of assumptions about how charter schools and public schools interact.First, the observation that charter school density, proximity, or market share is generating competition because of effects, does not account for any effects that could be generated through choice, autonomy, collusion, or cooperation.Second, the assumption that the presence of charter schools is inducing a competitive response is flawed as institutional factors and environmental factors may prevent a public school district or school from responding (Linick & Lubienski, 2013;Ni & Arsen, 2010).Lastly, though many studies claim charter school "competition" is generating effects, a more accurate description would be that charter school proximity, density, and/or market share is associated with a change of outcomes at district-run schools.As seen in Table 1, institutional factors have the potential to disrupt any measure of competition; methodologically, researchers approach these concerns and attempt to compensate for them through the use of quasi-experimental models.However, the environmental factors below complicate how studies have traditionally approached these measures and should be considered in future research.

Measuring Choice not Competition
While many charter school policies create new schools that are meant to generate competition with nearby public school districts, they are not necessarily creating competition, only choice.Admittedly, for some families, any given public school was one of many choices long before charter school policies first emerged.Parents with the necessary resources have had options of private schools, other public schools, or homeschooling.Granted, these choices open the potential for significant costs such as tuition, moving to a new district, or the time and energy investment of homeschooling; however, it is not clear whether public school districts feel the need to respond competitively or improve practices due to the competitive pressure of these forces.Though charter schools require less investment on the part of families, until the public school district is threatened by the charter school, it is simply generating another educational choice, not competition.Hoxby (2000) demonstrated that more educational choices led to increased productivity and efficiency in public schools, but there is no evidence that this is caused or not caused by competitive effects, only choice.
In fact, it can be argued that many studies of "charter school competition" are not actually measuring competition.The proximity of nearby charter schools (Holmes et al., 2003), or the density of charter schools (Bifulco & Ladd, 2006), is not an indicator of competition, but the number of educational options.Choice is often proffered as a proxy for competition, or as an ensured creator of competition (Ghosh, 2010); however, if we accept the economic assumption that increased competition leads to increased efficiency and effectiveness (Chubb & Moe, 1990;Friedman, 1955;Hoxby, 2003), and consider the history of conflicting findings, there is little evidence that measures of choice sufficiently capture whether or not charter schools are exerting actual competitive pressures on nearby public schools.
An additional concern, when comparing these studies, is the subject facing the competitive pressure of the charter schools.Different measures of competition measure different subjects; market share measures may be appropriate for measuring the level of competition felt by a public school district that is facing losing enrollment and the associated funds, whereas proximity to a given school, or density around a given school, is more appropriate for measuring the effects facing an individual school (Bifulco & Ladd, 2006).As charter schools primarily exert competitive pressure through shifting enrollment, and money, away from the public school district, the district may be more likely to "feel" the competitive pressure than the school.However, district's response to competition is more likely to be associated with movement of financial resources (Arsen & Ni, 2012), while a school's response through instructional changes or variation in teacher/administrator effort is more likely to impact achievement.
Considering the original assertion by Friedman (1955) that competition would improve efficiency, keeping these different subjects in mind is important.School efficiency at its most basic measurement is a function of dollars spent and achievement earned (Hoxby, 2003); however, the various measures of competition currently used, such as proximity, density, and market share are more closely associated with different aspects of that equation.Arguably, an increase in achievement or a decrease in spending will both result in efficiency gains.Doing more with less, is not required to improve efficiency, in fact, doing the same with less, or doing less with a lot less, can all represent increased efficiency, the outcome predicted by Friedman.A district may increase efficiency, but that is possibly done by decreasing spending on students, rather than increasing achievement.In order to make claims about the role of competition in studies of public school districts and district-run public schools, outcome measures, measures of competition, and the subject of study must be clearly distinguished.
Studies using both market share measures and measures of proximity or density (Bohte, 2004;Booker et al., 2008;Sass, 2006) include school and district-level measures of charter school presence, but make up a minority of the existing literature.Inasmuch as there is consistency of outcomes across the studies using measures of both density and market share, the variation across methods of measuring market share can further cloud any accumulated understanding of charter school second-level effects on public school districts and district-run public schools.Unfortunately, there is even differentiation in the way that market share is measured; some studies assume that every student in a charter school has left the district, but many students will attend charter schools in a completely different district, thereby further complicating the measure of market share (Ni & Arsen, 2010).
Additional concerns arise when considering market share as a viable measure for competition.First, public school districts, especially urban districts, have long split market shares of students with private schools-so any measure of charter competition using market share should incorporate previous measures of market sharing or only measure students leaving district-run schools for charter schools.Second, measures of market share are not sensitive to changes in the school age population.For example, if a percentage of students in a given district attending the district-run public school drops from, say, 70% to 64%, the competition generated by such a change may be drastically lessened if the total number of school age children is growing, just as the generated competition may be drastically increased if the population of school age children is dwindling-in either situation market share becomes a less consistent proxy for measures of competition.

The (Potential) Role of Economics in Education Research
Given the relative newness of charter schools and the growing literature surrounding their effects, it is important that policymakers, scholars, and stakeholders understand what is being discussed when examining the second-level effects of charter schools.As stated above, the majority of students is currently attending district-run public schools, and will be for the foreseeable future; therefore, the effects of charter schools on the educational landscape will be most widely felt through second-level effects on public school districts and district-run public schools.As such, it is important that we begin to develop as clear an understanding as possible about outcomes associated with these policies and reforms; however, current quantitative research has approached the concept of "competition" using various methods and measures, and in various contexts.
It is clear that some researchers (Ghosh, 2010;Hoxby, 2000) argue that choice is competition, or at the very least, leads to competition.This definition is too simplistic; arguably, competition only exists when choice is coupled with an effort by one or more party to produce results superior to other involved parties.Imagine a track on which a single person is running; this person is only compelled to run as fast as he wishes.Simply adding runners to the track does not create a race; it only creates possible distractions for the original runner.It is not until the original runner is compelled to run faster than another runner on the track that competition actually existsthough choice is a necessary component of competition, it is not a proxy for competition any more than three people jogging independently on a track is a proxy for a race.While this metaphor is not perfect for the educational landscape, it serves to distinguish the difference between choice and competition.
The concept of competition, while relatively new to educational policy, is not new to economics.In fact, there are existing measures such as the Herfindahl Index that are used to measure the size of agencies in a given industry and measure the competition between them.In fact, the Herfindahl Index is used by the US Department of Justice to examine market concentration and pursue anti-trust cases (The United States Department of Justice, n.d.).As seen above, the most consistent findings were generated by studies that included measures for both market share and density in the measure of competition-the two measures included in the Herfindahl Index.This index, used to calculate the level of competition in a given educational market, could be expressed as where is the proportion of students in a given market that attend educational agency i, and N is the number of educational agencies available to a student in the market.Changes in the Herfindahl Index would then indicate changes in the level of competition in a given market, and duration could easily be added to any measurement of magnitude to provide further detail about the effects on a charter school on a public district or school.Though this measure may not be capable of isolating true competitive effects, it may serve as a helpful starting place to begin the challenging work of identifying the true effects of competition on educational agencies.
To date, most studies on charter school competition do not use the measure of competition that is used in economics.However, this measure has been used in the study of school competition between public schools.Hanushek and Rivkin (2003) used the Herfindahl index to measure competition between public schools in metropolitan areas of Texas.Their use of the Herfindahl index measured the concentration of students by district and by schools in a given metropolitan area.Their results suggest that increased competition led to higher levels of teacher quality.However, as noted above, they suggest that the institutional structure of public schools raises concerns about a schools ability to respond and distinguish between choice and competition: "Although many simply assume that expanded availability of alternatives will lead to higher public school quality, the institutional structure of public schools raises some questions about the strength of any response" (p.22).
The Herfindahl index is not a perfect measure of competition, and Hanushek and Rivkin (2003) noted that it may be measuring choice rather than competition.Also a concern regarding this measure is that the size or scope of the market is not determined by the index itself.Indeed, this concern could pose serious issues to analysis and generalizability of findings; as stated previously, measures of market share that do not account for inter-district transfers and competition may fail to fully account for the second-level effects.As measures, such as the Herfendahl Index, are explored for use in this field, incorporating what is already known about how variation in the measure competition impacts outcomes is essential.Variations in distance and density used to measure educational competition led to different outcomes, though variations in local policy and context may lead to variations in the size and scope of the educational market.
It is important that the education reforms that tout the use of economic concepts such as competition and efficiency as drivers of educational improvement be examined as accurately as possible.However, the true impact of these reforms is obscured by inconsistency in measurement and definition.Future research must bring clarity to these concepts, and perhaps answer other important questions about the potential impacts of market-based reforms.Other fields of study may provide helpful insight to the growing literature examining the second-level effects of charter schools.Indeed, there are many aspects of the charter school movement that, while new to education, have been studied in detail in other fields such as the role of local context in studying a competitive marketplace, the impact of franchising on independent providers, and at what point a market become saturated.While respecting the impact of local context on educational outcomes, until there is consistency in what is being measured, and how it is being measured, studies of charter school competition are likely to continue generating noise but little light.

Table 1
Factors related to measures of competition.