Characteristics of the Opt-Out Movement : Early Evidence for Colorado Grant

Testing and accountability measures have continued to expand since the passage of the No Child Left Behind Act in 2001. In addition to school and district accountability, student test scores increasingly formed the foundation of teacher performance metrics. State participation rates exceeded the 95% minimum prescribed by law despite increasing opposition to many testing requirements. However, the rollout of the Common Core aligned PARCC tests in 2015 marked the start of a backlash against state mandated testing. The movement, commonly called opt-out, encouraged families not to participate in required tests. We use pooled OLS regression on statewide panel data from Colorado schools to examine school-level characteristics in one of the states with the largest declines in test participation. We find the prevalence of opt-out is largest in charter schools, suburban and rural areas, higher performing schools, and schools with a higher proportion of White students.


Introduction
For better or worse, standardized testing is one of the foundations of the modern accountability-driven educational system in the United States.Countless policies across the nation hold schools, and increasingly, teachers, accountable for student performance on such exams.Standardized test results are widely reported in the media and are seen by parents and policymakers as a reflection of school quality.
A major assumption underlying the use of standardized tests is that the group of students taking the exams represents the student body within the school or school system.It is for that reason that the No Child Left Behind Act required 95% participation rates on mandated tests.Typically, schools have achieved and exceeded this threshold (CDE, 2015;Chingos, 2015).Recently, however, a movement in which students do not take the exams has cropped up in some areas of the country.Starting in 2015, a number of locations began to experience significant levels of nonparticipation -an event that coincided with the implementation of the Common Core aligned PARCC tests for states such as Colorado.Typically known as "opting-out," this non-participation has been relatively unexamined in terms of the types of students and schools directly involved.

Background
High-stakes testing and widespread federal incentives to states regarding accountability measures can largely be traced to the second Bush administration's reauthorization of the Elementary and Secondary Education Act (ESEA) known as No Child Left Behind (NCLB).NCLB was clearly the largest expansion of ESEA and, for the first time, included required annual assessments for all students.Schools and districts from every state were now accountable to federal student performance mandates that aligned assessments to state content standards.Annual assessments in ELA were required for all students in grades 3-8 and once during high school (Egalite, Fusarelli, & Fusarelli, 2017;McGuinn, 2016).In an attempt to guarantee educational equity, states were required to report the level of student proficiency as measured by adequate yearly progress (AYP) for all relevant student subgroups, including gender, race/ethnicity, poverty, disability, and English language status (Haertel & Herman, 2005).The Obama administration's Race to the Top program further incentivized federal priorities including adoption of the Common Core and linking student test performance to teacher accountability (U.S.Department of Education, 2009).
In late 2015, the Obama administration passed the Every Student Succeeds Act (ESSA), the ninth major reauthorization of ESEA.Addressing concerns from states, districts, and schools, the act removed the AYP measure; however, the focus on rigorous content standards and the use of annual assessments established in NCLB remained unchanged.The new law returned a number of accountability decisions to the state and local level and provided greater flexibility with performance level indicators and school improvement interventions.The reauthorization also maintained a focus on the idea of educational equity through assessment, requiring the reporting of student subgroup performance on two new student subgroups: gender and migrant status (National Conference of State Legislatures, 2015).While there were some noticeable differences between ESSA and NCLB, using assessments to attempt to improve student achievement in underperforming schools remained a fundamental focus of federal education policy (Egalite et al., 2017;McGuinn, 2016).
Central to the increase in accountability measures was a high participation rate in state mandated testing.NCLB required, and ESSA retained, the threshold of 95% participation in large part to ensure valid inferences about the results for both the school as a whole and reported subgroups (Spellings, 2007).The failure to achieve high participation rates has been shown to negatively affect the ability of policymakers to achieve valid interpretations and correctly identify low performing schools (Baker & Linn, 2002;Betebenner & Linn, 2010).The issue of the diminished participation and the degree to which scores reflected the total population was further exacerbated when student-level test performance was used as part of the measure of teacher efficacy in systems such as Colorado's.

The Opt-Out Movement
Overall, limited research exists on the opt-out movement.Existing studies present a complex narrative.For example, a New York study using district-level data found a positive relationship between wealth/test scores and opting-out (Chingos, 2015).After controlling for free/reduced lunch, however, the relationship changed direction, showing that districts with lower scores had higher levels of opting-out (Chingos, 2015).There are other accounts, from the popular media and other sources, that suggest that most opt-outs are representative of a limited number of highly involved, affluent families in a limited number of schools (Gorski, 2015b;Robertson, 2015).This is supported by some existing research.A non-representative, online national survey, for example, found that activists in the opt-out movement tend to be "highly educated, white, married, politically liberal parent(s) whose children attend public school and whose household median income is well above the national average" (Pizmony-Levy & Saraisky, 2016, p. 6).Opt-out participants were well informed and represented groups that frequently test well -only a fraction of the respondents expressed concern about test performance or school quality as a reason for optingout.Indeed, in Pizmony-Levy and Saraisky's (2016) study, many opt-out participants indicated that their primary reason for opting-out was opposition to the use of tests to evaluate teachers.Others were concerned about how testing promotes rote memorization over critical thinking, and about growing privatization and the role of corporations in schools.In short, this research, and the accompanying media accounts, suggest that parents with knowledge about schooling and educational systems and the perceived power to question those systems -in other words, parents with high levels of cultural capital -are more empowered to opt-out.Still, some states -like New York and Colorado -have experienced relatively high rates of opt-out, suggesting the phenomenon may be more widespread and complex than a limited group of suburban districts.
Other related research suggests that for schools facing federal and state sanctions, such as school closure or change to charter status, opting-out could present an opportunity to artificially increase test scores in ways similar to cheating (Amrein- Beardsley, Berliner, & Rideau, 2010;Jacob & Levitt, 2002).For example, Amrein-Beardsley, Berliner, & Rideau (2010), found that a significant number of surveyed teachers had at least heard of another teacher helping students in some way on state-mandated tests.Relating this to the practice of opting-out, one teacher in Amrein-Beardsley and colleagues' study reported that she/he encouraged a student to opt-out, stating: The student did not speak the language so it was very unfair to take a test that one could not read.So I said stay home in the morning and come after lunch.I allowed the student to do the math section because he could do the problems.(p.15) Though there has been little research as to how widespread this issue is, and we could only find one example of this in a published study, this conduct warrants attention.While cheating artificially raises scores of select students, opting-out removes potentially lower scoring students from the testing pool entirely.Teachers and school leaders could be actively or passively encouraging students to test based on their perceived ability to meet achievement standards (Figlio, 2006;Freeman, 2015), which could mean discrimination along racial/ethnic lines and/or socioeconomic status.In this way, selectively encouraging a few students to opt-out could achieve similar aims as more complicated cheating programs.Indeed, researchers using Ohio data demonstrated that school rankings could be swayed by a change in the participation of as few as 11 students in a single school (Beaver, Westmaas, & Sludden, 2015).If this motivation is a primary reason for opting-out, we should expect schools with lower accountability ratings to experience higher levels of opting-out.
There is also the possibility that opting-out could reflect regional differences in the perceived utility of school accountability systems.Rural, urban, and suburban schools differ on many of the metrics that reflect geographic differences.This includes wide disparities in base funding, passing rates for bond initiatives, and different rates of attendance to higher education (Bowers, Metzger, & Militello, 2010;Byun, Meece, & Irvin, 2012;Gebert, Calkin, & Schuster, 2004;Reyes & Rodriguez, 2004).It should not be surprising that they respond to accountability measures uniquely based on the local context.This could extend to opt-out, though there has been no research to date that parses out these differences.
Finally, opting-out could be reflective of a broader, more complex social movement.According to Pizmony-Levy and Saraisky (2016), parents engaging in opting-out are taking part in collective action to "pursue their objectives mainly through non-institutional means of disruption of accustomed practices (e.g.standardized tests)" (p.10).If a broader social movement is the motivation for opting-out, then timing of opt-out would be expected to coincide with a critical event such as the implementation of the Common Core aligned PARCC tests or new statewide accountability measures for teachers (Klandermans, 1984).As a result, we would expect wider variation across types of students and schools engaging in opting-out, as the motivations are not well captured through school-level covariates.
In short, the research on opting-out is scant.Existing studies indicate that White, affluent, educated parents may be at the forefront of the opt-out movement, in opposition to high levels of testing, and the use of high stakes testing for teacher evaluation (Pizmony-Levy & Saraisky, 2016).
There is also the potential that lower-performing students could be encouraged to opt-out in order to raise mandated test scores.The gaps in the literature leave many questions to be answered: does opt-out vary across racial/ethnic lines?By socioeconomic status?By school-type?Turnaround status?Opt-out could be happening at different levels for specific sub-groups and school contexts.

Framing the Opt-Out Movement: Cultural Capital
The research outlined above suggests that the opt-out movement could be a reaction by families with the knowledge and perceived power to opt-out of testing without fear of repercussions.Research also indicates that opt-out may be a form of protest, with parents using opt-out to object to increased testing requirements, privatization, and how the tests are used, among other concerns (Pizmony-Levy & Saraisky, 2016).The idea of parents choosing to opt-out to help their children, or to protest testing and accountability could be seen as a manifestation of parents' cultural capital.In this paper, we link cultural capital (Bourdieu, 1977;1986) to this idea of increased parent knowledge about schooling and empowerment to question school practices, and use cultural capital as an explanatory framework for our findings.
According to Bourdieu (1973), cultural capital is the internalization of culture and traditions, usually passed on by the family.Cultural capital is acquired over time and is present in knowledge about school, appropriate attitudes and beliefs, personal style, and linguistic competence sanctified by the dominant culture (Lamont & Lareau, 1988).It is class-based, with the more privileged classes able to use their internalized resources and knowledge in exchange for benefits (Bourdieu, 1986).Cultural capital is also institutionalized as credentials and qualifications that can be used for influence and economic gain.In the education system, cultural capital (or perceived lack of cultural capital) is how social inequality and class structure are perpetuated.
In education, cultural capital also manifests in how parents interact with schools and educators -often, parents with high levels of cultural capital feel empowered to ask for exceptions, and they use their knowledge and resources to provide their children with educational advantages (Lareau & Calarco, 2012).Privileged parents also pass on internalized knowledge about appropriate school behavior to their children.Those students who know how to act in school -according to primarily White, middle-class norms -are better able to operate within that system.Scholars applying theories of cultural capital have also found that middle-and upper-class parents pass on notions of social class that provide advantages and encourage entitlement (Lareau, 2002).Indeed, students' advantages and the entitlement propagated by parents' levels of cultural capital may ultimately influence student performance -some research has also shown that cultural capital is related to student outcomes (DiMaggio & Mohr, 1985).
In relation to opting-out, cultural capital may manifest in two ways: parents' decision to optout in order to provide advantages to their children (e.g. to avoid "testing fatigue"), or the decision to opt-out in order to protest the very concept of testing (e.g. to object to using tests to evaluate teachers).In both scenarios, parents with higher degrees of cultural capital may have the knowledge and access to information about opting-out and feel sufficiently empowered to remove their children from required testing because they do not fear the repercussions of doing so.Parents with more cultural capital use their resources to gain advantages for their students in their interactions with teachers and schools (Lareau, 2011).It stands to reason then that if parents feel that their students are not gaining anything from high stakes testing requirements, or if they feel that the increased levels of testing are actively taking something away from their children, parents with high levels of cultural capital may try to assist their children by opting-out.Further, their experience with federal accountability measures reflects an increase in test preparation and narrowing of the curriculum to more accurately align with state and national standards such as the Common Core.Thus, they may want to opt their students out of testing to protest the idea of "teaching to the test."These parents' social class and cultural capital may also allow them to feel entitled to opt-out to protest increased testing requirements, using tests to evaluate teachers, or the increased level of privatization in schools.Cultural capital thus provides an explanatory framework within which to situate our findings on opt-out.

Colorado Context
We focus specifically on Colorado in our exploration of the opt-out movement.We do so because Colorado presents an important case study of the type of schools and communities that have thus far been affected by the opt-out movement in some way.The state is geographically and politically diverse with a strong history of local control in education.It was one of three states, along with New York and Rhode Island, to experience large scale opt-out on state testing in 2015 (Bennett, 2016).Colorado fits squarely within a national movement wherein a majority of Americans were aware of opt-out as it related to standardized tests and the Common Core, specifically (Pizmony-Levy & Cosman, 2017).
The implementation of the new PARCC tests in 2015 marked a change in the incremental increase in statewide testing and accountability measures.This change was characterized by an alignment to the Common Core and by an increased number of testing days.Moreover, this occurred against the backdrop of the baseline year of the new -and hotly contested -teacher accountability system, which required using student growth measures for 50% of the teachers' total score (CDE, 2015;Johnston, 2014).This perceived structural shift in the education landscape prompted increased criticism from a number of stakeholders, including parents and school leaders.Criticism was widespread both nationally (Bennet, 2016;Fairbanks, 2015;Taylor & Rich, 2015) and locally (Gorksi, 2015a(Gorksi, , 2015b)); community leaders from local and national groups such as United to Counter the Core and United Opt Out encouraged parents to opt-out of testing (Gorksi, 2015b;Robertson, 2015).
Policymakers in Colorado did attempt to respond to an apparent groundswell against the new tests.In January 2015, the Colorado State Board of Education (SBE) voted to allow local districts to opt-out of portions of state mandated testing -including the new PARCC assessments.This action subsequently called into question if the SBE had the statutory authority to release districts from testing and culminated in the State Attorney General issuing an opinion stating the SBE did not have the legal authority to grant such waivers (State of Colorado, 2015).Further complicating matters, the Commissioner of the Colorado Department of Education (CDE) issued a statement in February 2015 that districts would not be penalized for having participation rates fall below the required 95% (CDE, 2015), although it was silent on the impact on school performance rankings.The United States Department of Education, however, later rejected Colorado's request and stated districts would not be exempt from participation requirements (Deslile, 2015).
Colorado's statewide school performance framework (SPF) places an emphasis on five key performance indicators: academic growth, academic achievement, academic growth gaps, postsecondary workforce readiness, and participation.Each performance indicator is rated to form an overall school performance ranking in order of overall achievement: performance, improvement, priority improvement, and turnaround.Schools placed on turnaround status must submit annual plans to the state and face additional monitoring requirements for a period of three years.Turnaround schools that fail to meet performance indicators are required to move to at least priority improvement and may be closed or reconstituted as charters (CDE, 2015).By the spring of 2016, the bottom two categories, turnaround and priority improvement, equaled approximately 9% of schools in the state.
As they entered the 2015 testing window, schools had limited information as to the exact requirements for participation and specifically how the state would take opt-out into account in the school performance frameworks.For those schools in the turnaround and priority improvement categories, systematic non-participation held the simultaneous possibility of changing achievement rankings-up or down depending on which students failed to participate-and possible sanctions for failing to meet the 95% participation threshold.
Though declining participation rates in some Colorado schools is well known, prior to this study, we have little detailed information about the characteristics of the schools where the opt-out phenomenon is most prevalent.Is the decline in participation widespread across the state, or is it centralized in a few particular schools that receive attention?Have participation rates dropped similarly in urban, rural, and suburban settings?Is opt-out most prevalent in low-or highperforming schools?The purpose of this study is to address such questions and more clearly document the relationship between observed characteristics and declines in test participation.In so doing, we aim to examine the overall pattern of relationships between school characteristics and test participation rates.Formally we ask: Are there school-level characteristics that predict opt-out behavior in state mandated exams?

Sample and Methods
We examined the participation rate of public traditional and charter schools in Colorado schools for state mandated testing for English language arts (ELA) from the 2012 testing cycle through 2016.This was accomplished by assembling a number of publicly available datasets provided by CDE and the National Center for Education Statistics (NCES) into a single panel.From CDE, we obtained school performance framework (SPF) scores, enrollment characteristics, test participation in ELA and math, and school level characteristics.We merged these data with NCES urbanicity classification collapsed to the levels of urban, suburban, and rural.
This panel resulted in all public schools in Colorado.We then restricted our sample to traditional public schools and charter schools-excluding sites dedicated to special education, vocational, online, alternative education, or closed within the 2012-2016 period-leaving a final sample of 96% of public schools in Colorado.This yielded 1,049 elementary, 545 middle, and 390 high schools.Descriptive data is presented in table 1. CDE suspended calculations of SPF during the transition to PARCC testing in 2015 and did not release new rankings until spring 2017.This meant all schools retained their 2014 ranking during the entire opt-out period.Schools may have, however, responded to suspending the SPF differently based on their baseline ranking.In this study, we present results on the tested grades 3-9 in ELA.Although not reported here, results are similar when we look at the math exams.Our primary analysis was a pooled OLS regression model to measure the relationship between observed school characteristics and participation rates on state mandated tests in ELA.Formally, we used OLS to estimate: Participation rateit= β0+ β2X +β3 (X * opt-out years) + β4 opt out years+ε (1) where Participation rate is the percentage participation in ELA in year t for grade i; X is a vector of school characteristics including school racial and ethnic composition with White as the reference; charter; urbanicity with urban as the reference group; SPF with Performance as the reference; and percentage free/reduced lunch (FRL); opt-out years indicates the year is 2015 or later; ε is a robust term clustered at the school level; and β's are parameters to be estimated.Our primary interest was the interaction between selected observed characteristics and the opt-out years indicator although we present results for baseline years for comparison and descriptive purposes.
To examine various motivations for opting-out, we present a series of variations on Model (1).The first, presented in Table 3, is a pooled OLS of the baseline years 2012-2014.Next, we present a series of models interacted on a different set of covariates theorized to motivate opting out with the opt-out period.These interactions include urbanicity (Table 4), race/ethnicity (Table 5), SPF (Table 6), charter and proportion of FRL students (Table 7).All models met the assumptions for OLS including linearity, multivariate normality, and homoscedasticity.To further prevent issues with homoscedasticity, we used the robust standard errors command in Stata 14 (StataCorp, 2014).To explore concerns of multicollinearity, we conducted a variance inflation factor (VIF) tests for each model utilizing a VIF cutpoint of 10 to identify issues of multicollinearity (Mertler & Vannatta, 2013).

Results
We first set out to describe the basic patterns of participation rates during the time periods under consideration.Consistent with expectations from media reports, there was a substantial decline in the participation rates on Colorado's state standardized tests in the 2015 school year with a rebound in 2016 that did not, however, return to previous levels.
We present basic descriptive characteristics of the sample for ELA in Table 2.The table shows that while there was a decline in participation in all grade levels, the drop was sharpest in the middle and high school grades.In the ninth grade, participation dropped from 98% in 2014 to 75% in 2015.The remainder of the table illustrates differences in the changes of participation rates by SPF and urbanicity.The aggregate decline in participation, not controlling for any other factors, was similar across school performance ratings.However, it does appear that the decline in overall participation in 2015 was larger in rural schools than in the urban or suburban schools, although this gap closed the following year.We descriptively illustrate the relationship between SPF and participation rate in Figure 1.The figure shows testing participation rates for ELA in grades 5, 7, and 9 in 2016 for schools of increasing school quality as measured under the accountability system without any statistical controls.The horizontal axis is the percentage of possible points on the SPF, which is the metric used to assign the categorical rating, and each symbol is a unique school/grade combination.The downward slope of the fitted lines shows the negative relationship between higher performing schools and the participation rate for each grade.Also apparent in the figure is the exceptionally high rate of opt-out for ninth graders -with many schools falling below 60%-and the degree of slope on the fit line relative two the other grades.

Baseline Characteristics
We now turn to a basic regression model meant to describe the overall pattern of relationships between school characteristics and test participation rates accounting for other observed characteristics of the school.We first consider the relationship between school characteristics and participation rates prior to the advent of the opt-out movement.
Table 3 reports the results from the pooled OLS regressions where the dependent variable is the participation rate for the school on the ELA exam and the independent variables include the full vector of school characteristics in Model 1.In order to assess the relationships between these variables and participation rates at the baseline, the regression in Table 3   The upper panel of Table 3 compares participation rates prior to the start of the opt-out movement.Suburban and rural schools participated at slightly different rates than did the reference group urban, although the differences were rarely statistically or practically significant with differences in participation ranging from .033% in the ninth grade to -1.63% in the fifth grade.In baseline years, urbanicity was a significant predictor in the fourth grade with rural and urban schools participating at approximately 0.25% higher rates that suburban schools.
The second panel of Table 3 compares the relative proportion of school populations with White as the reference group.In baseline years, Race/ethnicity was not a strong predictor of participation in elementary grades.In the eighth and ninth grades, schools with higher proportions of Native American students participating at lower rates of -0.252 and -0.551 respectively.Schools with higher proportions of Asian had slightly higher participation rates of 0.082 in the sixth and 0.031 in the seventh.
The third panel of Table 3 compares baseline characteristics by school quality as measured by the state's School Performance Framework (SPF).Prior to opt-out, lower SPF ratings frequently corresponded to decreased participation.Schools at the lowest ranking-turnaround-had some of the largest differences in participation relative to the top ranking comparison group-performance.The results were not statistically significant for turnaround schools although by the ninth grade the difference was a practically significant 7.88% lower participation.Similarly, priority improvement and improvement category schools showed increasingly larger differences in participation in later grades with significant differences in the eighth and ninth grades with priority improvement schools have 2.488% lower participation levels.
Finally, charter status reflected greater variation by grade with 0.034% higher participation in the fourth grade and 1.699% higher in the ninth although results were only significant in the eighth and ninth grades.Increases in the proportion of FRL were rarely a significant predictor of test participation prior to the opt-out movement.

Opt-out Years
Next, we present the primary regression models that look at the differences in the relationship between the observed characteristics and participation rates before and after the opt-out phenomenon.Each table reports the estimates of interest from regression models by grade level that include interactions of particular categories.For space considerations, the table reports only results from the coefficients for the main effect and the interaction between the identified school characteristic and the opt-out period.Again, these are not fully interacted models.The models differ by the variables that are interacted with the 2015 and 2016 time periods when the opt-out movement appears to have begun.
Table 4 reports the interaction results for urbanicity with the results relative to the control group, urban schools.Compared to urban schools, rural and suburban schools participated at higher rates prior to the start of opt-out.When interacted with the opt-out period, however, the results change.The rate of participation for rural schools consistently and significantly decreased relative to urban schools during the opt-out period.The interaction of rural* opt-out shows the change in participation rates ranged from -1.491% in the third grade to a high of -4.602% in the seventh grade.A similar picture emerges for suburban schools starting in the fourth grade.Participation rates consistently decreased in suburban school from the sixth through ninth grades with ninth graders experiencing an 8% decline relative to their urban peers.This finding is consistent with the common perception that largely parents and students outside of the cities drive the opt-out phenomenon.Table 5 follows the same pattern, presenting the main effects of racial/ethnic composition of schools, opt-out and their interaction.The key outcome of interest is the interaction term showing the change in the effect of race/ethnicity during the opt-out time period.Before opt-out began there were few significant or practical differences based on race/ethnicity relative to the comparison group, White students.After opt-out began, increases in the percentage of Native American students consistently corresponded with a decline in participation rates although results were only significant in high school with a -1.182% change.More notable, however, is the result showing schools with higher proportions of Hispanic and African-American students relative to the proportion of White students, experienced increases in participation rates.Schools with larger proportions of African-American students showed positive and significant increases in participation during the opt-out time period.The effect was smallest in the third and fourth grades at 0.064% increase and highest in the ninth grade at 0.422% increase in participation.Schools with higher proportions of Hispanic students followed a similar patter with third grade showing the smallest increase at 0.046% and eighth grade showing the highest relative change at 0.178%.Table 6 shows interaction results for the relationship between SPF-the school's accountability rating-and participation rates.Prior to opt-out lower ranked schools consistently had lower participation rates relative to the top ranked Performance category.During the opt-out period, those results reverse sign with lower ranked schools showing an increase in participation rates relative to top ranked schools.The difference by school performance ranking was most stark in the middle and high school grades.For instance, controlling for other factors, the participation rates of turnaround and improvement schools increased by 13.51% and 10.27% in the eighth and ninth grades respectively.These regression results square with basic descriptive statistics that do not include statistical adjustments.The participation rates on the eighth grade ELA exam in 2014 and 2015, respectively, was 97% and 94% in turnaround schools, 99% and 91% in priority improvement schools, but 99% and 86% in performance schools.Note that there was a decline in participation in each of these school types at the time that opt-out began.This decline was just much larger in the higher rated schools than in the lower rated schools.Table 7 shows results for charter schools and those with higher rates of poverty as measured by participation in the Free/Reduced price lunches (FRL).During the opt-out period, increasing percentages of FRL students corresponded to significantly increased participation rates.The effect was smallest in the third grade at 0.021% increase and largest in the ninth grade at 0.178% increase.The main effect for charter schools was a consistent decrease in participation.During opt-out, charter elementary schools had larger drops in participation relative to traditional public schools with the most pronounced change in the third and fourth grades at -2.87% and -2.73% respectively.Curiously, this effect changes sign in later grades with charter schools showing an increase in participation relative to traditional public schools of 3.26% in the eighth grade.

Robustness Tests
Tables 3 through 6 present results for a specific source of motivation for opting-outurbanicity, race/ethnicity, SPF, charter, or Free/Reduced price lunches.They do not, however, present a fully interacted model, which we present in Table 8.The fully interacted model acts as robustness test of model specification.The limited interaction models in the previous tables do not include the full vector of school characteristics and their interaction in order to concentrate attention on individual sources of motivation for opt-out.
The results for the fully interacted model do reveal changes when including the full vector of covariates and their interactions.In most instances, interactions for the opt-out period lose their significance and reduce in magnitude.In some instances, the coefficients change direction as well.This reveals the model is not robust to model specifications and there may be other sources of variance that are not identified in this model.Further, a VIF test revealed an unacceptably high VIF for Free/reduced price lunch participation (r =0.60, VIF12.09) and the variable was removed from the final model.
The fully interacted model in Table 8 presents a more complex picture as many of the interactions lose significance.There were some consistent results across model specifications.Notably, charter participation rates showed similar, significant declines in participation in the elementary grades across both the limited and fully interacted models.In the fully interacted model, charter status corresponded to 3.334% decline in participation in the third grade and remained significant-although smaller-through the sixth grade with a 1.714% decline.Similarly, increasing proportions of Black and Hispanic students continued to correspond with significant increases in participation across all grades.In the fully interacted model, increases in the proportion of Black students corresponded to a significant increase in participation all grades ranging from 0.064% in the third grade to 0.422% in the ninth grade.Similarly, increases in the proportion of Hispanic students corresponded to significant increases in participation across all grades.In this case of Hispanic students the point estimates in Table 5 and Table 8 remained closer than for other races/ethnicities.
Results for SPF were not as robust in the fully interacted model.In Table 5, the limited interaction model showed consistent and significant effects in grades six through nine for turnaround schools and all grades for priority improvement and improvement schools.In contrast, the fully interacted model in Table 8 shows inconsistent effects for SPF with the exception of Improvement schools in the ninth grade still showing a 6.86% increase in participation relative to the highest ranked reference group (performance).Similarly, urbanicity was no longer significant with the exception of suburban third graders in the fully interacted model at 1.426% increase in participation.As a final test of robustness, we present Table 9 in which we computed a falsification test with all the same models using 2013 as the interaction year.Since 2013 is prior to opt-out we would not expect to see any of the same results.None of the individual 2013 interaction models for urbanicity, race/ethnicity, SPF, charter, or Free/Reduced price lunches were significant.For simplicity we present only the fully interacted model of the falsification test.In the fully interacted model, the only consistently significant result is for Hispanic students, who showed a decline in participation rates relative to the other baseline years.Unfortunately, results are not fully robust to changes in model specification.There are, however, significant predictors of changes in participation rates that remain across specifications.Notably, charter schools in elementary grades corresponded with large decreases in participation.Increases in the proportion of Black and Hispanic students consistently showed relative increases in participation and was robust to changes in model specifications.

Discussion and Conclusion
The opt-out movement has the potential to be one of the most important phenomena in education policy today.Test data is widely used by states, districts, and schools in order to find areas of excellence and weakness in public schools and can influence critical factors including school rankings, program evaluation, and merit pay.However, for these tests to be informative, they must present information representative of the students enrolled in the schools.The opt-out movement threatens the ability of tests to serve that task.
Our results extend the literature beyond the previous examinations of New York (Bennett, 2016;Chingos, 2015) through a systematic analysis of a state not previously published, Colorado.Further, ours is the first study, of which we are aware, that employs panel data instead of a crosssectional design.We illustrate the type of schools that have experienced changes in the proportion of students sitting for standardized exams.We find that opt-out is a widespread phenomenon within Colorado.However, the change in test participation was largest in suburban and rural areas, higher performing schools, schools with a higher proportion of White students, schools with lower proportions of students participating in Free/Reduced price lunches, and elementary students in charter schools.These results are at least consistent with the popular press narrative that opt-out is largely a response from parents whose children are likely to do well on the exams.For charter schools, it is not suprising that parents who were engaged in school choice were also more likely to make an affirmative decision not to take part in state required testing.
One of the motivations behind the 95% participation requirement was to ensure the validity of inferences behind exam results.School level results do not appear to show systematic nonparticipation in the types of schools most likely to benefit from by actively encouraging parents and students to opt-out -those lowest in SPF rankings.Instead, the results seem to support a narrative that opt-out is largely driven by high-achieving schools.This could be due to a perceived lack of benefit from school accountability mechanisms and/or a reaction to what is perceived to be onerous levels of testing (Gorski, 2015b;Pizmony-Levy & Saraisky, 2016;Robertson, 2015).
These results can also be explained, at least partially, by theories of cultural capital -more specifically, by differences in cultural capital reflective of social class.As noted earlier, families with more cultural capital perceive and interact with schools differently (Lareau, 1987;Lareau & Horvat, 1999).More affluent families express differing levels of expectations from schools and generally have higher levels of involvement, often irrespective of race (Lareau, 1987;Lareau & Horvat, 1999).Because of this increased involvement, parents may recognize that students' participation in standardized testing is not mandatory.Indeed, such parents may view school accountability measures and testing requirements as something being done to other types of schools, not the schools where their children attend.These parents may feel empowered to use their privileged position to protest high stakes testing -they may have concerns about over-testing and increased privatization, or they may reject the concept of using tests to hold teachers and schools accountable.This aligns with the (limited) existing research.It is also reflected in the more than two thirds of optout participants who gave their children's schools an A or B rating and also indicated that their primary reasons for opting-out was opposition to the use of tests to evaluate teachers (Pizmony-Levy & Saraisky, 2016).
Parents with higher levels of cultural capital may also use their privilege to provide their students with advantages (Lareau & Calarco, 2012), which in this case could include helping their children avoid "testing fatigue" or over-testing.Highly-informed parents with higher levels of cultural capital may be more concerned about over-testing or the perceived lack of utility in testing and may take steps to focus their students' energy on more important tests.For example, although participation rates for PARCC testing in high schools declined during this time, there was not a similar decline in participation in Advanced Placement (AP) tests.From 2014 to 2016 the number of participating schools in offering AP classes increased by 31% and the number of AP examinations administered increased by 9% (College Board, 2016).In the Pizmony-Levy and Saraisky survey (2016), 27% of respondents listed "standardized tests take away too much time" as a reason for opting-out (p.22).However, this does not appear to apply to AP tests, suggesting that parents may see these tests as more useful and more beneficial to students than high stakes accountability testing since AP tests reflect college bound behaviors.As a result, parents with higher degrees of cultural capital may see value in AP, but they may opt-out their students from the "extraneous" state tests.
Conversely, scholars have consistently shown that parents with less cultural capital may be less informed about educational processes more generally and are thus more likely to defer to teacher and school decisions and processes (Lareau, 1987;Lareau & Horvat, 1999).For example, parents with lower levels of cultural capital may be more likely to defer to school procedures even if they are aware they can withdraw their students from testing and may be less likely to know that opting-out is an option in the first place.In other words, these parents may feel that they do not know as much about education and/or what is educationally best for their children as the teachers and administrators at their children's schools and may thus refrain from opting-out.Alternatively, perhaps due to less involvement in the day-to-day processes of the school, these parents may not even know that opting-out is an option (something schools are unlikely to widely advertise, given the high stakes nature of testing).Thus, parents who opt-out must have outside knowledge of schooling and testing processes.Pizmony-Levy and Saraisky (2016) frame the opt-out movement in similar terms identifying it as social movement with necessary conditions of resources, changes in political structures, and communication of demands to achieve success.Two of the key elements they identify are "opposition to the Common Core" and opposition to the "the growing role of corporations in schools" (p.22), both of which require higher levels of knowledge about school processes and schooling in general.
The results of this paper are highly relevant for the policy conversation and for the general understanding of the nature and potential impacts of the opt-out movement, at least as it has been applied so far.Given that the idea of high stakes testing is reliant on those tests being representative of each school's student population, higher levels of opt-out from any group is cause for concern.Colorado is one of a few states to have experienced a large and sustained decrease in test participation starting in 2015 (Bennett, 2016).There is little reason to believe this movement would not spread to other states.Indeed, in 2016, the U.S. Department of Education warned 11 states about their falling participation rates (Pizmony-Levy, 2018).This study presents Colorado as an informative case study as well as a quantitative perspective to the growing body of research into the opt-out phenomenon.A single state failing to meet federal accountability requirements presents an interesting test for decentralization efforts of the Department of Education under the current administration.
From the perspective of those who want to continue to use testing and accountability as a way to hold ineffective schools responsible for poor performance, our finding that opt-out is much larger in schools that had previously received high performance ratings is somewhat comforting.However, the fact that participation rates do decline in lower performing schools, just not as much as in high-performing schools, suggests the possibility of broader policy implications.The examples of teachers and principals in lower performing schools responding to pressures of accountability are numerous (Blinder, 2015;Jacob & Levitt, 2002) and relaxing participation requirements may invite schools facing sanctions to encourage strategic opting-out.
Our work utilizes school-level data, which captures many of the important school and community effects.It may, however, mask many important characteristics associated with optingout such as prior student achievement and the relationship between demographic characteristics on the student level.Further research using student-level data is needed in order to assess which students within schools are more likely to opt-out.Such an analysis would be in a better position to understand whether there is any strategic opt-out according to prior student performance.Alternatively, a finding more consistent with the results in this paper may support the conclusion that opt-out is more likely among higher performing students that see the test as an unnecessary burden.

Figure 1 .
Figure 1.Percentage participation in ELA for grades 5, 7, and 9 in ELA 2016 by overall SPF points earned.

Table 1
Descriptive Statistics of the Sample

Table 2
Statewide Assessment Participation Rates in ELA By Grade Note: Mean participation rate(Standard Deviation) includes only observations from years 2012, 2013, and 2014.

Table 3
Regression results for ELA for Baseline Years

Table 8
Regression Results for Fully Interacted Model

Table 9
Regression Results for Falsification Test