The “ Power ” Of Value-Added Thinking : Exploring the Implementation of High-Stakes Teacher Accountability Policies in Rio de Janeiro

The purpose of this article is to ethnographically document the market-based ideological assumptions of Rio de Janeiro’s educational policymakers, and the ways in which those assumptions have informed these policymakers’ decision to implement value-added modeling-based teacher evaluation policies. Drawing on the anthropological literature on meaning making (Anderson-Levitt, 2012), the focus of this study is on the common understandings and ideological assumptions regarding “good” teacher education practice that undergird the policymaking decisions of Rio de Janeiro’s public education policymakers. On the basis of ethnographic interviews, I argue that the then-current Secretariat of Education in Rio was run primarily by people whose backgrounds in business and administration heavily influenced their ideological assumptions about good educational management. I further explore the ways in which Rio’s implementation of value-added modeling Education Policy Analysis Archives Vol. 25 No. 91 SPECIAL ISSUE 2 and high-stakes accountability-based teacher evaluation mechanisms reflects these latent ideological trends.


Introduction
Value-added modeling-based teacher evaluation policies have had a presence in Brazil for several decades.In 1990, the National System for Evaluation of Basic Education (Sistema Nacional de Avaliação da Educação Básica, or SAEB) was created.The SAEB is a biannually administered series of tests to evaluate the effectiveness of teaching in Brazilian schools, though high stakes (such as school closure) were not originally attached to the results of SAEB testing (Ferrão et al., 2001).During the presidential tenure of Fernando Henrique Cardoso (1995)(1996)(1997)(1998)(1999)(2000)(2001)(2002), perhaps the politician most identified with the implementation of business-friendly, market-oriented public policy in Brazil (Mollo & Saad-Filho, 2006), Cardoso advocated for a performance pay system for teachers, based on student performance on nationally standardized tests.Rhetoric supporting market-oriented moves in educational policy such as this were very present in the national media at the time (Fischman & Sales, 2012).However, such a system was never fully implemented during Cardoso's time in office.Rather, it was during the Lula-Dilma period (2002)(2003)(2004)(2005)(2006)(2007)(2008)(2009)(2010)(2011)(2012)(2013)(2014)(2015)(2016), when Brazil was led by a center-left coalition led by Brazil's ostensibly leftist Workers Party that value-added modeling, as well as many similar and related market-oriented education policies, went into effect (Evangelista & Leher, 2012).
Specifically, a number of policies that were adopted at the national level in Brazil in the 2000s paved the way for the implementation of value-added modeling in teacher evaluation.In 2005, the first national standardized test was developed by the federal Ministry of Education, commonly known as the Prova Brasil (or "Brazil Test") (Afonso, 2009).In order to maximize the perceived utility of this new test, that same year the Ministry also developed the Basic Education Development Index (Índice de Desenvolvimento da Educação Básica, or IDEB), a statistic which aggregated school-level performance and progress based on several metrics, most prominently Prova Brasil results (Frankin, 2011).
While there was some pushback to the promotion of high-stakes accountability measures of public school performance from city-level managers in secretariats of education and educators as a whole (Franklin, 2011), over this same period public sentiment largely supported heavier accountability measures in Brazilian public schools, due to perceptions of public sector inefficiency and waste.Over the same period, there was a steady rise of private-sector advocacy groups, most prominently Todos Pela Educação ("Everyone for Education"), who promoted such accountability measures through public media campaigns (Leme, 2011).Groups like Todos Pela Educação also promoted the adoption of value-added modeling and other market-derived policies at the regional and municipal levels 1 through promoting the careers of business and management professionals who transitioned into management positions in public sector education-most pertinent to the present case, Todos Pela Educação was very supportive when Claudia Costin, a technocrat with training in public administration and economics with previous positions at the World Bank, several prominent private foundations and in both state and federal government (Evangelista & Leher, 2012), was appointed Municipal Secretary of Education in Rio de Janeiro in 2009.
Costin was brought into Rio's Secretariat of Education by Eduardo Paes, the newly elected Mayor of Rio.Paes is a center-right politician, known at the time for his business-minded approach to government, prioritizing accountability and efficiency in public sector governance (Fajard, 2012).In her previous positions, Costin was known for prioritizing data in her administrative decisions, and using rhetoric of accountability and efficiency to challenge traditional understandings of tenure in public sector careers (Schwartzman, 2011).These priorities were reflected in the policy changes she brought to Rio, most pertinent to this article being the administration of a municipal standardized test, and the use of value-added modeling to determine bonus pay incentives for schools that reach growth goals relative to that test (Lucas, 2011;Lucas & Canen, 2011;Pessoa & Vieira, 2013) 2 .
1 While there is evidence of the role of Todos Pela Educação in other states and municipalities (Hattge & Lopes, 2013), my analysis here is limited to the city of Rio de Janeiro. 2 To understand how one particular secretary under one particular mayor could bring about such strong changes, it is important to know that Brazil is a federally structure constitutional republic, in which states and The trends towards high-stakes accountability policies in education reviewed in this introduction, which attempt to bring the efficiency and competition associated with free market capitalism into public sector management, are not unique to Brazil-an extensive empirical literature has documented the global trend toward market-oriented social policy (see Lauder, Brown, Dillabough & Halsey, 2006;Waldow, Takayama & Sung, 2014) and the privatization of public sector service provision across the globe (see Gluz et al., 2014), since the rise of the Washington Consensus (Gore, 2000) in the late twentieth century to the present day.What is less common are ethnographic explorations of the cultural and ideological norms that have both facilitated and arisen from these trends.In other words, despite extensive documentation of the spread of such policies, relatively little work as of yet explores why these policies have spread.
In this article, through analysis of ethnographic interviews with educational policymakers in Rio de Janeiro, I argue that the Municipal Secretariat of Education in Rio under Paes and Costin was run primarily by people whose backgrounds in the private sector and for-profit businesses heavily influence their notions of good management practice in public education.I further explore the ways in which Rio's implementation of value-added modeling and high-stakes accountability-based teacher evaluation mechanisms reflects these latent ideological trends.To make this argument, I will first review the theoretical framework undergirding this study and the methods I employed throughout before using my findings to articulate these arguments.

An Anthropological Approach to Meaning Making
This inductive study builds upon and contributes to the on-going anthropological debate on the meaning of culture, and the impact of various global cultural flows on the commonly accepted ideas and regular cultural practices of individuals in localized spaces.This debate dates to the earliest days of anthropology as a discipline.Early on, anthropologists often defined culture as the proprietary beliefs and patterns of behavior of any given societal group (Baldwin et al., 2006).This definition was nuanced with the passage of time in the academic literature, though in popular discourse this essentialized definition of culture remains common-so common that some anthropological thinkers (see Abu-Lughod, 1991;González, 1998) have proposed disposing of the entire construct of "culture" due to its potential for misinterpretation.
In this article, I draw on Anderson-Levitt's (2012) definition of culture as the "making of meaning" (Anderson-Levitt, 2012, p. 442), with meaning retaining a very broad definition that can include behavioral norms, common understandings, or any other type of shared knowledge or belief (see Anderson-Levitt, 2002;Fischer, 2007;Strauss & Quinn, 1998).While previous thinkers have emphasized behavior as the primary measure of culture (Erickson, 2011), others like Spradley (1979) include the thinking processes that lead to and inform behavior as important factors in the development of culture.García Canclini (2006) perhaps best incorporates these two elements in defining culture as "the whole of the social processes of signification" (p.121), those that are manifested in social action and those thinking processes that inform or direct such action.In this article, I build off of this definition that recognizes both action and thought as determinants of culture-though thought (at least as articulated in speech) remains the focus of the present analysis.
While anthropological fieldwork is traditionally carried out in discrete local contexts (Anderson-Levitt, 2002;Bartlett, 2010), the increasingly globalized nature of society has led anthropological thinkers (see Appadurai, 1990) to emphasize the ways in which global cultural flows municipalities are federal entities with relative autonomy (Geddes, 1990).It is due to this structure that a municipality like Rio de Janeiro can make drastic changes to its educational policies, while other states and municipalities may make very different decisions.
influence manifestations of culture that are reflected in localized spaces.One of the mediators of these flows are individuals in positions of power.While the definition of culture can occur at the individual level, and individual beliefs and actions can be influenced by multitudinous factors, some actors hold a greater level of power and influence in the "construction and diffusion of ideas around the globe" (Anderson-Levitt, 2012, p. 442).In the case of Rio de Janeiro's municipal public schools, the educational ideologies of city level administrators hold particular influence over the types of policies which are implemented in Rio-area schools.As stated previously, the purpose of this article is to investigate what those ideologies are, as expressed by such administrators in ethnographic interviews, and to use those interviews as a means of making sense of the rise of value-added modeling and similar high-stakes accountability practices in Rio's schools.

Methodology
For this study's methods I drew on my disciplinary training in anthropology and conducted an ethnography, including ethnographic interviews and participant observations.For the purpose of this article, I will focus specifically on 66 ethnographic interviews conducted with teachers, Secretariat officials and nonprofit workers, with a particular focus on Secretariat officials.

Site Selection
The interviews cited here are drawn from a larger year-long ethnographic study of education reform in Rio de Janeiro, which included smaller explorations of the role of teacher strikes in shaping education policy (as municipal teachers engaged in a multi-month strike during the course of the year-long study) and the role of private sector partners as both educators and curriculum developers in public schools (a research interest driven by Paes' and Costin's development of the Escolas do Amanhã program during the period of the study, which heavily involved the private sector in educational provision in low-income public schools).It is due to these other research questions, which are entirely dependent on particularities of the context of Rio de Janeiro, that this study was conducted in Rio rather than in any of the other states or municipalities in Brazil undergoing similar changes over this period of time.

Participant Recruitment
I located these district administrators, teachers and nonprofit staff primarily through the use of social networks.That is, upon arriving in Brazil for a year-long period of ethnographic fieldwork, I had initial contacts with the Secretariat of Education, with various public school teachers and with leaders of several educational nonprofits.I then reached out to other individuals that knew these initial contacts, and continued this "snowball" pattern with each new contact I met.In total, I interviewed five Secretariat administrators, 37 teachers and 26 nonprofit workers (though some of these were interviewed multiple times).The primary focus of this article will be my interviews with these five Secretariat administrators and 37 teachers.

Data Collection
With each interview subject, I began with open-ended, semi-structured interviews (Rubin & Rubin, 1995).On average, these interviews lasted two and a half to three hours, with several lasting up to five hours.All interviews were recorded and transcribed.Most subjects were only interviewed once-however, all five Secretariat administrators were interviewed twice.When subsequent interviews occurred they were unstructured, driven by the themes that arose from the coding and analysis of initial interviews and observations (Spradley, 1979).Having years of previous experience living in Portuguese-speaking contexts, I had sufficient Portuguese language ability to conduct and translate all interviews myself.

Data Analysis
After transcribing each interview, I translated them into English, typically one to two weeks after the interview was conducted.Every month, I coded my most recent interviews according to dominant themes that were arising within the data (see Emerson, Fretz & Shaw, 1995).Twice during the year-long period of data collection, I re-coded all transcriptions, to allow for the emergence of new trends and new codes within the larger set of fieldnotes.
When referring to these transcribed interviews in the following sections, I refer to each interview using the pseudonym given to the interviewee, and the month and day on which the interview was conducted.For instance, the reference Raquel (7/17) would refer to an interview with a woman I here call Raquel on July 17th.

Findings
Having previously documented the incorporation of value-added modeling and similar highstakes accountability measures under the Costin and Paes administration, I will here draw on administrator and teacher interviews to help make sense of why Rio's Secretariat officials turned to business-derived accountability models like value-added modeling.

The Roots of the Rio Secretariat's Business-Minded Thinking
For many of the teachers I interviewed, one large factor they saw as influencing Secretariat officials' policy decisions was their predominantly private sector backgrounds.As Raquel (7/17) stated, "Our education secretary, sadly, isn't an educator.She is an economist concerned with numbers, with statistical analyses."This was problematic to Raquel and other teachers I interviewed because not being educators, these managers of educational programs and departments weren't able to fully understand how schools work, what successful teaching looks like, and other forms of knowledge which teachers see as essential to managing education.As Paulo (7/2) put it: There are a lot of people today working in curriculum writing, in educational projects, even at the government level that were never teachers and have never worked in education previously….I've worked on lots of committees and teams at the Secretariat, and there is an incredible difference when the team is made up of schools and principals, people with experience, and when all the other people besides me have backgrounds in management, economics and so forth.These folks are very smart, they're very analytical, but they simply don't know what it is like.They don't know how to design something that will reflect the reality on the ground and work in that context, in an actual school, because they've never been there.
When I brought this up with Robson (7/2), a former school teacher and current mid-level manager within Rio's Secretariat, he was quick to defend the influx of management professionals as necessary: Truthfully, I think the ideal is to have equilibrium between these two strengths, those with management backgrounds and those with educational backgrounds.We really can't, and try hard not to, look down on the experience of those who really understand and know schools on the ground.After all, if we bring in someone who is wholly a manager, who has a lot of experience in the private sector but none in public schools, and don't bring in any educators to help him, he'll only be able to skate on the surface, as we'll say-he won't know what to do.He might have huge, wonderful ideas, but he won't know how to put them into practice.At the same time, though, we need people with management experience because we need to lead and manage this huge public school system we have.Rio's Secretariat is the biggest school district in Latin America, and it needs to be managed.Look at me as an example: I was in public schools for years as a teacher, then I came to the Secretariat, and now to combine these two worlds I'm getting an MBA in project management.I sought this training because I understand the need for this equilibrium, that it isn't enough just to understand schools, that I need to understand management techniques, that I need theoretical and academic knowledge to improve my practice as a manager.
This rhetoric of the necessity of managers is reflected in the reality that all of the current administration's leaders within Rio Secretariat, all of those in the highest positions, came from management backgrounds and were not educators: The Secretariat in large part is made up of people that are teachers.That is, they were teachers and now they are in bureaucratic "desk jobs."That said, all of those in leadership positions are people with degrees in management, economics, marketing.All of them have degrees outside of education, and the sense you get is that people with these backgrounds are thought to be more capable, more able to think big.(Andréia, 9/26) Specifically, if you look at Secretary Costin's background, and that of her main sub-secretaries, those that are responsible for the big projects and such, they have degrees in communication, administration, management, economics.So the secretary is surrounded by people at the closest level that think a certain way, that think like managers.You see this at lower levels, too, those that are brought in to run new projects…all of these new people are economists, marketing specialists and so forth.When all of the top positions are given to people with these backgrounds, people with particular training, and for that matter a complete lack of understanding of school routines, of what it is like to run a classroom-it sends a clear message.Those that really understand schools aren't invited to make the big decisions on how to run schools.(Lorena,10/12) This message that Secretariat officials prioritized management ability and experience over educational know-how was reflected in my interviews with Secretariat officials themselves.For instance: If you ask me, what's really lacking in schools is management.A bit more management among the educators.If you are an educator, but have a manager's background, then your school will function much better.When we have principals that have management training, that manage well their student organizations, their teachers, their parent committees, then that school gets along just fine.There is a clear difference, one you can see, when you compare a school with a principal that has a background in education, no matter how good they are at that, and one with a principal that has a background in management.The difference is night and day.(Vinicius, 7/10) Such statements sent a clear message: that management knowledge is more necessary for good school functioning than pedagogical knowledge or classroom experience, and that management knowledge is more highly prized and valued than that associated with education.They also sent a secondary message: that in order to function as the Secretariat desired, current teachers and principals should seek out opportunities to learn to become managers, to transition in their professional identity from educator to manager.Caio (8/5), a former teacher, embodied this transition: he was a former teacher, and yet the language he used to refer to teaching seemed very economic and market-based.For example, at one point he referred to the knowledge students must gain in the classroom as a "good" to be acquired, and then described students as "consumers" rather than learners (Caio, 8/5).This line of thinking reflects another influence of the business backgrounds of most Secretariat officials: that is, individuals with backgrounds in the private sector are assumed to be superior to those with backgrounds in public service in their ability to manage the public sector.According to this logic, then, the best way to improve the efficacy and efficiency of the public sector is to make it more like private enterprise, a task best suited to those with business backgrounds.

The Private Sector Roots of High-Stakes Accountability and Value-Added Modeling
It is precisely the influence of those business backgrounds, that, in the eyes of some Secretariat officials, led to innovations like high-stakes accountability testing and value-added modelinginnovations that have been perceived to have led to gains, and which would not have arisen from administrators with backgrounds in education.As Caio stated, Yes, there are management-types working here at the Secretariat, but ….in past administrations we didn't have a clear vision of what students were learning across the district, so Claudia [Costin] put together this new standardized test system, and she applied it to the whole district, so that with those test results we could have some sense of where we were having problems.But some teachers don't agree with these tests, they think it's an imposition, a means of control.But it is an imposition that facilitates management and good resource distribution.If you can see in the test scores that in a certain subject, one school is having trouble, then we can approach that school and try to understand what the problem is.Not to over-manage them, but to help the school meet their goal.And lots of times, teachers don't understand that this helps students, that this helps us improve educational quality.(Caio, 8/5) In truth, most of the teachers I interviewed did oppose these accountability reforms, in large part because they did not agree that such tests helped students or improved educational quality.To many of the teachers I interviewed, this business-minded focus on numbers and statistics as a means of measuring quality was the root of the problem: Public education today is obsessed with test results.They focus so much on testing students, and say it is for their benefit, but all they focus on then is test results.We don't talk about quality, about how to teach a good lesson, how to help students actually learn.Better test results don't lead to improvement in quality, as much as they might need these results to defend their practices politically.(Raquel, 7/17) The crux of this difference of opinion between Secretariat managers and teachers was a different understanding of what "quality" meant, and how to measure it.In the eyes of Secretariat managers like Robson, the metrics available from standardized test results, and the ability to follow those to measure improvements, are a crucial measure of improved "quality:" The Secretariat's priority is to improve student learning.This is all we work towards, all the time, making sure students learn what they really need for their lives.All of our actions, all of these tests, all of the measures to monitor results and use those results to determine what we need to work on, all of them are focused on improving educational quality so that we can improve student learning.All of our metrics, the new tests, the use of value-added modeling to reward high-performing teachers, all of it has been put in place to improve the quality of the product we offer to our clients, our students.(Robson, 7/2) In this quote, Robson made it quite clear that he perceived standardized test results to be an accurate and efficient measure of student learning, and teacher bonus systems based in value-added modeling to be an effective means of improving teacher performance.As outlined in the current literature (Auld & Morris, 2014;Fischman & Tefera, 2014), this is a common sentiment among education policymakers in the US, Europe and throughout the world.Robson's articulation of this claim, using references to the work done by the Secretariat as a "product" and students being referred to as "clients," implied a very market-oriented way of understanding education and the educational work done by the Secretariat.
It is precisely this market-oriented way of looking at education that many teachers I interviewed felt was at fault.Several teachers expressed feeling that a market or business-oriented approach to teacher evaluation, as exists in value-added modeling-based systems, did not fit in public education: This business system-I don't think it fits very well in education.I have issues with the current administration on this point.I don't see education in this way.I'll still work with them, I'll do what is possible that I can do, but I think that in this type of reform you end up doing a disservice to and leaving behind the children that most need it, because when you are measured and rewarded by your numbers it's easier to find a way to leave out those children rather than help them.(Amanda, 9/25) I agree with other teachers that criticize this focus on test results.You see clearly, here in Rio, at the state level, the national level, that the public common sense is getting closer and closer to what is considered common sense by the private sector, especially in this focus on using statistics and metrics to measure results.In education, statistics will never measure true results, they can't.Each child is singular, each person is different.There are structural differences between the contexts in which students learn, students begin at different points, they don't all start on an equal playing field.And they don't account for that in their metrics.Education for me is a process that permeates your entire life, and if you just focus on test results, what are you really measuring?When you reward a teacher just on test results, what are you really rewarding?Just what someone memorized in preparation for that testing moment?It doesn't work.For me we would be better served by teaching students how to address the different situations they will encounter in life, rather than just reducing them to numbers.(Jeferson,4/21) As Jeferson mentioned, one of the primary concerns of teachers is that the Secretariat's current focus on test results as an accountability measure has gone so far as to leave concerns relative to pedagogy and learning left unaddressed.While agreeing that management in general is helpful in school settings, several teachers felt that the focus on management and metrics had become myopic, leaving out any other considerations: This seems like a general trend, this focus on management, this idea that principals and teachers should all be managers.My current principal is a manager more than a principal-his degree is in management, and he forgets about pedagogy almost entirely.He just focuses on test results, sitting in his office worried about getting our numbers up as a school.He doesn't say things like, "Look everyone, let's do some interesting work that will help our students learn," he doesn't seem worried about that at all.He's focused on what is demanded of him by his superiors, and what might get us a schoolwide bonus-which means he's focused entirely on numbers.(Marina, 5/8) The Secretariat of Rio de Janeiro, in this quest for higher numbers, keeps harping on this idea that principals, and administrators in general, should all be managers.I agree to this to an extent, I think good administrators are good managers.But they can't just be managers, you know?(Renan, 9/3) In addition to their concern about the current Secretariat's focus on management and test results, use of value-added modeling to determine teacher bonuses in schools that reach their test-based growth goals was something many teachers identified as based in particularly faulty logic.This valueadded modeling system was generally referred to as the "meritocracy policy:" Rather than have one general growth goal for the all schools in the district, individual schools' growth goals were based on that school's previous performance, thus supporting the Secretariat's assertion that all schools meritocratically had an equal chance to earn the bonus3 .A number of teachers found the presumption that this system was truly meritocratic to be false and misleading, particularly Jeferson (4/21): This logic of meritocracy, man, meritocracy doesn't exist.Meritocracy for who?No-one begins in the same place-even if you use previous test scores as the basis for schools' goals, how can you take someone raised in Leblon [one of Rio's richest neighborhoods], who had parents who read to them at night every night, who heard high vocabulary from a young age, and who studied at Santo Inacio [a well-known private school], and say that the same test will equally measure their performance alongside someone who was born in a nearby favela, had parents that had to work all the time and were hardly ever able to be home, and studied at the local public school that didn't have running water and sent their students home half the time?I don't care what algorithm you use to try to even out those inequalities, the same test will never equally judge the knowledge of kids separated by that much social inequality.(Jeferson,4/21) Jeferson here exposes one of the flaws with market-based accountability policies in Rio: their failure to take context into account.That is, while it theoretically makes sense for the school district to use a single test to evaluate all students and reward all teachers, such a model fails to recognize how socially privileged students may have access to score-improving resources that poor students do not.As a result, even hard-working students may not be able to reach district goals due to circumstances beyond their control.
As another part of the meritocracy platform, the current Secretariat had begun publishing schools' test results in local newspapers.While defended as a transparency measure, teachers also took issue with the competition and stigma that this created for low-performing schools in poor neighborhoods: This kind of policy has horrible consequences.Rio has never had such an intense ranking policy before.Today they make the message very clear, "Just look at the rankings, School A is marvelous, School B is okay, School C is mediocre."And all on the basis of test results, test results are held supreme in this ranking, they are the basis for everything, for the school bonuses, for everything.That may sound like a good policy in a board room downtown, but at the school level, it means you have teachers who feel that one measure determines their value to society.It creates a lot of pressure to perform, and understandably, though unfortunately, teachers pass that on to students.You have teachers discriminating against students, telling parents that their kids are no good and should go elsewhere because they're worried about looking good, about the effects on their ranking.(Fabiana, 1/8) Some teachers worried about the effect such policies had on instruction, as school leaders felt pressure to push test preparation in the classroom as a priority over the regular curriculum, resulting in classrooms where students memorize facts for tests, but don't seem to really learn.As Renan (9/3) stated, "the problem with this focus on tests is that it leads students to memorize instead of learn.They just memorize, memorize, blurt out that information on the test, and then don't remember anything afterwards."Marina (5/8) similarly noted, "Students don't remember anything of what they memorize for these tests.If you do your job well and raise scores, sure, you get a higher ranking-but students don't really learn."One young man who tutored in low-performing schools saw himself in his students as he pushed test preparation skills, remembering his own experience of studying for a college entrance exam: I look at my students while I'm leading through old practice tests and I see myself.I was also turned into a number, when I was finishing high school and I was preparing for the vestibular4 , I took an expensive private study course.Several teachers worried about their students and the effect of high-stakes testing and value-added modeling on them.For one teacher, her concern was with students that don't test well, feeling that they are left behind and unaccounted for in the current political climate.Another teacher felt that high-stakes testing hurts all children, leaving behind a focus on learning.In their words, I know I have complete autonomy over my grades, but the thing is, if a kid in my class gets a grade that is significantly different than what they got on the test, I'm asked to come in and explain myself.They assume that either I made up the grade, or I'm crazy, or I'm incompetent because I didn't teach the material well enough for them to do well on the test.But it's not like thatthere are simply children that don't do well on tests, despite being brilliant, ones that get nervous, that even throw up on test days because they're so nervous.
Others whose parents get on their case when they learn there will be a test, and so the child shows up terrified about doing badly.We're terrifying our children with this pressure, when in reality some people just don't do well on tests-I've had friends that have tried to get public sector jobs time after time, and were really smart, but didn't get them because the public sector jobs always include a test, and they get nervous.Yet I passed and got this teaching job-does that inherently mean I'm smarter than them?No, it means I'm better at test-taking.There are so many people out there that are so much smarter than what they show on tests, yet all we're measuring, all that seems to matter, are the test results.It's scary, when you sit in your classroom and wonder: Am I wasting all this time on something that, with it comes to actual learning, what I came here wanting to help child do, doesn't even really matter?(Livia, 10/2) I realize that given current circumstances, I need to prepare students for these tests, I have to respond to these pressures to perform on the test, or I lose my job.I have to establish quantitative goals and work to reach them, I have to do whatever's necessary, whatever it takes.My question is, really, is this helping our students?Is it really helping anyone?With all this pressure we stop thinking of students as students, as people with individual needs and concerns.They become metrics to be increased.(Amanda, 9/25) What several teachers found most tragic and ironic was that the current Secretariat's focus on standardized test results, and the ranking and bonus policies that have come from that focus, do not even necessarily succeed in producing real gains in test scores.Rather, the high stakes associated with those tests have primarily produced a huge incentive to cheat: These reforms feel like impositions.Things didn't need to be like this, so extremely business-oriented, crammed into this business framework that doesn't really produce success in education.To be more specific, I'm talking about the business-based benefit system they use: that who produces results gets rewarded, and who doesn't receives no reward.Education doesn't work like that.Like I've told you, the teachers at our school, or any school, get an extra 14% annual bonus if the school reaches its growth goal.Is it nice to get a 14% bonus?Sure, it's great getting a 14% bonus.Who doesn't like extra money?The problem is that this confounds things, it's based on some pretty big assumptions about our motives.This whole idea is based on the idea that teachers aren't already doing their best for kids to raise their scores and reach their growth goals.And ironically, with these bonuses, you create your own problem, giving teachers a motive to make up scores, to make up grades.You see schools whose scores come out in the paper and they're incredible, the growth looks staggering, but it's all made up for show, it's as we say here, "for the Englishmen to see."When if you visit that school and really look at the quality of education in that building, you'll see it's nothing like what you would expect from their reported growth margin.(Bruna,11/13) Other teachers reported how they had seen individual teachers and school officials cheat so as to improve their test scores: You see this in classrooms, teachers giving students the answers, teachers changing students' answers before turning in the tests.Anything so that at the end of the day your school looks good when the results come out, and you're up on top.(Amanda, 9/25) You really do see this at the school level, schools that don't fail anyone.And you see the Secretariat applauding for them, because according to the logic of the business world, the way they think and work, this looks great, it looks like success, when in reality it doesn't mean you did anything.(Andressa, 12/10) I'm not naïve enough to think that everyone is honest with their test results.If your school earned its 14% bonus, that means nothing-it doesn't mean you really made a difference for your students.What it usually means is you cheated, you told the kids with difficulties to stay home and only invited the best students to come on test day, or you sent your slowest kids to another class at the beginning of the year so you wouldn't have to account for their test results.(Livia, 10/2) It's simple how it's done: kids that should be held back aren't, kids that didn't learn anything are passed along to the next grade, kids that won't do well on the test aren't allowed to take the test.This way, a school looks beautiful, 0% of students are held back, test results look good, at least for those that were allowed to take it.It all looks beautiful for the Secretariat.And everyone thinks that school is good.But nothing really happened, nothing really was taught, and next year no-one will know anything.But in the end, who cares?We got our 14%.(Luiza, 2/15) These teachers here articulate a clear critique of market-based monetary incentives for improving test scores: rather than promote real improvement, they provide a powerful motive to cheat.While the policy aligns with market logic (i.e. if bonuses work as an incentive to increase productivity in private business settings, they will similarly work in public schools), these teachers' personal experiences clearly show that market logic-based policy in this case does not necessarily result in genuinely improved test scores, much less increased learning.

Conclusion
As has been shown throughout this article, Rio teachers and administrators generally recognized that value-added modeling and other recent reforms put forward under Costin's administration of the Secretariat of Education have been based primarily in the perceived superiority of notions of management that come from business and the private sector.Specifically, teachers and administrators recognized that business experience was seen as superior to classroom experience in the then-current Secretariat, improved test scores were seen as an ideal measure of student learning, and value-added modeling-based monetary incentives were seen as an effective way to improve test scores.This preference for private sector-derived knowledge and experience was reflected in the fact that then-current positions in administration, curriculum-writing and so forth tended to be occupied by individuals with backgrounds in business and management rather than education.
These backgrounds were reflected in the ideological frameworks of Secretariat officials, as noted by officials themselves and classroom teachers.In the teacher interviews cited here, teachers felt strongly that administrators with business backgrounds did not understand classroom and school dynamics, that the standardized testing regime the Secretariat promoted did not effectively measure student learning, and that the value-added modeling-based monetary incentives used to promote improved test scores only resulted in falsified gains.In short, teachers felt that the reforms put forward by the current administration did not produce genuine improvements in school management or student learning.
In interviews with Secretariat officials, it was clear that they had already heard these criticisms.In response, Secretariat administrators defended their reforms on the basis of documented increases in test performance.In other words, it was not that teacher criticisms were not heard: Secretariat officials simply gave more weight in their decision-making to their own ideological biases, based in their own personal experience in the private sector.The Secretariat, in this case, clearly based their policy decisions in free market-based logic, enacting policies that emphasized quantifiable gains and that utilized monetary incentives to promote such gains.In other words, the Secretariat seemed fully committed to what Ball and Youdell (2007) have called endoprivatization, or the "importing of ideas, techniques and practices from the private sector in order to make the public sector more like businesses and more business-like" (p.9).While teachers understandably felt frustrated that their day-to-day experiences with these reforms were dismissed as anecdotal and subjective, that dismissal made perfect sense within an ideological framework that prioritized supposed objectivity and measurable improvement.
As one cultural microcosm reflecting larger global market-derived policy trends, this article provides insight into the types of ideological assumptions that likely facilitate the spread of valueadded modeling and similar policy frameworks in other contexts throughout the world where educational policymaking has become more and more the realm of technocrats with backgrounds in management and business rather than educators with backgrounds in classroom settings.It is also important to note that while this article has focused on the role of ideology in meaning-making, and the role of meaning-making in the creation and spread of public policy, other material factors (economic, structural and political) are also important in explaining the spread of value-added modeling and high-stakes testing accountability, throughout Brazil as well as in other contexts.While such material factors are not the focus of this article, they are particularly important to take into consideration given that similar educational policies have taken hold even in Brazilian states and municipalities with much more openly leftist leaders, such as Pernambuco (Brooke, 2016).While the findings presented here are not directly transferable outside of their specific cultural milieu, I hope this study promotes similar inquiry conducted in other public sector policymaking settings, particularly in other municipalities and states throughout Brazil, as well as other countries' regional and national Ministries and Secretariats of Education.

Tore Bernt Sørensen
University of Bristol t.b.sorensen@bristol.ac.ukTore Bernt Sørensen completed his doctorate at the Graduate School of Education, University of Bristol, UK, in 2017 with the dissertation "Work In Progress: The Political Construction Of The OECD Programme Teaching And Learning International Survey".Tore's research centres on comparative studies of education governance in a global context.Tore has a background as teacher and teacher trainer in Denmark.Before starting his doctorate, he worked in the Analysis and Studies Unit of the European Commission's Directorate-General for Education and Culture.

Antoni Verger
Universitat Autònoma Barcelona Antoni.verger@uab.catAntoni Verger is associate professor at the Department of Sociology of the UAB.A former postdoctoral fellow at the University of Amsterdam, Antoni's research analyses the relationship between global governance institutions and education policy, with a focus on the study of public-private partnerships and accountability policies in education.Currently, he is coordinating the research project REFORMED -Reforming Schools Globally: A Multiscalar Analysis of Autonomy and Accountability Policies in the Education Sector (ERC StG, 2016-2021).
This is, in my experience, a universal truth: good management makes all the difference.Good management lets you solve your problems in a reasonable time frame with reasonable costs.It makes it possible to reduce your costs.This is completely different from an administration mindset where you focus on other priorities, where you neglect planning, you neglect indicators and other measures, and you really don't have any idea whether your work is helping or not.(Robson, 7/2) They did this whole thing, almost like brainwashing, where you stay there all day memorizing what you need for the test, and then after the test you forget it all, because you don't need or use any of that information in your actual life.That's what I see happening in the low-income schools we work in, kids who are in most need of education are just being trained to answer standardized tests!What they really need is to learn to reason, to think critically, to see things from different perspectives, to understand differing arguments.The current test prep emphasis gives them none of that.(Jeferson, 4/21)