Perceptions of the Impact of Accountability on the Role of Principals

Calls for accountability in America’s schools have created increased responsibilities for educational leaders. In this article, we describe and discuss a study of elementary, middle, and high school principals’ perceptions of the state-wide educational accountability program in North Carolina. The respondents indicated that the state’s accountability program has had its greatest impact on how they monitored student achievement, aligned the curriculum to the testing program, provided student remedial or tutorial opportunities, assigned teachers to grades levels or subjects, and protected instructional time. Views of some components, such as measures of school effectiveness, school safety standards, expectations and promotion standards for students, and financial bonuses received by staff members in schools that meet expected achievement standards, were viewed favorably. In contrast, the No Child Left Behind Adequate Yearly Progress (AYP) requirement (incorporated into the state’s accountability program), testing requirements for Limited English Proficiency students and special education students, the sanctions applied to schools that do not meet expected growth, and the school status designation labels that are applied to schools based upon student achievement were perceived more negatively. The predictable and unpredictable outcomes of a mandated accountability program on the perceptions (and behavior) of school principals create important considerations which are discussed for policy-makers and other professionals dealing with standards-based reform. Education Policy Analysis Archives Vol. 14 No. 16 2 Opiniones del impacto de la rendición de cuentas en los roles de directores/as de escuelas Resumen Las propuestas para aumentar la de rendición de cuentas (accountability) en las escuelas de América han incrementado las responsabilidades de los directores/as de escuelas. En este artículo, describimos y discutimos un estudio de las opiniones de los/as directores/as de escuelas secundarias del estado de Carolina del Norte. Los respuestas indican que el programa de rendición de cuentas del estado ha tenido su impacto más grande en: cómo supervisar los logros de los estudiantes, alinear los planes de estudios con las evaluaciones, implementar tutorías y oportunidades para estudiantes con dificultades de aprendizaje, asignar profesores de acuerdo a temáticas y/o grados-niveles, y proteger los tiempos dedicados a la instrucción. Acciones como mediciones de la eficacia de las escuelas, establecimiento de estándares de seguridad, estándares y expectativas sobre el aprendizaje para los estudiantes, y estímulos financieros para los miembros del equipo de las escuelas que alcanzaron los estándares apropiados, recibieron evaluaciones positivas. En contraste, el nivel anual de progreso adecuado (Adequate Yearly Progress AYP) de ley federal Sin abandonar ningún niño (The No Child Left Behind) incorporado en el programa de la responsabilidad del estado), los requisitos de evaluación de los estudiantes que están aprendiendo inglés como segundo idioma y de educación especial, las sanciones que se aplicaron a las escuelas que no obtuvieron los logros previstos, y la designación de estatus a las escuelas basadas en los logros académicos de los estudiantes fueron percibidos negativamente. Finalmente pensando en aquellas personas que participan en la toma de decisiones y otros profesionales que se ocupan de reformas basadas en estándares, se discuten los resultados previsibles e imprevisibles de un programa de rendición de cuentas (accountability) implementado mandatoriamente, sobre las opiniones (y comportamientos) de los/las directores/as de escuelas.


Introduction
Educational reforms and concerns about outcomes and accountability have been changing what goes on in schools for the past twenty years.Goals, objectives, directives, roles, responsibilities, programs, and activities have been targeted for sweeping changes; educational leaders have driven and been driven by all the "movements."The insistence by policymakers and politicians that educators be held accountable for student outcomes has resulted in dramatic shifts in the responsibilities of department chairs, principals, and superintendents (Duke, Grogan, Tucker, & Heinecke, 2003;Goldberg, 2004Goldberg, , 2005;;Ladd & Zelli, 2002;McGhee & Nelson, 2005;Tucker & Codding, 2002b).For example, principals today, particularly those in public schools, have myriad and varied leadership and managerial responsibilities as they carry out their roles.None of the many expectations are more important than that of chief educational accountability officer.Perhaps more than any time in history, student academic achievement is now the raison d'être of schools.As noted by Elmore, Abelmann et al. (cited in Ladd, 1996), the emphasis in terms of the principal's role has now shifted from being accountable for money and other resources to being accountable for student outcomes and achievement.Hanson (2003) argues that instructional accountability has now been placed at the institutional (i.e., the individual school) level and the process of grading schools and rewarding and punishing them for performance has become commonplace.He further notes that "low performing schools that cannot find a way to pump up their tests scores might be 'reconstituted' by being taken over by the state or having administrators and/or teachers replaced-essentially sending the whole lot to the institutional guillotine " (p. 36).Yet, Hanson argues, schools are a reflection of their communities and that a failing school is often a symptom of a failing community (e.g., lack of proper health care, unemployment, street violence, or inadequate parent education).This presents a major dilemma, as some researchers such as Cuban (1988) and Hanson have observed that the vast majority of principals devote most of their attention to managing the school and pupil control, particularly in view of the post-Columbine heightened concerns about school security and safety.Given the many and often conflicting role demands under which principals work, it is little wonder that fewer candidates are applying for these positions and some writers are questioning why anyone would want to be a principal (Tucker & Codding, 2002b).Further, difficulties in finding educational leaders could not come at a worst time based on the importance of principals in creating effective schools (Cusick, 2003;Ladd & Zelli, 2002;Olsen, 1999).The emphasis on new roles and their relationship to the heightened focus on accountability were the target for this study.

North Carolina's Accountability Program
North Carolina has received considerable national attention for its state-wide public education accountability program (Ladd & Zelli, 2002).Established in 1995 by the General Assembly and initiated in the 1996-1997 school year, this program commonly known as the ABCs of Public Education was established to foster, support and monitor student academic achievement in all local school districts.The program was initially implemented in the elementary grades, and the high school component was added during the 1997-1998 school year.This initiative which moved accountability from the district to the school level was designed to more quickly identify students performing below grade level so that intervention strategies might be timely employed.The program includes Student Accountability Standards called "gateways" for promotion at grades 3, 5, and 8, which require that students demonstrate grade level performance in reading, mathematics, and writing.Each year, elementary students in grades 3 through 8 take multiple-choice, end of grade tests that are aligned with the state's standard course of study in reading and mathematics.Writing tests are also taken by students in grades 4 and 7. End-of-course tests are taken by high school students in 11 subject areas, including Algebra I, Algebra II, Geometry, English I, English II, U. S. History, Physical Science, Biology, Chemistry, Physics, and Economic, Legal and Political Systems.High school students are expected to meet local district and state graduation requirements and, effective 2005, to successfully pass an exit examination.A comprehensive overview of the general accountability standards is available online from the North Carolina Department of Public Instruction (http://www.ncpublicschools.org).
The program is grounded in six components: a clearly articulated curriculum with content standards, agreement on the subject-matter content to be included, a set of valid and reliable indicators of progress and mastery, indicators reflective of how successful schools are at increasing student learning, a system of incentives and rewards to encourage improvement, and, a system of sanctions or directions for low-performing schools (Ladd & Zelli, 2002).The three basic purposes of the program were to provide local school accountability, emphasize mastery of basic skills, and promote as much local school decision making as possible.Since its inception, the program has been modified and improved to better portray school performance and to ensure that its measures are as fair and accurate as possible.The 2002-2003 school year marked the seventh year of the ABCs program for K-8 grades and the sixth year for high schools.The effort was specifically designed to essentially end social promotion of students who do not perform at grade level on the tests.When students do not initially pass the exams, they are provided two rounds of re-testing and a formal review process; however, final decisions on promotions are made by building principals.Students who are not promoted to the next grade must receive extra help in smaller classes or additional instructional opportunities.They also may be provided a personalized education plan that outlines the intervention strategies to be undertaken.A description of this aspect of the ABCs is available from the North Carolina Department of Public Instruction's Student Accountability Standards Reports website (http://www.ncpublicschools.org/accountability/reporting/sasr/2005).Ladd and Zelli (2002) summarized responses of a sample of North Carolina principals to key features of the state's accountability framework, including "the state's goals, the nature of incentives, the impact on the working environment of principals, and the actions they took in response to the accountability system" (p.502).They found that the program was perceived by elementary school leaders to be a powerful tool for changing behavior in intended (i.e., increased emphasis on reading and math) and unintended ways (i.e., decreased emphasis on other content areas of instruction).Little other research has been completed regarding principals' perceptions of their changing roles related to the impact of accountability.In this regard, we surveyed North Carolina principals to determine their perceptions of the state-wide accountability system and how it has affected their role as principal.Since the program has been operational sufficiently long for principals to have had substantial experience with it, we believed that valuable knowledge could be gained about the program by comparing information from a sample of principals representing elementary, middle and high schools.We also were interested in identifying instructional leadership responsibilities that the principals deemed most important in terms of their impact on student achievement.

Method
A survey was used to solicit principals' perceptions on the key components of the state's accountability program.To generate comprehensive, in-depth, and useful data, it was necessary to obtain responses from the subjects on an individual basis through a questionnaire.To ensure that participants would be representative of leadership within the state, principals were chosen from large urban, medium size, and small school districts.To the extent possible, districts from which the respondents were selected represented each of the three regions of the state, which includes the eastern region, central/piedmont region, and western region.Since large urban districts are only located in two of the three regions of the state, respondents from these districts necessarily came from only two regions.

Participants
Responses were received from 45 of 90 principals within two weeks of the first mailing (a return rate of 50%).To assure the respondents' anonymity and confidentiality, no system was developed to follow-up with non-respondents.Therefore, no follow-up mailing or contact was possible with those who did not respond to the initial mailing.Should a second mailing have been made to non-respondents, a greater response rate would likely have been achieved; however, representativeness relative to the target population within the state was judged to be acceptable when compared to state figures (North Carolina Public Schools, 2003).
Of those principals who responded, 10 (22%) were elementary principals, 17 (38%) worked in middle schools, and 18 (40%) were high school principals.The gender distribution of respondents was similar to that of principals in the state: 23 (51%) of the respondents were male and 22 (49%) were female.The majority of elementary principals were female, the majority of the high school principals were male, and the numbers of male (9/20%) and female (8/18%) middle school principals responding were essentially the same.The masters' degree is the highest degree held by the vast majority (56%) of the respondents.Twenty-six (58%) of the principals had less than ten years in the position, 14 (31%) had between 10 and 20 years of experience, and 5 (11%) were in the position for more than 20 years.
The elementary principals' schools averaged between 300 and 899 students, the middle schools averaged between 500 and 1,099, and the high school student enrollments ranged between 1,100 and over 1,500.Only one elementary principal responded who was in a school with less than 300 students.Thirty-two (71%) of the schools had between 21% and 59% of their students on free or reduced lunch (a proxy for the general social-economic status of the school).Only eight schools had less than 20% (4) or more than 80% (4) of the students on special lunch status.The respondents' schools were selected to be representative of the varied school systems within the state.

Procedures
A questionnaire was mailed to a stratified random sample consisting of 10-12% of the elementary, middle, and high school principals in each of the selected districts.The population of principals in the state was divided into strata (groups) to ensure that principals from all across the state would be included.Through a random procedure, 25 school districts were chosen from which the respondents were selected.A letter was sent to each superintendent in these districts explaining the purpose of the study and requesting his/her written permission to sample 10-12% of the principals in the district.Of the superintendents contacted, 22 (88%) responded and provided written permission.Two superintendents did not respond and one superintendent elected not to provide permission.
To solicit data from the principals, a three-part questionnaire (available from senior author) was developed.In the first section, 13 items (see Table 1) were used to assess principals' levels of support (1=Strongly Oppose to 5=Strongly Support) for the key components of the state's accountability program (e.g., emphasis on student testing and performance, school report cards, special testing requirements).The items were compiled from Department of Public Instruction documents defining North Carolina's ABC's program.
In the second section, principals were asked to provide their perceptions of the level of influence (1=Little to 5=Substantial) of the state's accountability program on 17 specific instruction leadership responsibilities (e.g., selecting teachers, monitoring instruction, obtaining needed resources).These items (see Table 2) were compiled from lists of professional standards and expectations represented in recent literature (Interstate School Leadership Licensure Consortium, School Leadership for the 21 st Century Initiative).In the third section, principals provided demographic information to support description and generalization of the findings.A letter explaining the study and asking subjects to participate along with the questionnaire and a stamped return envelope was mailed to the 90 principals in the fall of the school year.They were asked to complete the questionnaires and return them within ten days.

Design and Data Analysis
The study was designed to illustrate perceptions of principals regarding key aspects of accountability and its impact on their roles.Content and face validity of the questionnaire were grounded in conformance with underlying documents defining constructs being measured and Cronbach coefficient alpha internal consistency estimates for responses on both sections of the questionnaire were high (r xx =.85 and .92).Responses were available from principals with varied demographic characteristics (e.g., school levels, gender, degrees held, years of experience, size of schools, and percentage of students on free or reduced lunch).Summaries and comparisons of perceptions were completed for different groups of principals (e.g., elementary, middle, and high school, male and female) and for the combined sample of respondents.

Results
No significant differences were found for any of the demographic variable comparisons.Responses were similar across school levels, gender, degrees held, years experience, size of schools, and percentage of students on free or reduced lunch.While these outcomes may be due to an absence of statistical power to detect differences, the sample was comparable to that represented in similar research (Ladd & Zelli, 2002, N=64) and more varied (i.e., elementary, middle, and high school vs. elementary school only).To illustrate the degree of similarity in responses evident across groups, perceptions are reported for elementary, middle, and high school principals as well as the combined group of respondents.

Principals' Perceptions of the ABCs Program Components
Means and standard deviations for principals' ratings reflective of the degree to which they support the primary components of the ABCs program are presented in

Influence of ABCs Program on Principals' Instructional Leadership Responsibilities
Perceptions of the degree to which the ABCs accountability program has affected how principals perform their instructional leadership responsibilities are presented in Table 2 in the order they appeared on the survey.The responsibilities that the respondents rated as most influenced by the program included the following: Monitoring Student Achievement (Item 7: M=4.69, SD=0.75);Aligning the Curriculum to the Testing Program (Item 9: M=4.62, SD=0.63);Providing Remedial and/or Tutorial Opportunities (Item 10: M=4.52, SD=0.63);Assigning Teachers to Grade Levels or Subjects (Item 2: M=4.45, SD=0.89); and Protecting Instructional Time (Item 5: M=4.43, SD=0.99).
Responsibilities related to specific and frequent activities were reportedly more influenced that those with more general or global aspects of school leadership.For example, the instructional leadership responsibilities that the respondents rated as least influenced by the ABCs student More favored components of ABCs accountability and leadership responsibilities less likely to be affected by them were those with universal promise and high general degrees of acceptability (e.g., high achievement standards, supporting teachers and parents, obtaining resources).General expectations (e.g., numbers of students requiring testing, web site reporting) were moderately supported and general leadership responsibilities (e.g., assigning teachers, developing school schedule) were seen as moderately influenced by accountability demands.Less favored components of ABCs accountability and leadership responsibilities more likely to be affected by them were those requiring more direct action and typically low degrees of acceptability (e.g., supporting classroom instruction, meeting requirements, and dealing with sanctions).

Discussion
Concerns about accountability have cast education into a damage control mode and educational leaders as well as those who prepare them bear considerable burden and blame as criticisms remain strong or escalate and efforts to salvage the system become ancient history (Berliner & Biddle, 1995;Bracey, 1997;Cotton, 2003;Goldberg, 2004Goldberg, , 2005;;Hill, 2002;Kelley & Peterson, 2002;McGhee & Nelson, 2005;Stein, & Gewirtzman, 2003).In this regard, Tucker & Codding (2002b) a vivid picture: Why would anyone want the job of principal?Many school principals we know have the look these days of the proverbial deer caught in the headlights.Almost overnight, it seems, they have been caught in the high beams of the burgeoning accountability movement.(p. 1) As never before, the education enterprise is being challenged by the public and its federal, state, and local agents to improve student achievement.The federal government is putting pressure on representatives of the states who are pressing local governing bodies and superintendents to raise school performance.Of course, local leaders are tightening the vise by calling on principals to "make a difference" in their schools and the job becomes nearly undoable as the limits of power settle far above the individuals charged with change: Imagine that you are the principal, this person who is being asked to produce great improvements in student achievement.You cannot select your staff.You cannot fire anyone who is already on your staff.You cannot award or withhold a bonus from anyone.Seniority rights for teachers means that overnight, you can lose people you have made an enormous investment in and have them replaced by people who couldn't care less about your agenda.You may have little control over the instructional materials that are used.Someone else controls the training agenda.Someone else controls how all but a small amount of your regular budget is spent.Someone else controls how the federal program money will be spent.Some people who work in your school report directly to people in the central office rather than to you.In some systems, you do not even have the right to assign teachers to classes because teachers' seniority rights govern assignment.Yet despite all this, if your students do not make progress on the state accountability measures, your school is likely to be put on a public list of low performing schools.If performance does not improve, your school could be closed, the faculty disbanded, and you fired.You will be held responsible for the whole mess.(Tucker & Codding, 2002a, p. 6-7) While statements like these are not grounded in research or empirical evidence, they illustrate the context in which perceptions of educational leaders are often formed and provide a backdrop for considering the many factors that drive accountability or lack thereof in America's educational system.
In this regard, "surprisingly little is known about the impact of school-based accountability systems" (Ladd & Zelli, 2002, p. 494).Our findings were based on perceptions of elementary, middle, and high school principals.Professionals with less than ten years in the position responded to the survey in greater numbers.High school principals tended to have more years of experience than elementary and middle school principals.Over half of the high school principals had more than 10 years of experience in the position.Our outcomes suggest that there are differences in the acceptability and effects of our state's accountability program, one of the oldest and most heralded in the country.Respondents reported five favorable components of the program: the measures of school effectiveness that form the basis for the program-high student performance; safe, orderly and caring schools; and quality teachers; safety standards/expectations for students; financial bonus for staff members in schools that meet student achievement expectations; student achievement standards: promotion standards at grades 3, 5, & 8; and intervention expectations for students not meeting student accountability standards.These areas of support were predictable and bear strong witness to the "nurturing, supportive, even ministerial" goals, ideals, and objectives that bring many successful teachers and professionals to the principalship (Graseck, 2005, p. 373).Unfavorable components were more representative of the dangers on the "high-stakes" side of program and their implications for school leaders: the expectation for schools to meet adequate yearly progress (required under NCLB), the testing requirements for limited English proficiency students, the testing requirements for exceptional students, the sanctions for schools that do not meet expected growth, and the school status designation labels assigned to schools based upon student academic achievement as measured by test scores.These findings were also predictable in light of continuing reactions from educators across the nation as NCLB is fully implemented and drawn from recent research in Texas (another state recognized for its accountability reputation): Regardless of prior success, principals may be removed from their positions solely as a result of accountability test scores; test scores trump all.Principals who serve as leaders of schools with diverse student populations may be especially vulnerable to removal; clearly, the risk of serving the at-risk is real.The educational tradition of defining people by test scores is particularly disturbing when consequences of failing to meet standards are viewed as "irreparable" by those who fall from grace.(McGhee & Nelson, 2005, pp.370-372) Similarly, principals believe that their accountability program has had differential influence on important instructional leadership responsibilities.They indicated that the ABCs program had the most influence on monitoring student achievement, aligning the curriculum to the testing, providing remedial and/or tutorial opportunities, assigning teachers to grade levels or subjects, and protecting instructional time.In contrast, the instructional leadership practices that the principals believed were least influenced included dealing with student, teacher, and parent stress, evaluating teachers, and obtaining needed resources.Clearly, the focus of the latest grand drive for making schools better is correctly placed: teachers and teaching (and all its accoutrements) are at the core of improved student achievement.Of course, the difference between dreams (what and why) and reality (how) is the block.
Taken as a whole, the most troublesome component of accountability as directed by NCLB requirements is measuring annual yearly progress (AYP) to identify schools in need of improvement, a practice with concerns for professional organizations, academics, and the general public as well (AYP status, 2004;National Education Association, 2004;Policy implications, 2004;Popham, 2004;Weaver, 2004).The arbitrary, unyielding nature of the index and its reliance on simplistic, single-measure notions of performance with varying technical adequacy create disillusionment, discomfort, and dilemmas that are difficult to overcome.When schools may fail to meet AYP by trivial degrees (e.g., one-tenth of a point), concerns are justified.When indicators are influenced by disproportionate numbers of students with special needs (e.g., at-risk, limited English proficiency, or disabilities) and special considerations or exemptions are needed but not forthcoming, concerns are justified.When consequences grounded in these concerns are punitive and powerful (e.g., negative media publicity, threat of outside assistance teams, administrative "reassignment), concerns are justified.

Implications for Improving Research, Policy, and Practice
Recently, Gibson, Caldeira, and Spence (2003) studied the legitimacy of the U.S. Supreme Court using a survey of confidence in the leaders as regarded by the American people.Grounding discussions about policy and improvement in practices in outcomes from surveys is common practice and polling public and professional attitudes has been a key component of efforts to improve public schools for some time (Elam & Brodinsky, 1989;Gallup, 1970;Langdon & Vesper, 2000;Rose & Gallup, 2002, 2003, 2004).Our findings reflect less than positive general attitudes of educational leaders toward the acceptability and effects of a well-known and long-standing accountability program; these outcomes parallel those reflected in other research on opinions about educational reforms (Ladd & Zelli, 2002;McGhee & Nelson, 2005;Tucker & Codding, 2002a, 2002b).
So what can be done about all this shame, blame, and discontent?Clearly, developing curriculum content standards in critical areas of the curriculum, developing valid and reliable indicators of performance across the standards, and developing incentives and sanctions for favorable and unfavorable achievement are sound education fundamentals and educational leaders have a clear mandate to ensure that they are happening with fidelity.Additionally, Tucker and Codding (2002a, b) discuss broader assignations required for governments or agencies creating the work by setting the rules: Opportunities for funding research for effective leadership programs and methods should be available on a broader basis than has been the case in education.America's finest companies plow considerable money into product development-most would not operate without it.They also have access to funds from many sources to help them do their work.This is not the case in education; but, "something has to give," if the past is to serve as more than a promise of continued failure to progress.
Efforts to prepare school leaders to lead and manage today's schools with increasing pressure to produce steady gains in student performance requires attention from personnel preparation professionals.Opportunities for blending professional development provided in colleges and universities with the day-to-day "business" that goes on in local school districts have real merit here and deserve attention, acceptance, and fiscal support.In addition to providing "laboratories" for the development of effective practices in new and seasoned leaders, schools district leaders should also be asked to play a greater role in deciding who receives training, what the form and content of the training should be, and how it can be linked to work being done in their schools.The collaboration should also include mentoring and monitoring as programs strive to prepare more effective school leaders; and, adversarial relationships between colleges, universities, and school districts are clearly obstructions to be eliminated.
Opportunities for funding higher education's consumers need to be expanded.This means that money for training goes to those being trained, not those doing the training.Again, the giants of business and industry recognized the importance of "continuing education" (although seldom referring to it as such) and the availability of leadership and management training has produced important allegiances and powerful outcomes in the private sector.All things considered, accepting the sins of the past and not repeating them appears the best, but most formidable, course of action.

Who Would Want This Job?
Why would anyone want this job?As the mandate pile grows and the outcome package shrinks, it is easy to forget why smart people enter education and stay with it.Despite all the mess, the joys of listening, supporting, and inspiring teachers and others to improve the lives of children of all ages is one of the most rewarding jobs on the planet.The outcomes of our study suggest that it is a job being taken seriously by educational leaders charged with increasing responsibilities for higher and higher levels of performance and accountability.

Table 1
Summary Statistics for Principals Perceptions of the Components ABC Accountability Program a Reflects overall Analysis of Variance for differences in means across levels of school; p > .05;b overall rating for item was less favorable (i.e., below midpoint on scale).The second line in each cell is the standard deviation.
Table 1 in the order they appeared on the survey.The top five program components supported by ratings across all of the respondents included the following: The Measures of School Effectiveness: High Student Performance, Safe Orderly and Caring Schools, and Quality Teachers (Item 1: M=4.24, SD=0.99);Safety Standards/Expectations for Students (Item 11: M=4.21, SD=1.12);Financial Bonus for Staff members in Schools that Meet Student Achievement Expectations (Item 7: M=4.20, SD=0.95);

Table 2
Summary Statistics for Perceptions of the Influence of ABCs on Instructional Leadership a Overall ratings for all items were above midpoint on the scale; b reflects overall Analysis of Variance for differences in means across levels of school; p> .05.The second line in each cell is the standard deviation.