Catching Up With the Vaughn Express : Six Years of Standards-Based Teacher Evaluation and Performance Pay

Traditional methods of paying and evaluating teachers in the United States are longstanding, but discussions about changing these systems to support teacher quality and student achievement goals are becoming more common. Efforts to make significant changes to these programs can be difficult and take many years to design, gain approval, and implement; thus, few examples of alternative teacher compensation and evaluation systems exist. Relieved from many of the restrictions and requirements associated with most traditional public school systems, charter schools often are better positioned to implement changes more quickly than a traditional school, yet their experiences can provide useful information for others who are engaging in similar activities. Thus, the experience of a large urban charter school that designed and implemented an innovative teacher compensation system and standards-based teacher evaluation system that has been in place for six years offers important lessons in designing, implementing, evaluating and refining these systems. The Vaughn Next Century Learning Center has been one of the CPRE-UW Teacher Compensation Project’s research sites since 1998, when the school first implemented knowledge and skill-based pilot program and a school-based performance award program. The purpose of this paper is to provide a brief overview of the school’s pay Kellor: Teacher evaluation & performance pay 2 and evaluation programs, their evolution and current status, and to highlight findings from the employee attitude surveys that CPRE-UW researchers administered to Vaughn teachers the past five years. Finally, the paper will note key themes that may be of use to others who are considering compensation or evaluation reforms. Although other areas of research and inquiry are logical extensions to the work reported here, this paper primarily provides a longitudinal review of one school’s teacher compensation and evaluation innovation, including design, implementation, and evaluation, drawing from multiple information sources at the school and individual level.

and evaluation programs, their evolution and current status, and to highlight findings from the employee attitude surveys that CPRE-UW researchers administered to Vaughn teachers the past five years.Finally, the paper will note key themes that may be of use to others who are considering compensation or evaluation reforms.Although other areas of research and inquiry are logical extensions to the work reported here, this paper primarily provides a longitudinal review of one school's teacher compensation and evaluation innovation, including design, implementation, and evaluation, drawing from multiple information sources at the school and individual level.

Background
The Vaughn Next Century Learning Center is a public conversion charter school in the Los Angeles Unified School District.The school became a charter school in 1993 after a majority of the school's teachers petitioned for that status.Each five-year charter includes specific outcomes that will be achieved during the charter period and those outcomes inform virtually every decision made at Vaughn.The current principal, Yvonne Chan, has been with the school since 1990 and was transferred there prior to the school's conversion.Chan is widely credited with a dynamic leadership and inclusive style that has contributed to improved student achievement.The school's immediate neighborhood has been somewhat revitalized by the school's creation of a "compound" that includes a Family Center, a "one-stop shop" that provides families with basic needs, such as prenatal care, family literacy, adult education classes, job referrals, and services provided in collaboration with various agencies.The school's current charter includes plans to add one additional grade level a year so that the school eventually will serve students pre-K through high school; grade seven was added for the 2003-04 school year.
Vaughn employs about 75 classroom teachers and 50 paraprofessional and certificated support staff.Maximum class size is kept at 20 students (PreK-5) and 25 students in middle school.Every family is expected to be involved in the school community in some way, including signing an annual home-school compact that the parents will spend at least 30 hours a school year in activities that support their children's education.The school's governance structure includes roles for teachers, administrators, and parents, reflecting that responsibility and accountability for the education of Vaughn's students is shared and that everyone in the school community has a stake.It was from within the context of accountability and desired outcomes that the performance pay and teacher evaluation innovations arose.
The development of the knowledge-and skill-based pay program began during the 1997-98 school year.Six main issues and concerns were identified that contributed to the decision to develop a new pay system.They were: 1) pay inequity among teachers, 2) a concern that the single salary schedule promoted mediocrity, 3) a perceived need to develop a more explicit internal accountability focus to support the external accountability required by the school's charter, 4) a need to retain and incent new teachers, 5) a concern that many classroom teachers perceived movement into administrative positions as the only way to make more money, and 6) an interest in making improvements to teacher's salaries directly related to increased student achievement (Kellor, Milanowski, Odden, & Gallagher, 2001).Implementation of the pilot knowledge and skill-based pay system began in 1998-99 for newly hired teachers and for current Vaughn teachers within their first five years of employment at Vaughn and volunteered to participate.A school-based performance award program covering all teaching staff also was implemented that year.

Overview of the Pay and Evaluation Programs
This section includes a summary of these programs for each of the past six years.Because the school-based performance award program underwent few changes after the second year, that program is summarized first in a separate section, followed by an overview of the evolution of the knowledge-and skill-based pay and evaluation programs.

School-Based Performance Award: A Bridge Between Old and New
Although the school-based performance award (SBPA) program was created as an adjunct to the school's knowledge-and skill-based pay program, the initial compensation design did not include a SBPA program.Early in the design process, senior teachers appeared to view the new pay structure development as a personal threat or as implying that Vaughn intended to pay its teachers less.In January 1998 a draft plan was presented to Vaughn's Business Committee, which is responsible for teacher evaluation and salary determination.The submission of the draft plan brought senior teachers more directly into the discussion and review of options.In May a task force of newer and veteran teachers met to work out more of the details of the new pay program, and that group raised the question of why all teachers would not be compensated for those items included in the draft pay system that were based on specific outcomes at either the school-wide, grade-level, or classroom level.
The program designers then decided to include a SBPA element as part of its new compensation program to support the school-wide goal of improved student achievement and to function as a bridge between those teachers on the traditional salary structure and those in the new pay system.The SBPA also was intended to strengthen the concept of internal accountability by linking some form of compensation directly to student achievement for all teachers.
The initial SBPA program included two elements of performance: school-wide student achievement and school-wide redesignation of limited English proficiency students to English language learners (ELL).These two performance measures were key parts of the school's charter plan for 1998-2003 and thus were chosen as the SBPA's performance measures.Due to difficulties in establishing a measurement method and goal and the introduction of a new statewide accountability program beginning the 1999-00 school year, the redesignation portion was never implemented.
For 1998-99, the school used three measures of student achievement: 1) school-wide improvement on the state-mandated Stanford 9 (SAT9) reading, math and language tests for grades 1-5; 2) school-wide performance on the Terra Nova (English and Spanish versions) reading and math tests for grades K-5; and 3) school-wide report card grades.To receive the school-wide performance award the school had to post an increase of at least three percentile points over the previous year's SAT9 school-wide average score and meet at least one of the other two achievement goals: an average school-wide score of 37 th percentile or higher on the Terra Nova post-test or a school-wide average report card grade of "C" or better.Due to concerns about public pressure and credibility and potential problems in relying on a single measure, the school chose a combination of standardized test scores and grade level performance, as measured by report card grade.Compared to some SBPA programs that use a combination of academic and non-academic measures such as attendance, Vaughn's plan was relatively straightforward.The school-wide student achievement goals were met for 1998-99 and all certificated staff, including administrators, received $1,500.
After the first year, Vaughn continued its SBPA program but modified it in response to a bonus award component of the State's new accountability program.A key piece of the Public Schools Accountability Act of 1999 required that the California Department of Education annually calculate an Academic Performance Index (API) for every public school in California, including charter schools, and to publish school rankings based on the APIs.The Act also established a minimum five-percent annual growth target for each school.Schools that met API growth targets could be eligible for awards under several state programs, with potential individual awards of up to $25,000.Rather than operate a separate program, Vaughn decided to tie its SBPA program directly to the API program, with eligibility for a SBPA dependent on the school meeting its API growth target in a particular year.Due to changes in the assessment tools and the methodology to integrate results from multiple assessment measures, the actual calculation of the API varied somewhat from year to year and thus is not included in this paper.(California Department of Education, 2003) Vaughn's growth exceeded the target goal in each of the years for which API data are available.Table 1 shows the API base, growth target, and actual growth from 1999-00 through 2002-03: For most California schools, the actual payment of the state-funded bonuses was delayed most years by funding complications and difficulties in verifying final student achievement data schools.However, Vaughn's teachers received their bonus payments under the Vaughn SBPA program almost as soon as the test data were available, because Vaughn's administrators had pledged to pay the bonuses from funds already under the school's control rather than to wait for the state to release the API award funds.Strategically, this allowed Vaughn to maintain teachers' trust in the SBPA program and the school's commitment to deliver what it had promised.After three years at $1,500, the bonus award was increased to $2,000 for the 2001-02 school year and has been maintained at that level.
Several other pay innovations are included in Vaughn's pay system.A contingencypay program gives participating teachers a $250 bonus at the end of the school year for each of the identified areas for which the stated goal was achieved; the contingency plan was revised in 2003-04.(Refer to Appendix A and B for details.)A "gainsharing" program was developed, in which participating teachers receive the savings from decreasing the use of substitutes for the respective teacher.The pay program also provides "expertise pay" for specific roles or responsibilities.In the second year, a new "stock options" program was created by investing a lump sum received from a new grant.A total of $290,000 was invested on behalf of all certificated staff participating in the performance pay program, with distribution policies still under development, but with the understanding that the "Employee Incentive Fund" is intended to be a long-term investment with no immediate payouts.
The next section summarizes the performance pay (knowledge and skill-based pay) plan from its development through the sixth year of program operation.

Evolution of the Performance Pay Program
The first year of the performance pay program was intended to introduce the new pay system to new and early career teachers, who were perceived to have lesser commitment to or investment in the traditional salary structure than more veteran teachers.By phasing-in the new pay system, the school also hoped that veteran teachers would gain trust in the new system as well as confidence that the new system could be a positive change force.Teachers would be evaluated on the competencies each semester and eligible to receive half of the annual amount for each competency each semester.The competencies included in the performance pay program were chosen because in addition to directly relating to instructional principles in the school's charter they were viewed as non-controversial.The designers also believed that they could be measured by existing assessments or that new assessment tools could easily be developed to measure them.
The base pay rate for a first-year teacher was set at $30,000 for an emergency credentialed teacher and $31,000 for a teacher with a California elementary teaching credential.Each year for the first five years a teacher would receive a base increase of $1,000, with an additional $1,000 annually if the teacher had a master's degree.Although the $36,000 after five years for a credentialed teacher was less than the rate for a five-year teacher under Vaughn's traditional schedule, $37,867, additional pay increments available under the pilot performance pay system offered the potential of $3,300 more for four competencies related to instruction, and up to $800 more for the four contingency-based items.Potentially, a first year teacher would receive a base rate of $31,000 and up to $4,100 in additional pay under the performance pay program.(See Appendix A for a summary of Vaughn's performance pay provisions for the first five years.) The plan also offered the prospect of higher total compensation for existing teachers who were early in their careers at Vaughn.A five-year teacher would receive $36,000 in base pay, and up to $4,100 in performance pay, several thousand dollars more than under the traditional salary schedule.Five early career teachers opted into the program the first year.They cited a variety of reasons, including the potential for more pay, the equity of a pay system that did not restrict pay opportunities for those who did not have the time and/or money to take advanced coursework needed for salary advancement under the traditional system, and to be a role model by being held accountable for student achievement.
Although the original plan was to gradually expand the pilot pay program over several years to include more senior teachers, changes were made effective the second year that would allow more senior teachers to opt in.This change in direction was in large part a response to demands by some veteran teachers that they be able to participate in the pay program after they saw the amount of performance pay that many of the teachers in the pilot program received.These teachers believed that they were already performing at the level that would be required to receive the additional pay and that they could increase their compensation without significant additional effort or change in their performance.The second force behind the earlier expansion related to changes at the state level in grant and other funding opportunities that Vaughn could receive for certain instructional changes, along with state-required changes to the English as a Second Language (ESL) instruction that would require all Vaughn staff to be more involved in and responsible for ESL instruction.Thus, effective with the 1999-00 school year, the performance pay program was offered to all Vaughn teachers regardless of their length of service at the school.Participation for new hires continued to be mandatory, and teachers who opted into the new program could not switch back to the traditional salary schedule.
The design team for the expanded pay program was essentially an ad hoc group that included some of the same people who designed the pilot, joined by some veteran teachers who wanted to help make decisions about a revised program that could impact and appeal to more veteran teachers.The principal was involved in the design process, but tried to give the teachers the lead role.She stated that after the fact, this might not have been the best strategy, because it might have put the teachers in an unreasonably awkward position by requiring them to deal directly with their peers on controversial and sensitive pay issues.The potential availability of state funds earmarked for specific educational reform goals also meant that the pay designers had to consider whether the state funds were related to the school's charter and if so, how they could best be used to support the new pay program.This required that the design team deal with school-wide issues of curriculum design and delivery, but did give the school access to some additional funds that were used to enhance the pay program.
The design of the initial pilot program was not without challenges and new issues continued to emerge as the designers sought to expand the program.The primary challenge was to define pay principles in a way that would not seriously disadvantage veteran teachers, particularly those near the top of the traditional salary structure.Pay plan designers knew that if movement to the new pay system required too large a financial penalty, teachers would be unlikely to take the risk.A second challenge arose as the potential value of the performance pay was considered.For teachers in the pilot program, the potential performance pay available was almost 25 percent of their base pay.That same dollar amount, however, was only 5-7 percent more than the base pay of a more veteran teacher at or near the top of the salary schedule.Thus, a fundamental question arose: should the potential pay be "indexed" to the base salary rate, so that the same potential differential was available to all teachers, or should the performance pay amounts remain a flat dollar amount-providing the same amount of compensation for the same demonstrated performance.Estimates of veteran teachers' pay under the new system suggested that the proposed pay system would net veteran teachers on average the potential for 17 percent more pay than they were receiving under the traditional pay system.
While recognizing the veterans' concerns that a new pay system should not drastically affect their "guaranteed" base salary, the design team believed that the new pay system must strike a balance between that concern and the need for a new pay system to foster and be supportive of individual accountability.The designers also believed that for the new pay system to have a motivational effect, the potential payout under the new plan, and the potential cost of not performing well enough to earn at least some of the performance pay, had to be considered.To help address the tension between the need to decrease base pay rates under the new system and the desire to make the new plan attractive to teachers, the plan's designers adopted a policy statement that whatever base rates were decided, monthly "off-sets" would be available to help veteran teachers make the transition to the new and likely lower base pay rate.
Another challenge to the expansion of the pay program was whether there should be additional or different knowledge and skills identified for experienced teachers.The designers agreed that the initial set of competencies should apply to all teachers, and that additional competencies should be identified that would presumably be applicable to more experienced teachers.The result was the creation of three levels.Level 1 was slightly reconfigured for the second year and deemed "depth in essentials," those knowledge and skills that all teachers at all levels of experience should be able to demonstrate in their teaching.Level 1 included: literacy, ESL/Sheltered English, technology, special education inclusion, classroom management, and lesson planning.Level 2 included four additional competencies (mathematics, social studies, science, arts) that represented the breadth of knowledge and skills required of all teachers who, as elementary school teachers, essentially function as generalists and are responsible for instruction in all content areas.Level 2 also included English learner's support (BCLAD), with three possible strands of expertise.To be eligible for any of the Level 2 performance pay, a teacher must have received an average rating of 3.0 on all Level 1 competencies and have a clear California teaching credential.The latter requirement also was intended to serve as an additional incentive for emergency credentialed teachers to obtain a clear credential.
Level 3 did not include any additional knowledge or skill areas, but rather required an overall level of teaching performance markedly higher than that required for Level 2 performance pay eligibility.To be eligible for Level 3 pay, a teacher must have received an average rating of at least 3.5 in all skill areas (Levels 1 and 2).Thus, an experienced teacher with above average teaching performance could earn up to $13,100 in additional pay under the new pay system ($3,500 for Level 1, $5,600 for Level 2, and $4,000 for Level 3).
The decision whether to stay with the traditional pay system or opt in to the new system seemed to reveal two disparate views among the veteran teachers.Some teachers saw the new pay system as a way to create a more equal playing field that compensated people in a manner that was more directly tied to their actual contributions to Vaughn and its students, rather than on how well they had "played the game" of taking advanced courses and acquiring degrees.Other teachers who had successfully moved up the traditional salary schedule felt they would be penalized for having played the game by the rules in place at the start and believed they should in essence be held harmless in terms of base salary rate, and not denied the opportunity to receive performance pay "bonuses."Ultimately, the pay program included annual base pay increases up to the 11 th year, and then an increase at year 15.The expanded pay program also modified the criteria for the Master's degree add-on, making it available to people who had obtained 30 educational units after attaining a clear California teaching credential but not a Master's.This was intended to help address the concern of some veteran teachers that the additional coursework they had taken would be completely disregarded under a new pay system.
The beginning of Year 3 (2000)(2001) marked completion of a year of experience with the expanded pay program and two years with the program's first cohort of newer teachers.Some relatively minor changes were made to the amount of performance pay for some competencies to more accurately reflect the relative value of the respective knowledge and skills.In addition, Mathematics was moved from Level 2 to Level 1 and Technology was moved from Level 1 to Level 2. These switches were made because after additional consideration, mathematics was viewed as a critical area that should be included at Level 1 to stress its schoolwide importance and fundamental place in the school's curriculum.Technology's movement to Level 2 was due to ongoing problems with making technology tools regularly available to all teachers to use in their day-to-day instruction, as well as a reassessment of the relative importance of Technology to the school's overall goals.
The fourth year (2001)(2002) the pay program saw few changes, and it was unchanged for the 2002-03 school year.Physical Education was added to Level 2, the amount paid for attaining Level 3 was increased, and a fourth level, Distinguished Teacher, was added, which would pay $2,000 to someone after five consecutive years of attaining Level 3 performance.For the fourth year, all but five veteran teachers had opted into the performance pay program.By the end of that year, three of the veteran teachers who had remained on the traditional salary system planned to return to Vaughn the following year; one other planned to retire and one to return to a LAUSD position Ultimately, of the three, one stayed and converted to performance pay, one switched to performance pay mid-year, and one moved out of town.Thus, by the fifth year, the traditional teacher salary schedule became part of Vaughn's history and all teachers, current and future, were included in the performance pay system.
The sixth year (2003-04) saw more significant changes to the performance pay structure.The addition of middle school staff who are subject specific teachers rather than generalists, and the school's goal to have one pay system applicable to all teachers, had made it difficult to use the subject-specific competencies of the former system.The school also felt that because most staff were earning the performance pay bonuses in every Level 1 competency area, the pay associated with those competencies should be reallocated to base pay, and the performance pay program should focus on more general instructional areas.Thus, although the pay program retained three levels, they became tied to performance on four domains-very similar to the original domains in Charlotte Danielson's Framework for Teaching-rather than to content-specific competencies.The total amount of performance pay potentially available to a teacher is $13,050; the revised pay schedule is shown in Appendix B.

Evolution of the Teacher Evaluation System
Teacher evaluation at Vaughn prior to implementing the performance pay system was taken seriously, although it was not tied directly to pay.Teacher evaluation responsibilities were a significant part of two administrators' jobs and Vaughn teachers were accustomed to being observed and evaluated.The development of a new teacher evaluation system at the same time the new pay plan was being implemented meant that there were only two months to communicate and educate teachers about the new evaluation system, conduct observations and make decisions regarding whether a competency had been earned.
The determination as to whether an individual had demonstrated a competency was based on the average of the teacher's self-evaluation, a peer evaluation, and an administrator's evaluation.The four-level rating scale (1, 2, 3, 4) from Danielson's Framework for Teaching was used, but was open to individual interpretation; there were no rubrics or guidelines to help evaluators determine the rating level.Instances where there were obvious discrepancies between the three ratings were handled on a case-by-case basis, e.g., if a teacher gave a self-assessment that was much higher than the peer and administrator's evaluations.The discrepancy was brought to the teacher's attention and an opportunity given to revise the self-assessment, although it was not required that the teacher do so.To receive pay for a competency, the average rating had to be 2.5 or higher.
Concerns emerged about the evaluation system during the pilot year.The generic fourlevel rating scale was not adequate to ensure either clear understanding of the expectations for each competency or for consistent rating by all evaluators on performance levels within each competency.The school committed to develop rubrics for each competency area for each level for the second year of the performance pay pilot.The main goal of more detailed rubrics was to provide clearer guidance to both teachers and evaluators at all points of the evaluation process.
Beginning with the 1999-2000 school year, all teachers were evaluated under the new evaluation system, not just those teachers in the performance pay program.This was also intended to encourage more teachers to opt into the performance pay plan, because if they were going to be evaluated under the same system as those on performance pay, they would probably want to be eligible for the same pay enhancements.By the end of the first semester of the second year, over $107,000 was paid to teachers in the knowledge-and skill-based pay program, including payments of over $5,000 to several veteran teachers who attained the Level 3 rating and thus received all of Level 1 and Level 2 pay, as well as the Level 3 pay.Some operational and administrative changes were made to the evaluation system in the third year.Probably the most visible one was the creation of a specific committee, the Performance Assessment Review (PAR) Committee.Rather than relying on an ad hoc group of people with no clearly defined set of responsibilities or relationship to the school's governance structure, this committee was formally charged with ongoing responsibility to work on the evaluation system, including broad policy issues, as well as the administrative procedures and process.
As a result of its work that year, numerous changes were made to the evaluation system, including creating a pre-and post-conference system, an appeals process for evaluations, methods for evaluating team teachers who do not teach all content areas, and addressing inter-rater reliability.CPRE-UW interviews with teachers near the end of that school year revealed that they clearly viewed the PAR Committee and its responsibilities as important to the overall successful functioning of the evaluation system.The school's quick response to teachers' concerns about the evaluation system showed the importance of the evaluation system to the successful operation of the performance pay program and that the school was serious about creating a fair and trusted evaluation process.
A lingering concern on the part of both teachers and evaluators was the time required to accurately evaluate a teacher.The plan was to have two evaluation "windows" each semester, for a total of four evaluations from each evaluator each year.For each of the first core subject areas in which a teacher was evaluated, both the peer and administrator were to conduct a longer evaluation of up to 30 minutes of a specific lesson in the core area, supplemented by several shorter walk-throughs.During the subject-area observation, the evaluator would also look for evidence of the other competencies, e.g., classroom management or lesson planning.Teachers being evaluated for Level 1 competencies thus could expect evaluations in three core areas (Literacy; ESL/Sheltered English; Mathematics) four times a year, by two evaluators.For Level 2 or 3, total observation time in the five core areas (Social Studies and Science, as well as the three previously noted) could be up to about two hours by two observers, four times a year.This became too burdensome for both the teachers and evaluators, and mid-way through the second semester, the fourth observation was made optional.Although most teachers waived it, a few requested the fourth set of observations, especially if they thought an additional observation would improve their overall evaluation score.
Despite the amount of time spent on classroom observations, teachers still had concerns about whether all aspects of their teaching could be evaluated accurately.The introduction of pre-and post-conferences was intended to help assuage some of these concerns by providing an opportunity for teachers to bring the evaluators' attention to aspects of the rubric that might not be readily observable from a routine classroom observation, and for evaluators to ask for evidence of aspects of teaching that were not present in a particular observation session.
Feedback after the observation also was addressed the third year.Because Vaughn already had a separate mentor system for new teachers the evaluation system initially did not include feedback and was intended to be more of a summative than a formative review.Internal comments and responses to the employee attitude survey after the first two years indicated that teachers and evaluators both wanted to be able to give and receive more feedback as part of the evaluation process, and in particular, feedback about areas in which performance could be improved or evidence provided to support items in the rubric.The post-conferences introduced the third year offered a partial solution to this problem by providing a specific time where the feedback could be provided.The evaluation system also was modified to allow evaluators to provide feedback about how to improve one's performance.
During the fourth year (2001-02), work began on developing rubrics that would be needed when the middle school expansion began.The PAR Committee believed that even though only one grade level would be added each, beginning with sixth grade, rubrics for all three middle school grade levels should be developed so there would be a clear set of expectations for all the grade levels from the outset.There also was a belief that the teacher and student standards should be developed concurrently to ensure consistency and alignment between the standards and curriculum.A set of draft standards for middle school teachers were developed and were intended to be used the following school year when the school added grade 6.
The subsequent year, the effort to develop performance rubrics for all grade levels and content areas at the middle school experienced a setback when the sixth grade rubrics proved difficult to administer.An interim solution was devised for the remainder of the school year: continuing faculty were paid on the basis of their previous year's evaluation, and the one new staff member was evaluated based on classroom observations by another sixth grade teacher and an administrator.The entire set of performance rubrics, both middle school and elementary, were put under review and revision by the PAR Committee, with the middle school rubrics being the highest priority.New rubrics applicable to all grade levels were completed for use for the 2003-04 school year and paralleled the changes made to the performance pay program described earlier.

Teacher Evaluation Scores and Student Achievement: Preliminary Findings
Perhaps the most-asked question about any education reform is whether it improves student achievement.Clearly, test scores at Vaughn have risen dramatically during the last several years, but the role or influence of the performance pay plan or evaluation system in this improvement can't be presumed.There are inherent difficulties in attempting to measure any particular teacher's effect on student achievement, but one method that is growing in popularity is value-added analysis, wherein pre-and post-test results are used to identify student achievement growth over a specified period of time.Vaughn's standards-based teacher evaluation system and the availability of several years of test scores for a large number of Vaughn's students presented an opportunity for CPRE-UW to conduct a value-added analysis of student achievement at the classroom level and the relationship of classroom performance, if any, to teachers' evaluation scores.
In brief, the study used a sophisticated statistical technique, hierarchical linear modeling, to compare subject area teacher evaluation scores and student subject area standardized test scores for reading, math, and language arts for thirty-four 2 nd -5 th grade teachers and all of their students for whom two years of student achievement were available.Teacher performance was measured by using teachers' average score in each evaluation domain across the three observation windows in the 2000-01 school year.A pre-and post-test model was used, measuring value-added achievement on the SAT9 test as administered in California.Of the three subject areas analyzed, a statistically significant correlation was found between the teachers' average score on the literacy domain and average gains in classroom level student achievement and positive relationships in the other two subjects.The demonstrated positive relationship between a teacher's evaluation score in the subject area domain and student achievement in that subject area is an important finding.(Gallagher, 2004)

Vaughn Teachers Reactions to Pay and Evaluation Innovations
The introduction of a new pay system or changes to an existing pay system can cause turmoil in an organization, and schools are no different in that regard (Odden & Kelley, 2002).Because not all of Vaughn's staff embraced the idea of a new pay system, the effects of the new pay system on the school's culture and staff's responses towards the new pay system were of interest to both the school and researchers, especially since there are relatively few opportunities to examine the effects of compensation changes in the K-12 public school context.At Vaughn, the teachers' responses to a new evaluation system were of special significance because of the possible overlap into their perceptions and acceptance of the new pay system.Changes to an evaluation system also can be controversial, especially so if the evaluation outcome has some influence on pay or pay increases (Kimball, 2002;Odden & Kelley, 2002).
The context of Vaughn's new pay system was even more interesting because the impetus for change came primarily from staff, making it more of a bottom-up change than often is the case when compensation or evaluation systems are changed.Similarly, because the new evaluation system evolved out of a need created by the new knowledge and skill pay system, it could be considered more of an "organic" change than is typical when evaluation systems change.Vaughn's leadership and staff were willing to allow the school to become an ongoing research site for the CPRE-UW Teacher Compensation Project, offering a rare opportunity to track the evolution of these innovations from the first year of implementation to the present.The school's leadership also used the results of each year's survey as a form of ongoing evaluation, leading to changes and improvements.CPRE-UW researchers agreed to conduct an employee attitude survey each year, which was supplemented in most years by on-site interviews of representative teachers and administrators.
The survey was designed to address themes of interest related to introducing performance pay systems, as well as focusing each year on a question or area of interest to the school.The original themes identified for the survey were: 1) the new process to assess teacher competencies and performance levels; 2) the effect of the pay system on the school's sense of community and overall morale; 3) the alignment of related systems, such as professional development, with the demands of the new pay system, and 4) the extent to which the new pay system was perceived as influencing teachers' decisions to work at Vaughn.
Overall, the surveys had exceptional response rates, ranging from a low of 84 percent to a high of 99 percent.The surveys were distributed to staff at a staff development retreat just before the start of the school year so even though participation in the survey was not mandatory, teachers typically had work time to complete it.The school's culture supported its role as an innovator in education reform, and it is likely that many of the school's teachers shared that belief and thus were more likely to participate in the survey, knowing that the results could help others who were making changes to teacher pay and evaluation systems.After the first and each subsequent survey, the researchers prepared a report for the school.(Kellor & Milanowski, 2004;Kellor & Milanowski, 2003;Kellor & Milanowski, 2002, September;Milanowski & Gallagher, 2001;Milanowski & Kellor, 2000;Milanowski & Kellor, 1999).

School-Based Performance Award Program
The questions in this section remained the same each year and are relatively easy to summarize and note general trends.Overall, the bonus program generally was accepted quite well from the outset.A few questions showed dips and rises from year to year, but there were no questions for which a majority did not agree every year, and never any indication that the program should be abandoned.From the first year on, a majority of the respondents agreed that it was fair to hold teachers accountable for student achievement, with 20 percent disagreeing the first year and only 2-7 percent disagreeing in subsequent years.Similarly, a strong majority, 80 percent or more each year, agreed that it is fair for teachers who increase student achievement to receive a bonus, with the percent agreeing going as high as 98 percent one year.
The shift to using the API as the indicator for bonus payment eligibility beginning the second year did not appear to have any apparent effect on how Vaughn's teachers felt about the performance award program.Although respondents agreed with the statement that they "would work just as hard to achieve the school-wide outcome goals even without the possibility of getting a bonus," a large majority of respondents also agreed each year that as a way to measure the school's performance, the school-wide outcome goals are fair; they have a clear understanding of the school-wide outcome goals; the bonus provides a clear goal that encourages school staff to work together; the possibility of a bonus provides a reason to make the extra effort needed to meet the goals; the possibility of a bonus motivates to work toward the goals; the amount of bonus money is enough to motivate; and the school-wide bonus should be continued.
These responses suggest that even though teachers indicated they would work just as hard to achieve the goals if there was not money associated with it, they seemed to place some value on the prospect of receiving a bonus.
Two questions relating to expectancy theory, a motivation theory, were asked the first four years.The first asked the probability or likelihood of the school meeting its outcome goals if the teacher and his/her colleagues worked together as hard as they could.The mean rating each year ranged from 78.3 percent to 87 percent, suggesting that there was a relatively high degree of confidence that the goal could be achieved.A similar question was asked of teachers who participated in school-based performance award programs in Kentucky and Charlotte, North Carolina.Every year, Vaughn's teachers were consistently much more positive about this outcome than either the Kentucky respondents (52.7 percent) or Charlotte respondents (62 percent).(Kelley, Heneman III, & Milanowski, 2001) The second question asked the probability or likelihood that the teacher would receive the pay bonus if the school met the outcome goals; the mean rating ranged from 81.1 percent to 90 percent.In an ideal world, the response would be 100 percent-every teacher believing that the bonus would be received if the goal were met.Nonetheless, the responses from Vaughn teachers suggest a relatively high degree of confidence that the money would be paid, perhaps reflecting the degree of trust the staff in the school's principal and the financial ability of the school to pay the bonus.In contrast, teachers in Charlotte and Kentucky responded quite differently to a similar question.Although almost three-quarters (72.8 percent) of respondents in Charlotte believed they would receive the bonus if the goals were met, just over half (54.8 percent) of Kentucky respondents believed they would receive the bonus if the goals were met.There are many possible interpretations for the strikingly different responses in respondents, including size (school, district, state) and previous experiences of something being promised and not delivered, which was perceived as a factor contributing to Kentucky teachers' perception that the probability of receiving the money was only slightly better than chance.(Kelley et al., 2001)

Performance Pay and Teacher Evaluation Programs
Each year the survey included questions about aspects of the performance pay and evaluation programs.The first survey did not address evaluation issues as explicitly as did subsequent surveys.Responses the first year and feedback from interviews and sources within the school made it clear that there were strong concerns especially about the evaluation process, so specific questions about the evaluation process were added to the survey beginning the second year.
Responses to the survey for the first year of the new pay and evaluation programs (1998-1999) revealed something of a fissure between the general understanding and acceptance of the knowledge and skill pay plan, and how teachers felt about how it was being implemented, especially how the knowledge and skills were evaluated.Although a majority of respondents believed they had a clear understanding of the knowledge and skill areas, could explain them to others if asked, believed the knowledge and skills were important to good teaching at Vaughn, and were motivated to increase their own knowledge and skills in those areas, the positive responses generally did not carry over to how the knowledge and skills were evaluated or what supports were available to teachers to improve their knowledge and skills.Some striking findings were that less than 30 percent of respondents believed that the knowledge and skill assessment process was valid, that the assessment process was fairly run, and that the way the knowledge and skills were measured was fair.Curiously, although only about 40 percent of the respondents said that the knowledge and skill pay greatly influenced what professional development the person took the previous year, more than 80 percent said they made a lot of effort the previous year to develop the knowledge and skills that were rewarded by the pay system.The mismatch between the professional development and skill development responses suggest that people may have made individual, personal efforts to improve their knowledge and skills, rather than integrating the effort into formalized professional development opportunities.However, almost three-quarters (72 percent) of the respondents believed that the pay program should be continued.(Kellor & Milanowski, 1999) Most of the open-ended responses to the question why people chose not to participate in the knowledge and skill pay system the first year revolved around process fairness issues, including the absence of specific procedures and policies relating to the evaluation system.Other frequently noted concerns had to do with the applicability and availability of the pay program to veteran teachers and the negative effect the new pay program would have on individual's base pay.These concerns, along with the other sentiments expressed through the survey and other feedback mechanisms, were all influential in determining the features of the performance pay program and evaluation system when it was extended to veteran teachers the following year.
Survey results for the second year of operation (1999)(2000) suggested that staff had become much more comfortable with the knowledge and skill standards.In general, a higher percentage of respondents indicated that they understood and accepted the standards and felt the process was fair.For example, 71 percent (vs.44 percent) agreed that they knew what to do to increase their knowledge and skills, and 73 percent (vs.47 percent) agreed that they had a good understanding of how the knowledge and skills were measured.The perception of fairness of the evaluation process also improved, with 72 percent (vs.28 percent) agreeing that the way the knowledge and skills were established was fair, and 74 percent (vs.28 percent) agreeing that the way the knowledge and skills were measured was fair.Despite these improvements, less than half (46 percent) agreed that overall, the assessment process was fairly run, and only about twothirds agreed that they got a fair evaluation and that it was accurate.
As part of an effort to examine the assessment process fairness and validity perception, the second year survey included specific questions about the evaluator.In general, respondents agreed that evaluators were knowledgeable and provided positive recognition.They were less likely to agree that the evaluators communicated expectations, gave timely feedback, and provided coaching.Because it is not clear that these areas were explicitly included as part of the evaluation process, in retrospect it is not surprising that they were identified as areas of concern.
The responses to questions about the pay program itself were similar to the previous year.Most respondents felt that the pay amounts were motivating, but only a minority agreed that the pay system influenced their professional development choices.Fewer respondents than the previous year (64 percent vs. 83 percent) agreed that they made an effort to develop the knowledge and skills rewarded by the program.Because the program was expanded to include more experienced teachers, these responses could reflect an overall higher degree of confidence about one's teaching skills and less of a perception that changes in practice were needed to meet the teaching standards.The expansion of the program to include more veteran teachers did not seem to have a negative effect on how the pay system was perceived overall.
The relatively minor changes that were made to the pay and evaluation programs after the second year reflected the generally high level of acceptance of the pay and evaluation programs by staff.The creation of the Peer Assistance and Review Committee that began to operate during the third year probably was the most obvious response to the ongoing concerns about assessment process fairness and validity, and the responses to the survey after that year (see below) seemed to suggest that formalizing responsibilities for the oversight of the performance evaluation process paid off.(Milanowski & Kellor, 2000) After the third year (2000)(2001), a large majority of respondents indicated that they understood the system, accepted the teaching standards, and found the process and results fair.However, while 76 percent agreed that their own evaluation was fair, only 65 percent felt it was accurate.Some persistent areas of concern continued, with just over half of the respondents agreeing that they had the professional development opportunities needed to develop the knowledge and skills, and 59 percent agreeing that they had the opportunity to develop the skills.
Evaluators were perceived somewhat more favorably than in previous years, with about 80 percent of the respondents agreeing that the evaluators were knowledgeable about good teaching, but only about 40 percent agreeing that evaluators did a good job of communicating expectations and gave coaching that was needed to improve one's teaching.Respondents were split almost equally in terms of how they perceived the timeliness of feedback and whether the feedback helped to improve their teaching.Thus, although the respondents' perceptions of the evaluation process seemed to have improved in the aggregate over previous years, there were still lingering areas of concern.
The responses to questions about the pay program were similar to the previous year.Most respondents felt the pay amounts were motivating and fair, and for the first time, a majority of respondents indicated that the pay system influenced their professional development choices and that they made real changes to their teaching practice to receive the additional pay.A higher proportion of respondents (87 percent vs. 64 percent) also agreed that they made an effort to develop the knowledge and skills rewarded by the pay system.These are noteworthy responses in that they suggest that the pay system was beginning to have a direct influence on teachers' behavior.
The responses to questions about the effect of performance pay overall were similar to previous years' responses.A large majority of respondents agreed that performance pay had encouraged teachers to focus on improving instruction, improving student achievement, and to help each other develop skills.However, about a quarter agreed that performance pay had reduced the sense of community at the school, had hurt morale and there was more rivalry among teachers as a result of the performance pay program.About a third of the respondents believed that teachers resented other teachers who received more performance pay.These responses suggested that although a majority of Vaughn's staff had a generally positive reaction to the performance pay program, there clearly were areas of concern and the pay and evaluation programs were not universally accepted within the school.(Milanowski & Gallagher, 2001) The survey conducted after the fourth year (2001)(2002) of the new pay and evaluation programs did not allow direct comparison of the prior years' responses because many of the questions were changed to conform to similar questions being used at other CPRE-UW research sites.However, broad themes can be compared, and overall, responses were similar to previous years.The changes to the teacher evaluation system seemed to have positively affected teachers' perceptions of it.About three-quarters of the respondents continued to agree that: the processes and procedures of the evaluation system were fair; the respondent's own evaluation was conducted in a fair way; the evaluation process was run fairly; the evaluation scores the respondent received were fair, and the evaluation process helped the respondents improve their teaching.However, there was a noticeable decrease from the previous year (77 percent vs. 91.3percent) in those who agreed that working towards improving one's performance on the scoring matrix would really help teachers teach better.
In general, respondents reported that they had a good understanding of the scoring matrix and of the evaluation system itself.Almost two-thirds of the respondents agreed that professional development opportunities were available to develop the knowledge and skills rewarded by the pay system, and more than three-quarters agreed that they had the opportunity to develop the knowledge and skills that she or he needed to do well.This suggested that although professional development opportunities were not always perceived as being available, the majority of respondents apparently found ways to develop the necessary knowledge and skills.
Respondents' attitudes towards the evaluation process generally were favorable, although all aspects of the peer evaluator part of the process were perceived less favorably than the administrator parts of the process.The reasons for this disparity were not clear.For example, it could be that feedback and ratings by peers inherently are perceived differently by teachers than feedback and ratings by administrators, but that can not be determined without thoroughly reviewing the evaluation process to identify whether there were real differences between the two categories of evaluators.
New teachers appeared to be somewhat more positive about the influence that performance pay has on teachers' behaviors and actions.They were more likely to agree that performance pay encouraged teachers to focus on improving instruction, that it encouraged teachers to help each other develop their knowledge and skills, and that it encouraged teachers to focus on improving student achievement.
The effect of knowledge and skill-based pay on professional development activities and actual changes to instruction was reported as somewhat less than the previous year.As the school's staff became more familiar with the teaching expectations and more experienced in teaching in that manner, they might have felt less of a need to change their instruction.Vaughn also tried each year to offer school-wide professional development activities that supported development of the desired knowledge and skills, so perhaps staff did not feel as great a need to identify such professional development on an individual basis.
Despite the bottom-up origins of the new pay system, the perception that it reduced the sense of community among teachers continued, with 15 percent (vs.27% the prior year) of veteran teachers agreeing with that statement.Nonetheless, a strong majority of respondents agreed that the performance pay system should be continued, suggesting that the performance pay system is relatively well accepted by the school's teachers.
Of special interest to Vaughn administrators the fourth year was whether teachers were concerned about the potentially varying income that they could receive each year under the performance pay system.Two questions addressed this concern.Teachers were asked how much more or less stable they think their future income is under Vaughn's pay system, compared to Los Angeles Unified School District's traditional salary system.About a quarter of veteran staff felt that their income was somewhat or much less stable, whereas less than ten percent of new staff responded in that manner.At the same time, 43 percent of new teachers and 30 percent of veteran teachers felt that their income was about the same, and about half of both new and veteran teachers felt that Vaughn's pay system was somewhat more or much more stable than the Los Angeles district's.(Kellor & Milanowski, 2002) After the fifth year , survey results relating to pay satisfaction continued to show that the performance pay system seemed to have more influence on new teachers' decisions to work at Vaughn than it did for veteran teachers, with 57 percent of new teachers agreeing that the pay system was an important reason why they chose to teach at Vaughn, compared to 25 percent of veteran teachers who agreed with that.All of the new teachers who responded agreed that they have the opportunity to make more at Vaughn than they do elsewhere, compared to just over half of the veteran teachers who agreed.Finally, while 79 percent of the new teachers agreed that the performance pay system made working at Vaughn more attractive to them, only 49 percent of the veteran teachers agreed with that statement.It is hard to determine whether the pay system's appeal to new teachers reflected primarily the structural differences between the traditional pay scales and the performance pay system, which has a competitive starting base rate and offers more opportunities for new teachers to earn additional pay, or if it reflected philosophical differences between how new teachers and veteran teachers view their pay opportunities.(Kellor & Milanowski, 2003) As noted previously, significant changes were made to the PAR process and the performance pay program for the 2003-04 school year.Overall, the survey responses showed mostly favorable attitudes towards the new PAR system, and most areas showed an increase in the percentage of favorable responses.One potential area of concern was in response to the training on the new PAR system.Just over half felt they received enough training, and about a quarter were neutral about the amount of training provided.Given the direct linkage of the evaluation system to the performance pay system, it is important to provide adequate training on the PAR system, and the responses suggest the need to improve the training.However, the responses to questions about the fairness of the evaluation process and the clarity of expectations were generally favorable, and 70% agreed that the new evaluation system was an improvement over the previous one.If the increases in favorable responses are attributable to the changed rubrics, it could suggest that teachers are more comfortable thinking of their teaching in a more general and holistic way, rather than the more specific strategies and concepts that were associated with the content-specific rubrics used in previous years.Overall, the evaluation system continued to have a high degree of acceptance, with just over threequarters of respondents indicating that the school has an excellent performance evaluation system.
One of the reasons cited for changing the teacher evaluation rubrics from content specific to more general teaching domains was to address concerns that only particular "types" of teaching were recognized under the previous system.Thus, a new question asked whether changes to the PAR system in 2003-04 made it easier for the respondent to receive the score needed for performance pay; 60% agreed that it was easier, with 29% neutral.An increase in the number of teachers receiving performance pay could be an indicator of whether the perception of the new PAR being "easier" than the old one was valid, although the school has not yet conducted an analysis of the actual pay pay-outs under the two systems.
Overall, both veteran and new staff had positive attitudes about the performance pay system.The responses of new staff were supportive of the pay program but the extent to which it influenced their decision to teach at Vaughn wasn't clear.For example, although 76% of the new teachers agreed or strongly agreed that they "would have the opportunity to make more money teaching at Vaughn than . . .elsewhere," only 34% agreed that "the performance pay system is an important reason [they] chose to teach at Vaughn."For veteran teachers, the responses generally were about the same as in previous years, although fewer teachers agreed that performance pay increases rivalry between teachers, and that teachers who get more performance pay are resented by those who don't do as well.The majority of veteran teachers, 73%, agree that the system should be continued.(Kellor & Milanowski, 2004)

Wrapping It Up: Lessons from Vaughn's Experience
A standard phrase in any version of Vaughn's history is that it is a "conversion charter school."After six years of a knowledge and skill-based pay program and associated standards-based teacher evaluation system, it could rightfully be called a "conversion performance pay school," with all staff now paid on performance pay systems.Although Vaughn is an exceptional school in many ways, its transformation to a performance pay-driven organization has not been without some difficulties, some of which the pay plan advocates expected and others which were surprises.One of the reasons the Vaughn staff pursued a new pay system was to use the advantages of flexibility and innovation that being a charter school gave them, so that the lessons they learned from their pay experiment could be shared with others who might be in settings where the ability to take risks and be innovators was less supported.This last section highlights key points that could help make the journey to knowledge and skill-based pay or standards-based teacher evaluation a bit less arduous for others who are ready to move towards those education reforms.
Expect changes, but don't change too much or too often.From the outset, Vaughn's pay plan developers and designers of the teacher evaluation system knew that changes would be necessary, both to refine rough edges as well as to address unforeseen complications or situations.Some years saw more significant changes than other years, but efforts were made to make only those changes that were essential each year.This was done for several reasons, including the ability to accurately project costs for future years, to help teachers gain experience and trust in the new programs, and to be certain that only those changes that truly were necessary were made.
Encourage involvement of representative people in designing the system, but recognize the need for a "change champion" at times.Vaughn's governance structure requires teacher, administrator and parent involvement in almost every important decision relating to the school.The knowledge and skill pay plan and new evaluation system, although not initially designed by a formally designated group, followed the same inclusive principles.This was important because it ensured that the people most directly affected by the new programs-the teachers-were directly involved in key decisions about the programs.The governance structure included an approval process that ensured that broader concerns of all teachers, and more important, of students, would be considered before the pay and evaluation programs were put into place.The principal played an important role throughout this, providing support, direction and encouragement, along with ensuring that any data or assistance the designers needed was provided.On the flip side, the principal believed that she might have stepped back too soon, or inappropriately, at a time when the designers were facing controversial and emotional issues.Although she did this to empower the teachers and make it clear they had authority to design the programs, her absence of leadership at a critical time might have made the expansion of the program to veteran staff more difficult.However, it also is possible that the heated discussions that took place when the program was designed and the "true colors" that were exhibited at this time could have had a cathartic effect in that people's positions and concerns were made clear.Others involved in designing a new compensation system should consciously consider the positive and negative implications of a leader stepping away from a leadership role at critical points.
Even in a collegial setting, expect some "fallout" from changes for which there is no objective way to determine what's "best" or "fair."According to interviews and anecdotal comments made to researchers, the Vaughn staff generally considered themselves to be a collegial group that regularly put students' interests ahead of other interests, evidenced in part by the fact that the staff who remained at Vaughn when it became a charter school made a conscious choice to stay rather than return to a district position.Yet when it came to bottomline decisions that had to do with individual pay levels, some of that collegiality seemed to disappear and differences began to emerge about how people felt about important things like how individual effort should be recognized and compensated, and if describing the elements of "good instruction" took away from individual teacher autonomy and diminished the "art of teaching."This probably should not have been a surprise, especially when changing something as long-standing as the single salary schedule, but it seemed to take many people by surprise.Over time, Vaughn's sense of collegiality and morale seem to be recovering, perhaps aided by the gradual attrition of those staff who did not agree with the changes.Whether the school's sense of community would have been restored if a significant number of performance pay "dissenters" had stayed at the school is unknown, but places contemplating changes to their pay systems should consider the possibility that there will be splits among the staff and develop strategies to rebuild the community.
Communication is essential.Encourage the free flow of information, do what you can to ensure that accurate information is being presented, and establish and document clear, written policies and procedures so that the rules and expectations are known from the outset.Vaughn's pay plan and evaluation system designers did an admirable job of identifying important issues in advance and addressing them, either by a specific decision or by establishing policy directions to guide future decisions and make the programs' goals clear to teachers.Yet in some important areas the planners knowingly proceeded without specifying details (e.g., implementing the performance pay program at the beginning of the school year and working out the details of the evaluation process during the first several months of school) and in other areas (e.g., creating descriptions of performance levels in the evaluation system) its effort fell short.In Vaughn's environment, the lack of specificity created tension the first year and was a key priority for the second year of the program, but for many schools, the lack of specificity in areas as critical as these could have been a fatal flaw.The quick response to resolve the problems probably helped give the new programs credibility and to gain teachers' trust, but implementing a program without knowing and/or communicating details and policies introduces an even greater element of risk to the program's success and is not the preferred method for an effective implementation.
Transition details from one pay system to another are important; minimizing risk to veterans while meeting the goals of a new program can be a delicate balance to reach.Vaughn's approach to implementing a new pay system was to begin with the group of teachers who were likely to have less invested in the current pay system, i.e., new and early career teachers.Although new teachers were required to be part of the new pay system, even early career teachers were given the option to stay under the existing pay plan or to become part of the pilot.When the program was offered to veteran teachers sooner than originally planned, attention to the details of the transition were an important part of the design process, from establishing base rates that were somewhat reflective of the veterans' pay under the traditional salary schedule, to setting up a method to essentially keep a teacher's monthly pay close to its previous level on at least a temporary basis so that there would not be a precipitous drop in income before the performance pay was paid out.Another way that Vaughn dealt with transition issues was in structuring the pay system for veteran teachers so that some vestiges of experience were retained in the salary structure, both in the form of annual increases up to some point and by creating advanced performance levels that realistically would only be attainable by the more experienced teachers.(Vaughn's approach was still quite aggressive in that a teacher could end up with net compensation lower than the previous compensation, but given the overall confidence in teachers' ability to meet the performance pay standards the risk of losing net pay did not seem as real as it might have in other school settings.) Accommodating veterans' interests in the new salary structure in a way that was still compatible with the long-term goals of the performance pay program and the evaluation system, and identifying transition issues and providing ways to deal with them, very likely made the transition to the new systems more attractive and easier than if the pay system completely ignored the lingering influence of the traditional pay structure.
In brief, Vaughn's multi-year experience with its knowledge and skill-based pay and standards-based teacher evaluation system provide useful insights to others who are considering or are in the midst of teacher compensation and evaluation reform projects.Although the school's status as a charter school makes it difficult to generalize the exact circumstances of its compensation and evaluation reforms, the lessons derived from its experiences the past several years are useful regardless of the context.The evidence of the predictive ability of its teacher evaluation system on student achievement (that the higher the evaluation score, the higher the learning gains produced in that teacher's classroom) is exciting and the fact that the standardsbased evaluation system was introduced to meet the needs of the knowledge and skill-based pay program lends credence to the recommendation to aligning internal systems and resources so they support the same goals.  Teachers with satisfactory evaluation (2.5) will advance yearly 2 To receive Year 6 and above pay levels, must possess a clear California elementary teaching credential.

(depth +breadth after achieving 3.0 in all Level 1 areas)
From year 4 on, must have an evaluation of "average" or better to receive annual pay increase 2 To receive Year 6 and above pay levels, must possess a clear California elementary teaching credential 3 Beginning 1999-2000, paid for holding a Master's degree in education, OR for having 30 units after obtaining clear credential 4 Called "English language development" beginning 1999-2000. 1