Assessing Pre-service Teachers Prior to Certification: Perspectives on the Performance Assessment for California Teachers (PACT)

This study focuses on pre-service teachers’ perspectives regarding how the process of completing the Performance Assessment for California Teachers (PACT) affected them academically, professionally, and personally. Pre-service teachers’ perspectives were acquired using a survey instrument comprised of open-ended questions. In addition, pre-service teachers’ selfconfidence levels pertaining to assessment task components of PACT (i.e., planning, instruction, Education Policy Analysis Archives Vol. 21 No. 56 SPECIAL ISSUE 2 assessment, reflection, and academic language) were measured prior to the execution of PACT portfolio assessment and these levels were compared to the actual scores on PACT. This study concludes with implications for teacher educators and teacher education programs implementing pre-service teacher assessments; these implications include policy level suggestions as well as a discussion of intended and unintended consequences of the PACT assessment on the pre-service teachers.


Introduction
For more than a decade, the No Child Left Behind (NCLB) Act served as a catalyst for rigid accountability measures and altered not only the way individual states educate their students in public schools but also how they prepare their teachers. Heavily facilitated by significant changes to states' K-12 academic standards, deliverables these standards aim to achieve, and pupils' academic outcomes with an underlying intent to hold the nation's schools and, ultimately, teachers accountable, this reform has introduced a new challenge for both teachers and teacher education programs.
The K-12 reforms advocating stronger accountability and achievement among nation's pupils are not novel to the Education community and have taken prominence in policy circles long before NCLB. An introduction of minimum competency tests in the seventies, a more blatant documentation concerning lack of achievement among nation's pupils in the Nation at Risk report of 1983, and standard-curricular alignments as well as other systemic reforms in the nineties were coupled with teacher certification reforms. Supported by the extensive literature linking quality teaching to student academic achievement (McCaffrey, Lockwood, Koretz, & Hamilton, 2003;Rivkin, Hanushek, & Kain, 2000;Sanders, 1997) as well as long lasting effects on learners (McCaffrey et al., 2003;Mendro, Jordan, Gomez, Anderson, & Bembry,1998;Sanders & Rivers, 1996), this shift toward stronger accountability has inevitably impacted teacher licensure and professional development measures for practicing educators. Notably, the introduction of basic-skill and content-competency tests in the subject matter brought forth in the eighties was the first step toward pre-service teacher assessments. Over time, basic-skill tests and content-competency assessments became embedded in teacher preparation programs either as a prerequisite for admission or in order to obtain a license upon completion. Although perceived as a correct course of action (more so in policy circles), the arguments arose stating that such an approach (i.e., heightened focus on assessment measures for pre-service teachers) neglects a need for the multifaceted set of skills that the teaching profession requires, especially in light of the complex standards-based reforms (Corcoran, 1995).
More recently, calls for "highly qualified teachers" have morphed to encompass a nationwide initiative for teacher exit exams. To this end, prior to 2010, conversations about and mandates pertaining to pre-service teacher exit assessments had taken place primarily at a state level. However, the current landscape on teacher preparation and accountability has shifted to a more centralized national initiative. As part of President Obama's goal to strengthen and support teacher evaluation systems, the not-so-long-ago concept of pre-service teacher exit exams has gone beyond just individual states and become a matter of federal interest (U.S. Department of Education [USDE], 2010). This movement arose among a highly politicized debate focusing on further reforming teacher education with the help of exit exam assessment measures. Over the years, teacher assessments and various appraisal systems designed to measure effectiveness have encountered much scrutiny documenting the impacts of and questioning the measures used to determine whether any given teacher is effective (Ludwigsen, 2009;Okhremtchouk, Seiki, Gilliland, Ateh, Wallace & Kato, 2009;Ovando & Ramirez, 2007). Additionally, these newly introduced assessment measures have been synonymously aligned with calls for deregulation of teacher education programs, which have long permeated the ongoing debate of what makes a teacher effective. Ballou and Soler (1998) and Fordham Foundation have been strong advocates for deregulation of the way in which teachers are prepared ranging from more practice-oriented alternative programs for teacher preparation to questioning a need for teacher licensure altogether (in the name of improving the quality of the nation's teachers and therefore improving pupils' achievement).
California, the state that employs and prepares roughly 10% of all teachers in the nation (U.S. Department of Education, 2009), serves as one of the first states both to implement preservice teacher assessments and to enact induction support measures. These changes have lead to elevated expectations placed on the pre-service teachers in California as well as added an additional accountability measure (i.e., Performance Assessment for California Teachers portfolio assessment) for the state's teacher preparation programs. This development directly affects pre-service and beginning teachers as the required performance assessments and long induction periods mandated by California Senate Bill 2042 (1998) made the acquisition of teacher certification in California more challenging than before.
While a fairly recent phenomenon in teacher education, the practice of exit assessments prior to licensure of professionals has been widely used in the United States, as evidenced by the history of the Bar Examination in Law and the Medical License Examination in Medicine. The argument for current policies requiring a similar accountability measure for the teaching profession appears valid and reasonable, yet, as with any change that adds layers of assessment and accountability, how this type of change affects the profession itself and those who are seeking it must undergo careful examination.
In this year-two survey study we examine pre-service teacher perceptions on the Performance Assessment for California Teachers (PACT), a recently mandated accountability measure, and how this portfolio assessment impacted pre-service teachers' academic and personal lives. The results of this examination will help inform the field about the impacts this nowwidespread accountability measure has on pre-service teachers. As part of the discussion of the results of this study, we also include considerations of ways to structure teacher education programs in order to better prepare pre-service teachers for high-stakes assessments. Furthermore, through further exploration of the effects of PACT on pre-service teachers, this study adds an additional layer to previous instruments, such as PACT Candidate Survey (Pecheone & Chung, 2006, and a year-one study (Okhremtchouk et al., 2009).
We start by providing an overview and background on the development and implementation of teacher performance assessments, focusing on PACT. We then describe the study itself and report results. Lastly, we conclude with a discussion of crosscutting themes that emerged from the data and suggestions for teacher education programs.

Background
Since the beginning of Normal Schools, education programs have played a significant role in teacher preparation resulting in much attention attributed to how teachers are prepared. Over the years, various studies have investigated different dimensions of pre-service teacher experiences, including their development of literacy practices (Estrada & Grady, 2011), concerns (Capel, 2001), mentoring experiences (Edwards, 1998), and how stress affects them (Head, Hill & Maguire, 1996). In a more recent investigation pertaining to pre-service teacher expectations, researchers have found that expectations appear to influence how pre-service teachers perceive preparedness at the end of their courses (Hobson, Giannakaki & Chambers, 2009). However, limited research exists on preservice teachers' perceptions regarding the effectiveness of their training (Ingvarson, Beavis & Kleinhenz, 2007;Okhremtchouk et al., 2009). As is true in this case, the vast majority of teacher education studies often had limited access to teacher preparation programs and showed data from individual teacher preparation schools. Giannakaki, Hobson & Malderez (2011) analysis of data from the "Becoming a Teacher" study, a large-scale longitudinal (2003)(2004)(2005)(2006)(2007)(2008)(2009) study of trainee and early career teachers in England, serves as one of few examples of scholarly works that went beyond single teacher preparation programs.
Coupled with the nationwide standards-based reform efforts, the complexities associated with educating an educator have expanded over the last decade. Using California as an example, current teacher credential candidates are held to a higher standard than ever before when acquiring licenses to teach. Senate Bill 1209 (2006, Chap. 517) mandated that each teacher preparation program embed a Teaching Performance Assessment (TPA) by July 1, 2008, as part of their graduates' completion of the credentialing process. Since the requirement for high-stakes pre-service teacher exit exams is a fairly recent phenomenon in the state, policymakers and education professionals need to investigate pre-service teachers' perceptions relating to this new introduction of assessments due to the national attention the PACT assessment has received. This study of pre-service teachers' perceptions pertaining to the PACT assessment has allowed us to do that investigation.

Teacher Preparation and Pre-service Teacher Exit Assessments
In the current era of accountability and standards-based reforms, teacher preparation programs as well as state and federal standards are being reshaped (Tellez, 2003;USDE, 2010). As part of these reforms, federal and state policies have mandated requirements for the assessment of pre-service teachers across teacher preparation programs (Darling-Hammond, 2006). Mounting concerns regarding the preparation of effective teachers in the nation's teacher education programs have resulted in the establishment of Teacher Performance Assessments (TPAs) (Ahlquist, 2003;Darling-Hammond, 2006;Sleeter, 2003). The federal Higher Education Act calls for evaluations to be partially based on graduates' performance and test scores (Darling-Hammond, 2006). Additionally, the National Council for Accreditation of Teacher Education requires programs to align pre-service teacher instruction and assessment with teacher accreditation standards (Darling-Hammond).
Given these realities, California's implementation of PACT plays a small part in a larger movement of teacher education reform in the United States. These reforms have caused major structural and organizational rearrangements in how California and other states prepare teachers. Since its development, the use of PACT has been expanding across California's teacher education institutions and beyond. Those institutions that use PACT assessment have amounted to roughly 30% of various teacher education programs (Pecheone, 2007). However, the implementation of the PACT assessment is no longer limited to the state of California and two out-of-state institutions of higher educations. Presently, twenty-four states in the nation are piloting a modified version of the PACT assessment, TPAC (Teacher Performance Assessment Consortium), to evaluate their preservice teachers prior to licensure. The consortium includes representatives from state education agencies and more than 140 institutions of higher education (American Association of Colleges for Teacher Education [AACTE], 2011).

California Context
In 1998, Senate Bill (SB) 2042 passed with the intention of establishing a new accountability system for teacher preparation (Pecheone & Chung, 2006). The bill included a change in requirements for the California Commission on Teacher Credentialing (CCTC) that entailed an implementation of new teacher education curricula and program standards that aligned with the state-adopted K-12 content standards. These standards, along with professional standards and preservice teacher assessments, now serve as the basis for California's teacher preparation programs (Selvester, Summers, & Williams, 2006). In response to SB 2042, teacher education programs across the state have reframed their curricula to reflect the new requirements established by the CCTC.
With the passage of SB 2042, debate has taken place over the type of assessment to be used in qualifying teacher credential candidates for teaching licenses. The CCTC hired the Educational Testing Service (ETS) to develop the California Teacher Performance Assessment (Cal TPA) to evaluate skills that candidates are expected to have learned before graduating from a credential program (Pecheone & Chung, 2006). However, teacher education programs were allowed the option of developing their own assessments based on the thirteen measures outlined in the state Teacher Performance Expectations (TPEs) (Selvester et al., 2006).
As an alternative to Cal TPA, Stanford University led a consortium of teacher preparation programs across California to develop collaboratively the Performance Assessment for California Teachers (PACT). The initial PACT consortium was composed of eight University of California campuses, San Jose State University, San Diego State University, Stanford University, and Mills College. In under a decade, the consortium has expanded in size both in and beyond California. To date, CCTC has approved three different exit assessments for pre-service teachers. However, only two can be used across teacher education programs: PACT and Cal TPA. A third, the Fresno Assessment of Student Teachers (FAST) 1 , is specific to one CSU campus.
The PACT assessment itself examines pre-service teachers' planning, instruction, assessment, and reflection skills (with close attention to academic language embedded throughout the assessment) using professional standards of practice (Darling-Hammond, 2006). These skills are documented through a portfolio of lesson plans, analysis of student work, and videotaped clips of pre-service teaching, all of which are accompanied by reflective writing (Pecheone & Chung, 2006). Trained readers score PACT portfolios using a task-based rubric (Pecheone & Chung, 2006). In October 2007, CCTC approved PACT as a measure to qualify teacher credential candidates for teaching licenses (CCTC, 2007b). The national interest and conversations centered on PACT could eventually translate to the assessment tool (or a version of it) being implemented in every state of the nation.

High-Stakes Assessment and Implications
Existing TPAs aim to engage pre-service teachers in an authentic exercise, which involves a development of teaching practices and reflection. Sternberg, Torff and Grigorenko (1998) and Sternberg (2002) stress that preparing pre-service teachers is a process rather than a task and involves building pre-service teachers' abilities to make teaching decisions based on the solid practice and abilities centered on designing, assessing, and executing lessons in various ways. As a result, these scholars assert that credential programs need to be tailored to ensure that the authentic process of skill development where the formative assessment of one's own skills or reflection processes is treated as the foundation of teacher preparation (Elmore, 2002).
In 2003-04, the second year pilot of the PACT assessment, thirteen institutions of higher education participated in administering PACT to their students. A portion of PACT submissions was double-scored, and results showed a high degree of inter-rater reliability. In its first pilot year, 2002-03, inter-rater consensus exhibited very little variation, and in the 2003-04 study 91% of double scored documents were in exact agreement or in agreement within one point (Pecheone & Chung, 2007).
As part of this pilot review, researchers administered an on-line survey to ask participants about their experiences with PACT as well as to acquire demographic information. The results from the PACT Participant Survey found that the majority (60%) of pre-service teachers felt they learned "important skills" in their preparation of PACT assessment portfolio (Pecheone & Chung, 2006). Pre-service teachers reported that the PACT assessment preparation process "improved their ability to reflect on their teaching" and "their assessment of student learning" (Pecheone & Chung, 2006, p. 11). The study also found that the support provided by teacher education programs is critical to successful completion of PACT. Pecheone and Chung (2006) point out that pre-service teachers who "receive targeted support in their development of the TE [teaching event] view their experiences more positively and report that the process of constructing their TEs strengthened their teaching" (p. 11).
The 2003-04 survey results indicated that candidates teaching in urban settings reported the presence of limitations on their "teaching decisions related to district mandated curricula" (Pecheone & Chung, 2007, p. 29). Further analysis of scores showed that these reported limitations were associated with lower scores (Pecheone & Chung, 2007). No significant variation between candidate groups was found in grade levels taught or in reported numbers of students who are English learners present in the student teaching placement classrooms; however, candidates teaching in suburban schools received higher scores on PACT than those in urban placements. Moreover, females and males demonstrated marginal differences on PACT scores, with females scoring higher (Pecheone & Chung, 2007). Selvester, Summers, and Williams (2006) conducted another study at a California State University campus to assess the effects of a locally developed TPA. The researchers sought to determine the impact of a TPA on faculty as well as its ability to rate pre-service teachers' skills. They found that teacher performance assessments do benefit teacher education programs. Most significantly, the pre-service teachers' questionnaire comments revealed desire for greater support in the form of models, mentoring, and direction during the TPA implementation. After identifying those needs, the faculty improved the articulation of their program courses to better support the needs of their pre-service teachers, resulting in an overall improved teacher education program (Selvester et al., 2006).

Background, Program, and Participants
This study is a year-two study of pre-service teachers' perspectives regarding the PACT assessment. The study was conducted during the second half of a pre-service teacher academic year at a University of California campus that has piloted the PACT assessment since 2002. Lead faculty in each specific subject area participated on the initial development of the PACT assessment or were identified PACT bench-markers and statewide trainers at this institution. The teacher credential program at this university remains very selective, and every qualified applicant is interviewed prior to being accepted to the program. At the time of the study, the school enrolled a total of 134 preservice teachers in six different programs in both elementary education (Multiple Subject) and secondary education (Single Subject Agricultural Education, English/Language Arts, Math, Science, and Social Studies/History). The instructor-to-student ratio in each cohort was roughly one-to-eight. The program required all pre-service teachers to complete PACT in order to be recommended for a California teaching credential.
To narrow the scope and increase the depth of the year-one study (Okhremtchouk et al., 2009), in this year-two study we have focused on two secondary cohorts: Single Subject English (SSE), and Single Subject Social Science (SSS). Out of the total 36 students enrolled in these two cohorts, 20 pre-service teachers participated in this study, resulting in a 56% participation rate.

PACT Rubric for English Language Arts and History Social Studies
Our reasons for the cohort selection were driven by the PACT rubric similarities between these two subject areas; these similarities assisted in the development of the pre-PACT pre-service teachers' confidence survey. The History-Social Science and the English-Language Arts rubrics are very similar. Each of the twelve-rubric tasks uses almost the exact same language; only when the issue of the content being evaluated arises does the wording change. This close alignment allowed easy comparison across these two rubrics, which is described in Table 1.

Table 1 Alignment Between English Language Arts and History Social Studies PACT Rubrics
Note. The above synopsis is based on the English Language Arts and History Social Studies PACT rubric.

Study Instruments
Two instruments were used in the course of this study.
(a) A pre-PACT survey collected information on the pre-service teachers' confidence levels of their skills. The survey posed eighteen questions pertaining to the five areas of the PACT evaluation rubric (described above) and asked the participants to evaluate themselves on a fourpoint scale similar to the evaluation used in the actual PACT rubric.
In developing this survey instrument and prior to implementation, the researchers asked three impartial teacher education faculty members who have also served as PACT scorers for over six years to inform the survey in order to ensure that the self-confidence survey accurately captures the PACT rubric. Once the teacher educators' feedback was incorporated, the researchers conducted a pilot run with several Masters degree students who recently completed their credential year (which required completion of the PACT assessment) in the same program in order further to inform the survey and adjust as needed.
(b) A second survey focused on pre-service teachers' perspectives and was a close replica of the prior year's survey (Okhremtchouk et al., 2009). In addition to an open-ended questionnaire to ensure the participants' freedom of response, students were asked to make suggestions pertaining to

Focus PACT Section Focus Asks
Planning Section one: Establishing a balanced instructional focus How the plans support student learning of how to apply the content material Section two: Making content accessible How do the plans make the curriculum accessible to the students in the class Section three: Designing assignments What opportunities do students have to demonstrate this understanding of the standards/objectives Instruction Section four: Engaging students in learning How the candidate actively engages students in their own understanding of how to apply the content material Section five: Monitoring student learning during instruction How does the pre-service teacher monitor student learning during instruction and respond to student questions, comments and needs Assessment Section six: Analyzing student work from an assessment How the candidate demonstrates an understanding of student performance with respect to standards/objectives Section seven: Using assessment to inform teaching How the candidate uses the analysis of student learning to propose next steps in instruction Section eight: Using feedback to promote student learning What is the quality of feedback to students

Reflection
Section nine: Monitoring student progress How the candidate monitors student learning and makes appropriate adjustments in instructions during this learning segment Section ten: Reflecting on learning How does the candidate use research, theory, and reflections on teaching and learning to guide practice Academic Language Section eleven: Understanding language demands and resources How does the candidate identify the language demands of learning tasks and assessments relative to the students' current levels of academic language proficiency Section twelve: Developing students' academic language repertoire How do the candidate's planning, instruction, and assessment support academic language development the support received and to identify one component of PACT that was most benificial to them as developing teachers. The survey questions were intended to assess the participants' affective reactions to the process of completing PACT amid the context of their lives and the teacher education program. The questionnaire consisted of three open-ended response items with seven (also open-ended) sub-items, constructed in an unstructured item format survey in which the participants had complete freedom of response.

Procedure
Prior to completing the PACT assessment, pre-service teachers were asked to fill-out a pre-PACT self-confidence survey. The self-confidence paper-based survey was administered in class by one of the researchers. Self-ratings for each area of PACT assessment and participants were then recorded and later compared to the actual PACT scores received in each subcategory of the PACT assessment.
The students were surveyed once again after completing their PACT assessment. Teacher Education faculty were asked to send the link to an online-based survey to all pre-service teachers enrolled in the SSS and SSE cohorts. All thirty-six students enrolled in SSE and SSS programs participated in the second survey. However, only twenty completed both surveys.
To ensure interpretive agreement (Teddlie & Tashakkori, 2003) and accurately portray the meanings expressed by the participants, the researchers reviewed data collected in the second survey individually for inter-rater reliability. The analysis proceeded as follows: (a) individual review of the transcribed data and consideration of emerging themes and patterns; (b) collective discussion of the found themes and patterns for inter-rater reliability; (c) coding of data according to the key themes/patterns found; (d) quantification of the themes/patterns; and (e) analysis of the data by looking at frequencies and variation in responses to statements.

Design
This study generated both qualitative and quantitative data. The research model adopted for this study is Sequential Exploratory Design (Creswell, Plano Clark, Gutmann, & Hanson, 2003). The qualitative data collection and reviews (individual rater review and interpretation of participant responses and subsequent group review and coding) were then followed by quantitative categorization and analysis. Since the researchers (n=3) reviewed and coded original data individually and then met to discuss their findings in order to reach agreement, this review process contributes to the interpretive validity of the study (Onwuegbuzie & Teddlie, 2003).
As the goal of this study is to formulate hypotheses based on conceptual ideas and to discern the participants' main concerns, multi-step coding process informed by Grounded Theory (Glaser, 1992) was used to analyze the qualitative and quantitative data generated by the self-confidence level survey and the actual score received on the PACT assessment.. The qualitative data in this study was weighted more heavily than quantitative data due to the nature and size of the study.

Self-Confidence Levels v. the Actual Scores Received
The results show an overall gap between pre-service teachers' self-confidence ratings and the actual scores received on the PACT assessment. As shown in Figure 1, across all areas, on average, pre-service teachers appeared to be more confident in their skills measured by the PACT assessment prior to completing PACT than the actual scores earned. The Academic Language category of the PACT assessment shows greater gap in self-confidence levels as compared to the actual scores received. In breaking down these differences further, while the most common official score received for any given PACT assessment category was a two, on average, pre-service teachers rated their skills higher in all areas measured by the PACT assessment as shown in Figure 2. Although the survey questions did not ask pre-service teachers to comment about the clarity of the rubric, several responses reflected that the PACT rubric was unclear. Below are the examples that speak about the PACT rubric and lack of clarity: The education-specific vocabulary of the PACT requirements made specific expectations of the project [they were] very difficult to understand/visualize. It seemed the requirements offered very few specifics.
PACT questions were very wordy-and didn't always make sense.
The comments pertaining to the PACT language outlining directions and requirements revealed that the participants were unclear as to what was asked of them. This lack of clarity, including clarity as it relates to the specifics of what is required, introduces a twofold challenge for the pre-service teachers encompassing an authentic demonstration of teaching skills by the candidates and, ultimately, successful completion of the assessment.

Effects of PACT on Personal Time, Student Teaching, and University Coursework
Although overall confident in their abilities, pre-service teachers identified a number of areas that have affected them to a great degree and proven to be "not helpful" prior to completing the PACT assessment.
As shown in Figure 3 over 90% of the participants reported that the PACT assessment had significantly affected their personal time. While the four day teaching segment enhanced my student teaching one major drawback of the PACT was that the subsequent two weeks or so while I was having to write and analyze for PACT my teaching preparation time diminished greatly and the amount of sleep I was able to obtain diminished to the point that I was no longer able to be the teacher PACT is designed to check for.
Completing PACT meant that I had a lot less time to spend giving students feedback on a major research paper. Classroom management was more difficult as a result of my feeling less prepared each day.
During the writing process [of PACT], both classroom management and lessons went down in quality.
The main focus of any professional program, including that of teacher preparation, is to prepare competent professionals. Many examples (similar to the ones shown above) stress that the practical component of the teacher preparation program suffered significantly due to the high-stakes assessment (PACT) taking place in the middle of the practical training resulting in diminished performance by the pre-service teachers in their student-teaching placements.
In addition, over 50% of the participants indicated that interactions with their resident teachers had diminished during the PACT assessment preparation period and became "less frequent." The following responses express this issue very clearly: I focused so much on PACT that I ended up not meeting with my teachers that often.
My interactions [with the resident teacher(s)] became less frequent. I didn't talk with my resident teacher about PACT at all.
Similar to earlier examples that revealed a significant impact on classroom practices during PACT preparation, the collaboration between the resident teachers and pre-service teachers at student-teaching placements was diminished or "became less frequent." Collaboration component of practical training plays a critical role in developing the skills necessary to address the many challenges teachers face once in their own classrooms. The mentor-mentee professional collaboration is one of the paramount essentials in acquiring such skills in order to meet the needs of today's diverse classrooms. Teacher preparation programs should focus on ensuring that pre-service teachers' practical training is not mitigated or interrupted by the highstakes assessment.
Moreover, roughly 70% of the participants stated that the university coursework required by other courses, which did not cover the PACT assessment, was considerably impacted by the timeconsuming task to fulfill the PACT portfolio requirements. As with the classroom management component, the university coursework question received no "helpful" responses. That said, it is important to note that the current program structure requires pre-service teachers to assemble their PACT portfolios while engaging in full-time graduate coursework in addition to participating in student teaching for at least three hours a day.
Work produced for PACT as well as other courses was sub-par. In essence, I did things to get them done.
It left little/no time for other coursework.
The time I spent on other work was greatly reduced.

I definitely focused most of my energy into PACT. I left the work for my research class until the very end.
As evidenced by their comments, the participants felt that it was very difficult to fulfill satisfactorily all the program expectations, which designed to provide theoretical knowledge and avenues for discussion. Once again, suggesting that the teacher preparation programs must carefully consider the timing of when the PACT assessment takes place in the course of pre-service teacher preparation. Figure 3. Affects of PACT on university coursework, student-teaching placements and personal time as reported by the participants.
In summary, pre-service teachers repeatedly stated that the PACT portfolio of skills takes a significant amount of time to develop. The intricate preparation process requires technological and pedagogical knowledge, extensive planning, and a successful implementation of the lessons depicted in PACT. These processes impacted the pre-service teachers' already busy schedules, which included a full complement of graduate-level courses, unpaid student teaching, and lesson planning five days a week, as well as maintenance of their personal lives.

PACT Helped Pre-service Teachers Learn About Their Teaching
Despite the drain on their time for completing their coursework while student teaching and planning lessons, pre-service teachers do appear to have acquired knowledge about their teaching practice from completing the PACT assessment. In their answers to the questions asking about impacts of the assessment on their instructional practice in the purview of PACT, many participants reported having favorable experiences.
My PACT lessons were some of my best lessons.
I think that the [PACT] analysis section has really helped me because I now have a better idea of how to take a piece of work and figure out what my students are still struggling with.

Good lesson planning [during PACT]. Coming up with meaningful lessons is always important.
[PACT] improved my lesson plans. I felt like I reflected a little more on what I was teaching and how I was teaching.
As reflected in the examples above, it is evident that the PACT assessment has a positive impact on the reflective component of practical preparation and quality lesson planning. In part, these responses suggest that more attention was given due to the high-stakes nature of the PACT assessment. Although positive, considering other factors i.e., subpar teaching at student-teaching placements, lack of collaboration between resident teachers and pre-service teachers as well as impacts on personal time, teacher preparation programs must be cognizant as to how they evaluate pre-service teachers' practice throughout the program. Conceivably, these responses speak more to a need for heightened rigor and accountability, which inadvertently the PACT assessment introduced, as compared to the benefits of the PACT assessment itself.
Furthermore, pre-service teachers believed PACT's requirements for designing a unit and, more specifically, a requirement of differentiating instruction to benefit students of varying abilities helped participants become better teachers.
The one thing that I found most useful is the intense reflection that you have to do for PACT. I now constantly reflect on my lesson and how they can be tweaked to better fit my students needs.
I think that the analysis/reflection section has really helped me because I now have a better idea of how to take a piece of work and figure out what my students are still struggling with.
The assessment portion opened my eyes to something I need to work on in the future. I think that the videotaping portion of PACT has made me more aware of my mannerisms in the classroom, and has also demonstrated that I need to work on my "universal eyes and ears." I realized the importance of periodically videotaping lessons to have a better idea of class environment.
As suggested by the excerpts above, the participants reported that their reflection skills on their own teaching practice and focus on various assessment strategies have improved. As found in year-one study (Okhremtchouk et al., 2009), the participants overwhelmingly reported the positive impacts of the videotaping component of the PACT assessment, which allowed pre-service teachers to authentically reflect on and improve their professional skills.

Support Received
The support received and sources of support proved to be another significant factor in the pre-service teachers' experiences during the PACT preparation. Over 70% of participants indicated that they relied on their university supervisors and their counterparts for sources of support ( Figure  4). The information acquired as part of the university coursework was reported as helpful or mixed for over 50% of pre-service teacher participants. However, another potentially critical source of support, resident teachers, was not reported as a source of support for the majority of participants. As evidenced in the year-one study the critical site-specific component of potential support, resident teachers, is yet to be improved (Okhremtchouk et al., 2009).
Reflections about support received from university supervisors and counterparts included comments such as My supervisors were more than helpful.
[Supervisors] were very supportive and accommodating.
The most helpful thing was discussing our ideas, plans, theories, etc.
[with my peers]. This is where I learned the most.
The responses pertaining to lack of support received from the resident teachers, on the other hand, varied from "none" or "not at all" to She was sympathetic but had no clue.
I was really upset. They [resident teachers] did not understand how difficult this was, and they didn't offer any advice or help.
Those pre-service teachers who expressed that their resident teachers were in fact supportive stated that the support was primarily exemplified in freedom to select the topics for their PACT assessment segment (otherwise restricted by the school departments' pacing guides) and in accommodations such as, "bringing coffee" or "finding a TA to help with PACT." The following comment serves as a good representation of those pre-service teachers who felt supported by their resident teachers.
My resident teacher was very supportive during the process. I was able to have free reign with my ideas for the lessons. Reflecting Back… As indicated in Figure 5, when asked to reflect back on their PACT experience and make suggestions pertaining to additional support needs, 68% of the pre-service teachers indicated that advising and modeling were the two key areas where they felt that more support was necessary. The other two components were the early introduction to PACT and mentorship. Figure 5. Types of support needed as reported by the participants.
Roughly forty percent of pre-service teacher participants had expressed that they would have liked to examine more models during the in-class discussions about PACT. I would have loved to see more models and been able to discuss the assignments with more depth.
Models of each task would have been helpful... especially the assessment task.
More models of effective PACT responses… as specific requirements [of the PACT rubric] often seemed vague.
As reflected above, in their statements, the pre-service teachers indicated that having good examples of each component required by the PACT assessment would have provided a better understanding of the assessment along with giving them a more in-depth understanding of each task.
Moreover, the participants suggested a peer-review type of an arrangement for the development of the PACT portfolios. This suggestion was offered in multiple participants' responses, which included several statements indicating that advisors and course instructors could only offer limited feedback pertaining to the PACT portfolio during the preparation stages in order to ensure authenticity of the assessment. These suggestions carry much merit and should be strongly considered by teacher preparation programs that administer PACT. Due to the restrictions imposed by the PACT assessment to ensure authenticity, peer-review process would guarantee an additional layer of support without compromising the authenticity of the PACT assessment portfolio.
Additionally, pre-service teachers shared that the type of support received and when they received it influenced their PACT portfolio development. The participants (21%) stated that they would have liked to receive information about the PACT assessment and examine the models of the PACT portfolios early on in their credential year as compared to halfway through their credentialing program.
Introduce PACT at the very beginning so it will not be this dreaded thing that we know very little about until January.
Explanation at the beginning of the year instead of at the semester.
[ Overall as suggested by the participants, the common theme centered on timing of the assessment as well as preparation prior to engaging in the development of the PACT portfolio have been reported to be paramount. Lastly, the participants suggested a mentorship or buddy system, which would allow pairing with Masters students who had undergone the process of completing the assessment a year before. As expressed by the pre-service teachers, such an arrangement would provide guidance while not violating compliance issues associated with receiving feedback from teacher educators and supervisors. #1 [suggestion] assign a mentor who is not supervisor to help with PACT… maybe MA students? Who have gone through this process before.
My advisors explained PACT during the courses. And had components to help prepare us. But since they were grading it we received very little if no feedback on whether we were doing it right or on the right track.
These suggestions call for program administrators and teacher educators to think creatively about the high-stakes assessment and scaffold the PACT process into pre-service coursework in order to ensure ongoing formative feedback early in the credential year, which is an essential component to guarantee much needed supports in development of the PACT portfolio.

Discussion
The impacts of the PACT assessment described above provided the basis for the major themes identified below. These themes appeared frequently in the pre-service teachers' responses throughout the survey. Another finding, lack of alignment between pre-service teachers' self confidence pertaining to the areas assessed by PACT and the actual scores received, added an additional layer to the year-one study and provide valuable information to the teacher education programs.

Disconnect Between Self-confidence Levels and the Actual Scores Received
As indicated in the results, a considerable disconnect exists in pre-service teachers' perceived abilities and the actual scores received. On average, the difference amounted to one point, which is all a candidate needs to pass or not pass a section of the PACT assessment (i.e., a score of "2" is passing and a score of "1" is a failing score).
This difference in the pre-service teachers' own perceived abilities could be attributed to the assessment of other tasks performed as part of the university coursework where the pre-service teachers' scores were inflated as compared to an independent evaluation of a PACT scorer. On the other hand, this gap could also be attributed to the requirements of the rubric and how it is worded, which might have affected the way pre-service teachers structured their write-ups of the assessment, resulting in a disconnect between the two. In either case, the lack in consistency could potentially result in pre-service teachers doing subpar work on their PACT assessments through no fault of their own due to heightened perceptions of their own abilities or by preparing write-ups for the tasks based on unclear wording in the rubric. In addition, this discrepancy could also be a product of a disconnect between what was actually executed (i.e., teaching behaviors in the classroom, evaluation assessments, planning) and the students' ability to clearly articulate those behaviors in their final write-up of the PACT assessment portfolio.
Considering the above reasons, it is important for pre-service teachers to have a realistic understanding of their abilities in order to be successful in completing all the tasks of the PACT assessment. Teacher educators and supervisors must incorporate necessary tools into the coursework not only to educate their students but also to ensure that the assessment tools offer pertinent feedback for the development of teaching skills in order to ensure professional growth. Also, teacher educators should also incorporate assessment and evaluation tools that provide the pre-service teachers more accurate and honest feedback on their work. One of the suggestions to teacher educators would be to provide avenues for the pre-service teachers to improve their expressive writing skills in order to develop skills necessary to clearly articulate best teaching behaviors evidenced in their teaching segments of their PACT portfolio.
In addition, the majority of the first half of pre-service teacher education program focuses on completing tasks and assignments with a considerable amount of formative feedback. Preservice teachers' scores are often based on completion of the task and not necessarily on its mastery. The start of the second half of teacher preparation program shifts from task completion to mastery. As the pre-service teachers are asked to complete the PACT assessment at the beginning of the second half of their preparation, they were still adjusting from formative to summative feedback. This may also be why students have an inflated sense of execution, which is both a timing (of when the pre-service teachers are asked to complete their PACT portfolio) as well as a programmatic issue.
Moreover, a more rigid focus on the academic language component of the PACT assessment is needed and must be examined further. The scaffolding of this subject requires pre-service teachers to invest significant time and care due to the diverse nature of today's public schools and should not be taken lightly. In presenting the skills relevant to the development of addressing the academic language skills in the classroom, pre-service teachers must learn authentic ways of how instruction in their content area could be differentiated to address the needs of all subgroups of students. Such knowledge takes time and is a process rather than a learned set of skills (Sternberg, 2002;Sternberg, Torff & Grigorenko, 1998). As a result, the instructors and supervisors of preservice teachers need to structure components of teacher preparation courses to address academic language needs posed by today's public school classrooms.

Multiple Effects of PACT
A realistic interpretation of the pre-service teachers' own skills and a clear understanding of the rubric could potentially result in lowering the stress factors as these relate to the completion of PACT portfolio--concern most commonly expressed by the participants. The PACT assessment itself is designed to evaluate the knowledge pre-service teachers acquired during their credentialing year. As a result, in theory, the process of completing the tasks of the PACT assessment should be fluid and second nature to the pre-service teachers; however, the results of this study show otherwise. As expressed by the pre-service teachers, the stress factors are many and resulted in subpar teaching practices outside of the PACT assessment at their student-teaching placements, considerable toll on their personal lives, and substandard work submitted for their university coursework.
In view of all of the above, the teacher education programs as well as PACT evaluation centers should reconsider the timing of when the PACT assessment is administered. Since the typical administration of PACT is in the early winter/spring, the pre-service teachers are only halfway through their teacher preparation programs and are still in training. Asking pre-service teachers to complete the PACT assessment tasks at later stages in their teacher preparation programs would not only ensure a better product but would also help alleviate the multiple stress factors expressed and would result in higher quality portfolios. However, we do recognize that such a change may be challenging, as teacher education programs must provide enough time for students to re-take the PACT if they fail. As a result, the timing of acquisition of teacher certification must be addressed at both policy and programmatic levels. In other words, we are suggesting that perhaps one year is not a sufficient enough period of time to learn all skills necessary as well as to assess authentically pre-service teachers' skills evaluated by the PACT assessment.
We further suggest dividing various tasks of the PACT assessment and requiring pre-service teachers to complete them at different times during their teacher preparation program rather than all at one time. For example, the context/background and demographic sections of the PACT portfolio should be developed and completed ahead of other sections. After completing these sections, the pre-service teachers should select topics for the teaching segment while developing, practicing, and building mechanisms for the assessment and academic language development tasks that are relevant to the teaching segment selected but not yet executed by the pre-service teachers. The teaching component of the segment should come last and toward the end of the student teaching year. This change will push back the final submissions of the PACT assessment to the end of the year as compared to the winter/early spring, which would considerably affect the amount of time given to scorers of the assessment. However, the benefits of the scaffolding and partitioning of the tasks required by the PACT assessment would result in a more holistic and authentic PACT portfolio as well as allow more time for pre-service teachers to develop their skills while alleviating the stress factors associated with fast-paced PACT preparation practices.

PACT and Teaching Practices
As evidenced in the results above, the PACT assessment did contribute to the development of pre-service teachers' teaching practices, especially in the area of reflection aided by the videotaping of their teaching. Given the impacts of the reflection practices on one's own teaching and the substantial contribution to the development of teaching skills, we argue that the videotaping component should not be limited only to the PACT assessment. Incorporating an approach similar to the PACT assessment task of videotaping into university coursework would contribute to more authentic and substantive reflections by pre-service teachers on their own teaching practices.
We suggest introducing the videotaping exercises at the beginning of teacher preparation programs and a wide use of this practice at the student-teaching placements to allow an ongoing development of the pre-service teachers' reflection skills, which in turn will contribute to both more authentic PACT portfolios and professional growth. The wide use of this practice in teacher preparation coursework will accomplish a threefold goal of preparing better professionals: (a) ensuring well-informed and better quality PACT assessment portfolios, (b) minimizing the stress factors associated with PACT preparation due to the videotaping section of PACT, and (c) graduating professionals who could authentically engage in formative assessment of their own teaching practices. By accomplishing the threefold goals listed above, the new teaching professionals will have the tools needed to engage in their own professional development, which could potentially aid in higher beginning-teacher retention rates.

Need for Support Mechanisms
In part, given the current landscape, the complex process of educating an educator is closely intertwined with the pre-service teacher assessments. A few key elements stand out as core necessities for producing a competent practitioner. The mere knowledge of principles of effective lesson development obtained from university coursework does not ensure the use of best practices or the presence of support mechanisms needed within the classroom. These important factors directly relate to the second key element of teacher preparation that is evaluated by the PACT assessment, practice.
The current arrangement focuses on providing teachers an integrative system with tiers that progress from classroom aid to acting teacher. By following through these tiers, pre-service teachers are given the chance to develop skills and experience failures that support their learning. These tiers belie an integral philosophy of the current State's system, namely that teacher preparation relies on an apprenticeship model. Such a model requires that apprentices have highly skilled and wellinformed mentor teachers --who, indeed, carry primary responsibility for skill development during the student-teaching component of teacher preparation.
The support mechanisms in place (or lack thereof) could potentially "make or break" the path new-to-the-profession pre-service teachers have chosen for themselves. Although the participants reported favorably regarding the support received from their university supervisors and instructors, pre-service teachers indicated that resident teachers' contribution of offering needed supports is lacking. Provided that pre-service teachers spend a significant amount, if not the majority of their time, at their student-teaching placements, strong local support mechanisms where studentteaching takes place is an integral component of any teacher preparation program.
As documented in the year-one study (Okhremtchouk et al., 2009), "the local factor" is essential as it relates to the development of needed professional skills as well as successful execution of the PACT assessment portfolio. As reported above, veteran teachers, who are typically selected to serve as mentors, have little to no familiarity with the PACT assessment requirements due to the recent implementation of pre-service teacher exit exam assessments. Because of the novelty of these assessments (and that they differ significantly from their own experience), site mentors lack necessary knowledge and experience to establish an environment conducive to pre-service teachers' development of the professional skills required by the PACT assessment.
Perhaps the most pressing recommendation for improvement directly relates to existing student-teaching support mechanisms and site level experiences for pre-service teachers. As has been found by Pecheone and Chung (2006), targeted support does make a considerable difference in fostering positive pre-service teacher experiences. While the number of mentor teachers needed will likely produce logistical problems, providing masters in the field for pre-service teachers to collaborate with is of the utmost importance to developing competent practitioners.
We suggest incorporating an early overview session centered on the set of skills assessed by the PACT assessment for potential mentor teachers to collaboratively discuss pre-service teacher needs and site support mechanisms in order to provide essential information to as well as establish an early vetting process for mentor teachers. In addition, inviting and training mentor teachers to participate in the scoring of PACT would provide an opportunity for veteran teachers to become knowledgeable about the assessment. Such changes would maximize the benefits of the apprenticeship model and significantly improve the level of proficiency as well as the professional knowledge needed to successfully execute tasks required by PACT.

Pre-service Teacher Suggestions
As discussed above, site support and ongoing advising at the site level serve important roles in ongoing and successful development of skills assessed by PACT. However, additional factors of desired support needs should not be overlooked. Pre-service teachers voiced a need for more modeling by their university mentors as well as a desire to examine models of the PACT assessments, as was also found by Selvester et al. (2006). Teacher educators should consider expanding their instruction by engaging in more modeling of successful teaching practices, which in turn will ensure skill development and strong preparation for PACT. Moreover, introducing multiple examples of successful PACT portfolios and allowing pre-service teachers ample time to examine exemplary work would adequately inform pre-service teachers of what is expected of them.
Furthermore, as has been discussed in this section, the early introduction to the skills assessed by PACT serves an important role in pre-service teacher skill development and their familiarity with the assessment itself. The practice of early introduction of PACT would shape preservice teachers' perceptions of what is expected of them, provide ample time to develop needed skills, and ensure that the process of completing PACT is more familiar and less stressful. Lastly, we suggest establishing a system of mentorship in order to create a forum where the beginning teachers and pre-service teachers would be given a chance to interact and share ideas. In addition to the above results, the mentorship component has appeared as one of the pertinent pre-service teacher suggestions noted in Selvester et al. (2006). Creating an avenue for authentic interaction as well as exchange of ideas and skills will prove beneficial for both pre-service teachers and those entering the profession.

Conclusion
In closing, we would like stress that introducing a new accountability measure to any profession is a challenge for that field as a whole and for those who are seeking to enter that profession. Changes in assessment practices require programs to spend significant time deciding how to reorient their work and investing a considerable amount of effort to adjust practices, which require much time and resources. The PACT portfolio assessment was intended to provide a structured way to assess the mastery of teaching skills acquired (i.e., planning, instruction, assessment, and reflection) during the teacher preparation program (Darling-Hammond, 2006). However, as the results of this study show, this assessment carries a noteworthy number of unintended consequences. Perhaps the two major consequences of the PACT assessment that continue to surface in this year-two study are the stress factors pre-service teachers experience and the issue of adequate support mechanisms, which are not mutually exclusive.
The current practice, as documented in the accounts of these pre-service teacher participants, and the actual intent of the PACT assessment seem to contradict each other. If the intent is to get the pre-service teachers through the program and to ensure that they pass all the steps necessary to be recommended for a teaching credential, then the current practices do suffice. On the other hand, if the PACT assessment is intended to prepare competent professionals who can demonstrate their well-developed set of skills and knowledge of teaching and learning, then a number of adjustments are necessary. To those who are in charge of making legislative decisions and programs that use the PACT assessment, piloting it, or contemplating adopting this or other preservice teacher assessment tool, we suggest addressing the issues discussed above in the following order:

Teacher Preparation Programs and Teacher Educators
The first and perhaps simplest adjustment would be to introduce and scaffold the PACT assessment early in the program. Providing pre-service teachers with a better idea (as part of university coursework) of what is required, sooner rather than later, will allow them time to integrate and put to practice the important components the PACT assessment aims to underscore. The second adjustment would be to assess closely and thoroughly pre-service teacher coursework assignments in order to ensure realistic awareness among pre-service teacher of the skills necessary as well as to allow time for improvements. The third suggestion is to expand, establish, and improve support mechanisms in place at the school sites where student-teaching takes place. Finally, the timeline for completing the PACT assessment must be revisited and reassessed as possible within the required course of study and other programmatic constraints.

Policy Level
When addressing the issue of timing, we suggest adjusting deadline requirements for preservice teacher assessment portfolios. To this end, we argue that assessment portfolios should be assembled at the end of a teacher preparation program as compared to the start of a second half. This adjustment will help with stressful situations in which pre-service teachers find themselves when attempting to compile the evidence for their PACT portfolios during the second half of a very busy program when they have developed only limited theoretical knowledge and practical skills. Delaying the exit assessment will result in more skilled authentic portfolios and a more accurate evaluation of teaching skills.
After discussing multiple and, at times, complex components encompassing pre-service teacher skill development and assessment, teacher educators as well as policymakers must be cognizant not to lose sight of those who are directly affected by this process -pre-service teachers. As we ask pre-service teachers to engage in ongoing formative assessment of their own skills and teaching practices we would seem completely inauthentic if we did not undergo a similar focused reflection on the processes involved in preparing them to teach. schools, teacher practices in the context of urban school reform and democratic education, and comparative and international education. She teaches courses on critical pedagogy, multicultural and bilingual education, and educational research. From 2000 until 2008, she was the Project Director for the Equity Network, an urban school reform and teacher preparation partnership that included 12 professional development schools, 5 districts, 2 teachers' associations and Sacramento Area Congregations Together and was supported in part by grants from the U.S. Department of Education and the California Post-Secondary Education Commission. She is a member of the Committee on Accreditation for the CA Commission on Teacher Credentialing and currently serves as co-chair of the Teacher Advisory Panel, convened by the CTC. Dr. Wong has collaborated with Brazilian scholars on school reform in Brazil and has served as a consultant to the Secretariats of Education in the states of Minas Gerais and Paraná. She co-edited (with Ronald Glass) Prioritizing urban children, their teachers, and schools through professional development schools (2009)