This article has been retrieved
times since March 31, 1999
Education Policy Analysis Archives
Volume 7 Number 11
March 31, 1999
ISSN 1068-2341
A peer-reviewed scholarly electronic journal
Editor: Gene V Glass, College of Education
Arizona State UniversityCopyright 1999, the EDUCATION POLICY ANALYSIS ARCHIVES.
Permission is hereby granted to copy any article
if EPAA is credited and copies are not sold.Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education.
First Year Implementation of the
School to Work Opportunities Act Policy:
An Effort at Backward MappingArthur M. Recesso
Valdosta State University
Abstract This study examined the intent of federal policy and the actual implementation within local school districts. Specifically, the focus is on the Federal School to Work Opportunities Act of 1994 and its implementation in 47 school districts in upstate New York as part of a consortium during the 1995-96 school year. The purpose of the study was to determine 1) the extent to which an agreement to participate in a consortium arrangement designed to facilitate the implementation of a specific Federal or state policy resulted in the active implementation efforts by individual consortium members, and 2) how a high school setting where School to Work activities were perceived by local stakeholders as having great specific and important effects differed from a high school setting where School to Work activities were perceived by local stakeholders as having some or no effect. A bottom-up backward mapping policy analysis model was used for the purposes of this study. Local level data was used to create performance, environment, technology implementation, and School to Work implementation profiles of local high schools. Regression and correlation analyses were used to determine the relationship between stakeholder perceptions and local high school characteristics. Results of the study were reported and interpreted with the aim of furthering research and knowledge of policy analysis and the use of local level data to determine the success of policy implementation. This study found that variation between federal intent and local adaptation is explained by characteristics of the high school and perceptions of stakeholders. School to Work policy implementation, perceived by the high school administrator as a stakeholder, varied significantly by high school student performance, environment in which the high school operates, and level of technology implementation in the high school. Results indicated that the backward mapping policy analysis model is effective in determining the actual levels of policy implementation. Backward mapping results in a definitive explanation of the role of the local actor and the use of discretion in decision making. The final analysis as a result of backward mapping goes beyond the measurement of policy objectives being met and explains the meaning of local level participation.
Introduction
This study compared the original intent of a specific educational reform policy to its local adaptation. Specifically, it focused on the implementation of the Federal School to Work Opportunities Act of 1994 in the forty-four public high schools participating in the New York State Mohawk Valley Workforce Preparation Consortium in the 1995-96 school year.
Policy analysis assumes implementation can be assessed by describing the reported activities of the "local integration" and comparing such findings to the original intent of the policy (Elmore 1982, Ingram & Schneider 1990). The difference between original intent and reported local integration or adaptation were interpreted as "discrepancies" in fulfilling the purpose of the particular policy. This study argued a more inclusive approach. This study was designed to examine these discrepancies in local level implementation. To this end data were collected on both the perceptions of the local stakeholders implementing the policy and the characteristics of the schools in which the implementation took place. To meet the stated objectives and to delineate the patterns of local characteristics and stakeholder perceptions of School to Work implementation, a rank ordering of the forty-four school settings that participated in the Mohawk Valley Workforce Preparation Consortium during the 1995-96 school year was undertaken. The rank ordering of perceptions of School to Work implementation ranges from sites where perceptions of having "little or no effect" on the on-going activities in the high school to perceptions of "great influence". The rank ordering on the basis of local characteristics (performance, environment, and technology implementation), showed real deficiencies among sites. The result of the rank ordering of districts on these characteristics served as the basis for the analysis of the relationship between local characteristics and perceptions of implementation.
A second phase of the study described the specific organization, curriculum, instruction, and the community actions related to local implementation in four of the high schools under study. Such description focused on (a) the use of local discretion, (b) the intent of the original policy, and (c) the significance of organizational characteristics. These four sites are described in some detail. The schools were selected on the basis of their ability to elaborate cases of little or no effect, great effect, and a mixed or midrange effect of School to Work as discovered in the first phase.
Background
A Nation at Risk (1983) broadcast a message that schools were not preparing students adequately for the workplace. It sharpened the on-going policy debate about the most effective means for preparing students for the workplace and/or college. The United States was portrayed as uncompetitive in the global marketplace. Students were trained for 1940s and 50s workplaces, not for those of the new millenium. Other reports noted employer complaints that too many recent graduates lacked even the most basic reading, writing, and communication skills (U.S. Congress 1997, America's Choice: High Skills/Low Wages 1983, Nation at Risk 1983, SCANS Report 1983). These reports asked for a policy to address the needs of employers and of students as future employees.The Role of the Federal Government
The Congressional intent of the School to Work Opportunities Act of 1994 was expressed in fourteen parts to provide an overarching guide for States plans for implementing a sustainable School to Work system. It required the involvement of all key stakeholders, especially employers, in planning the system, implementing it, and supporting its continuation. The School to Work Opportunities Act, as written, was intended to be administered in a way that provided for State and local discretion in implementing the system, as long as the efforts addressed local capacity building, minimized program overlap, used scarce resources effectively, established clear goals, and provided the flexibility to meet these goals (School to Work Opportunities Act of 1994, 108 Stat 568, p.6-7).
In the School to Work Opportunities Act, each State was required to plan three general areas. First, Federal government selection of states to receive funding under the Act was based, in part, on how well this system implementation plan built upon past support for School to Work-type initiatives. (STWOA 1994) Each state's proposal was required to explain how components would be linked together within a five year time-frame to establish a statewide School to Work system. Each state's system implementation plan also was required to show timelines outlining the implementation of all components of the system and the relationship of each to other reform efforts. Second the state must explain how all policy components, including those not supported in the past, were to be supported throughout the state. Thirdly, the state agencies named in the Act by the Federal policy makers to be critical partners in School to Work (i.e., Department of Labor and State Education Agency) were required to be named as having an active role within the State's implementation plan. States were to be funded that demonstrated past practice, a commitment to School to Work-type activities, and planned for the future successful statewide implementation of a School to Work system that met Federal guidelines.
In the first year of approving statewide implementation plans, eight states received funds to implement a statewide School to Work system. State governments decided how funds would be distributed, according to flexible Federal guidelines. The state was responsible for evaluating local progress towards full implementation of the system, ensuring that all students were being served and all stakeholders were involved in the process.New York State
New York was one of the initial eight States to receive implementation funding. It also was the recipient of the largest grant, over eight million dollars in year one, and more than $60 million over five years. The funds were considered "seed money' and "venture capital" for the development of a School to Work System. (STWOA 1994) New York State submitted a proposal for the implementation of a School to Work System over a ten year period, with five years of funding to be provided in part by the Federal Government (New York State Implementation Grant p.1). Commitment to school reform was established through documentation of workforce development prior to the School to Work Opportunities Act of 1994. Three-and-three-quarter ($3.75) million dollars had been dedicated to Workforce Preparation sites (involving 129 school districts) in the previous year (1993-94). The Governor's School and Business Alliance was created in 1987; twelve regional sites were provided funding by the Governor's Executive Budget. Both of these initiatives had been working towards building the system called for in the STWOA.
New York State's approved proposal provided the federally mandated framework. Skill standards, skill assessments, and skill certificates were to be integrated into performance-based assessment. Teachers, counselors, and administrators were to be prepared for the implementation of new strategies for curriculum, instruction, and assessment. And parents, employers and other community members were required to be involved in the decision making process for determining appropriate preparation for the workplace or college (New York State Implementation Grant p.6-30). Efforts supported by allocated funds were required to provide opportunities for all students.Local Partnership
In 1995 the State of New York, through the NYSED, selected fifty-two partnerships throughout the State for funding as planning or implementing sites for School to Work initiatives. These partnerships consisted of school districts, employers, higher education institutions, community members and organizations, and localized State agencies (e.g., County Departments of Social Services). Each partnership had to identify how many and what types of activities were taking place already in the schools, workplaces, and school/workplace projects. They were asked to explain how each activity was to be replicated in non- project sites in the local area and how new activities were going to be developed and supported.
Programs were to be designed and supported for all students, K-12+, including at-risk, gifted, and out of school youth. The program and activity design was to be based on the needs of the local population of students. The needs of the local target population, in turn, was to be determined by student performance on statewide testing, dropout rates, attendance rates, discipline problems, college bound rates, and also local unemployment rates. One result was staff development programs created to address ways to improve student performance. For example, the Teacher Job Shadowing Program connected staff with local employers and employees. Teachers visited business sites, spoke with employers and employees, and toured facilities. Subsequently, each teacher designed a classroom lesson incorporating work-based learning. Some participants were teachers of Regents classes. Their students were to apply knowledge gained through the Regents syllabus to an actual workplace problem. The premise was that students would more thoroughly understand of how the classroom knowledge was applicable to actual problems, resulting in better performance on Regents exams. This was an important point because Regents exams, and the rigors of the courses, were often cited as barriers to change.
Only one region, the Mohawk Valley Workforce Preparation Consortium, submitted a proposal on behalf of the entire region that had been originally designated for funding distribution by the statewide advisory council. Most of the other partnerships in the State were connected and funded through a local BOCES. Other initiatives were funded as either one of the big five school districts (Buffalo, New York City, Syracuse, Rochester, and Yonkers), as a group of school districts participating as a non-BOCES consortium, or as specialized magnet schools like those found in New York City.Conceptual Framework
Many policy studies focus on the role of information and analysis in decision making, others on the role of political bargaining and power in policy decisions (Elmore, 1979, Lindblom & Woodhouse 1993, Wildavsky & Pressman 1971). The actions of local actors and their use of discretion in decision making is often a critical component of these studies (McLaughlin 1987). At any given time, individuals at the local level deal with a myriad of issues and policies related to school reform. Mandates and limited resources force local actors to decide which policies will be implemented. The role of the local actor in implementation may be overlooked in forward mapping approaches to policy analysis. Local actor perceptions and their relationship to decision making and implementation deserve attention that typically does not take place in the traditional top-down approach. If it is agreed that the policy making process yields unintended results (including the failure to implement), then it is desirable to utilize other methods that may aid in determining predictors of success. Important in this approach is the ability to discover forces influencing policy acceptance or integration (Lindblom & Woodhouse, 1993, p.1-32).
Literature Review and Conceptual Framework
Policy Analysis: Forward Mapping v. Backward Mapping
Policy development, implementation, and analysis are extremely complex processes. "As a 'science' policy analysis is concerned with predicting cause and effect relationships implicit in policies" (Elmore 1987 p.174). A traditional approach to policy analysis is the top-down process of forward mapping. Through the traditional lens, the view is of a relationship between policy creator and policy implementation and outcomes. Richard Elmore suggests an approach to policy analysis that goes beyond the traditional top-down analyses used in many studies. Elmore's interest is not limited to determining success or failure based on the measurement of objectives, but seeks to understand why and under what conditions policies are adopted locally. Elmore introduces another vantage-point from which policy is viewed from the locality and by grassroots stakeholders. Both views tell parts of a complex story. Thus a case can be made for studying policy with both the traditional forward lens (from design to local implementation) and a backward lens (from implementation to original intent).Forward Mapping
The most widely recognized type of analysis is the "top down" approach. This begins with the objectives and goals of the policy and works its way towards the outcomes to be measured in determining success. This process often begins with legislation that outlines the congressional intent of the policy and determines success or failure by measuring reported outcomes in terms of the original objectives.
Supporters of the top down approach assume the clear delineation of goals from policy makers will lead to more effective support, more effective implementation, and greater success of policies in addressing problems. But many authors have reservations about the policy maker's power to affect local implementation process (Ingram & Schneider 1990, McLaughlin 1987). The top down approach assumes the policy makers have control, that they can affect the implementation process, and that their decisions have some bearing on local actors. Authors such as McLaughlin, Ingram, Schneider, and Sabatier fault the top-down approach for not considering the discretion of the local actor as an important component of policy analysis. Ingram and Schneider (1990) cite the work of Elmore for its emphasis on the "organizational structure and personal factors" (e.g., perceptions and discretion) in determining the actual policy impacts and in explaining their success or failure. McLaughlin and others indicate that a need to understand that many factors are involved in the success or failure of a policy, and many of them involve the actions of the "street level bureaucrat" (McLaughlin 1987, Wildavsky & Pressman 1971).
Pressman and Wildavsky's study of the Oakland Project in the 1970's is heralded as a pioneering study of implementation. Their assessment of the Federal implementation of employment programs for severely disadvantaged adults raised questions about why the programs failed. The Federal Government had established very clear goals for the public works projects, the Oakland government officials supported the initiatives, private companies wanted the projects, and the people of Oakland wanted the projects to be a success. Yet the policy failed for reasons that were beyond the control of the policymaker (Wildavsky p.87-124). In their final analysis, Pressman and Wildavsky determined that the relationship between the number of transactions required to implement the policy and the perception that the implementation would be successful were inversely related (Wildavsky 1971). This eventually lead to the understanding that policy decisions are not self- executing (McLaughlin 1987). The policy is created and exists for the local actor to follow as a guide for implementation. The local actor has perceptions of the policy. Those perceptions have a relationship to decision making. Decision-making results in implementing or non- implementing. Therefore, the notion that clearer goals or objectives should have any impact is thwarted and the stance in support of hierarchical procedures is challenged. Pressman and Wildavsky used a forward mapping approach and did not clearly determine the cause of the policy's failure. Their approach does not consider the local actor, the characteristics of the organizations, or the local environment beyond the obvious unemployment rate. Their final analysis does mention the need for a process that takes into account each one of these items. Backward mapping does consider internal and external environment of the local environment and local discretion. And backward mapping completes the analysis by looking back at what was originally intended.
There is no single rule for creating and implementing new policies that will be automatically effective at the local level. Pressman and Wildavsky's study demonstrated that even the best of intentions and the clearest of goals can prove to be ineffective in achieving desired or expected results (Ingram and Schneider,1990, p.69). Still, many analysts believe it is the policy maker and the clarity of the goals and objectives of policy that make the difference at the point of implementation. Ingram and Schneider (1990, p.69) argue that if policy makers framed better statutes, implementation would improve. Expanding on the work of others such as Sabatier, Pressman, and Wildavsky, this theory follows the belief that implementation itself is hierarchical in nature. The focus is on a sequence of events rather than a determination of actual influence or impact (Elmore, Studying Implementation, p.24).
Different processes are required if existing policy analysis approaches are not successful in determining the actual effects of policies (Lindblom & Woodhouse 1993). Often these different, even non-conventional, processes must go beyond the lens of the policymaker. Even though a state or federal government official may play a pivotal role, local influences have an effect on the implementation of the policy. Knowledge of this aspect of implementation may contribute greatly to improving future policy development. Providing data that policy implementation is not necessarily made up of leaders (policy makers) and followers (policy implementors) would be an important step towards establishing policy implementation as an interactive process (Lindblom & Woodhouse p. 4-6). Just as important as following the conventional government process is the investigation into the influences that distort these processes (Lindblom & Woodhouse p.11-12).Backward Mapping
Backward mapping is a form of policy analysis that acknowledges that there may be variables that exist outside the treatment program, and that some of these variables may have something to do with the organization's characteristics (Elmore 1979, p.601-616). Considering the organizational characteristics of an organization (the school district in this study) could help to explain why policies fail or succeed. While policy makers define the arena in which the implementation process will take place, and determine the actors and the roles each may play, determine the level of resources to be allocated, the impact of environmental influences on implementation cannot always be foreseen (Pressman and Wildavsky p.174, 126). These include organizational, political, and technical conditions (Elmore, Backward Mapping, p. 603).
Backward mapping challenges some of the most basic assumptions of the top down or traditional approach. It does not assume that all organizations are the same. Each has varying environments (e.g., local unemployment, student performance) that affect its decision making related to addressing local problems. The approach does not assume that all organizations are even interested in implementation. The backward lens places value on the role of the local actor and on the local organization that is focusing on resolution of the problem. The process determines if the school district does not identify itself as having the problem, or has the problem but is not concerned about it. In either event, backward mapping sets the stage for further analysis to determine why either case occurred.
Elmore provides the analysis of the Youth Employment Policy as an example of the shortfalls of using the top down approach (Elmore, 1988, p.29-33). The analysis process begins with the goals and objectives of the policy and summarizes the level of success of the policy by measuring perceived attainment of goals. Similar to the Oakland Project, the results of this analysis may raise concerns that there may be variables outside the process that have a direct impact on the success of the policy (Pressman & Wildavsky, p. 191). However, these other variables go undetected without the use of another approach. McLaughlin adds support for the need of added "bottom up" approach and calls this the micro-level of policy analysis (McLaughlin, Learning from Experience, p.177-178). That is, the focus is on the local actor and the utilization of discretion, acknowledging that the policy is not the only determinant affecting the actions of people at the focal point of the problem (Elmore, Studying Implementation, p.21).
The logic of the analysis used by Wildavsky and Pressman in Oakland and Elmore's assessment of Youth Employment policy indicate the need for further study. It is clear that policy analysis involves not only the measurement of outcomes based on policy goals, but also an understanding of the environment in which the problem resides and within which the implementation takes place. At the local level where decisions are made, the discretion of local actors come into play. It is also where organizational characteristics may have some role in determining the success or failure of the policy. The variation of organizational characteristics, local market conditions, local preferences and prior commitments, and other perceived impacts on the organization will help to determine the relationship between the policy itself, the nature of implementation in the school district, and the actual success of failure of the reform effort.
Design of the Study
The purpose of this study was to compare the original intent of a specific educational reform policy to its local adaptation. The focus was on the implementation of the Federal School to Work Opportunities Act of 1994 in the 44 public high schools participating in the Mohawk Valley Workforce Preparation Consortium during the 1995-96 school year.Research Question One
To what extent did agreement to participate in a consortium arrangement designed to facilitate the implementation of state or federal policy result in the active implementation efforts by individual consortium members?
Research question one, and its subparts, were developed based on the work of several authors such as Pressman and Wildavsky (1971), who raised questions about the meaning of participation and the relationship to implementation. The question also addressed Elmore's (1983,1988) backward mapping model that explains the relationship between the organization's characteristics and implementation. The political bargaining, organizational process, and rational decision making models also partly contributed to the development of research question one. The work of Simon (1993), Allison (1991), Gieseck (1995), and others delineate the need to understand why and under what conditions decisions had been made while implementing the policy.Research Question Two
How did a high school setting where School to Work activities were perceived by local stakeholders as having great and specific important effects differ from a high school setting where School to Work activities were perceived by local stakeholders as having some or no effect?
Similar to research question one, research question two and its subparts were based on the backward mapping model of Richard Elmore (1988). However, unlike research question one, this question more directly addressed the relationship between local characteristics and implementation of the policy. Research question two was based on the work of Elmore (1988), McLaughlin (1987), Ingram and Schneider (1990), and Lindblom and Woodhouse (1993), all of whom addressed or identified the need to research the differences between local organizations and the relationship of those differences to the level of implementation. In addition, it was within the work of Pressman and Wildavsky (1971) that the question of the relationship between local level characteristics and implementation arose. The research of Rogers (1963, 1988, and 1995) and Tornatzky and Klein (1982) addressed the issue of innovations and their implementation at the local level. Their attention to the organization's involvement in multiple innovations and the relationship between organizational characteristics and implementation of innovations was also a basis for research question two.Methodology
Forty-seven school districts entered into a partnership to implement a Federal School to Work policy. The question of the match between the federal policy intent and the actual "grassroots" adaptation rested with the level of specific and important changes that can be attributed to four characteristics of the school. School to Work implementation, student performance, environment, and technology implementation as four characteristics of a high school were addressed using a backward mapping policy analysis model.
The characteristics of the schools, to be discussed for the purposes of this study as "profiles," were developed for this study to delineate information for district comparisons, predict changes, profile system strengths and weaknesses, and identify policy improvement strategies (Oakes, 1986). To provide information about the high schools, the profile had to have a reference point, a measure by which to judge the indices used to compare the high schools. The reference points by which to measure the indices were the substantive and technical criteria (Cooley, 1992). Each profile did provide at least one of the following recommended substantive criteria: information that described student performance in terms of achieving outcomes, information that described the central features of the high school, or information that was policy relevant (Oakes, 1986). In addition, each profile was developed having the following technical characteristics: it measured features that can be found in all schools being studied, the data collected can be traced over time, the profiles can be understood by practitioners and policymakers, and the profile indices were generally accepted in educational policy analysis (Cooley, 1992; George, 1993; Oakes, 1986; Windham, 1990). The purpose of developing these profiles was to delineate the local conditions of the high schools implementing federal policy, provide a common means for comparing those schools, and provide a final analysis that improved policy decision making.
The meaning of participant and non-participant as defined by implementing and non-implementing schools was explained using the School to Work implementation profile. The profile described to what extent federal policies were adapted. The School to Work implementation profile also described to what extent the high school implemented school- based learning, work-based learning, and connecting activities components of the School to Work system. The components were required by the federal STWOA for schools using funds to implementing School to Work.
The following data were collected as indices of the level of School to Work implementation achieved in each district: if the school described itself as being involved in School to Work during the 1995-96 school year; the priority of staff development overall and specifically connected to school based learning, work based learning, and connecting activities; the number of partnerships established locally; and the level of commitment to required activities as described in the legislation and regional proposal (e.g., job shadowing, school based learning, tours) as demonstrated by implementation by grade levels. These data were used to compare schools within the consortium. Correlation methods were used to determine the relationship between implementation level, the performance of the school's students, the environment in which the school operates, and the level of technology that has been implemented in the school.
The performance profile of the students in the high school was used to explain the school's educational quality and its effectiveness in meeting desired educational outcomes (Cooley, 1992; George, 1993; Windham, 1990). The New York State Regents exams and diploma are benchmarks by which students and schools were measured to determine academic success. A broad audience of educators and policymakers understand and generally accept the Regents system to be a consistent measure of student achievement (Wiles, 1996). In developing the performance profile, the following data were used: attendance rate, dropout rate, suspension rate, percentage of students graduating with a Regents Diploma; percentage of students enrolled passing the Regents exams in English, Math III, Chemistry, US History and Government; and the percentage of students passing the Occupational Education Proficiency exam. The performance profile also was used to determine a relationship between past performance and present participation rates. The performance profile was designed to measure the school's ability to prepare students to meet expected academic outcomes. This performance measure was compared to the school's ability to meet policy implementation outcomes as described by the School to Work implementation profile. In this analysis, the school's effectiveness in meeting expected outcomes and the level of School to Work policy implemented by that school were tied to the economic viability of the student. Educational outcomes (employment, achievement in subsequent education and training, and admission to further training upon graduation) were all indicators of school effectiveness in meeting educational outcomes (Windham, 1990). These outcomes matched goals of the School to Work policy implemented within these schools (STWOA, 1994). The School to Work policy was developed to better prepared students for the workplace so they could enter the workforce more easily and prepare for better jobs (Jennings, 1995). Jennings found the better the student preparation programs and the better students perform academically, the more likely they are to be employed after high school. This established a link between School to Work and student performance and substantiated the use of performance profile. The performance profile also explained the similarities and/or differences between schools in terms of their effectiveness, and it was used to explain the relationship between student performance and the School to Work implementation profile.
The environment profile was a composite of indices of the external environment and external forces acting upon the high school. These environmental indicators describe the demographic, social, and economic influences of the local communities in which the high schools are located (Bryson, 1988; Cook, 1990). Influences such as types of employers and the number of businesses located in the community were important because School to Work policy makes specific mention of the relationship between successful School to Work implementation and these influences (Grumman, 1994; STWOA, 1994). For example, the school was required to be supporting the work-based learning component. Therefore, employers were required to be involved as partners in School to Work activities. A locality with numerous employers was potentially able to support numerous work-based learning activities.
The availability of resources in the community also described the context in which the schools operate and were represented in the Environment Profile (George, 1993). Levels of per capita income and number of people employed directly influenced school budgets and the funding available to support innovative programs (George, 1993). While School to Work did provide implementation resources to the schools, schools did incur expenses during implementation. Therefore, schools with more resources had the potential to support more School to Work activities.
The environment profile was composed of the following items: percentage of free and reduced lunch (a local wealth indicator), average household income, percentage of people with bachelors degrees, and percentage of youth at risk. These indicators of school and community were then linked to desired outcomes. Each of the profile indicators could be measured for each school district in the School to Work consortium and throughout the state of New York (Oakes, 1986). They capture enduring features that are recognizable to both practitioners and policymakers and generally accepted as being valid and reliable items (State Education Department, 1993; Wiles, 1996).
School to Work took place within the broader context of school reform. A high school that was involved in School to Work was more likely to also be involved in another systemic reform initiative (Grumman, 1994). The School to Work policy was being implemented at the same time as at least one other innovative reform effort in all of the Mohawk Valley Consortium schools. Computer-related technology was also infused in the Mohawk Valley Consortium schools, and its study was completed for most of the same schools during the 1996-97 school year (Mann, 1997). The Mohawk Valley region technology implementation study considered the policy implications for a large-scale commitment to and investment in computer technology. Mann collected detailed information from superintendents, high school administrators, teachers, and students. Data was compiled and manipulated in an effort to determine the impact of technology on student performance (Mann, 1997, p.2).
Technology was used in this study as a means of profiling and comparing schools for several reasons. The data set collected by Mann covered a majority of the same school districts as this study. Similarly, Mann's survey results also relate to level of participation, level of staff commitment, participation in staff development, and the intent of technology integration. And Mann's study sought to explain, in part, how stakeholders perceived the effect, importance, and level of technology implementation. Like this study on School to Work, the Mann study took the approach of addressing implementation at the grassroots level and compared it to the intent of policies in place. Therefore, the fourth and final characteristic was the technology implementation profile. Technology implementation in this study was defined as the level of computer-related technology that had been implemented for instructional use (Mann, 1997). As innovations, School to Work and technology have some distinctive similar qualities that are called the critical mass aspect (Rogers, 1988). Neither innovation reached its potential for utility unless other people or organizations implemented the idea. Technology can operate stand-alone, but its' uses increased as it was connected to other people, the Internet, and networks of software and information. Similarly, School to Work could have been implemented with just one employer or in just one school. But it reached more students and a wider variety of students' needs with multiple participants and supporters. A collaborative effort was required to establish a system that serves all students.
School to Work and technology are similar in the critical mass aspect in many other ways. In order for other organizations and people, other than those directly involved in initial implementation, need to be networked into the continuation of implementation and programs being supported in order to replicate new ideas and see that the innovation works (Rogers, 1988). In School to Work, the more employers and teachers that were involved, the more students that were being served by its initiatives. Similarly, the more technology that was made available to teachers and students, the more access students have leading to a higher level of integration into the learning episode (Mann, 1997). Therefore, the technology profile was an indicator of the extent to which the school had implemented another innovation. Determining the relationship between the technology implementation and School to Work implementation profiles explained the variance attributed to the level of technology implementation. The technology profile was also combined with the environment and performance profile to explain the level of variance in School to Work implementation.
The specific items used to develop the technology implementation profile were as follows: perception of whether the technology importance was linked to improved curriculum or to changes in teaching; and the number of computers installed, networked, accessible to teachers, accessible to students, and the number of Internet accounts in the classroom. These were exactly the same items selected, in part, from the Mann study (1997) and replicated in eleven other high schools that had not participated in that study. (Note 1)
All four profiles of the high school were completed within the framework of a backward mapping policy analysis that took into consideration the conditions of the high school as it implemented the policy. The analysis of the School to Work policy implementation compared the Act's original intent with local implementation that was supported by the organization. The variance between intent and local adaptation was to be explained by the characteristics of the local high school.
The "mapping" process began at the local level and traced the actions back to the beginning. A backward mapping lens, defined as stakeholder perceptions of specific and important effects and local organizations' characteristics within a community environment, was offered as an alternative to traditional top-down methods (Elmore, 1979; McLaughlin, 1987). Stakeholders were considered as any group or individual who was affected by or who could affect the future of the organization (Bryson, 1990). For the purposes of this study, the focus was on local stakeholders with a vested interest in activities that impacted the school district and the student population. These included superintendents, school district administrators, teachers, employers, students and community members. School district administrators provided leadership and decision making for the schools. It was their duty to see that programs were preparing all students leaving school for college or the workplace. The teachers prepared students along a continuum of learning that culminated in graduation from high school and entrance into higher education of the workforce. Whichever path was taken, the students must have had attained the skills and knowledge necessary to perform in their chosen environment. Community members that provided representation on local partnership committees (e.g., private employers) had an interest in students becoming contributing members of the community. The student's contribution could be as a future employee with a good work ethic or as a good citizen that values a role in community efforts. While the forward map would stop at the program administrator's response to the level of integration, the backward map looked for evidence of change in the organization, such as methods used in the classroom, and the interpretation of integration from the viewpoint of participants and stakeholders.
Measuring the level of specific and important changes is an element of the backward mapping model. Grassroots level investigative procedures were used to determine perceptions of School to Work policy implementation resulting in changes in the high schools of Mohawk Valley Workforce Preparation Consortium during the 1995-96 school year. Interpretation of stakeholder perceptions were interpreted to determine if the federal policy intent was similar or significantly different from local implementation. These perceptions were examined in the context of the on-going high school operations, organizational characteristics of the school district, and the local community.
The development of the performance, environment, technology implementation, and School to Work implementation profiles provided a means for describing groupings of data collected in support of the backward mapping model. It enabled the backward map to factor in the impact of external forces such as the local community, other government agencies, and private employers. Local adaptation of program intent, and the conditions within which it took place, was discussed in terms of how the School to Work System was implemented within the local school district. Representative stakeholders of the groups involved in the implementation process described to what extent they perceive School to Work to have been integrated into the schools.
The level of implementation sought by the federal government was delineated to guide the states and schools in their efforts to implement a School to Work. Full implementation was defined in the School to Work Opportunities Act legislation as having these components: (STWOA, 1994)
- All students are participating in School to Work activities,
- All levels of education are supporting an integrated School to Work System,
- All staff have seamlessly integrated School to Work into their classroom,
- All stakeholder groups have been involved in the process, and
- All initiatives and partnerships are tied together in one cohesive system across the State and country.
Schools were differentiated based on the varying degrees to which they were able to achieve full implementation. These levels of implementation were to be determined by: the number of School to Work initiatives that were implemented grades nine through twelve; the number of staff development related activities that were in place; the number and types of partnerships that have been established with local employers; and the number of school-based, work- based, and connecting activities that were supported. The Performance Profile as an independent variable construct was developed to represent the educational quality of the school and its effectiveness in meeting state educational benchmarks. Schools accumulated points in the form of data items on student performance. The total number of points for each high school established a ranking by which to compare the schools. The comparison discussed the success of the school in meeting NYSED-established benchmarks as a member of the consortium and an implementor of School to Work. Higher profile rankings indicated schools having greater success meeting the statewide student performance standards. The success of the school in meeting these standards and, therefore, attaining a higher ranked profile spoke to the educational effectiveness of the school. The Performance Profile was utilized to determine this independent variable's ability to explain the variance in the dependent variable, School to Work implementation. The variance was measured directly and in combination with the two other independent variables.
The Environment Profile was a second independent variable. This profile was a representation of the external factors effecting the school. Highly ranked schools were operating in a healthier (e.g., more opportunities for kids with more employers) environment than other schools participating in the consortium. The availability of employers and employees, community financial resources, and availability of people to serve as appropriate role models could impact the school's ability to support School to Work activities.
Technology Implementation, a third independent variable, was developed based on the responses from high school principals on the level of computer-related technology and training present in each building. The highest ranked district had the most involvement in implementing technology, demonstrating support and dedication of resources to another reform effort.Sample Selection
The Mohawk Valley Workforce Preparation Consortium consisted of a population of 47 school districts. Forty- four school districts had full secondary programs, grades 9 through 12 inclusive. This was a study of the population of school districts, as partners in the Mohawk Valley Workforce Preparation Consortium, which support high school-level implementation. Therefore, three districts without a grade 9 through 12 program configuration were not surveyed. One district provided schooling up to the ninth grade and transported students to a nearby district for tenth through twelfth grade. The two other districts did not provide a high school program at all. They provided K-8 and K-6 programming and transported students to other districts for further schooling.
One high school administrator from 44 high schools in the Mohawk Valley Workforce Preparation Consortium was surveyed. High school administrators who were not employed by the school district for a full school year during the first year of implementation (1995-96) were not sent surveys. Out of the 44 possible districts, 33 superintendents and 39 high school administrators were surveyed.Response Rate
High school administrators returned 24 of the 39 surveys, a 62% rate of return.Summary of Data Collection
To complete this study using backward mapping, data that describe the characteristics of the local organization, the perceptions of local stakeholders, and the adaptation of intent of the policy at the local level was collected. Three types of data collection sources were used. Original surveys were developed and sent to stakeholder groups. Follow-up interviews with stakeholders provided more in- depth information and explanation for the survey data. Existing data sets were readily available to build the profiles that characterize the high schools' student performance, environment, technology implementation, and School to Work implementation. All three data sources proved to be useful and effective in supporting the analyses to be used in this study using the backward mapping process.Data Analysis
The data collected were analyzed in stages. In the first stage, the data were compiled into profiles, labeled as independent and dependent variables, and analyzed using regression and correlation analysis. In a second stage, the aggregate data from specific questions posed to a surveyed stakeholder group and specific items from the existing data sets were used for analysis. For example, specific questions in the survey were asked to determine if high school administrators responsible for policy implementation described School to Work as being specific and important. Correlation analysis was used to determine the extent to which the administrators' perceptions of importance were associated with the level of implementation. In a second step, regression analysis was applied to determine the extent to which the level of implementation in high schools varied by the perceptions of the high school administrator with significance. The third stage of analysis involved selecting high schools for follow-up interviews by placing the high schools on a quadrant grid. Based on the school's placement on the grid the school was selected for a follow- up interview. Interviews provided more detailed explanations about perceptions, participation, and implementation.Compiling the Profiles
The profiles (Performance Profile, Environment Profile, Technology Implementation Profile, and the School to Work Implementation Profile) were developed to serve as a means for explaining how organizational characteristics in terms of student performance, external environment, and technology implementation predicted or were associated with the level of School to Work implementation. The profiles developed in this study were essentially constructs. They served the purpose of grouping many different but related variables into one term for ease of analysis and subsequent discussion. The profiles were used to explain to what extent student performance, the environment in which the school operated, and whether the level of technology implemented had an effect on the adaptation of intent at the local level.
The Performance Profile was developed for each high school participating in the Mohawk Valley Workforce Preparation Consortium to determine if school organizational effectiveness characteristics varied systematically with stakeholder perceptions of specific and important effects of School to Work policy. The Performance Profile was a construct consisting of the following variables: percentage of graduates receiving Regents Diplomas, dropout rate, attendance rate, suspension rate, percentage of enrolled passing the Math III Regents Exam, percentage of enrolled passing the English Regents Exam, percentage of enrolled passing the Chemistry Regents Exam, percentage of enrolled passing the U.S. History and Government Regents Exam, and the percentage of number tested that passed the Occupational Education Proficiency- Introduction to Occupations State Exam.
All of the performance data was collected from the New York State 1995-96 Report Card on the Schools released in February of 1997 (Report Card on the School, 1997). Developing the Performance Profile, the reported figures, as raw data, for each item (e.g., percentage of graduate receiving Regents Diplomas) were summed into a total of performance profile points for each high school (See Table 1). The dropout rate and suspension rates were added to the schools total by subtracting the reported rate (e.g., 5% drop-out rate) from 100% and resulting in a positive figure (e.g., 95% did not drop-out) to be added into the schools total (See Table 1). This process thus penalized schools for higher dropout or suspension rates in either category. None of the variables were weighted because there was no evidence of one item having more importance than another (e.g., Math III Regents Exam compared to the Chemistry Regents Exam). The state reporting does not differentiate the items as being more or less important. The total for each high school was rank ordered. The high school with the highest number of total points was given the top ranking. This resulted in a rank order listing of the highest- performing to lowest-performing high schools in the Mohawk Valley Consortium.
Table 1. Compiling the Performance Profile ITEM CALCULATION RESULT % Grads receiving Regents Diplomas Actual % (0-100%) Dropout Rate 100% - rate (0-100%) Attendance Rate 100% - rate (0-100%) Suspension Rate 100% - rate (0-100%) % of Enrolled Passing: Math III Regents Exam Actual % (0-100%) English Regents Exam Actual % (0-100%) Chemistry Regents Exam Actual % (0-100%) US History and Govern Reg. Exam Actual % (0-100%) % of Number Tested & Passed Occ Ed Intro to Occ State Exam Actual % (0-100%) TOTAL Sum of figures 0 to 900
To determine if community characteristics varied systematically with stakeholder perceptions of specific and important effects of School to Work policy, the Environment Profile for each high school participating in the Mohawk Valley Consortium was developed. The Environment Profile was a construct of the following variables: percentage eligible to receive free and reduced lunch, percentage of households on public assistance, percentage of residents with bachelors degrees, and the percentage of youth at risk.
All of the environmental data was collected from the United States Census Bureau: 1990 Census Data. The sole exception was the free and reduced lunch eligibility data, which was obtained from the 1995-96 Report Card on the Schools. The data were summed for a total of environment profile points for each high school (See Table 2). The totals for each school were then ranked. A lower total number of points indicated a healthier and more resourceful environment in which to operate. Therefore, schools with lower totals were ranked higher: the highest ranking was number one, the lowest, forty-four. The average household income and the percentage of residents with bachelor's degrees had to be resorted to fit the pattern of the other variables. The percentage of residents with bachelors degrees was subtracted from the highest possible percentage (100%). The result was summed into the total environment profile points (See Table 2). The data items were not weighted because none of the data items carried more importance in characterizing the environment of the high school. Data was available for all of the participating high schools.
Table 2 Compilation of the Environment Profile ITEM CALCULATION RESULT % Eligible for Free and Reduced Lunch Actual % (0-100%) % of Households on Public Assistance Actual % (0-100%) % of Residents with Bachelors Degrees 100 - %w/ BS (0-100%) % of Youth at Risk Actual % (0-100%) TOTAL Sum of figures 0 to 400
To determine if technology implementation varies systematically with stakeholder perceptions of specific and important effects of School to Work policy, a Technology Profile was developed for each high school participating in the Mohawk Valley Consortium. The Technology Profile consisted of the following variables: number of computers installed in the high school, number of computers with a 486 megahertz or faster processor, number of computers networked, number of teachers with access to computers, level of demand from teachers, level of demand from administration, level of demand from community, importance of improving curriculum, and the importance of improving teaching (See Table 3). Each item was summed into a technology profile points total. Similar to the performance and environment profile, no weighting was used with the technology profile. No item was determined, based on the literature reviewed, to be in need of weighting. All items were treated with equal importance.
The level of technology implementation was used as a predictor of involvement in and commitment to reform efforts. The assessment of each school's integration of computer-related technology was intended to explain the level of technology present in the district, the knowledge and attitude of school district staff towards innovations, and more significantly, how technology was being utilized in the learning environment. The variance between the school's technology profile and the level of implementation of School to Work was discussed to explain any patterns or distinct occurrences.
Technology implementation levels were based on participation in the activities that were supported by the Mohawk Regional Information Center (MORIC). The MORIC is the vehicle through which each district purchased computer- related technology, integration strategies, planning services, network support services, and leveraged State Aid for purchases of equipment and services. High implementors had participated in a variety of activities, such as teacher training, curriculum integration of technology, and administrative use of technology. Low implementing schools had few activities or none at all.
Table 3 Compilation of the Technology Implementation Profile ITEM CALCULATION Number of computers in the high school Actual Number Number of computers with 486+ processor Actual Number Number of computers networked Actual Number Number of teachers with access to comp. Actual Number Level of demand from teachers Survey Number Level of demand from administration Survey Number Level of demand from community Survey Number Importance of improving curriculum Survey Number Importance of improving teaching Survey Number TOTAL Sum of figures
To determine how School to Work implementation varied systematically with stakeholder perceptions and/or the performance, environment, and technology implementation profile, the School to Work Implementation Profile for each participating high school was developed. A School to Work implementation profile was created based on the survey results of the high school administrators.
The High School Administrators' School to Work implementation profile was a construct of the high school administrators' perceptions of the extent to which School to Work implementation took place (See Table 4). It was developed from the survey of high school administrators and used the following variables: provision of multiple learning opportunities (applied academics, career academies, work sites as classroom, work-site simulation, community service projects, internships/apprenticeships, and co-ops/youth-run enterprises), number of grade levels School to Work activities took place (employer visits, job shadowing, simulated work environment, school assignments in the workplace, youth-run enterprise, community service projects, internships, work-linked learning, work-study co-ops), and the level School to Work staff development was supported (active/hands-on learning, multiple learning environments, partnership strategies, development of world-class standards, integrating career development). The high school administrators' total points for School to Work Implementation was summative of all the variables for each high school. The legislation and literature available do not differentiate on the importance of each item. Therefore, no weighting was used in the calculation.
Table 4 High School Administrators' School to Work Implementation ITEM RESPONSE RANGE Provided multiple learning opportunities Applied academics 0 to 4 Career academies 0 to 4 Work sites as classrooms 0 to 4 Community service projects 0 to 4 Internships/apprenticeships 0 to 4 Co-ops/youth run enterprise 0 to 4 Number of grade levels STW activities offered Employer visits 0 to 4 Job shadowing 0 to 4 Simulation of work environment 0 to 4 School assignments in the workplace 0 to 4 Youth-run enterprise 0 to 4 Community service projects 0 to 4 Internships 0 to 4 Work-linked learning 0 to 4 Work-study co-ops 0 to 4 School to Work Staff Development Supported Active/hands-on learning 0 to 10 Multiple learning environment 0 to 10 Partnerships strategies 0 to 10 Developing world-class standards 0 to 10 Integration of career development 0 to 10 TOTAL (sum of response range figures) 0 to 114
Methods Summary
This was not a traditional forward mapping (top-down) analysis. If this study had used the forward mapping approach, the data collection methods would have used the objectives stated in the original policy to determine what data should be collected. This study used the backward mapping (bottom-up) approach. Backward mapping is concerned with the situation in which the policy was being implemented. Therefore, methods different from the top-down approach were used. The data collection methods gathered information about the actual activities at the local level. A final analysis was completed of the perceptions of local stakeholders, the competition for resources, and the environment in which the school was operating to determine their effect on implementation of the policy.
The methods of analysis used in a forward mapping process could have measured the locality's level of implementation by the objectives of the original policy. But that process would not describe the contributing factors or barriers to implementation. Nor would it describe actual implementation and what it meant to a high school. To follow-through on the backward mapping approach, a different method of analysis was used. The perceptions of the local actor as a stakeholder and/or implementor were compared to the original intent of the policy. The extent of variance that can be explained by these perceptions was measured. This analysis of perceptions was in combination with the measurement of variance of School to Work implementation levels attributed to the environment in which the school operates, the performance of its students, and the level to which it has been active in implementing technology.Results of Data Analysis
Within the context of this section of the article descriptive, regression, and correlation statistics are used to analyze the data collected from the surveys and data sources described. The data include regional and local level data collected from the New York State Education Department (NYSED) and the New York State Department of Labor (NYSDOL); the stakeholder survey responses; and the data collected to build the Environment Profile, Performance Profile, Technology Implementation Profile, and School to Work Implementation Profile.
Findings
High School Administrators: Description of Survey Responses
The high school administrators' information about their school's involvement in School to Work was used to create the second of three subgroups for the School to Work Implementation Profile. The profile was created using the same method as for the superintendents. Each high school received a total number of points based on the high school administrator's reporting of School to Work implementation activities (See Table 4). The distribution of high school for each point range is reported in Table 5.
Table 5 HS Administrator: STW Implementation Points per High School TOTAL POINTS NUMBER of SCHOOLS 1995 1996 0 -11 5 3 11-20 6 6 21-30 3 2 31-40 5 5 41-50 3 4 51-60 0 2 61-74 2 2 TOTAL 24 24
High school administrators were asked to identify the number of grade levels involved in various types of multiple learning environments during the 1995-96 and 1996-97 school years. On average, 2.06 grade levels per school were providing out of class experiences in 1995-96; 2.28 were in 1996-97 (See Table 6). Schools, on average, did not support a single grade level for work site simulations, internships/apprenticeships, or youth-run enterprises in either school year. High schools averaged over two grade levels of employer visits, and over 1.5 grade levels of job shadowing and community service projects for each year of the program (1995-96 and 1996-97). Simulated work environments, school assignments in the workplace, youth-run enterprises, internships, work-linked learning, and work study co-ops did not average one grade per school and did not take place at all four grade levels in any of the schools.
On a 1 to 10 scale (lowest to highest), high school administrators were asked to estimate their commitment to staff development related to School to Work. Active/hands- on learning averaged 4.33 for 1995-96 and 5.11 for 1996-97, with a range of 0 to 9 and 0 to 8, respectively. Developing multiple learning environments, partnership strategies, world class standards, and integrating career development in 1995-96 and 1996-97 all had a mean score of less than 5. The exception was integrating career development, it did achieve a mean of 5.56 for the 1996-97 school year.
Most administrators did not report that a majority of the activities increased in occurrence from year one to year two (See Table 6). The most common activities that took place were out of class experiences, employer visits, and job shadowing. Each individual activity had an average range of occurrence of .5 to 2.44 grade levels. There was an average one grade level of support per School to Work activity for each high school.
Each high school administrator was asked to estimate his/her perception of the importance of different types of staff development. All of the staff development areas shown in Table 7 were emphasized in the federal legislation, as well as the regional partnership State-approved proposal, as necessary for all implementation. On a scale of 1 to 10 (lowest to highest), the highest average commitment to any single staff development component was the integration of career development, which had a mean level of commitment of 5.6. High school administrators demonstrated a low level of commitment to all staff development components related to School to Work. Partnership strategies, an activity used to develop relationships with local employers and organizations to get them involved in School to Work, had a level of commitment just above a 3.
There was a high level of agreement among high school administrators that School to Work should be a part of curriculum and instruction (See Table 8). High school administrators on average perceived School to Work as a more important activity than did superintendents. In contrast, high school administrators and superintendents agreed that technology should be a major component of School to Work. On a scale of 1 to 10 (lowest to highest), high school administrators also were asked to indicate their level of agreement with four statements about the 1997-98 school year. On average, School to Work as a part of high school curricular and instructional efforts was ranked 8.22. The actual range for responses from administrators to this statement was from 6 to 10. The statement that business and industry should be active in the development and integration of School to Work resulted in the mean response of 6.28, with a range of 1 to 10. The statement that community and parents should be active in the development and integration of School to Work had a response mean of 5.83, with a range of 1 to 10. Technology and electronic learning as a major component of School to Work received a response mean of 7.72, also with a range of 1 to 10.
Table 6 High School Administrators: Responses to Survey Number of Grade Levels Min. Max. Sum Mean Out of class experiences 1995-96 0 4 37 2.06 1996-97 0 4 41 2.28 Work-site used as classroom 1995-96 0 2 6 .33 1996-97 0 2 10 .56 Internships or apprenticeships 1995-96 0 2 12 .67 1996-97 0 2 12 .67 Co-ops of youth run enterprises 1995-96 0 4 10 .56 1996-97 0 4 10 .56 Employer visits to classroom 1995-96 0 4 39 2.17 1996-97 0 4 44 2.44 Job shadowing 1995-96 0 4 28 1.56 1996-97 0 4 35 1.94 Simulated work environment 1995-96 0 4 14 .78 1996-97 0 3 11 .61 School assignments in workplace 1995-96 0 2 5 .28 1996-97 0 3 9 .50 Youth-run enterprises 1995-96 0 2 6 .33 1996-97 0 2 6 .33 Community service projects 1995-96 0 4 27 1.50 1996-97 0 4 30 1.67 Internships 1995-96 0 2 11 .61 1996-97 0 2 10 .56 Work-linked learning 1995-96 0 4 9 .50 1996-97 0 4 9 .50 Work-study co-ops 1995-96 0 2 11 .61 1996-97 0 2 13 .72 Table 7 High School Administrators Responses: Commitment to STW Staff Development (1 to 10 scale) Maximum Mean Active/hands-on learning* 1995-96 9 4.3 1996-97 8 5.1 Development multiple learning environments 1995-96 8 2.9 1996-97 8 3.4 Partnership strategies 1995-96 9 3.2 1996-97 9 3.3 Develop world-class standards 1995-96 10 3.6 1996-97 10 4.2 Integration of career development 1995-96 9 4.4 1996-97 *Response was 1 for all items Table 8 High School Administrators' Response: Actions for 1997-98 Low High Mean STW should be part of curriculum and instruction 6 10 8.2 Business and industry should be active in integration 1 10 6.3 Community and parents should be involved in integration 1 10 5.8 Technology should be a major component of STW 1 10 7.7 * level of agreement 1=none 10=completelyHigh School Administrators: Analysis of Responses as a STW Implementation Profile
As shown in Table 9, there is a strong positive association between the high school administrators' School to Work implementation profile and the amount of job shadowing and simulated work environment activities. The more job shadowing that was taking place in the high school, the higher the overall level of implementation of School to Work. There is no association between the high school administrator's and the superintendent's estimation of the level of implementation of School to Work.
High school administrators were asked to estimate their level of commitment to School to Work staff development activities as described in the Federal legislation and the State-approved regional partnership proposal for the implementation of School to Work policy. Based on high school administrators' responses, there is a high association between the level of implementation of School to Work and the administrators' commitment to active/hands-on learning, development of multiple learning environments, utilization of partnership strategies, development of world class standards, and integration of career development staff development activities (See Table 10). Table 11 shows the positive high correlation between the High School Administrators' School to Work Implementation Profile and the percentage of youth-at-risk, the number of years the superintendent has been employed in that title in the current district, and the total number of years the superintendent has held that title.
Table 9 High School Administrators: STW Profile & Specific Responses High School Supers Job Simul. Administrators Rank Shadow Environ. Admin Profile .320 .546* .537* Of STW '95 Admin Profile .451 .735* .575* Of STW '96 School assign Youth Comm. Intern- in workplace Enterp. Serv. ships Admin Profile .305 .086 .570* .331 Of STW '95 Admin Profile .221 .029 .439 .335 Of STW '96 *p<.05 **p<.01 Table 10 High School Administrators: Commitment to STW Staff Development Active/ Develop Partn- Develop Integrate hands-on multiple ership World Class Career learning learn Strat. Standards Develop. environ. Admin STW .828** .907** .765** .839** .758** Profile in '95 Admin STW .774** .931** .698** .819** .461 Profile in '96 * p< .05 ** p< .01 Table 11 High School Administrators Profile of STW Implementation % YAR 1995 Superintendent: Superintendent: Yrs in district Total # of yrs HS Admin STW .442* .623* .792** Profile 1996 * p< .05 ** p< .01
Regression analysis, using the high school administrators survey data, was used to determine the level of variance in high school administrators' perceptions of School to Work implementation explained by the performance, environment, and technology implementation profiles. The results are different from the analysis using the superintendents' responses. The level of variance in the High School Administrators' School to Work Implementation Profile is explained significantly (p <.10) by the high school's environment, performance, and level of implementation of technology. Roughly 50% of the variance can be explained with less than ten percent chance for error (See Table 12).
Table 12 Regression Analysis Predictors: Environment Profile, Technology Profile, Performance Profile Depend. var.: H.S. Administrators Profile of STW Implementation '95 R=.707 Rsquare=.499 F=2.99 sig=.088 Depend. var.: H.S. Administrators Profile of STW Implementation '96 R=.718 Rsquare=.515 F=3.184 sig=.077High School Administrators: Summary
According to these high school administrators, the level of participation in School to Work increased from the 1995-96 to the 1996-97 school year. Each of the required School to Work activities was taking place in at least one grade level in every school in the region. In terms of staff development, integration of career development had the highest level of commitment. The development of partnership strategies, however, was among the lowest concerns. High school administrators saw a need for involvement in workforce development activities and the staff development for developing those activities in general. But they did not follow through with the training of staff or program implementation across all grades.
High school administrators extended strong support for the continuation of School to Work into the 1997-98 school year. In addition, it was important business and industry and community members remain involved in the implementation process. They also wanted technology to be a major component of the activities supported.
The higher the percentage of youth-at-risk in the school district, the higher the implementation of School to Work as described by the high school administrator. The greater the length of time the superintendent was employed by the district in his/her current title, the greater the level of School to Work implementation. The greater the total number of years the superintendent has held that title, the greater the level of School to Work implementation that took place in the high school.
Student performance, the local environment, and the level of technology implementation reached in the high school helped to explain the variation in the level of School to Work implementation perceived by the high school administrators. Specifically, there was a strong and significant relationship between the local conditions under which the school was operating while implementing the policy and the success of actual implementation.School Selection and Follow- up Interview Process
To understand more completely what affected the school district participation in School to Work, follow-up interview methods were used. The intent of the interview was to gather data to determine why schools implemented the School to Work policy at varying degrees although they were part of the same consortium. High schools in the region were selected based on their performance profile and School to Work implementation profile, determining the school's placement on a grid (See Figure 1). The performance and School to Work implementation profiles were chosen as selection criteria because of the availability of data.
Figure 1. Placement of School Districts on the Grid
Summary of the Interview Process
The interviews with stakeholders provided information that could not have been collected through surveys. Interviewees shared what had happened during the decision making process when considering involvement in School to Work. All agreed the initiative was worthwhile, but there were common issues keeping schools from becoming involved: lack of resources to become involved, lack of resources for continuation, and competition with other efforts. Participating schools saw the School to Work initiatives as a means for schools to prepare and engage students in career preparation decisions. They also recognized the desirability of creating connections between the community and teachers, thus exposing them to new ideas.Summary of Three Stages
Four profiles were created to determine to what extent characteristics of the local organization explained the variance in implementation of Federal policy. The performance, environment, and technology implementation profile of a local organization significantly predict the variation in School to Work implementation, when the high school administrator described the implementation. According to the survey results of superintendents, high school administrators, high school teachers, and employers, School to Work implementation levels increased from 1995-96 to the 1996-97 school year. During the same period, commitment to School to Work activities also increased. The majority of stakeholders surveyed felt very strongly that School to Work should continue during the 1997-98 school year.
Conclusions
Summary of Findings
This study compared the original intent of a specific educational reform policy to its local adaptation. Specifically, it focused on the implementation of the Federal School to Work Opportunities Act of 1994 in the 44 public high schools participating in the New York State Mohawk Valley Workforce Preparation Consortium during the 1995-96 school year. To compare the original intent of the School to Work policy to the local program adaptation and explain their differences two questions were answered. Participation in a consortium arrangement designed to facilitate the implementation of State or Federal policy resulted in active, if minimal, implementation efforts by individual consortium members. Stakeholders from the high schools studied described their schools as having been active in the implementation of School to Work during the 1995-96 school year. However, the level of implementation throughout the region was limited and minimal in some high schools.
The organizational and community context descriptions of those high schools in which School to Work was perceived as having both specific and important policy effects differed significantly from those where the policy was perceived as having little or unimportant effects. The greater the perceived effect of School to Work on operations, the higher the level of School to Work implementation in both 1995-96 and 1996-97. The greater the perceived importance of School to Work, the higher the level of implementation. However, in no high school had the program been implemented in all four high school grades. A high school setting where School to Work activities were perceived by local stakeholders as having great specific and important effects differs from a high school setting where School to Work activities were perceived as having little or no effect. The differences in organizational demographics and community characteristics that contributed to the high school administrator's description of School to Work implementation are statistically significant. Profiles of high school student performance, high school operating environment, and the level of technology implemented predicted the level of School to Work implementation described by the high school administrator. However, none of the individual profiles explained the variance in School to Work implementation by themselves. Therefore, the extent to which there were differences in perceptions of the enriched or impoverished status of technology did not contribute to perceptions of School to Work. Similarly, there were no significant differences in perceptions of curriculum scope and sequencing or in instructional strategies seen as being influenced by School to Work.
Based on the findings of this study, implementation was assessed by describing the reported activities of the local integration and comparing them to the original intent of the policy. The difference between original intent and reported local adaptation were interpreted through the backward mapping analysis as discrepancies in fulfilling the purpose of the particular policy. This study argued a more inclusive approach.
Backward mapping includes an explanation of local conditions and expands policy analysis beyond the determination of success or failure of a policy based on the measurement of the objectives. This bottom-up approach explained the conditions under which the policy was implemented. The process was more than simply measuring success by meeting policy goals; it explained the relationship between characteristics of the organization and local level policy implementation. Thus, there was consideration for the context in which implementation took place. In terms of context, backward mapping delineated both the conditions prior to and during implementation and their relationship to the variance in the level of implementation.Backward Mapping as a Method of Policy Analysis
Backward mapping was effective in accomplishing three important objectives: analyzing policies that have been created to have an impact at the local level, providing contributions to the knowledge of the policy implementation process, and enabling a determination and explanation of the success of the studied policy. The process takes into account the difference in localities. Therefore, the analyst takes into consideration that the many differences between the localities--including leadership, environment, student performance, involvement in other innovations, and stakeholder perceptions--contribute to the differences in policy implementation among these localities. Consequently, when the implementation process is to take place over multiple years, or if it is slated for replication in other localities, the analyzed policy can be adjusted or improved based upon the findings. Forward mapping, as a comparison, explains to what extent the implementor has met established benchmarks. Variation in implementation is only considered more or less successful compared to a set goal, which is frequently complete integration. Backward mapping takes the analysis an additional step and explains the extent to which the locality's perceptions influenced variations of implementation. By addressing perceptions and the conditions of the environment, backward mapping removed the focus from the policy objectives in determining implementation success. Rather, success is expressed in terms of the effect of the local implementing organization. Backward mapping was useful for determining and explaining the relationship between the locality prior to policy implementation and how policy, as intended to be implemented by the federal government, was adapted to those conditions. During the process the direct analysis of the policy was shifted to focus on the high school. The organization's impact on the policy guiding the implementation was explained through an analysis of its characteristics. The result was the delineation of predictors associated with the successful implementation of the policy.
However, the comparison of intent and actual implementation is only one step in the backward mapping process. Backward mapping is also useful for analyzing why the implementation did or did not take place. Answering this question requires a process which enables the researcher to examine the conditions surrounding the policy. Because schools do not operate in a static environment, the policies being implemented by them are subject to forces which can cause the school itself to change. Simply measuring the ability to meet objectives is often worthwhile and appropriate. But for explaining varying degrees of implementation and how mediating variables may be dealt with by future policies, the information compiled in the backward mapping process is beneficial.
Student performance, environment, and technological characteristics of the high school explained the variation between Federal intent and local adaptation. Stakeholders were asked to provide their perceptions of the implementation process. Concurrently, the data collected about the organization explained what had been taking place at the local level, irrelevant of the policy implementation. For example, the organization may have been involved with other important and specific challenges that precluded an implementation at a pace similar to other organizations. Likewise, similar groupings of organizations may have differentiated levels of implementation due to a common factor, such as the participation in a particular event. Backward mapping was an effective method for delineating that implementation variance was explained by characteristics of the local organization.
Backward mapping ascertained that organizational leadership, past performance, stakeholder perceptions, external environment, and involvement in other efforts were all associated with policy implementation success. Both forward and backward mapping explained the level of implementation attained. But backward mapping also explained why the levels of implementation were reached. Backward mapping, therefore, was an effective tool in explaining the relationship between the original policy, the implementing locality, and the actual implementation.Backward Mapping and Local Level Characteristics
Backward mapping elicited information which supports an understanding that successful implementation is not based solely on well-written policy. Characteristics of the high school which impacted the implementation process were identified and measured. Their association with variation of school implementation from established goals was explained in the final analysis of the process.Perceptions and Participation
It was found that the district and building level leadership had an impact on the level of policy implementation. Each leader, as a stakeholder, had a different perception of the policy's interpretation and level of importance. Participation in terms of the policy implementation meant signing as a consortium partner. But the variation of implementation among schools could not be explained simply by agreement to join a consortium. All of the schools identified themselves as partners in the submission of the grant to the State, but not all of them implemented the same level of School to Work initiatives.
Instead, interpretation of the policy influenced actual participation. Some school leaders interpreted School to Work as being career exploration. Those schools implemented basic activities such as tours, speakers, and job shadowing. But they were less likely to have implemented components of the policy that took a higher level of commitment of resources, such as work-based learning. At least two schools excluded themselves from initially participating because of their leaders' limited interpretation of the legislation. Not until there were definitive examples of the policy being implemented in other school buildings did these leaders "buy-into" the concept.Decision Making
The original School to Work policy provided the flexibility to adapt implementation to meet local needs. It was at the discretion of the consortium to determine how the framework of support and implementation would be provided. The local school determined how the policy was actually implemented. This study found that the way in which a superintendent perceived multiple reform efforts, as well as the way the high school administrator perceived the School to Work policy, explained the discretion used to adapt the policy to local need. For some schools, it meant implementing School to Work components as described within the language of the legislation.
Other schools participated for different reasons. One school stated that it used School to Work support as a means for exposing its staff to new ideas and environments with the expectation that participation would effect change for the benefit of children. There is evidence that this happened, both in that school and others.
High school teachers involved in School to Work activities described changes in curriculum scope and sequence or in instructional strategies. Teachers described how classrooms and individual lessons had been affected positively because of their involvement in staff development programs funded by School to Work. In many cases, they became more aware of the resources (e.g., lessons being used by local teachers) around them. They discovered how the application of learning had a positive effect on the students. These teachers had previously believed that the application of knowledge did not have to take place in the classroom for students to learn and develop skills. Teachers also discovered that there were many opportunities for students to test their knowledge in surroundings not previously considered learning environments, such as factories, farms, stores, and the community.Competition for resources
While this study did not set out to analyze the issues of resources, there are some interesting findings are worth mentioning. The competition for scarce resources contributed to the variation of policy implementation levels between school districts. Poor schools described two areas of concern: the level of resources necessary to become involved and the resources needed to sustain the effort. Poor schools with a small local employment base that perceived School to Work merely as a career awareness strategy had low rates of participation. These educators did not see how they could become involved when there were so few community resources. Other poor schools were concerned about sustaining the effort after funding was no longer available. Either they did not feel the school had the fiscal resources to continue the initiative, or they perceived School to Work as one more initiative that would simply disappear.
Resource allocation also impacted the decisions of more affluent schools. Some of these schools and their communities put high demands on the students to perform academically. Their expectation of student preparation was for college and highly skilled careers. In some cases, this expectation resulted in School to Work implementation at higher levels. It was a means of exposing students to the demands of the workplace and the high level of skills required to be successful in the positions these students sought. For other schools with the same high expectation of their students, the perception of School to Work as applicable only for the "career-bound" was a detriment. Their perception was the student should focus solely on academic preparation leading to college.Summary of Backward Mapping Policy Analysis
By using backward mapping in this study, attention was focused on the importance of the locality during the policy implementation process. Backward mapping established a broad framework within which an analysis of the actual implementation process was carried out. Within this process, the analyst is able to discover and explain what determines the level of success of the policy. By choosing any single Federal policy, one could determine the actual impact on the local school district. Using this process, it can also be determined the role of the locality in relation to the policy. The policy exists to create change at the local level, but it is the locality which ultimately decides how the policy's intent will be adapted to meet its needs. The key elements of backward mapping in policy analysis are as follows:
- It is effective in describing actual versus perceived implementation.
- It is effective in explaining actual versus perceived participation.
- The effect of the locality is considered to help to explain variation in implementation.
- Perceptions of importance and policy effect are measured.
- A more complete picture of policy implementation is presented.
- The thoroughness with which policy implementation information is provided is at a high level.
Backward Mapping, Forward Mapping, and School to Work
In the case of School to Work, a form of forward mapping analysis had already been completed by the Mohawk Regional Consortium, the New York State Education Department (NYSED), and the Federal government. Each level of governance had used previously set criterion established within the original Federal legislation, approved regional implementation plans, and state level implementation plans to determine the success of the policy. These studies found that the schools had the required representation in place and that they were engaged in the required activities. Thus, the state and the region appeared determined to have successfully implemented School to Work policy. Objectives of the policy had been met to varying degrees in each school, and continuation of the project was contingent on the objectives set for the following year of implementation. According to Mohawk Valley Consortium members, there had been little feedback given to the region from these studies upon which to base continuation on a commitment of resources to areas in need of improvement.
The data collected during this process that can be labeled as an example of forward mapping was for the most part unidimensional. There was snapshot of activities on a given day. But the discussion of the data collected assumed that all the activities were the result of School to Work participation and use of funds from the program. None of the data was in a format enabling the determination of association or any such relationship linking policy to actual implementation. Much of the data was compiled into anecdotal stories that spoke of program objectives being met in various schools.
There were 44 school districts with high schools that agreed to be partners in the Mohawk Valley Workforce Preparation Regional Consortium to implement the Federal School to Work Opportunities Act of 1994 (STWOA). Consistent with other non-academic studies involving this region, this study found that schools participated in varying degrees. Therefore, the agreement to participate in a consortium arrangement designed to facilitate the implementation of Federal policy did not result in identical implementation levels being achieved by individual consortium members.
By using backward mapping, it was found that the level of participation is best explained by the perceptions of the high school administrator. In addition, the perceptions of the high school administrator are predicted by the characteristics of the school. The high school administrators' perceptions of School to Work implementation vary based on the performance of the students, conditions in the community, and level of technology implemented in the school. The better the students are performing, the more resourceful the external environment. The more technology present in the school, the higher the level of policy implementation. Therefore, the level of school participation in the policy's initiatives varied according to perceptions and characteristics of the locality.Local Characteristics
Characteristics of the local school districts in the Mohawk Valley Consortium were measured to explain the variation of high schools' involvement in School to Work policy initiatives. The school's characteristics and stakeholders' perceptions were measured to determine their impact on high school participation. By grouping characteristics into three broad areas called profiles (performance, environment, and technology implementation), the combination of profiles were found to explain the variations in success of School to Work implementation.Performance Profile
The performance profile was used as a grouping that characterized student performance as a means for measuring the school's effectiveness. When the performance profile was combined with both the environment characteristic profile and the technology implementation levels profile, the variation of implementation between schools was explained to a high degree. Therefore, the combination of previous student performance, the conditions of the community, and the level of participation in other reform efforts combined predict the level of implementation of the School to Work policy.
The performance profile of the high school was found to have a few specific items that were significantly associated with the level of School to Work implementation. The greater the percentage of the total number of students receiving NYSED Regents Diplomas, the higher the level of School to Work implementation. This becomes an important statistic considering the perception of the teachers interviewed. Teachers attributed low implementation levels to the lack of flexibility of students working towards a Regents diploma. The teachers' cited specific problem areas, such as preparing students specifically for items on the Regents exam, and the lack of time for "extra- curricular" and exploratory activities that prevented Regents student participation. The teachers' statements are contradictory to the finding. Therefore, a question is raised as to what was happening in the classroom where teachers were asked to participate in School to Work but did not because of Regents requirement on students. Were the teachers' perceptions inaccurate? Were there a greater number of non-Regents students actively participating in School to Work and more School to Work activities in place in schools where higher percentages of students receive a Regents diploma? If so, could this be for the purpose of providing more opportunities in those schools for those non- Regents students if they were assumed to be workplace bound? These are questions to be addressed in further studies of the policy.
Schools within the Consortium with high percentages of Regents students and a high level of School to Work implementation had perceived School to Work to be something more than career exploration. In these schools, there was evidence of career awareness activities such as tours, speakers, and job shadowing all of which were typically already taking place, and which took less commitment by the teacher and the school to integrate. But there was also evidence of higher levels of integration taking place within the curriculum, instruction, and even the assessment. Schools graduating 85% or more students with Regents Diplomas had created interdisciplinary projects in areas such as Math, Science and Technology. They also changed methodologies in the classroom, incorporating applied learning techniques and utilizing the community and the workplace as learning environments. Schools had even begun to assess student knowledge through hands-on application and problem solving using actual workplace situations. These schools did not perceive the Regents diploma as a barrier; they used School to Work initiatives as a means for creating new ways to meet its rigors. According to this study's findings, these occurrences can be attributed to two factors: the leadership provided by the high school administrator and the support of multiple reform efforts by the superintendent.Environment Profile
The environment profile was a grouping which characterized the conditions of the environment in which the local high school operated. Schools located in communities of high and low unemployment and wealth were able to implement School to Work policy initiatives. Measures of association were applied to determine the extent to which the level of School to Work implementation was explained by specific community characteristics. It was found that the types of business in the community and the percentage of youth at risk were highly correlated to levels of implementation. The higher the percentage of people in the community employed by agriculturally-based businesses, the higher the level of implementation of School to Work. Even though agriculture was the lowest percentage of the total businesses, no such significant correlation could be found for manufacturing, retail, or service-based businesses. The analysis also determined that the higher the percentage of youth who are "at-risk", the higher the level of School to Work implementation. Several factors were discovered during the quantitative follow-up interview process that could explain this fact. Prior to the implementation of the STWOA of 1994, several State and Federally funded programs were addressing the needs of the youth-at-risk population through School to Work- type activities. The Governor's School And Business Alliance (SABA), Private Industry Councils (PICs), and Workforce Preparation Pilot projects were all based heavily on the needs of high school children from low wealth families. Twenty-seven of the 44 Mohawk Valley Workforce Preparation Consortium school districts were located in a BOCES service area that also had a SABA. Fifteen of the 44 school districts were involved in the original New York State Workforce Preparation Pilots funded through the Governor's office. Twenty-four of 44 school districts were located in a BOCES that was involved in the second round of Workforce Preparation grants funded by the Governor's office and managed by NYSED. All of the schools were supported by a regional PIC. Therefore, all of the schools had been participating in a combination of School to Work-type initiatives based on their at-risk youth population prior to the implementation of STWOA of 1994.Technology Profile
The third local characteristic groupings to be measured were those related to participation in local reform efforts. There were actually two measurements of commitment to other reform efforts. One was the level of implementation of educational technology within the school, which was determined from an existing data set. The second was stakeholder perceptions of the importance of involvement in other reform efforts that was collected from this study's survey. The level of technology implemented in a school was not significantly associated with the level of School to Work implementation. However, the more important the involvement in multiple reform efforts, the higher the level of technology integration into a school. There was found to be a strong association between technology implementation level and the level of importance of participation on multiple reform efforts as indicated by the school superintendent. In addition, there was a strong and significant association between the level of School to Work implementation as perceived by teachers and the level of importance of participation in multiple reform efforts as indicated by superintendents. The more important a superintendent perceived participation in multiple reform efforts, the more School to Work activities and technology were implemented by the teachers. Therefore, it was not the actual participation in another reform effort that explained future participation in other reform efforts. It was the perception of the superintendent of the importance of involvement in other reform efforts that was associated with implementation of other Federal policies.
Stakeholders' perceptions of involvement in multiple reform efforts were measured as a local characteristic that was associated with the level of School to Work implementation. The perceptions of the superintendent were not indicative of the level of School to Work implementation in the high school. The analysis of local level data did find that there was a significant and very strong relationship between the number of years the superintendent has been employed in the district under his/her current title, the total number of years he/she has held the title, and the level of School to Work implementation as reported by the high school administrator. Therefore, the longer the person held the title of superintendent, the more likely School to Work was reported by the high school administrator to have been implemented.
In comparing the perceptions of educators and employers involved in School to Work, it was found that neither superintendent's nor employers' perception of an effect on organizational operations was associated with involvement. This can be attributed superintendents perception of School to Work as means for creating change in the classroom and providing opportunities for students, rather than as a means for impacting the operations of the organization. But part of the impetus for the region to become involved in School to Work was to work with employers to improve the future workforce. However, the results showed teachers knew very little of the partnerships that existed with business. There was ample documentation in the region of partnerships including directories of partners, teacher/student documented activities with business, and a computerized database of available employer partners and the types of activities they would support. But the teachers and administrators did not identify them as existing in their building.
It was assumed, based on the available literature, that businesses were involved, at least partly, because they were concerned for their future employees and how well they were being prepared. However, employers participated in the activities for reasons other than organizational impact. During the follow-up interview process employer-partners indicated there had not been enough time invested in the implementation of School to Work to realize impact on their own business. But it was expected by employer-partners interviewed that employers would be able to see an impact in the long term.Continuation of Implementation
Most schools indicated that it would be important to participate in School to Work activities during the 1997-98 school year. The level of previous implementation was not associated with the level of stated importance of participating the following year. A majority of the schools indicated a high level of agreement that the integration of School to Work into curriculum and instruction with the involvement of community members and employers was very important.
The business and industry members of the community described School to Work involvement during l995-l996 as being important. All of the respondents strongly agreed that School to Work should continue, but at the same time they did not perceive the policy as having any impact on the operations of the business. A few employer partners did expand on this point. Their view was that School to Work is a means for helping students learn, which was their rationale for participating. There would need to be a longer-term interaction before they could assume that the policy would have an impact on the operations of the business. In the long-run they did assume that students would be better prepared for work with such programs. For that level of impact to take place, they asserted that the School to Work implementation would have to be an integral part of all schools.Local Characteristics Summary
By using backward mapping this study found that the perceptions of stakeholders in district and building level educational leadership positions were an important factor in determining the level of implementation of School to Work policy in 9-12 high schools located in the Mohawk Valley Region. The high school administrators' reported level of implementation varied according to local characteristics and conditions. When superintendents found involvement in multiple reform efforts to be important, teachers reported higher levels of School to Work implementation. Actual implementation was best predicted by the high school administrator. Yet, only one BOCES in the region documented providing technical assistance and training specifically to high school administrators. Had the significance of the high school administrator been known the region and its leadership may have placed greater emphasis on staff development for high school administrators. It may have marked resources dedicated to improving the leadership and understanding of School to Work and prepared them to provide more effective support during the implementation of this policy. This should not be considered a weakness on the part of the regional leadership, but a lack of information available to policymakers who required superintendents to sign-off and teachers to implement School to Work. Backward mapping has provided the means for collecting this information that, if used, could impact the School to Work implementation policy for the last two of the five-year policy implementation phase in New York State and subsequent years in other states.Using Backward Mapping Process for Policy Analysis
Policy analysis through a backward mapping lens has proven to be an effective process for determining the predictors and identifying items that explain the success of implementation. Previous analyses of School to Work implementation in the Mohawk Valley Region followed a forward mapping and top-down process. These attempts to analyze implementation of the Federal policy determined the level to which the region met goals and objectives and how much School to Work was implemented. It did not explain why the implementation took place, or who or what contributed to the level of implementation. The results of the forward mapping process did not explain the variation of implementation levels between schools. The forward mapping process focused on the measurement of the volume of activities and the documentation of the involvement of stakeholder groups. It is the backward mapping approach which addressed how their involvement impacted decision making and final implementation in the form of local adaptation.
The backward map explained first, that schools participated at varying levels and secondly, why schools participated at varying levels. To reach these findings, the process required an initial analysis to determine that there was in fact a difference between the schools and the environments in which they operated. A determination was then needed that each school agreed to and followed the same guidelines, adapting the initiatives to meet it's students needs. Each individual school then participated at a level which best suited it's stakeholders. The stakeholders' decisions and perceptions impacting implementation are attributed to a combination of pre-existing characteristics of the school's internal and external environment. Pre- existing conditions in and around the school and the perceptions of stakeholders were measured by the backward mapping process, and resulted in an awareness that such actions are valuable to policy analysis. If the policy analysis process had stopped at measuring goals and objectives, there would have been no determination of the reasons some schools contributed to meeting those objectives. It would not have explained how the attainment of some objectives was actually due to schools participating in other initiatives.
It is the backward mapping of the policy that enables as analysis recognizing that the locality may interpret the intent of the policy differently from that of the policymaker. Forward mapping assumes the failure to meet objectives is due to the locality's inability to implement. Backward mapping asks the local actor as implementor how they interpreted the policy. It assumes a variation in interpretation results in a variation in adaptation and implementation. If School to Work was locally defined as a career exploration program, then implementation would reflect activities to support student career exploration. In that case, school-based and work-based learning activities would be less likely to take place. Therefore, a school's lack of implementation is not related to its ability; it has to do with discretion based on perceptions. Choices were made because of forces acting upon the school that impact local actors' decision making.
The analysis was built from a local-level perspective to explain local actions to meet a Federal policy guideline. The survey instrument of the backward mapping process asked the district level, building level, classroom level, and employer partner to describe their perception of School to Work, what it meant to them, how important it was, and to what extent had it really taken place. To take the analysis a step further, the process used this data to identify special cases. These cases either met or went against expectations and were slated for a follow-up process. This second step looked for actual documentation and reviewed initial findings to gather a clearer understandi