This article has been retrieved
times since March 14, 2000
This article has been retrieved
Education Policy Analysis Archives | ||
Volume 8 Number 17 |
March 14, 2000 |
ISSN 1068-2341 |
|
Editor: Gene V Glass, College of Education Arizona State University
Copyright 2000, the
EDUCATION POLICY ANALYSIS ARCHIVES. Articles appearing in EPAA are abstracted in the Current Index to Journals in Education by the ERIC Clearinghouse on Assessment and Evaluation and are permanently archived in Resources in Education. |
The Appalachian Rural Systemic Initiative:
|
|
Abstract This article was written in response to "Top-Down, Routinized Reform in Low-income, Rural Schools: NSF's Appalachian Rural Systemic Initiative, by Robert Bickel, Terry Tomaskek, and Teresa Hardman Eagle which was published in the Education Policy Analysis Archives as Number 12 of Volume 8 on February 21, 2000. |
The Real ARSI ProjectThe Appalachian Rural Systemic Initiative (ARSI) has made a major contribution in education reform through the implementation of a truly systemic school and district improvement model. Improved student achievement is being realized as ARSI focuses on K-12 students through the development and support of catalyst schools designed to serve as models for other schools in their district. The resulting catalyst districts serve as leaders for reform efforts throughout the region.The ARSI model is based on a "bottom-up" team approach to school reform. A key component of the model is the development of teacher partners, who are designated by their schools as mathematics and science leaders. The teacher partner's work is supported by a team of professionals at the building and district level including the building principal, ARSI district liaison, and district superintendent. External support for the teacher partners and the development of catalyst schools and districts comes from five resource collaboratives located at university sites across Appalachia. These collaboratives are staffed by a director and mathematics/science specialists who, with support from university mathematics and science educators, provide training for teacher partners and direct services to catalyst schools in their region. Each catalyst school, led by the teacher partner, develops its own school improvement plan based on needs assessments, data analysis, and assessment of the instructional program. Implementation of the ARSI model has proved to be effective in providing both direction for school reform and a mechanism for technical assistance to catalyst schools. ARSI has provided assistance through the development of school leadership, access to national and regional resources that support mathematics, science, and technology reform efforts, and improvement of the community support base. ARSI has made a major contribution through the development of standards-based curricula, science/mathematics content and pedagogy development workshops for teachers, identification of high quality instructional resources, while providing extensive support for the key ingredient of the ARSI model, the teacher partner. One of the tools used for assessing program improvement needs has been the Science and Mathematics Program Improvement Review. This instrument is used to assess the program's effectiveness against a set of standards developed around "best practices" which are consistent with mathematics and science state and national standards. Needs assessment data gathered through this process has been utilized in both school and district strategic planning efforts. ARSI as a "Bottom-up" Reform InitiativeThe ARSI project utilizes a school-based approach to program improvement. The basic premise of the ARSI model is that reform and improvement of science and mathematics programs is best done in rural schools through the teachers and principals in each school. The ARSI team, consisting of the teacher partner, ARSI district liaison, principal, and superintendent, has been the primary planning group in each district and is supported by the resource collaborative housed at an area university. The ARSI emphasis has been on the identification of program needs, assistance in developing both short range and long range improvement plans, and in the provision of technical assistance in the development of curriculum and selection of appropriate resources. Professional development has been primarily "job-embedded." The primary functions of the teacher partner have included such activities as mentoring of other classroom teachers, modeling inquiry teaching strategies, and assisting teachers plan for inquiry based instruction.A major service provided by the ARSI staff has been to assist schools and districts with strategic planning. The Program Improvement Review has been a welcomed source of needs assessment data from which the teacher partner, principal and other science and/or math teachers have constructed their own improvement plan. Based on the needs assessment data, ARSI has facilitated school and district reform efforts by providing professional development, assisting in the identification of resources, and providing guidance in regard to curriculum development and instructional improvement. In no case, as implied in the article "Top-down, Routinized Reform in Low- income, Rural Schools: NSF's Appalachian Rural Systemic Initiative," has ARSI dictated how a participating school or district proceeds with their science and/or mathematics program reform efforts or constructed a "one-size fits all" approach to school/district assistance efforts. After a review of the first four years of the ARSI project, Inverness Research Associates, the ARSI project external evaluator, made the following statements concerning the ARSI approach to school reform: "The ARSI model is developmental and works from the inside out. That is, ARSI starts by identifying and building leadership within the district through its work with teacher partners. The teacher partner, with the help of the district liaison, then builds a core group of teachers and administrators who are committed to the reform effort. Eventually the reform effort may move to the level of district policy—curriculum, professional development, etc.—and then out to the community and national scene." |
The Program Improvement Review: One of Many Means to ARSI's GoalsThe statement "The primary means of accomplishing ARSI's aims is a one-day-one-school visit," indicates a lack of knowledge regarding the ARSI project. (Bickel et al., 2000) ARSI incorporates a wide variety of interventions and assistance to schools in their reform efforts. The primary means of accomplishing ARSI's aims is the utilization of "teacher partners" to mentor other teachers, provide professional development, coordinate curriculum development efforts, obtain quality resources, and work with parent and community groups to promote science and mathematics education. The teacher partner is selected on the basis of his/her general leadership ability, skill as a mathematics or science teacher and potential for providing assistance to other teachers. Teacher partners receive monthly training in both content and pedagogy through the ARSI resource collaboratives. In addition to the training and support provided by the teacher partner, professional development is being provided for teachers in participating district schools by both the ARSI curriculum specialists and university math and science educators. Training is being provided in inquiry instructional techniques, authentic assessment strategies, data analysis, and standards-based mathematics and science content. In all cases, the training provided at the school level has been requested by the school on the basis of needs identified at that level.The Program Improvement Review is but one tool, of many, utilized by ARSI to provide needs assessment data to schools involved in the ARSI project. In fact, the Program Improvement Review is not a requirement for participation in the ARSI program and is utilized only at the request of the individual school. The process has proved so beneficial, however, that most schools have voluntarily participated in the process and in several cases, districts (ARSI and non-ARSI) have requested that the process be completed in all schools to provide data for program planning. ARSI Project Potential to Improve Student Achievement in Rural Counties in AppalachiaDuring the four and one-half years of the ARSI project, it has become clear that the school districts in Appalachia differ widely in their "readiness" and ability to participate in significant reform efforts. At the outset of the project none of the participating schools had district-wide curricula in science or mathematics aligned with their state or national standards. School leaders lacked a "vision" of quality mathematics and science programs which would provide direction for reform efforts. Professional development was primarily district based and generally focused on generic topics such as improving school discipline or improving student safety in schools. Although these topics are certainly important, teachers also need a consistent, well-planned professional development program focusing on both content and pedagogy.Professional development, through the ARSI teacher partner has been one of the major foci of the ARSI program. There is clear evidence that the quality of instruction is improving as a result. Improved instruction, use of standards-based materials designed to promote student inquiry, and well defined curricula focusing on state and national standards are now commonplace in ARSI schools and the student achievement data, included in this document, show clearly that use of the ARSI model has resulted in positive results across the region. 
Another focus area for ARSI has been the development of
policies, at both the school and district level, which increase
mathematics and science learning opportunities. Policies
designed to increase the breadth and rigor of programs and the
support for mathematics and science in Appalachian schools, have
been implemented in many ARSI districts. See Figure 1.
|
ARSI: Positive Results Across the Appalachian Region and in States Served by ARSIOne of the most positive results of the ARSI project has been the development of skilled and committed leadership for mathematics and science program improvement. "There is no doubt that the greatest contribution of ARSI lies in this area: ARSI is helping districts identify, train and support local leaders who are knowledgeable about math and science reform and empowered to work towards change in schools and classrooms." (Inverness Research Associates, External Review Report, 2000) ARSI's efforts in training teacher partners, ARSI catalyst school principals, and ARSI district liaisons have resulted in a district team that has provided extensive leadership for science and mathematics program reform efforts.Student achievement data for ARSI catalyst schools validate the impact of the ARSI model. Catalyst schools that started the program during its first year (having had ARSI interventions for two full years), show a dramatic increase in student achievement in both mathematics and science. In science, students scored above the combined states' average and were significantly higher than comparison districts in the Appalachian region. Mathematics scores were slightly below the states' combined average, although the gap was significantly reduced, and students scored well above their Appalachian region counterparts. As would be expected, the gains for schools involved with the ARSI project for only one year are not as dramatic although ARSI catalyst schools that started the program in its second year demonstrate similar trends. Student achievement in science shows a similar percentage of improvement, as did the students from the inaugural year whereas the mathematics performance increased only slightly. See Figures 2 and 3. 
|
In examining individual school data, the results are even
more dramatic. See Figure 4. An ARSI school that has had a full
range of interventions in science demonstrates the type of
results achieved through the project. The school started with
the Program Improvement Review which identified several
weaknesses including lack of a curriculum in science and little
emphasis on inquiry-based instruction.
After the implementation of an aligned, standards-based curriculum and extensive staff development in inquiry based instruction, student achievement in science exceeded the state average in all assessed sub-domain areas whereas student achievement in all other content areas was below the state average. The data for another ARSI district with nine (9) elementary schools is equally impressive.  As in the previous example, the
ARSI catalyst school implemented an aligned, standards-based
curriculum and provided inquiry-based instruction professional
development for teachers through the ARSI teacher partner. As
can be seen in the graph to the right, the ARSI catalyst school
scored above all other district elementary schools in every
science sub-domain area. See Figure 5 above.
These data are not unique. 1999 ARSI schools' state assessment data is currently being analyzed. The preliminary results indicate substantial improvement for nearly all ARSI schools since the inception of the ARSI project in 1996.
ARSI Project Focus on the Uniqueness of Rural Schools"There is something about "rural-ness" that is important. These are small, closed communities. So, any effort to change the mind set, or to change the value system or the valuing of things, is difficult because it is a closed system. I think what we are seeing is a slow, steady battle to win hearts and minds--and having a local, well respected, well trained, well supported, well chosen teacher partner is the way to go about it. As one district superintendent said, 'Mountain people are just old mules--it is easier to lead them than it is to push them." (Inverness Research Associates, ARSI External Review, 2000)The ARSI project has been sensitive to the characteristics and needs of rural communities since its inception. Characteristics common to rural communities have long been known to researchers and ARSI is cognizant of the necessity of attending to the specific needs of these communities if the school reforms initiated are fully implemented and persist beyond the years of ARSI involvement. In addition to being rural, the Appalachian region school districts participating in the ARSI project are similar in that they reside in counties with poverty levels of school age children greater than 30% (according to the 1990 census) and USDA Beale Numbers 6 or higher. The principal ARSI goal, "to accelerate performance in science, mathematics, and technology in Central Appalachia," addresses one of the major educational challenges of rural communities. Formal education attainment tends to be lower in these areas. High school completion rates are lower than those in metropolitan areas and fewer rural students complete college (Herzog & Pittman, 1995). Rural students are also less likely to take college preparatory classes (Stern, 1994) often resulting in the need for remedial classes in science and/or mathematics upon their entry into a community college or university. Another goal for the ARSI project, "to develop a sustainable system providing students and teachers with timely, coordinated access to educational resources and services ...." addresses the "isolation" of these communities. Fewer institutions of higher education are located in rural areas and educators feel more professionally isolated than their metropolitan counterparts (Massey & Crosby, 1983; Stern, 1994). "Through ARSI, each of these districts, especially the teacher partners and district liaisons, have become affiliated with at least one university as well as other state resources such as national education laboratories, museums, and other NSF projects." (Inverness Research Associates, ARSI External Review, 2000) Rural areas often have difficulty attracting and retaining mathematics and science teachers. This results in a large number of teachers teaching "out of field" and generally these teachers are unfamiliar with current resources for standards based mathematics and science instruction. A recent study by the Kentucky Department of Education showed that fully a third of the teachers in Kentucky lack the necessary mathematics background and certification to teach middle school content (Clements, Hartanowicz, and White, 1998). In many of the ARSI districts, the percentage is even higher. The ARSI teacher partner has been a major factor in improving the qualifications of mathematics and science teachers in the participating school districts. The social norms of rural areas value family, place, and community over other priorities. The school in a rural community is often the "center" for community activities. (deYoung & Lawrence, 1995, Herzog & Outtnmabm 1995, Nachtigal, 1982, Stern, 1994) Recognizing this importance, increasing "community engagement," has also been a major objective of the ARSI project. |
The Program Improvement Review: A Tool for Assisting Schools in Identifying Science and Mathematics Program NeedsSince the article, "Top-Down, Routinized Reform in Low-income, Rural Schools: NSF's Appalachian Rural Systemic Initiative," was primarily a critique, be it uninformed, of the Program Improvement Review process, it is important that the procedures utilized and the training program be explained.The Program Improvement Review is a program assessment process developed to provide schools an "outside" view of their programs as measured against a set of clearly identified standards. The process involves a site visit to the school by a team of trained observers who collect data through interviews with the school principal, teachers, parents, and students, classroom observations, review of the school's curriculum, review of instructional resources, and review of testing procedures and data. A classroom observation instrument is used in the Program Improvement Review which guides the reviewer's observations related to student-teacher and student- student interactions. Student engagement and interaction, as well as the teacher's questioning strategies, are critical pieces of the data collected related to inquiry based instruction. Following the site visit, the school is provided a written summary of the site visitors' observations including recommendations for making improvements in the instructional program. Debriefing with the site visit team occurs immediately following the visit. It takes approximately 10-12 hours to draft a report. After meeting with team members, editing, and publishing the report, the report is delivered to the school in 2-4 weeks. Although reported in the article that "The final report, usually written overnight and presented the next day," there has never been a case in which the report was generated overnight and presented the next day. The Program Improvement Reviews are based on "recognized good practice" and national standards as identified in a set of clearly defined look-fors. The look-fors are translated into a set of standards which help the reviewer collect data from a variety of sources. The procedures utilized are modeled after the procedures designed by Fenwick English in his Curriculum Auditing process as utilized by PDK and site visit procedures developed as part of the U.S. Department of Education's Blue Ribbon Schools Program. The approach is not unlike the procedures utilized by the Southern Association of Colleges and Schools (SACS), North Central Association, or other such accrediting agencies. The primary difference between the Program Improvement Review and these types of programs are the Program Improvement Review's specific emphasis at the program level. No claims regarding "...easy-to-understand, easy to evaluate nature of education achievement in rural Appalachian Schools," have ever been made by ARSI or the developers of the Program Improvement Reviews. Quite the contrary. The reviews are only one piece of assessment data utilized in assisting schools develop both short-range and long-range plans for improvement. The Program Improvement Reviews were developed as a result of a specific need identified by local school districts. The standards and sub-standards are based on the classroom practices of experienced math and science educators and are consistent with standards as specified by NCTM and National Research Council. The Program Improvement Reviews, as designed and utilized in the ARSI project, have never been used to evaluate a school or a school program. The ratings, comments, and recommendations are a synopsis of the "one-day snapshot" and designed to provide schools with insight not normally found by "self-evaluations," questionnaires, or other routinely used procedures. The instrument utilized in West Virginia was developed by West Virginia educators. The procedures described in the article, "Top-down, Routinized Reform in Low-Income, rural Schools: NSF's Appalachian Rural Systemic Initiative," are specific to the West Virginia process which, as initially implemented, is vastly different from the Program Improvement Review process utilized in other ARSI states. The project team at Marshall University developed their own procedures and instrument specific to West Virginia. ARSI gave permission to this team to adapt the instrument and, although much different the West Virginia instrument is referred to as a Program Improvement Review. Because of the relatively short time that Program Improvement Reviews have been utilized, approximately 5 years, definitive results are just now being identified. Data are being compiled which shows clearly the impact of the Program Improvement Review Process on individual school reform efforts as part of the ARSI project. In addition to individual school and district data, a database is currently being developed to identify trends among all schools reviewed and the specific needs of schools across Appalachia. As stated, the Program Improvement Review process is an evolving one, based on identified best practices and formulated with much input from school clients, both present and future. Science and Mathematics Program Improvement Review Training ProgramThe "formal" training session consists a 6-hour session focusing on the various aspects of the process including interviews, classroom observation, and data analysis. The training day begins with an introduction to the process including the assumptions as well as the research and practice basis for the procedures utilized. A simulation is utilized to prepare reviewers for conducting the on-site interview sessions. To insure consistency in classroom observation reports, a significant amount of time is spent on the observation and scripting of a classroom setting via videotapes. This is followed by a comprehensive analysis of the participants' observations, a review of student assessment data and how this data is utilized, and a time for reflecting on actual school data for the purpose of preparing a summary report. In regard to the extended description of a "videotape segment" in the training tape in "Top-Down, Routinized Reform in Low-Income, Rural Schools: NSF's Appalachian Rural Systemic Initiative", it is important to note that this part of the scripted observation is approximately 2 minutes long out of a 30 minute training tape.This formal training session is followed by a "shadowing experience" in which the "trainee" participates in the data collection process and assists with writing various sections of the summary report. In regard to the quality of the report provided the school, it has proven to be very important that potential reviewers participate in all phases of the site visit and report writing process prior to assuming the role of a program reviewer. It is also important to note that the West Virginia project (described in "Top-Down, Routinized Reform in Low-Income, Rural Schools: NSF's Appalachian Rural Systemic Initiative") requested that they be allowed to deviate from the normal training program. Although against its better judgement, ARSI complied with this request. ConclusionsARSI's model, using a team approach to systemic reform, has produced desired results, namely, standards based instruction in mathematics and science, implementation of supportive policies, convergence of resources for mathematics, science and technology education improvement, a broader base of community support, and increased student achievement.The four main intervention approaches—Catalyst Schools and Teacher Partners, Program Improvement Reviews, Community Engagement, and Resource Collaboratives/University Partnerships—recognize the importance of "bottom-up" strategies for school reform in rural schools. Among these interventions it has been stated: "The Program Improvement Review and Planning Process may be the most important of all the intervention strategies used by ARSI." (Smith, 1999-2000) The Program Improvement Review does not operate in a vacuum. ARSI has focused on "school-based" leadership in the form of the ARSI teacher partner supported by the local district team consisting of the school principal, ARSI district liaison, and district superintendent. The ARSI resource collaboratives have served this model through the provision of professional development for the teacher partners, assistance in the identification of quality mathematics and science instructional resources, provision of leadership training for principals, and development of networks with universities and other professionals who can assist in school reform efforts. The development of a skilled and committed leadership for mathematics and science program improvement has been one of the most significant results of the ARSI project to date. Because of ARSI's training, the district teams now have a "standards- based vision" of mathematics and science instruction which is providing direction for district reform efforts. It is also apparent that ARSI's focus on K-12 students through the development and support of catalyst schools and leadership of the teacher partner has resulted in improved student achievement. Both aggregated state data and individual school data indicate the positive effects of the ARSI project. Because of the success obtained, ARSI catalyst schools are beginning to serve as models for other schools in their district further validating the project's potential for school reform in the Appalachian Region. The data, obtained after four and one-half years of ARSI activity, clearly indicate that ARSI is a major partner in the school improvement process for low-income rural schools in Appalachia. ReferencesEnglish, Fenwick. (1988). Curriculum Auditing. Lancaster, PA. Technomic Publishing Co., Inc. Kannapel, Patricia J. and DeYoung, Alan J. (Fall, 1999). The Rural School Problem in 1999: A Review and Critique of the Literature. Journal of Research in Rural Education. Volume 15, No. 2, 67-79. Smith, Keith. (Winter, 1999-2000). Implementing Systemic Change for Math and Science in Rural Appalachia: The Appalachian Rural Systemic Initiative. The Rural Educator. Volume 21, No. 2, 3-8. St. John, Mark, Brown, Samantha, Carroll, Becky, Hirabayashi, Judy, Horsch, Elizabeth, and Howard, Mike. (2000). Appalachian Rural Systemic Initiative (ARSI): Final Report. External Evaluation Report. About the AuthorsDr. Stephen A. HendersonDr. Henderson is currently Director for the Appalachian Rural Systemic Initiative. Dr. Henderson has extensive experience as a classroom teacher, school administrator, and university science educator in Michigan, Virginia, and Kentucky.
Dr. Wimberly C. Royster |
Copyright 2000 by the Education Policy Analysis ArchivesThe World Wide Web address for the Education Policy Analysis Archives is epaa.asu.edu General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu . EPAA Editorial Board
|