Introduction

SBIA is an ideal–a set of principles to strive for. There are many obstacles to realizing the ideals of SBIA in schools, but progress has occurred both because of education reform initiatives and developments in information technology in education: namely state standards-based reform policies and web-based systems for reporting standards and assessment results. After elaborating more on SBIA, this paper explains and provides examples of how progress is occurring at the state, district, and school levels. At the same time, there is still far to go and obstacles to overcome before more fully realized versions of SBIA operate effectively at the classroom level. The last section of this paper examines some of these obstacles.

The Standards-based Instruction And Assessment Model

Assessment is integral to effective instruction. Good teachers assess the outcomes of their instruction frequently by questioning students and with more formal assessments of student learning, and they use the assessment information in the design of subsequent instruction. On a moment to moment basis, in the context of instruction, good teachers use a variety of questioning techniques to assess student comprehension and developing proficiencies (Angelo, 1998; Tanner, 2001). Over the longer term, from lesson to lesson, unit to unit, and marking period to marking period, good teachers systematically collect and manage assessment data on their students. Good teachers also evaluate their impact on students over the course of the year, by examining individual and aggregate growth of their students based on appropriate local and norm-referenced and criterion-referenced assessments. Appropriate assessment data include scores on tests, quizzes, and other diagnostic and standardized assessments, as well as grades and other forms of student information useful for instruction. When assessments reveal students are not achieving desired learning outcomes, instruction is revised accordingly. Planning, instruction, and assessment should constitute an ongoing, integrated cycle.

Standards-based instruction and assessment requires that teachers teaching at the same grade, whether in different classrooms or schools, be guided by the same content and performance standards. As asserted earlier, this is an ideal, a set of principles to strive for. Ideally, expectations for student achievement and standards of curriculum content should not be dependent upon what classroom or school a student happens to be in. The planning and delivery of instruction should be guided by and aim toward clearly specified standards. Whether standards prescribe how well students should be able to read or write by certain grade levels, what topics they should cover in particular courses, or what competencies they should demonstrate at graduation, in principle the standard is the same for all students–that is what “standard” means. The antithesis of standards-based instruction is each teacher independently deciding what will be covered in his/her classroom and the level of achievement expected of the students. Without standards, expectations for student knowledge and achievement and instructional goals are necessarily based on the idiosyncratic preferences of individual teachers.

Fully realized at an organizational level, standards-based instruction and assessment (SBIA) requires that the planning, instruction, and assessment cycle be vertically integrated among the school’s different levels: student, classrooms, supra-classroom groups, whole school (McEwan, 1998; NEC, 1985; Otto, Wolf & Eldrigde, 1984). Standards of achievement should be clearly understood and operate school-wide, with adequate data to monitor performance and processes at organizational and sub-organizational levels. The model assumes that teachers have a common vocabulary to plan instruction and reflect on practice; plan instruction from a shared understanding of content and achievement standards; evaluate students based on principles and expectations that are consistent across students, groups of students, and periods of time; and participate in instructional management, such as monitoring the effectiveness of instructional programs and identifying instructional and resource allocation priorities for short and long-term planning. Research shows that schools vary widely in their ability to achieve SBIA at an organizational level–with leadership being a critical variable–and that schools that are able to implement effectively principles of SBIA are likely to be better schools (Archbald & Newmann, 1992; Kaufmann, 1992; Levine & Lawrence, 1992; Leithwood & Aitken, 1995; McEwan, 1998; Sanders, 1999; Senge, 2000; Wohlstetter, Kirk, Robertson & Mohrman, 1997).

A Gap Between the Standards-based Instruction and Assessment Model and Practice

The standards-based reform movement has occurred in large part because many people believe there has been, and perhaps continues to be, a significant gap between instruction in practice and the principles of SBIA. Building in the mid-80s and continuing throughout the 90s, the education standards movement resulted in widespread adoption in almost all states of written standards of content and performance and standards-based testing programs to measure student achievement and school performance. Before turning to some of the accomplishments of standards-based reform in education, let us examine some of the conditions that lead to it.

Certainly one of the most significant factors has been the glaring disparities in achievement and presumably academic standards among schools, especially between schools serving lower-income and/or minority populations and schools with more middle-income, non-minority populations. While these disparities have been fodder for numerous newspaper articles and government reports, one notable study with evidence of disparities in standards among schools was reported by the U.S. Department of Education’s Office for Educational Research and Improvement. This study (OERI, 1994) found students in low-income schools getting “mostly As” in reading and math, while students matched on reading and mathematics ability but enrolled in schools with low percentages of kids in poverty were getting “mostly Cs.” The obvious implication is that reading and mathematics ability levels sufficient to get high grades in lower-income schools, would likely produce only "Cs" in schools with more higher-income children.

Other studies have examined more directly variation in teacher expectations and instructional practices among schools and classrooms. A review of observational research on reading instruction found “substantial variation across teachers” on such variables as academic focus, classroom management problems, and time-on-task in reading, with predictable relationships with reading achievement (Rosenshine & Stevens, 1984: 782). Likewise, a national survey of reading practices found “considerable variation” among teachers in reported teaching philosophies, learning goals, and instructional practices (Baumann, Hoffman, Duffy-Hester & Ro, 2000).(Note 2) A study of a large sample of teacher-made tests and quizzes from middle school general mathematics courses found teachers’ tests varying widely in their level of alignment with the standards published by the National Council of Teachers of Mathematics. Some teachers’ tests were composed of more than two-thirds “single-step/single-solution” computation problems, while for other teachers the percentage was as low as one-third, with the majority of their test questions involving higher levels of mathematical reasoning and problem solving. Teachers varied even more in the extent to which test questions involved contextualized problems, mathematical representations, and written explanations of mathematical reasoning (Archbald & Grant, 2000). A similar study by Senk, Beckman and Thompson (1997) found that teachers ranged greatly in their reliance on multiple-choice formats for their tests from a low of 3% to a high of 42% of tests used in their courses. The percent of “low-level” items (answers can be computed in one or two steps) among tests ranged from 55%–90%. The data clearly suggest that teachers’ views of what constitutes adequate achievement and their instructional and assessment practices vary widely.

As the above studies indicate, the particular teacher to which a student is assigned matters a lot. A growing body of research indicates that “differential teacher effectiveness is a strong determinant of differences in student learning, far outweighing the effects of differences in class size and heterogeneity” (Darling-Hammond, 2000). The teacher to which a student gets assigned to a large degree determines the content s/she will cover, the pedagogy s/he will experience, and the academic standards s/he will be held to. In the absence of clear and consistent standards and standards-based assessment within the school, it stands to reason that there is likely to be more idiosyncratic–“standards-free and data-free”–instructional planning and assessment, contributing to disparities in achievement expectations and the quality of instruction (Otto, Wolf & Eldridge, 1984; Valencia & Wixson, 2000). SBIA should be able to reduce the number of students held to substandard academic expectations, subjected to poorly planned and sequenced instruction, and socially promoted through the grades with inadequate skills and undiscovered learning deficiencies.

Another factor fostering support for standards-based education reform policies has been the perception of deficient instructional management at the school and district level, perceived in part to be a product of the absence of agreed-upon standards and measurement of results, and limited accountability for results.(Note 3) An example of deficient measurement systems is revealed in this anecdote from an administrator in a New York district. She describes the following attempt to use data to make an important decision related to the effectiveness of a sixth grade accelerated math program:

To find out about the students in accelerated math, I had to pursue several data sources: the current roster of students (middle school database), sixth grade scores on state exams (cumulative folders in the middle school guidance office), designation of mastery for eighth-graders on their sixth grade tests (testing and assessment database, located in the high school), and analysis of the scores against the current class lists.

That work, which required several weeks of effort, suggested some interesting findings. The initial insight was technological in nature: We lack an integrated database that could help us answer questions such as who has access to accelerated courses. …A question from a parent opened the door to review of our math program and the realization that otherwise helpful data were too scattered to be immediately useful (King, 2000: 19).

Lacking clear standards and good assessment data, the concern is that too many teachers and administrators fall prey to easy, but unexamined, assumptions that standards are high, teachers are teaching effectively, and students are achieving at acceptable levels (Litow, 1999; Powell, Farrar & Cohen, 1985; Sanders, 1999; Sizer, 1984). If a school has neither clear grade level achievement standards nor standards-based assessments of achievement, it is hard to know whether instruction is effective. It also creates conditions in which ineffective teachers can be buffered from accountability or pressures to improve, while exemplary teachers go unrecognized, with the outcomes of their efforts remaining invisible within the organization. Substandard instruction and learning outcomes are not easily changed if they are not easily identified. Without benchmarks against which to gauge performance and data to support analysis of practice, many schools are rudderless and unable to do much about it.

Whether standards-based reforms can fully resolve these problems remains unclear, and is probably unlikely. There is little research specifically on this question, but findings of studies which can speak to this question support that the absence of clear standards and instructional planning information is an obstacle to school effectiveness. A study comparing successful and unsuccessful schools implementing school-based management found that the successful managers of reform had “access to a wide variety of information on student, staff, and school performance, and used the information to guide decision making, to provide feedback to school constituents, and to enhance organizational learning” (Wohlstetter et al., 1997: 213). Other studies emphasize the costs deficient information and bad management: A large study of a systemwide school-based management initiative in Chicago delegating more authority and management responsibility to schools found that most schools did not improve, and a very large proportion, as much as one-third, actually got worse with planning and decision-making in schools crippled by conflict, inadequate information, and poor decision-making (Bryk, 1999). The efficacy of standards-based reform is still more assumed than documented, but there is hope that the gap between SBIA and practice in schools may be lessened by state laws implementing clearer and more ambitious standards, improved assessments of achievement, and stronger accountability sanctions.

Standards-based Reform and Information Technology:
Making Standards and Performance Outcomes Clearer

Educational System Goals Are Clearer

As states have rewritten and revised these standards over time, the progression has been inexorably one way: states have made content and performance standards more comprehensive and more specific and strengthened their statutory authority. Over that past several decades state-prescribed education goals have become more comprehensive and specific. Content and performance standards have been prescribed at increasing numbers of grades in the K-12 sequence, in increasing numbers of subjects, and with broader scope of coverage at each grade level. In addition, state curriculum documents and policies are increasingly prescribing instructional procedures with examples to illustrate exemplary forms of student achievement (Archbald, 1999). To illustrate, compare a set of goal statements excerpted from U.S. History from Texas’s standards, 1985 versus their current standards:

This kind of change has occurred in virtually all states (additional examples below). Educational goals, then, have been made clearer, more specific, and more uniform throughout state education systems. In fact, research indicates goals have become more uniform among states within the country (Gandal, 1997; Joftus & Berman, 1998; Raimi & Braden, 1998). A major component of standards-led education reform has been the production of influential national curriculum reports.(Note 4) As state curriculum committees across the country worked on revising and upgrading their own documents and standards they often looked to the national curriculum reports for guidance while sharing information through networking, conferences, and the like – activities which many observers believe have lead to greater uniformity of standards among states.

Performance Measurement Has Improved

First, there was growing recognition of the limitations of exclusive reliance on norm-referenced comparisons (Linn, 2000; Shepard, 1990). State testing programs became more criterion-referenced, with scores of students and school mean scores reported in relation to fixed benchmarks derived from published content and performance standards. In addition to reporting, “we are at the 55th percentile,” a school would report, “our average score is 3.2 on the 5 point scale, where 3.0 is the threshold described as ‘at standard’.”

Second, exclusive reliance on multiple choice formats gave way to mixed formats, tests with both multiple choice items and constructed response items. This in theory makes the tests more valid for measuring problem solving and knowledge integration. Many more states also started using writing tests in which students produce short essays graded by readers.

Third, states moved toward more analytical measures of school performance reporting. Educators and researchers have long understood that a school’s test scores are a function not just of the quality and effectiveness of its staff and programs, but also of the socioeconomic backgrounds of its students. Therefore, comparing schools on “raw” descriptive statistics, such as the mean of their students’ scores on a particular test, is not particularly useful (Meyers, 2000; Willms, 1992). High scoring schools almost always have low percentages of low-income students; vice versa for the low performing schools. By the late 80s many states began using more analytical measures of school performance. One approach is to adjust for student socioeconomic background and other characteristics considered exogenous to the school, so schools’ scores would be compared only to other demographically similar schools. Another approach (not mutually exclusive with demographic adjustments) is to report and evaluate schools’ scores on the basis of achievement gains (or losses) over time. Within these broad analytical reporting strategies there are a variety of more specific statistical methods. The main point, though, is over that last fifteen years state education agencies shifted toward the reporting of measures of school performance that are significantly more advanced than mere reporting of “raw” descriptive statistics (Linn, 2000).

Fourth, school performance measurement has increasingly included information beyond test scores. Other measures of performance have also been added to states’ systems for evaluating schools. These include graduation rates, percentages of students taking the SAT, and enrollments in college prep courses.

Clarifying Goals, Disseminating Performance Information, and Creating More Accessible Information

Prior to the advent of the Internet and computer networking for electronic data interchange, information about goals and performance was distributed exclusively in paper reports. Curriculum guides were printed and bound by state education agencies and then distributed to school district office personnel by mail or at conferences of district curriculum coordinators; district personnel in turn had the responsibility of distributing these documents to schools. It was generally expected that at least one copy would be available in schools’ administrative offices and hoped that each teacher would have his/her own set, but this often depended upon the administrators’ predilections and photo-copying resources. The same general approach operated with respect to reports of school and school district performance on state tests, although unlike curriculum documents, test data (typically in the form of magnetic tape) would eventually also be sent to school districts.

Paper-based, manual distribution of information still occurs–however, there is a parallel process today of electronic information exchange. Virtually all school and district level decision-makers now have access to electronic information. Over 95% of schools have computers connected to the Internet and local area networks (NCES, 2000; NCES, 2001).(Note 5) The infrastructure is in place, then, to profoundly improve access to decision-support information about educational goals, standards, and performance. Throughout the nation, states and districts are using this IT infrastructure to do just that. The examples below are illustrative of typical web-based EDI concerning educational standards and school performance.

Information on state content and performance standards is accessed typically through a four step process of web navigation: (1) the state education agency website; (2) the office responsible for curriculum/instruction; (3) the link(s) to “standards;” (4) the link(s) to particular content areas and grade levels. Increasingly, standards documents are displayed in portable document format, but are also in rich text and ascii text formats. This maximizes their accessibility.

Figure 1 shows Virginia’s web-based menu box for specifying the subject area and grade level standards.

Figure 2 shows a selection from the “Grade Eight Standards for Learning” from Virginia’s content/performance standards documents, which are accessed on the web in portable document format.

Most states with web-based access to standards documents have relatively similar processes of access.

Distributing, revising, accessing, and using standards documents on the web is far easier and more efficient than managing paper reports. There are obvious efficiencies of distribution and revision. The information moves electronically instead of in cartons in trucks, and can be revised without incurring large re-distribution costs. (This is not to say frequent revision is a good idea–far from it.) More important from the perspective of SBIA, is that users always have access to the information. Teachers planning lessons or committees planning curriculum can immediately access the specific standards and areas of content they need for guidance. They can do it from home. Parents also can access the information. No one has to try to remember where their curriculum document is stored and ask for a new copy when they cannot find it.

Information on student and school performance is also increasingly on-line. Following are a few examples. Figure 3 shows the Delaware State Testing Program home page. The home page provides a great deal of background information on the state testing program, including an explanation of the scales used to represent student and school performance, sample items used in the state tests, and an on-line newsletter about the testing program and related assessment topics. Figure 4 shows the menu of selections for users to produce on-line reports of school level test scores. A subsequent menu (not shown) allows users to select particular schools and to disaggregate results by student categories (race, gender, special ed, etc.). Figure 5 shows a report from a similar system in Arizona. The portable document format report of a selected school provides not only test score information, but information on staff, student enrollments, curriculum specialties, the school academic calendar, expenditures, discipline, and a other items of information. Figure 6 shows a hyper-text markup language report produced from data for the Chicago school district. This web-based system produces reports reflecting the results of value-added analyses computed at the school-level. The reports show whether a school’s productivity trend over the last nine years is upwards, downwards, or flat. These examples are illustrative of the kind of school performance information that has become widely available on the web.

The web-based information is intended for anyone with an interest in the performance of schools in a given locale (region or district) or in the performance of a particular school. Teachers, principals, and parents can evaluate the performance of “their” school in relation to standards and in relation to the performance of other schools; school board members, civic leaders, and policymakers can evaluate the performance of groups of schools in which they have an interest. The information reveals how well students are achieving in schools, identifies needs for improvement, and depending upon the quality of the measurements, which schools are performing adequately and which schools are not. This creates a more efficient and rational basis for instructional management decision-making, and for allocating rewards, sanctions, and resources aimed at school improvement.

Before computer networking and electronic information exchange, performance data on schools was locked exclusively into paper reports that moved slowly through the organization, typically down the organizational hierarchy, from one office to the next. Often it was four or five months after testing before test score reports and other school reports were distributed throughout the organization. At the school level, administrators would typically have responsibility for disseminating test results among school faculty–a process carried out in a wide variety of ways, with widely varying efficiency, and often involving a lot of photo-copying. The same tasks and difficulties of paper report management described earlier with standards documents also affects the distribution and use of test score reports.

The web and EDI have changed this. Now, routinely, information from statewide testing is available over the web at the same time to everyone–teachers, parents, school principals, guidance counselors, district office curriculum coordinators, personnel supervisors, superintendents, and school board members. Many states and districts also make individual student achievement information available to authorized users (e.g., the child’s teacher, guidance counselor, principal, and district data specialists). This has created substantially improved access to information for teacher-parent conferencing, for IEP meetings for special education students, and for instructional planning.

Some Conclusions About Progress Toward SBIA

The Next Challenge: Integrating SBIA Into Instructional Practice

We do know that IT can play an important role in facilitating standards-based reform and the implementation of SBIA within schools at the level of instructional practice. Relational data-base management technology and computer-based instructional management systems have gone far toward making SBIA technically feasible. The rapid rise of vendors and software programs for instructional management attests to this. There are a variety of instructional management and information technology systems now available. The goal of these systems is to use computer technology to improve the development and management of lesson plans, student academic records, and information about students’ instructional experiences. Reflecting principles of standards-based reform, these instructional management systems almost always have features to link lesson and unit objectives to larger curriculum content and performance standards, whether these standards come from district, state, or national sources. These systems are also designed to manage student achievement records, such as grades and test scores, as well as to record instructional experiences such as lessons and tasks completed, books read, essays written, and the like.

While there is very little research specifically on the extent to which instructional management systems are effective in achieving the goals of SBIA within schools and classrooms, available research and theory suggest effectiveness will depend upon several factors:

Leadership And School Culture

Often, these conditions do not occur. Instead of trust, teachers or principals may be suspicious that information will be used against them or simply unmotivated to pursue the advantages computers may bring (KHEC, 2000; Stiggins, 2001). Such fears are reinforced when test scores are publicly reported and lead to simplistic media proclamations about “poor teaching and failing schools” or are used inappropriately by uninformed district officials. Another factor militating against the development of cultures conductive to SBIA is past negative experiences with instructional management or student information technologies (Rosen & Weil, 1995). It was not until well in to the 90s that faster CPUs, greater storage capacity, faster connections, better graphical user interfaces and other IT advances made more user-friendly systems possible. During the developmental days of computers in the 80s and even into the 90s untold many schools and districts across the country hastily adopted computer-based instructional management and student information systems sold with promises and attractive presentations, only to find the systems actually complicated, cumbersome, time-consuming, and difficult to use. Frustrating early experiences with computers have left many personnel in schools skeptical about new “technology solutions” promised by technology advocates outside the school.

Personnel Technical Proficiency

Despite the ubiquity of computers in schools, teachers and administrators are still predominantly of the pre-computer generation. Many are still reluctant computer users and limited to fairly simple procedures, such as word processing or web-browsing (Cooley, 1998; OTA, 1995). Compared with prescribed standards of proficiency (e.g., Coughlin, 1999; ISTE, 2000), the proportion of teachers and principals proficient for more advanced computing uses remains relatively small. With respect to the proficiencies required for data-driven decision-making, for instance, a prominent ULCA education research center describes school decision-makers as “woefully underprepared to use data [to document results] and for planning and decision-making purposes” (Linn & Baker, 1999).(Note 7) Several reports indicate that school districts generally give insufficient attention and resources to training teachers to gain needed proficiencies with educational technology–about half as much as they should (MDR, 2000; OTA, 1995).

Technical Obstacles

All schools have some turnover of students over the course of the school year and many schools have high levels of student turnover. According to one national estimate, one out of six children will attend three schools by the time they are in third grade (GAO, 1994). It is therefore essential to develop procedures to insure that student databases are current and easy to update. When a new student arrives in a school, the registration information must be entered immediately into the data system and appropriate records up dated. It is a waste of time and discourages reliance on data for planning and decision making if teachers or administrators attempt to use a system and frequently encounter missing or out-of-date information.

A closely related issue is linkages between databases. Functionality decreases to the degree that an information management system is unable to pull together current information from different databases. If student demographic and registration information cannot be easily linked with student achievement information and student achievement information cannot be linked with curricular or discipline information, the usefulness of the data declines markedly. It is not enough that the data exist, they must be cross-referenced and easily queried by users.

A third issue concerns the quality of instructional and achievement data. Relatively detailed information is needed for instructional planning. Ideally, SBIA requires at least three types of information: standards-based assessment information, at least once per marking period, from tests and tasks closely aligned with the school’s curriculum; state/district achievement test information; and instructional information at the student level related to curriculum objectives. State testing information by itself is inadequate. As Clements (2000) observed:

The decision support systems now under development generally contain the accountability data, because they are the data that are available. These data are considered to be of sufficient quality to warrant widespread use for school and district comparisons and to highlight where effective practice seems to be occurring. While these data may be useful to some decision-makers looking for places where assistance is needed, the data appear to have limited use to teachers and principals seeking to improve what they are doing. (p. 3).

The variables above are some of the most important that will determine the benefits to be derived from the “next generation” of IT-based instructional management systems for schools. Most schools and districts are still struggling with the challenges of converting existing paper-based data systems to electronic formats and connecting multiple, separate databases to create more integrated, warehoused data management systems. Today in most districts, school-based personnel can access electronically (a) information on state curriculum content standards and schoolwide profile reports and (b) information on individual students from student records. However, the information of type (a) is in aggregate form and has been compiled by others, while information of type (b) is typically limited to registration, scheduling, and report card information and is difficult to aggregate and analyze. Further, it is typically not the case that all teachers can access information of types (a) or (b) easily from their classrooms on an “as needed” basis. The goal of “next generation” IT systems is to surmount the school-level leadership, personnel proficiency, and technical variables to implement systems that can place at the teachers’ fingertips the information necessary to fully realize the vision of SBIA (Means, 2000). This can bring standards-based reform more effectively to the classroom level. Then, perhaps, we will see payoffs in rising student achievement.

Notes

General questions about appropriateness of topics or particular articles may be addressed to the Editor, Gene V Glass, glass@asu.edu or reach him at College of Education, Arizona State University, Tempe, AZ 85287-0211. (602-965-9644). The Commentary Editor is Casey D. Cobb: casey.cobb@unh.edu .