Introduction

The Ad Hoc Committee is an independent group, whose formation is described in this report. The drafting and publication of this report have been solely the responsibility of the Committee. Nonetheless, we would like to acknowledge the support and help of numerous people and institutions in preparing this report. These include the Commonwealth Education Deans' Council, Boston College, Bridgewater State College, Elms College, Framingham State College, Lesley College, Salem State College, The University of Massachusetts at Boston, and Westfield State College. In particular, we would like to thank the Center for the Study of Testing and Public Policy, which allowed us to use its address as a temporary mailing address for the Committee. Among individuals who have generously assisted us are Irwin Blumer, Mary Brabeck, Joseph Caruso, John Cawthorne, Richard Clark, Bill Dandridge, Patricia Delaney, Anne Harrison, Virginia Harvey, Catherine Horn, Bailey Jackson, Diane Joyce, Joanne McCourt, Patricia O'Brien, Joan Rasool, Maria Sachs, Bob Schaeffer, Kelly Shasby, Dennis Shirley, and Michael Thomas. Also, we thank Larry Ludlow, Ron Hambleton and Dan Koretz who provided helpful reviews of statistical analyses recounted in this report. The report itself is entirely the responsibility of the authors, and has not been sponsored, funded or endorsed by any institutions or individuals who have aided our inquiry. Reviewers of drafts of this report have been generous in offering comments and suggestions, but naturally not all have agreed with all that is written here.
The Ad Hoc Committee was formed out of concern that important decisions were being based on Massachusetts Teacher Tests (MTT) (Note 1) scores without reasonable evidence on their reliability and validity--a clear violation of professional standards concerning testing. These standards, the widely recognized 1985 Standards for Educational and Psychological Testing (Note 2), have been in existence for almost 50 years (in current and previous editions) and have been relied upon in numerous legal proceedings that involved testing in state and federal courts. The very first provision of these standards deals with test validity, requiring that:

Evidence of validity should be presented for the major types of inferences for which the use of a test is recommended. A rationale should be provided to support the particular mix of evidence presented for the intended uses. (Standard 1.1 p. 13).

Other standards call on test publishers or developers to document the reliability of test scores for each total score, subscore, or combination of scores that is reported (Standard 2.1, p. 29); to clearly describe scales used for reporting scores (Standard 4.1, p. 33); and to document the reliability of classification decisions based on licensure or certification tests (Standard 11.3, p. 65). Moreover, Standard 5.1 requires that: "A technical manual should be made available to prospective test users at the time a test is published or released for operational use" (p. 35).
In contravention of these requirements, MTT results are being used to make decisions about prospective teachers in Massachusetts, and about educational policies in the Commonwealth. Our Committee therefore set out to gather evidence on the technical merits of the MTT. Before we recount what information we have gathered and what has been learned from it, we describe the background to our inquiry.