Less Truth Than Error: Massachusetts Teacher Tests

Authors

  • Walt Haney Center for the Study of Testing, Evaluation and Educational Policy, Boston College
  • Clarke Fowler Center for the Study of Testing, Evaluation and Educational Policy, Boston College
  • Anne Wheelock Center for the Study of Testing, Evaluation and Educational Policy, Boston College
  • Damian Bebell Center for the Study of Testing, Evaluation and Educational Policy, Boston College
  • Nicole Malec Center for the Study of Testing, Evaluation and Educational Policy, Boston College

DOI:

https://doi.org/10.14507/epaa.v7n4.1999

Keywords:

Large Scale Assessment, Standardized Tests, Measurement Model,  Authentic Assessments, Performance assessments, Portfolio assessments

Abstract

Scores on the Massachusetts Teacher Tests of reading and writing are highly unreliable. The tests' margin of error is close to double to triple the range found on well-developed tests. A person retaking the MTT several times could have huge fluctuations in their scores even if their skill level did not change significantly. In fact, the 9 to 17 point margin of error calculated for the tests represents more than 10 percent of the grading scale (assumed to be 0 to 100). The large margin of error means there is both a high false-pass rate and a high false-failure rate. For example, a person who received a score of 72 on the writing test could have scored an 89 or a 55 simply because of the unreliability of the test. Since adults' reading and writing skills do not change a great deal over several months, this range of scores on the same test should not be possible. While this test is being touted as an accurate assessment of a person's fitness to be a teacher, one would expect the scores to accurately reflect a test-taker's verbal ability level. In addition to the large margin of error, the MTT contain questionable content that make them poor tools for measuring test-takers' reading and writing skills. The content and lack of correlation between the reading and writing scores reduces the meaningfulness, or validity, of the tests. The validity is affected not just by the content, but by a host of factors, such as the conditions under which tests were administered and how they were scored. Interviews with a small sample of test-takers confirmed published reports concerning problems with the content and administration.

Downloads

Download data is not yet available.

Author Biographies

Walt Haney, Center for the Study of Testing, Evaluation and Educational Policy, Boston College

Walt Haney is a professor in the School of Education and Senior Research Associate in the Center for the Study of Testing, Evaluation and Educational Policy at Boston College. He is former editor of the journal Educational Measurement: Issues and Practice, advisor to the committee that developed the 1985 Standards for Educational and Psychological Testing and author of numerous articles concerning educational testing and evaluation. He has also served as an expert witness in numerous court cases concerning testing.

Clarke Fowler, Center for the Study of Testing, Evaluation and Educational Policy, Boston College

Clarke Fowler is a professor in the Education Department at Salem State College. He has taught both preschool and kindergarten and currently teaches courses in early childhood education.

Anne Wheelock, Center for the Study of Testing, Evaluation and Educational Policy, Boston College

Anne Wheelock, an independent education policy writer and researcher, works for several national foundations and is the author of Safe To Be Smart: Building a Culture for Standards-Based Reform in the Middle Grades (1998).

Damian Bebell, Center for the Study of Testing, Evaluation and Educational Policy, Boston College

Damian J. Bebell is a doctoral student at Boston College where he is employed at the Center for the Study of Testing, Evaluation and Educational Research. His research interests include educational philosophy, alternative forms of assessment, and homeschooling.

Downloads

Published

1999-02-11

How to Cite

Haney, W., Fowler, C., Wheelock, A., Bebell, D., & Malec, N. (1999). Less Truth Than Error: Massachusetts Teacher Tests. Education Policy Analysis Archives, 7, 4. https://doi.org/10.14507/epaa.v7n4.1999

Issue

Section

Articles