Interrogating the Generalizability of Portfolio Assessments of Beginning Teachers: A Qualitative Study

Main Article Content


This qualitative study is intended to illuminate factors that affect the generalizability of portfolio assessments of beginning teachers. By generalizability, we refer here to the extent to which the portfolio assessment supports generalizations from the particular evidence reflected in the portfolio to the conception of competent teaching reflected in the standards on which the assessment is based. Or, more practically, “The key question is, ‘How likely is it that this finding would be reversed or substantially altered if a second, independent assessment of the same kind were made?’” (Cronbach, Linn, Brennan, and Haertel, 1997, p. 1). In addressing this question, we draw on two kinds of evidence that are rarely available: comparisons of two different portfolios completed by the same teacher in the same year and comparisons between a portfolio and a multi-day case study (observation and interview completed shortly after portfolio submission) intended to parallel the evidence called for in the portfolio assessment. Our formative goal is to illuminate issues that assessment developers and users can take into account in designing assessment systems and appropriately limiting score interpretations.


Download data is not yet available.

Article Details

How to Cite
Moss, P. A., Sutherland, L. M., Haniford, L., Miller, R., Johnson, D., Geist, P. K., Koziol, S. M., Star, J. R., & Pecheone, R. L. (2004). Interrogating the Generalizability of Portfolio Assessments of Beginning Teachers: A Qualitative Study. Education Policy Analysis Archives, 12, 32.
Author Biographies

Pamela A. Moss, University of Michigan

Pamela A. Moss is an Associate Professor in the School of Education at the University of Michigan. Her areas of specialization are at the intersections of educational assessment, validity theory, and interpretive social science. She can be reached at 4220 School of Education, University of Michigan, Ann Arbor, MI 48109-1259 (

LeeAnn M. Sutherland, University of Michigan

LeeAnn M. Sutherland is an Assistant Research Scientist at the University of Michigan ( Her work focuses on adolescent literacy and identity, particularly as students make sense of school discourse vis-à-vis their everyday experiences.

Laura Haniford, University of Michigan

Laura Haniford is a doctoral candidate in Educational Foundations and Policy at the University of Michigan ( She specializes in teacher education, especially multicultural education. Her research focuses on the ways in which classroom discourse influences learning opportunities.

Renee Miller, University of Michigan

Renee Miller is a doctoral candidate in the School of Education at the University of Michigan ( She specializes in Science Education and Museum Studies.

David Johnson, University of Michigan

David Johnson is a Ph.D. student in Educational Studies at the University of Michigan ( His research interests include the influence of government policy on teachers and students and how students make meaning of their state-mandated testing experiences.

Pamela K. Geist, Denver, Colorado

Pamela K. Geist is an educational consultant in Denver, CO ( She specializes in mathematics education.

Stephen M. Koziol, University of Maryland

Stephen M. Koziol, Jr. is Professor and Chair of the Department of Curriculum and Instruction at the University of Maryland ( He specializes in English Education, program design and policy in teacher education, and teacher assessment.

Jon R. Star, Michigan State University

Jon R. Star is an Assistant Professor in the College of Education at Michigan State University ( His research focuses on students¹ learning of middle and secondary school mathematics, particularly the development of mathematical understanding in algebra.

Raymond L. Pecheone, Stanford University

Raymond L. Pecheone is an Academic Research and Program Officer in the School of Education at Stanford University ( He specializes in the design and implementation of complex performance assessment systems. Previously Dr. Pecheone was Bureau Chief of Curriculum, Research, Testing and Assessment for the Connecticut State Department of Education. He was co-director of one of the first assessment development labs for the National Board for Professional Teaching Standards.