Reasonable Decisions in Portfolio Assessment: Evaluating Complex Evidence of Teaching

Main Article Content

Abstract

A central dilemma of portfolio assessment is that as the richness of the data available to readers increases, so do the challenges involved in ensuring acceptable reliability among readers. Drawing on empirical and theoretical work in discourse analysis, ethnomethodology, and other fields, we argue that this dilemma results, in part, from the fact that readers cannot avoid forming the data of a portfolio into a pattern—a coherent "story" or "stories"—in order to evaluate it. Our article presents case studies of readers independently evaluating the same portfolios. We show that even readers who hold a shared vision of effective teaching and who cite much the same evidence can, nonetheless, develop significantly different "stories." Our analysis illustrates that some portfolios are more ambiguous than others and are thus more likely to result in such divergent readings. We argue that more fine grained understandings of portfolio ambiguities and disagreements between readers over "stories" can help us respond to the challenges posed by the rich data of portfolio assessments.

Downloads

Download data is not yet available.

Article Details

How to Cite
Schutz, A., & Moss, P. A. . (2004). Reasonable Decisions in Portfolio Assessment: Evaluating Complex Evidence of Teaching. Education Policy Analysis Archives, 12, 33. https://doi.org/10.14507/epaa.v12n33.2004
Section
Articles
Author Biography

Pamela A. Moss, University of Michigan

Pamela A. Moss is an Associate Professor in the School of Education at the University of Michigan. Her areas of specialization are at the intersections of educational assessment, validity theory, and interpretive social science.

Most read articles by the same author(s)