Predicting Variations in Math Performancein Four Countries Using TIMSS

Authors

  • Daniel Koretz Harvard University
  • Daniel McCaffrey RAND Education
  • Thomas Sullivan RAND Education

DOI:

https://doi.org/10.14507/epaa.v9n34.2001

Keywords:

Performance, Classrooms, Third International Mathematics and Science Study (TIMSS), Variations, Math

Abstract

Although international comparisons of average student performance are a staple of U.S. educational debate, little attention has been paid to cross-national differences in the variability of performance. It is often assumed that the performance of U.S. students is unusually variable or that the distribution of U.S. scores is left-skewed – that is, that it has an unusually long ‘tail' of low-scoring students – but data from international studies are rarely brought to bear on these questions. This study used data from the Third International Mathematics and Science Study (TIMSS) to compare the variability of performance in the U.S., Australia, France, Germany, Hong Kong, Korea, and Japan; investigate how this performance variation is distributed within and between classrooms; and explore how well background variables predict performance at both levels. TIMSS shows that the U.S. is not anomalous in terms of the amount, distribution, or prediction of performance variation. Nonetheless, some striking differences appear between countries that are potentially important for both research and policy. In the U.S., Germany, Hong Kong, and Australia, between 42 and 47 percent of score variance was between classrooms. At the other extreme, Japan and Korea both had less than 10 percent of score variance between classrooms. Two-level models (student and classroom) were used to explore the prediction of performance by social background variables in four of these countries (the U.S., Hong Kong, France, and Korea). The final models included only a few variables; TIMSS lacked some important background variables, such as income, and other variables were dropped either because of problems revealed by exploratory data analysis or because of a lack of significance in the models. In all four countries, these sparse models predicted most of the between-classroom score variance (from 59 to 94 percent) but very little of the within-classroom variance. Korea was the only country in which the models predicted more than 5 percent of the within-classroom variance in scores. In the U.S. and Hong Kong, the models predicted about one-third of the total score variance, and almost all of this prediction was attributable to between-classroom differences in background variables. In Korea, only 19 percent of total score variance was predicted by the model, and most of this most of this was attributable to within-classroom variables. Thus, in some instances, countries differ more in terms of the structure and prediction of performance variance than in the simple amount of variance. TIMSS does not provide a clear explanation of these differences, but this paper suggests hypotheses that warrant further investigation.

Downloads

Download data is not yet available.

Author Biographies

Daniel Koretz, Harvard University

Daniel Koretz is a professor at the Harvard Graduate School of Education and Associate Director of the Center for Research on Evaluation, Standards, and Student Testing (CRESST). His work focuses primarily on educational assessment and recently has included studies of the validity of gains in high-stakes testing programs, the effects of testing programs on schooling, the assessment of students with disabilities, and the effects of alternative systems of college admissions.

Daniel McCaffrey, RAND Education

Daniel McCaffrey is a Statistician at RAND. His areas of concentration are education policy and the analysis of hierarchical data and data from complex sample designs.

Thomas Sullivan, RAND Education

Thomas J. Sullivan is a statistical programmer/analyst with RAND and a doctoral candidate in Computational Statistics at George Mason University.

Downloads

Published

2001-09-14

How to Cite

Koretz, D., McCaffrey, D., & Sullivan, T. (2001). Predicting Variations in Math Performancein Four Countries Using TIMSS. Education Policy Analysis Archives, 9, 34. https://doi.org/10.14507/epaa.v9n34.2001

Issue

Section

Articles