We are pleased to announce that Svend Kreiner from the Dept. of Biostatistics, University of Copenhagen will be presenting his DIF study of the PISA data at the Rasch Day in March.

Is the foundation under PISA solid? A critical look at the scaling model underlying international comparisons of student attainment.

Svend Kreiner

Dept. of Biostatistics, University of Copenhagen

This talk addresses methodological issues related to the scaling model underlying the international comparison of student attainment in the Programme for International Student Attainment (PISA) and, in particular, the question of whether PISA’s ranking of countries is confounded by differential item functioning (DIF). To address these issues, we reanalyze data on reading from the 2006 survey. The analysis provides strong evidence against the fit of PISA’s scaling model and very strong evidence of DIF. The second part of the talk addresses the question of whether country ranks are robust towards the errors of the scaling model. We do this in two different ways. First, by analysis of invariance across different subscales defined by items and second by comparison of ranks based on the scaling model and ranks based on more appropriate models where some of the flaws of PISA’s scaling model have been taken into account. Our results do not support claims that country rankings reported by PISA are robust.