MODELING NONIGNORABLE MISSING DATA WITH ITEM RESPONSE THEORY (IRT)

<jats:title>ABSTRACT</jats:title><jats:p>Large‐scale educational surveys are low‐stakes assessments of educational outcomes conducted using nationally representative samples. In these surveys, students do not receive individual scores, and the outcome of the assessment is inconsequential for respondents. The low‐stakes nature of these surveys, as well as variations in average performance across countries and other factors such as different testing traditions, are contributing factors to the amount of omitted responses in these assessments. While underlying reasons for omissions are not completely understood, common practice in international assessments is to employ a deterministic treatment of omissions, either as missing observations or as responses that are considered wrong. Both approaches appear problematic. In this project, we analyzed the effects of treating omitted responses either as missing or as wrong, as is done in the majority of international studies, and compared these data‐treatment solutions to model‐based approaches to treating omitted responses. The two types of model‐based approaches used in this study are: (a) extensions of multidimensional item response theory (IRT) with an additional dimension based on response indicator variables defined and calibrated together with the set of items containing the observed responses and (b) multidimensional, multiple‐group IRT models with a grouping variable representing the within‐country stratification of respondents by the amount of omitted responses. These two model‐based approaches were compared on the basis of simulated data and data from about 250,000 students from 30 Organisation for Economic Co‐operation and Development (OECD) Member countries participating in an international large‐scale assessment.</jats:p>