Mode equivalency in PAT: Reading Comprehension

This report details the results of an investigation into the school-based assessment, PAT: Reading Comprehension, which has recently gone online. We wanted to know whether the test items behaved in similar ways in the online and paper-based modes, and whether there was any difference in student scores.

There were two main findings:

1. There was very little difference between how items behaved online and how they behaved on paper. That is, the relative difficulties of items (their locations on the scale) stayed the same, regardless of whether the student read and answered the items online or on paper.
2. While the difficulty level of items relative to each other remained constant across modes, students taking the assessment online scored lower on average than those who took the assessment on paper. The difference was slightly bigger for Māori students and students in decile 1 and decile 2 schools. 

The discussion section of the report looks at possible contributing factors for the difference in average scores. There is also a summary below with advice for schools.

We are happy to answer any questions schools might have about the research. Please contact Cathie or Julie on educationadvise@nzcer.org.nz