Beyond Proficiency

Using Log Files to Understand Respondent Behaviour in the Survey of Adult Skills

In series:OECD Skills Studiesview more titles

Published on April 18, 2019

Computer-based administration of large-scale assessments makes it possible to collect a rich set of information on test takers, through analysis of the log files recording interactions between the computer interface and the server. This report examines timing and engagement indicators from the Survey of Adult Skills, a product of the Programme for the International Assessment of Adult Competencies (PIAAC), both of which indicate large differences across countries and socio-demographic groups, in the amount of time spent by respondents and their levels of disengagement, which reduce the probability of giving a correct answer and consequently reduces measured performance. Such insights can help policy makers, researchers and educators to better understand respondents’ cognitive strategies and the underlying causes of low and high performance. This, in turn, can help improve the design of assessments and lead to more effective training and learning programmes.


Executive summary
What log files are and why they are useful
Timing indicators in the Survey of Adult Skills
Allocation of time to different items in the Survey of Adult Skills
Measuring disengagement in the Survey of Adult Skills
The PIAAC LogDataAnalyzer
Powered by OECD iLibrary

Education GPS - The world of education at your fingertips

Survey of Adult Skills (PIAAC 2015): Full selection of indicators

Adult Skills in Focus

Adult Skills in Focus is a series of policy-oriented notes designed to describe the results from the OECD Survey of Adult Skills in a concise, user-friendly way.


How computer-based tests are enriching education research