This report summarises the results of a year-long collaboration between the OECD Secretariat and researchers in 14 countries (16 country-language contexts) focused on rescoring a sub-sample of student responses to select tasks from the PISA 2022 Creative Thinking assessment. The purpose of the “PISA CT Rescoring project” was to conduct further research on the rich data contained in student responses to the open-ended PISA creative thinking items, as well as to pilot alternative scoring methods that can be used to measure creative thinking in these types of task and that can serve as examples for national and international assessment approaches.
The PISA CT Rescoring project involved over 100 judges that applied three different scoring methods – a holistic judgment approach, a criteria-based analytical approach, and a conceptual-knowledge mapping –to nearly 38,000 unique responses from 14,830 students that took the PISA 2022 test in 16 country-language contexts. These three approaches conceptualise and prioritise creative value in different ways to each other and to the international scoring rules applied in the PISA 2022 CT assessment. In particular, the new scoring methods enable the most creative responses within this sample to be identified, while the scoring method applied in PISA 2022 primarily focused on distinguishing creative (original) from non-creative responses. As such, the PISA CT Rescoring project does not aim to replicate the PISA 2022 results but rather to explore alternative ways of evaluating open-ended tasks that ask students to think creatively.
The first results from this project were described in an earlier report published by the OECD titled Creative Minds in Action, which showcased genuine student responses to the PISA 2022 creative thinking tasks that were judged to be amongst the most creative in the sample across country-language contexts.
In this report, Seven Questions About Creativity and Creative Thinking, we analyse the results from the PISA CT Rescoring project, together with the PISA 2022 test and questionnaire data. These data allow us to investigate key questions related to measuring creative thinking across different socio-cultural contexts. While cultural differences exist in judges’ subjective evaluations of students work, overall, judges across the country-language groups involved in the Rescoring project all assigned strong importance to the essential criteria defining creativity – that is, appropriateness, originality and value. The rescored data also allow us to examine the extent to which key findings from the report are replicated when focusing on different dimensions of creative value. For example, the rescored data show that girls performed better boys, on average, even when changing how their responses are evaluated (e.g. considering the overall creative value of their responses). Socio-economic differences in performance also remain marked across the different scoring methods, with disadvantaged students particularly struggling in tasks that required longer written responses. These and other findings described in this report confirm the value of the data generated by the PISA test, and indicate that further research using the PISA data can uncover other important lessons for education policy and practitioners that wish to nurture creative talent.