Share

How we measure social and emotional skills

The Study will take a single snapshot of two cohorts of primary and secondary school students, at ages 10 and 15. It will assess students' social and emotional skills directly but also obtain information from their parents, teachers and school principals. This will allow us to understand the home and school contexts in which these skills develop.

Our formula for data quality

.

 

Who are the Study's respondents?

.

How do we assess social and emotional skills?

Triangulation

The Study is using a triangulation approach to assess social and emotional skills of students. More specifically, in evaluating students’ social and emotional skills we combine information from three separate sources of information about these skills: reports provided by students themselves but also by their parents and teachers. This triangulation is an important part of the study for several reasons. First, collecting information from multiple sources and across multiple contexts improves the representation and understanding of students’ behaviours in the most important contexts for school-age students.

Students may behave differently in different settings and choosing information from any one of those settings may provide a somewhat biased representation of students’ social and emotional skills. Additionally, obtaining information from parents and teachers allows us to control for measurement error in self-reports, such as social desirability and unrealistic self-perceptions.

.

 

What are the types of assessment instruments?

 

The Study’s assessment instruments are self- (student) and others’ (parents and teachers) reports on assessed students’ typical behaviours, thoughts and feelings. All three respondents answer the same items but teachers have only three items per assessment scale while students and parents have eight items per scale.

Questions/items are in the form of simple statements such as “I like learing new things” (item assessing students’ creativity) and “I stay calm even in tense situations” (item assessing stress resistance). We use a 5-point Likert type agree/disagree response scale, with answers ranging from 1 – completely disagree to 5 – completely agree. All of the 15 assessment scales use positively and negatively worded items.

These methods are used the most frequently in social and emotional skills assessments. They provide a simple and efficient way to collect information from a large number of respondents, are cost efficient and simple to administer, tend to produce consistent results, and in many cases provide a remarkably high approximation of objective measures.

 

How do we collect information on students’ environment?

.

The study collects information on students' and their parent's background characteristics, as well as on family, school, and community learning contexts through four contextual questionnaires developed for: students, parents, teachers and school principals.

The contextual questionnaires aim to capture the most relevant information that influences students’ social and emotional skills development in line with characteristics of this study that tend to be more responsive to policy interventions and adapting teaching methods.

Sample design

.

How is the Study administered ?

Administration method

• The study is administrated to groups of students belonging to the same cohort in a controlled school setting. 

• A trained study administrator delivers the study, with school staff present. 

• Parents, teachers and principals complete their questionnaires individually.

Assessment mode

• The students fill out the questionnaires online through desktop or laptop devices.

• Parents, teachers and school principals also fill out questionnaires online, but in some participating sites parents can choose a paper and pencil option in case of necessity or personal preference.

• All instruments are provided using a centrally managed online platform. 

online-paper

What are the innovative features in the Study?

Most comprehensive study on social and emotional skills

The Study on Social and Emotional Skills represents the largest assessment of its kind to date. Three separate informants assess 15 social and emotional skills separately for two cohorts of students. This means that the study includes 15 x 3 x 2 = 90 skill assessment scales. Moreover, the study also includes four contextual questionnaires with more than 500 variables, again collected separately for two cohorts, resulting in more than 1 000 contextual variables. Thus, there are about 10 000 bivariate structural relations between skills, various contextual factors and life outcomes for each of the 10 participating cities and countries.

Anchoring vignettes

The Study includes anchoring vignettes in order to enhance the results' cross-country comparability. Anchoring vignettes are a set of questions designed to account for reference bias, which is one of the sources of cross-country incomparability of self-reported measures. When people exhibit reference bias, they answer the same question using different reference points. For example, respondents answer the question “I see myself as someone who tends to be lazy” differently depending on the respondents’ reference point of what it means to be lazy. Therefore, anchoring vignettes are designed to identify a respondent’s reference point when answering the survey questions. Based on the responses to the anchoring vignettes, the respondent’s answers to the survey questions are adjusted to account for these reference points. This could help reduce possible reference bias due to respondents’ different referent points from different countries when answering the same survey questions.

Behavioural indicators

The Study also asks students, teachers and parents to report on students’ behaviours. These indicators are relevant for policy makers as they are concrete manifestations of students’ social and emotional skills, and can be influenced directly. These indicators complement the student, parent and teacher assessments of students’ social and emotional skills, and include questions on classroom behaviour, absenteeism, disruptive behaviour, behaviour with peers and parents, and health-related behaviour.