This chapter explores students’ learning environments, both within and outside school. The learning environment at school includes the role of teacher support, classroom disciplinary climate, student engagement and parental involvement at school. In Eastern Partnership (EaP) economies, students who receive greater teacher and family support tend to achieve higher scores in the OECD Programme for International Student Assessment (PISA). The chapter also looks at the use of digital devices for learning and leisure at school, assessing whether they act as a distraction during lessons. Additionally, it describes how EaP countries managed school closures during the COVID-19 pandemic and highlights the main issues students face in adapting to remote learning.
Education in the Eastern Partnership: Findings from PISA
Abstract
The learning environment encompasses what happens in classrooms and schools, from the disciplinary climate to teaching practices. What happens outside of school, during students’ free time and with family at home is also an important component. This chapter investigates the learning environment from both perspectives, both within and outside of schools. Finally, it draws lessons from the way learning continued during the COVID-19 pandemic (Figure 4.1). Different aspirational benchmarks, such as Finland and Japan, are used depending on the topic discussed as in this report’s other chapters. The chapter also shows countries that illustrate the range of possible values (e.g. highest and lowest) to put results for Eastern Partnership (EaP) countries/economies into context.1
The first section of this chapter provides insights from the OECD Programme for International Student Assessment (PISA) 2022 on the learning environment in EaP countries/economies. According to students, in EaP countries/economies, teacher support and disciplinary climate in mathematics lessons are, on average, better or similar to the OECD average. An exception is the disciplinary climate in Baku (Azerbaijan)2, which is worse. However, more students skip classes or whole days of school and arrive late – which is strongly associated with lower student performance – in EaP systems than on OECD average. While student truancy and lateness have improved since 2018, further improvements are possible. The use of digital devices at school does not appear to be as conducive to learning as it could be in EaP countries/economies. Limiting distractions is important for student performance and well-being.
The chapter highlights parental involvement as a strength in EaP countries/economies. According to school principals, it shows that parental involvement in students’ learning at school is higher in EaP countries/economies than in OECD countries, except in Moldova. Students in EaP countries/economies, most noticeably in Baku and Georgia, perceived their families as more supportive than did students in most OECD countries. However, parental involvement at school decreased substantially between 2018 and 2022 in all EaP countries/economies except Georgia. EaP countries/economies should aim to strengthen school-family partnerships and keep parents involved in students’ learning.
The second section of this chapter analyses how learning continued during the COVID-19 pandemic. This is an important context for understanding student learning and well-being, as analysed in this report. According to student reports, the analysis shows that in EaP countries/economies, fewer students experienced short school closures during the COVID-19 pandemic than on average across OECD countries. This is positive because it likely contained the extent of learning losses in EaP countries/economies. As highlighted in Chapter 2, performance generally declined in EaP systems between PISA 2018 and PISA 2022. In addition, EaP countries/economies had stable or improving trends in terms of their sense of belonging at school, whereas, in countries with longer school closures, the sense of belonging dropped between PISA 2018 and PISA 2022.
Looking at students’ experiences during the pandemic also allows drawing lessons on strengths and weaknesses that are still relevant today for building more resilient and inclusive education systems in EaP countries/economies in the future. Students in EaP countries/economies faced different kinds of problems with remote learning when their school building was closed due to the COVID-19 crisis Although EaP students tend to feel confident in self-directed learning, many still struggle to motivate themselves to do schoolwork and understand school assignments. A comparatively smaller but still substantial share of students had problems with logistical aspects such as having Internet access or finding a quiet place to study. This shows the importance of further preparing students for autonomous learning, which includes students’ self-directed learning skills. As shown in this chapter, supportive teaching practices can nurture students’ confidence in their capacity for self-directed learning. Developing more supportive teaching practices in EaP countries/economies could entail strengthening personalisation in teaching and learning and improving relationships between teachers and students.
Teacher support, disciplinary climate and student truancy
Teacher support during classroom lessons
In 2022, PISA surveyed students about teacher support in their mathematics lessons. Students were asked how often their teachers show interest in each student’s learning, give extra help, assist with learning and continue teaching until understanding is achieved. Responses ranged from “never or hardly ever” to “every lesson”. These responses were combined to create a teacher support index. The average of this index is 0, with a standard deviation of 1, across OECD countries. A higher index value indicates that students perceive their mathematics teacher as more supportive.
Students in EaP countries/economies generally felt similarly or more supported by teachers in their mathematics lessons than students in OECD countries. For example, 75% of students in Baku and Georgia responded that teachers show an interest in every student’s learning, compared to 63% across OECD countries (Figure 4.2). Costa Rica and Poland (included in the figure) were the OECD countries with the highest and lowest values respectively, in the teacher support index in PISA 2022.
Students who feel more supported by their teachers in mathematics lessons score higher in mathematics and experience less anxiety towards mathematics. A one-unit increase in the teacher support index corresponds to an average improvement in mathematics performance of five score points across OECD countries. Teacher support has a similar positive association with mathematics performance in Baku, Georgia and Moldova, although the results were not statistically significant in Ukrainian regions (18 of 27)3. Additionally, more teacher support is linked to reduced mathematics anxiety in all EaP countries/economies except Moldova.
Disciplinary climate during classroom lessons
In all EaP countries and economies, students in classes with a better disciplinary climate generally outperformed those in classes with frequent disciplinary issues. This is the picture observed in most countries and economies that took part in PISA 2022. Moreover, countries and economies with better average disciplinary climates showed better overall mathematics performance. This was the case even after accounting for countries’/economies’ level of economic wealth in gross domestic product (GDP) per capita (OECD, 2023[1]).
Percentage of students who reported that the following happens in their mathematics lessons for most or every lesson
Note: Countries and economies are ranked by ascending order of the percentage of students who reported that their teacher shows an interest in every student’s learning for most or every mathematics lesson.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Change in mathematics anxiety and mathematics performance associated with a one-unit increase in the index of teacher support
Notes: Statistically significant values are shown in darker tones.
The results are based on linear regression analysis that accounts for students' and schools' socio-economic profile. The PISA index of economic, social and cultural status (ESCS) measures the socio-economic profile.
Countries and economies are ranked in descending order of the change in mathematics performance associated with a one-unit increase in the index of teacher support.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
According to the PISA index of disciplinary climate,4 students in Georgia and Ukrainian regions benefit from a better disciplinary climate in mathematics lessons than the average OECD student. However, their climate was not as good as that of 12 other countries and economies, including Japan. By contrast, students in Baku suffer from a worse disciplinary climate, even though it is still better than 29 other countries and economies, including Finland.
As shown in Figure 4.4, comparatively few students in Georgia and Ukrainian regions suffer from noise and disorder in their classrooms or feel that students do not start working for a long time after the lesson begins. In Baku, by contrast, about a third of students have to wait a long time for teachers to start class while students quiet down and feel that students cannot work well.
Percentage of students who reported that the following happens in their mathematics lessons during most or every lesson
Note: Countries and economies are ranked in order of the percentage of students who reported that there is noise and disorder in most or every lesson in their mathematics classes.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Student truancy and lateness
Between 2018 and 2022, EaP countries/economies saw a notable decrease in student truancy, particularly in Georgia and Ukrainian regions. A significant decrease in lateness is also observed in all EaP countries/economies, except Georgia. Decreases in truancy and lateness during this period are also observed on average across OECD countries.
Nevertheless, students in EaP systems tend to miss school or come late to class more than the average student in OECD countries (Table 4.1). Among countries and economies participating in PISA 2022, Japan had the lowest levels of lateness and truancy, while Montenegro had the highest level of lateness.
Percentage of students who reported that the following happened at least once in the two weeks prior to the PISA test
Note: Countries/economies are sorted in descending order of students arriving late for school in PISA 2022.
Decrease in PISA 2022 compared to PISA 2018.
Difference is not significant.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
In EaP countries/economies – except for Baku – student truancy and lateness were strongly associated with lower mathematics performance, as in most countries and economies that took part in PISA 2022 (Figure 4.5). This highlights that students who miss school and arrive late lose critical learning opportunities and support. In Moldova, Ukrainian regions and on average across OECD countries, students who skipped a whole day of school at least once in the 2 weeks prior to the PISA test scored about 30 score points less than their peers after accounting for their socio-economic profile. When it comes to lateness, students who reported that they had arrived late for school at least once in the 2 weeks prior to the PISA test underperformed their peers by between 9 (Georgia) and 20 (OECD average) points.
In high-performing countries such as Finland, Japan and Singapore, the difference in mathematics performance among students who skip school or arrive late to school is even more significant than the OECD average. Top-performing countries likely use time more effectively in the classroom; this could partly explain why learning losses are greater for students who skip school or arrive late. It is also possible that the pressure on students to excel academically can exacerbate the consequences for those who disengage by skipping or arriving late.
Baku, together with Türkiye, is one of two education systems in PISA 2022 where, contrary to expectations, students who skip school perform better than their peers in mathematics, even after accounting for students’ and schools’ socio-economic profiles. While PISA data do not allow for a conclusive explanation, one possible reason is that rules about truancy are not clear for students or are not effectively enacted by schools in Baku. This hypothesis is supported by the very large share of students (one in four) who skip school and by the fact that truancy in Baku is pervasive across socio-economic groups (i.e. no significant differences between advantaged and disadvantaged students). Furthermore, Baku is one of the few education systems where students in higher grade levels (i.e. upper secondary education) are significantly more likely to be truant than students in lower levels (i.e. lower secondary education). This suggests that truancy worsens as students’ progress in their education.
Change in mathematics performance when students reported that they skipped a whole day of school and arrived late for school at least once in the two weeks prior to the PISA test
Notes: All changes in mathematics performance are statistically significant before and after accounting for students’ and schools’ socio-economic profiles, except for Baku where lateness was unrelated to mathematics performance.
Countries and economies are ranked in ascending order of the change in mathematics performance before accounting for students’ and schools’ socio-economic profiles.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Use of digital devices
Use of digital devices for learning and leisure at school
How education systems adapt to technological change and whether policy makers can balance risks and opportunities is crucial in defining the effectiveness of education systems (OECD, 2015[3]). The use of digital devices in schools varies widely across education systems. In EaP countries/economies, overall usage was highest in Ukrainian regions and lowest in Georgia. Students in Georgia used digital devices for 2.6 hours per day, whereas those in Ukrainian regions used them for 4.6 hours per day (Figure 4.6). For comparison, the usage of 7 education systems, including Finland, exceeded 4 hours per day. Meanwhile, in 14 education systems, including Japan, students spent less than 2.5 hours per day on digital devices. In Ukrainian regions, the high number of hours on digital devices can be attributed to the school closures to ensure student safety in the context of war. While schools started to reopen in early 2022 after the school closures to respond to the COVID-19 pandemic, the country’s school system operated again on line from February 2022 to October 2022.
In Baku and Ukrainian regions, the two countries with the highest total time students spend on digital devices, students spend more time on digital devices at school than on regular school lessons. This might again be related to the need for students to use digital devices to participate in remote lessons in Ukrainian regions. The same picture is observed in ten countries/economies participating in PISA 2022. It is likely that students who spend more time on their digital devices than on regular classroom lessons are using their devices during most of their classroom time, which in turn can lead to distraction from instruction. This might partly explain that in all ten countries/economies where this was the case, student performance in mathematics is significantly below the OECD average.
What students do on their devices in this time at school also differs (Figure 4.6). They may use them for learning or leisure. In Baku and Ukrainian regions, the two EaP economies with the highest amount of time spent on digital devices, students spend a slightly higher share of their total time on digital devices for learning. By contrast, students in Georgia and Moldova, the two EaP countries/economies with less total time on digital devices, spend a slightly lower share of their time using digital devices for learning activities. Across OECD countries, students spend more than twice as much time on digital devices for learning than for leisure, at an average of 2 hours per day compared to 1.1 hours.
Looking at the use of digital devices for leisure at school, all EaP countries/economies spend between one and one and a half hours per day on digital devices, which is about the same as the OECD average.
Time spent per day by students (in hours)
Notes: Time spent in regular lessons at school per school day refers to the time spent in regular lessons per school week divided by five (assuming there are five days per school week).
Countries and economies are ranked in descending order of the time spent using digital devices at school for learning and leisure.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
How does using digital devices for learning or leisure relate to learning outcomes? Moderate use of digital devices in school for learning is related to higher performance across OECD countries. Students who spend up to 1 hour per day on digital devices for learning activities in school scored 25 points higher in mathematics than students who spend no time on such devices. This positive relationship between one hour of learning using digital devices and performance is also observed in Moldova, as well as in over half of the education systems with available data. However, it is not associated with higher performance in other EaP countries/economies. In Baku, Georgia and Ukrainian regions, there is no difference in performance between students who do not use digital devices for learning and students who spend up to one hour per day on digital devices for learning.
The relationship between digital devices and academic performance becomes negative, however, when students spend more than 1 hour per day on digital devices for learning in school; for example, on average across OECD countries, students who spent between 5 and 7 hours per day on digital devices for learning activities in school scored 12 points lower than students who spent between 3 and 5 hours per day. Students who spent over 7 hours per day on digital devices for learning activities in school scored even lower. A similar pattern is observed in all EaP countries/economies.
More time using digital devices for leisure rather than instruction is also associated with poorer results. On average, across OECD countries, students who spent a small amount of time (i.e. up to 1 hour per day) on digital devices for leisure activities scored more than 20 points higher in mathematics than students who spent more than 2 hours and up to 3 hours, and 40 points higher than students who spent more than 3 hours and up to 5 hours using digital devices for leisure at school. A similar pattern is again observed in all EaP countries/economies.
Distraction from digital devices during classroom instruction
One of the reasons why only moderate use of technology contributes to learning is that using digital devices at school can lead to distraction in the classroom. Three out of ten students reported that students in their classes get distracted using digital devices in most or every mathematics lesson in Baku, Georgia, Moldova and, on average, across OECD countries. Furthermore, a somewhat smaller but still significant proportion of students reported digital distraction in most or every mathematics lesson by other students who are using digital devices (Figure 4.7). Distraction due to using digital devices is very rare in Japan and more frequent in Finland.
Percentage of students who reported that the following happens in every or in most of their mathematics lessons
Note: Countries and economies are ranked in ascending order of the percentage of students who reported that students get distracted by using digital devices in every or most of their mathematics lessons.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Parental involvement at school and family support at home
Parental involvement at school
Parental involvement in students’ learning at school is generally higher in EaP countries and economies than in OECD countries, except for Moldova, according to school principals. In Baku, Georgia and Ukrainian regions, more than 35% and almost 60% of students were in schools where most parents initiated discussions about their child’s progress with a teacher (Figure 4.8).
By this measure, Georgia is among the countries and economies with the greatest parental involvement at school. Parental involvement is still only higher in Kazakhstan, the Philippines and Viet Nam. By contrast, Finland and Japan, which are used as benchmarks for a number of indicators in this report, have rather low levels of parental involvement at school.
Between 2018 and 2022, however, parental involvement in students’ learning at school decreased substantially in all EaP countries/economies except Georgia. The share of students in schools where most parents initiated discussions about their child’s progress with a teacher dropped by about 30 percentage points in Baku and Moldova and 17 percentage points in Ukrainian regions, versus 10 percentage points on average across OECD countries (Figure 4.8).
Percentage of students in schools whose principal reported that at least 50% of students’ parents are involved in discussing their child’s progress with a teacher on their own initiative
Notes: Changes between PISA 2018 and PISA 2022 that are statistically significant are shown in a darker tone. In addition, they are shown in brackets under the country/economy name. In Georgia, the change is not significant.
Countries and economies are ranked in descending order of the percentage of students in schools whose principal reported that at least 50% of students’ parents are involved in discussing their child’s progress with a teacher on their own initiative in PISA 2022.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Education systems with positive trends in parental engagement in student learning between 2018 and 2022 showed greater stability or improvement in mathematics performance. This was particularly true for disadvantaged students. These figures show that the level of active support that parents offer their children might have a decisive effect.
Family support at home
Students in EaP countries, most noticeably in Baku and Georgia, perceived their families as more supportive than students in most OECD countries.5 Only students in Portugal and five other countries felt their families were even more supportive than in Baku. By contrast, the country with the lowest level of perceived family support was Japan (Figure 4.9).
Index of family support (based on students’ reports)
Note: Countries and economies are ranked in ascending order of their mean value in the PISA index of family support.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Students with more family support at home perform better in mathematics in all EaP countries and economies, similar to what is true on average across OECD countries. A one-unit increase in the index of family support is associated with an increase of between 7 and 13 score points in EaP countries and economies (Figure 4.10). Only in Albania and five other countries/economies is the increase in performance steeper than in Moldova, the EaP country with the strongest positive association.
However, family support at home is not always associated with higher performance. In Japan (also in Germany and Sweden), students with more family support at home actually perform worse in mathematics than their peers. This might seem counterintuitive. While PISA data do not provide evidence of a causal relationship nor the potential direction of the relationship, in some contexts, such as possibly in Japan, parents might provide more support at home when their children perform poorly in school.
Change in mathematics performance associated with a one-unit increase in the index of family support
Notes: All changes in mathematics performance associated with a one-unit increase in the index of family support are statistically significant.
Countries and economies are ranked in ascending order of the change in mathematics performance.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
PISA 2022 took place as countries were still grappling with the impacts of the COVID-19 pandemic. The pandemic presented a major challenge for education, testing the extent to which systems, schools and students around the globe were able to adapt to sudden and profound changes in how students are taught and how they learn. It is thus important to understand how it affected teaching and learning practices as a contextual factor. At the same time, it provides insights into potential strengths and weaknesses that might need to be addressed to build resilient and inclusive school systems in the future.
School closures, defined as the period in which school buildings were closed to students, were one of the most common responses to the pandemic to contain the spread of the virus. PISA 2022 collected information on learning during school closures from both students and principals. However, data on school closures collected through the PISA student questionnaire suffer from specific limitations, namely low response rates and different biases. Readers should consider these limitations when drawing conclusions from the results presented in this part of the chapter on learning during school closures. Annex 4.A explains these limitations in detail, and given these, readers need to take care in drawing definitive conclusions from this analysis.
School closure policies differed between countries/economies (OECD, 2021[4]).6 As a result, the duration of school closures, defined as the closure of the building itself, as analysed here, does not capture all the time that individual students were not permitted to enter the school building. As mentioned above, the situation in Ukrainian regions is particular, as COVID-19-related school closures were compounded by school closures related to the war in the country. As a result, students in Ukrainian regions studied remotely for a longer period than in other countries. The support schools provided to students also varied, depending on when and for how long schools were closed. Schools in education systems where closures were relatively rare and brief may have provided fewer supportive actions since schools may have resumed in-person classes before support was considered necessary. In these cases, the values on the indicators for school support may be low.
Table 4.2 shows how EaP countries/economies supported students and schools during the pandemic. In some countries, notably Ukraine, the response was defined at the local level, in others, such as Moldova, at the central level. In Baku and Georgia, some measures were dropped as the pandemic progressed, while in Moldova, measures were largely maintained and new ones were put into place.
Information for school years 2020/21 and 2021/22
x: Not applicable; .. : Missing value or not available.
Source: OECD (2023[5]), Annex B3, PISA 2022 system-level indicators. PISA 2022 Results (Volume I): The State of Learning and Equity in Education, PISA, OECD Publishing, Paris, https://doi.org/10.1787/53f23881-en.
The relation between school closures and student performance and well‑being
According to students, in EaP countries/economies, fewer students experienced short school closures than on average across OECD countries. To measure the length of school closures, PISA 2022 asked students whether their school building was closed to them for more than a week in the previous three years due to COVID-19(some schools closed and reopened multiple times during the period). Schools in most countries/economies were closed for several months because of the pandemic.
PISA 2022 then makes a distinction between “short” and “long” school closures and assesses the share of students reporting “short” closures. Short closures are defined as those for less than three months and long closures for more than three months. On average, around half of the students across OECD countries experienced short closures.
In Baku, Georgia and Ukrainian regions, less than half of students experience short closures (Figure 4.11). Conversely, more students in Moldova experienced short school closures than their peers across OECD countries and more than 60% of students experienced closures for 3 months or less. When interpreting these results, keep in mind that it is not possible to establish with confidence whether these are accurate estimations of the length of school closures in EaP countries/economies: only 33% of students in Baku and less than 70% of students in Georgia and Ukrainian regions (60% and 67% respectively) responded to the question on the length of school closure during the COVID-19 pandemic (see Annex 4.A).
Length of COVID-19 school closures as reported by students
Note:
* Caution is required when interpreting estimates because one or more PISA sampling standards were not met (see Box 1.1 in Chapter 1).
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
While no causal relationships can be established with PISA data, the results suggest that the duration of school closures is related to differences in student performance. Education systems that spared more students from long closures (longer than three months) showed higher average mathematics performance than those with more schools closed for longer periods (Figure 4.12).
Furthermore, shorter school closures are positively related to students’ well-being, as measured, for example, by their sense of belonging at school. Countries/economies that avoided long school closures for more of their students, according to students, had more stable or improving trends in their sense of belonging at school (Figure 4.13). A clear example is Japan, which closed its schools for only 3 months or less to 84% of its students, as reported by students, had one of the greatest improvements in students’ sense of belonging at school, reaching a level above the OECD average in 2022.
EaP countries/economies deviate somewhat from the general pattern. The sense of belonging at school did not change between PISA 2018 and PISA 2022 in Moldova, the EaP country where a smaller proportion of students suffered from long school closures. In Baku and Georgia, where a larger proportion of students suffered from longer school closures, sense of belonging at school improved by a small yet significant margin during this period.
An even larger difference from the general pattern can be observed in Ukrainian regions, where a sense of belonging at school improved greatly between 2018 and 2022; only Japan and Montenegro saw a greater sense of belonging than in Ukrainian regions. It is plausible that schools and school communities became a source of social-emotional protection for students in the context of war, which could explain the increase in student sense of belonging in Ukrainian regions; in addition, it is possible that this increase is due to changes in the composition of the PISA sample between 2018 and 2022. The high non‑response rate to the question on school closures can also partly explain this finding.
COVID-19 school closures and change between 2018 and 2022 in the sense of belonging
Note:
* Caution is required when interpreting estimates because one or more PISA sampling standards were not met (see Box 1.1 in chapter 1).
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Students’ learning experience during school closures
To avoid severe learning losses during school closures due to COVID-19, systems and schools ensured that education continued effectively in remote mode. Remote education forces students to learn more independently – and to draw on self-directed learning skills (Bond et al., 2021[5]). The following sections describe the problems with remote learning and the levels of self-directed learning skills in EaP countries and economies, as reported by students in PISA, and what EaP countries/economies might learn from this in the future. As the analyses suggest, this includes further supporting students’ motivation and engagement in their learning but also helping families address logistical aspects that might hinder student learning, such as adequate Internet access at home or the lack of a quiet place to study. Nevertheless, students in EaP countries/economies report feeling relatively confident in their ability for self-directed learning and are similarly supported by teachers as in OECD countries. The pandemic highlighted teacher support’s role for students, which positively correlates with self-directed learning and mathematics performance.
Problems with remote learning
Students faced different kinds of problems with remote learning when their school building was closed because of COVID-19. Students’ ratings of how often they had various problems completing their school work (e.g. “problems with Internet access”, “problems with understanding my school assignments”) while their school building was closed due to COVID-19 were combined to create the PISA index of problems with remote learning. Higher values in this index indicate that students faced more problems with remote learning during the time schools were closed during COVID-19. Students in all EaP countries and economies had more issues with remote learning than their peers across OECD countries, as indicated by higher values in this index. Problems with remote learning were particularly acute in Baku, which had the second-highest value in this index out of all PISA 2022 participating countries and economies, only after Mongolia. Japan had the least problems among all countries/economies (OECD, 2023[1]).
Percentage of students who reported that, when their school building was closed because of COVID-19, they had the following problems when completing their schoolwork at least once a week
Note: Countries and economies are ranked in ascending order of their value in the PISA index of problems with remote learning.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Figure 4.13 shows the incidence of specific problems with remote learning. Most noticeably, many students struggled to motivate themselves to do schoolwork and understand school assignments. In Baku, almost half of students had problems at least once a week motivating themselves to do schoolwork. A similar share of students had self-motivation problems on average across OECD countries, while only about three in ten students had self-motivation problems in Korea and Moldova. Furthermore, four out of ten students in Baku reported having problems at least once a week with understanding school assignments and finding someone to help them with their schoolwork. In the other EaP countries and economies, and on average across OECD countries, about a third of students had problems at least once a week with understanding school assignments during school closures.
A comparatively smaller but still substantial share of students in EaP countries/economies had problems with logistical aspects such as having Internet access, finding a quiet place to study or finding time to study because of household responsibilities.
Student confidence in self-directed learning
To measure the extent to which education systems prepared students for self-directed learning, PISA 2022 asked students to report on their confidence in their capacity for self-directed learning in case their school building would have to close again in the future. Self-directed learning encompasses two components: using digital technology for learning remotely and taking responsibility for their own learning.
In EaP countries/economies, most students feel confident in their capacity for self-directed learning. Unlike in OECD countries, students feel similar levels of confidence in both their capacity to use digital technology for remote learning and to motivate themselves to do schoolwork. In Baku and Moldova, seven out of ten students felt confident in using digital technology to learn remotely and taking responsibility for their own learning. In Georgia, this was six out of ten students.
On the other hand, students in OECD countries felt significantly more confident about using digital technology for learning remotely than they felt about taking responsibility for their own learning. This is also the case for students in Ukrainian regions, where students feel more confident about using digital technology than motivating themselves. The additional months of school closures in relation to the war could explain why Ukrainian students are more confident with technology but also have lower motivation than others.
Percentage of students who reported feeling confident/very confident in taking the following actions if their school building closes again in the future
Note: Countries and economies are ranked in descending order of the percentage of students who reported feeling confident or very confident in using a video communication programme.
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
Supportive teaching practices can nurture students’ confidence in their capacity for self-directed learning. In EaP countries/economies, and on average across OECD countries, students who responded that their teachers were available for help when they needed them when schools were closed during COVID-19 are more confident in their capacity for self-directed learning, even after accounting for mathematics performance and students’ and schools’ socio-economic profiles (OECD, 2023[1]).
Furthermore, as shown in Figure 4.15, greater average levels of student confidence in capacity for self‑directed learning are found in education systems where a greater share of students reported that their teachers were available when they needed help during COVID-19 school closures. In EaP countries and economies and on average across OECD countries, seven out of ten students reported that their teachers were available when they needed help.
Based on students’ reports of their experience during COVID-19 school closures
Note: Positive values on the vertical axis mean students are more confident in their capacity for self-directed learning.
* Caution is required when interpreting estimates because one or more PISA sampling standards were not met (see Box 1.1 in Chapter 1).
Source: OECD (2022[2]), PISA 2022 Database, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
The strongest link to mathematics performance during COVID-19 school closures was teacher availability to help students when they needed help. After accounting for students’ and schools’ socio-economic profiles, students with access to teacher help when needed scored 15 points higher in mathematics on average. This is compared to students with less supportive teachers across OECD countries (OECD, 2023[1]).
PISA 2022 data on COVID-19 school closures have two main limitations: i) high non-response rates; and ii) response biases. Readers should keep these limitations in mind when drawing conclusions from the results presented in the first part of this chapter on learning during school closures (OECD, 2023[1]).
When responding to a survey, some participants might not respond to a question as they might not have time or prefer not to answer, for example. In the PISA 2022 student questionnaire, many students did not respond to questions about COVID-19 school closures because the questions on this topic were placed at the end of the student questionnaire. This limits the representative nature of these data.
In EaP countries/economies, rates of non-response to the question on the length of school closure were higher than on average across OECD countries, except in Moldova. On average, almost 25% of students across OECD countries did not answer the question about the duration of COVID-19 school closures. In Baku, 67% of students did not answer the question, the largest share among all PISA participating countries. The share of missing data for this question was also very large in Georgia (40%) and Ukrainian regions (33%). Only in Moldova was the share of students who did not answer this question (19%) lower than on average across OECD countries.
A comparison of the characteristics of students who responded to the question on the duration of COVID‑19 school closures with those who did not respond shows that both differ in important ways in EaP countries and economies and, on average, across OECD countries. Non-responding students scored lower in mathematics, reading and science, were of lower socio-economic status and were more often boys than girls.
In addition, the high non-response rate affects the precision of the data since standard errors are higher than for other parts of the questionnaire.
Students’ responses to the questions on school closures are subject to various biases, such as any information from questionnaires. Common biases include social desirability – where students might downplay negative feelings to appear more positive, for example – and cultural bias – where different cultures have varying views on education and school closures. These biases may be more pronounced given the debates about the appropriateness of school closures and other measures to address the COVID-19 pandemic.
In addition, students retrospectively answered these questions on school closures in 2022, possibly recalling events from early 2021. Some students might struggle to remember details about their school’s closure, especially if it happened early in the pandemic. Systemic bias should also be considered since school closures varied in timing and duration across countries. For example, students from countries with long closures might have a different perspective than those with shorter closures.
References
[5] Bond, M. et al. (2021), Global Emergency Remote Education in Secondary Schools during the COVID-19 Pandemic, Center for Open Science, https://doi.org/10.31219/osf.io/7k59g.
[1] OECD (2023), PISA 2022 Results (Volume II): Learning During – and From – Disruption, PISA, OECD Publishing, Paris, https://doi.org/10.1787/a97db61c-en.
[2] OECD (2022), PISA 2022 Database, OECD, Paris, https://www.oecd.org/en/data/datasets/pisa-2022-database.html.
[6] OECD (2021), The State of Global Education: 18 Months into the Pandemic, OECD Publishing, Paris, https://doi.org/10.1787/1a23bb23-en.
[4] OECD (2021), The State of School Education: One Year into the COVID Pandemic, OECD Publishing, Paris, https://doi.org/10.1787/201dde84-en.
[3] OECD (2015), Students, Computers and Learning: Making the Connection, PISA, OECD Publishing, Paris, https://doi.org/10.1787/9789264239555-en.
← 1. The EaP countries/economies forming part of this report are Baku in Azerbaijan, Georgia, Moldova and 18 of the 27 regions in Ukraine. Any reference to EaP countries/economies, as well as the EaP average, specifically pertains to Baku, Georgia, Moldova and Ukrainian regions. Armenia is also part of the EaP but has not yet participated in PISA, although participation is underway for PISA 2025.
← 2. Azerbaijan as a whole country participated in PISA 2006 and PISA 2009, 2006 but has only participated with its capital city, Baku, since PISA 2018.
← 3. Ukraine joined PISA for the first time in 2018. However, in the context of war, only 18 of the country’s 27 regions were able to participate in the 2022 assessment. Box 1.2, included in Chapter 1, provides detailed information on Ukraine’s participation.
← 4. In PISA 2022, students were asked how often (“never or hardly ever”, “some lessons”, “most lessons”, “every lesson”) certain things happen in their mathematics classes (e.g. “students do not listen to what the teacher says” and “there is noise and disorder”). These statements were combined to create the index of disciplinary climate (DISCLIM) with an average of zero and a standard deviation of one across OECD countries. Positive values on the index mean that the student reported a better disciplinary climate in mathematics lessons than did students on average across OECD countries. Mean values in this index for all countries and economies in PISA 2022 are included in PISA 2022 Results (Volume II): Learning During – and From – Disruption (OECD, 2023[1]), Table II.B1.3.9. For EaP countries/economies, mean values in the DISCLIM index are the following: -0.06 in Baku, -0.01 in Moldova, 0.13 in Georgia, and 0.31 in Ukrainian regions (18 of 27).
← 5. In PISA 2022, family support (FAMSUP) was measured by asking students how often (“never or almost never”, “about once or twice a year”, “about once or twice a month”, “about once or twice a week”, “every day or almost every day”) their parents or someone in their family do different things with them indicative of family support (e.g. “discuss how well you are doing at school”; “eat the main meal with you”; or “spend time just talking with you”). An index of family support with an average of zero and a standard deviation of one across OECD countries is formed by combining students’ responses to ten scenarios. Students with positive values on this index perceived their family as more supportive than did students on average across OECD countries.
← 6. In many countries, schools opened for certain grades, levels of education or age groups, often giving preference to students in earlier years (OECD, 2021[6]). School closures were often only imposed in affected regions, schools or classes, not nationwide (e.g. teaching shifted to remote mode for classes where COVID-19 cases were detected or for contact cases within these classes). In some education systems, half of the student body alternated with the other half in attending classes in person.