Low response rates are a potential source of non-response bias. Non-response bias can occur if non‑respondents are systematically different (in terms of their level of education, for example) from those who agree to participate in the survey.
Achieving strong survey response rates is a growing challenge around the world. In addition, the nature of the ECEC sector in many countries creates further challenges for surveying this population. For instance, in many countries, the ECEC system is fragmented, with oversight and responsibility for the sector spread across several ministries and across levels of governance, with municipalities often having an important role. This creates challenges for even establishing a sampling frame at the national level in many places (as reflected in the high exclusion rates, in some cases) and complicates the process of contacting ECEC settings once the frame has been established. In addition, the private sector has a significant role in providing ECEC in many countries, which can limit the ability of central authorities to motivate participation. These challenges are compounded by the fact that countries often face ECEC staff shortages and ECEC settings are typically small: these features place an additional strain on recruiting ECEC settings to participate, as ECEC leaders may be wary of asking their staff to take the time to complete a survey.
Despite the many challenges, countries and subnational entities participating in TALIS Starting Strong 2024 invested important efforts in reaching their populations of ECEC staff and leaders, ultimately achieving a range of response rates (see Annex B). Nonetheless, data for ECEC systems that did not meet technical standards with respect to response rates are annotated (*) in tables and figures:
pre-primary settings (ISCED level 02): Germany, Ireland and Norway
settings for children under age 3: the Flemish Community of Belgium, Ireland, New Brunswick (Canada), Norway and Quebec (Canada).
New Zealand also did not meet the TALIS 2024 technical standards with respect to participation rates; data from New Zealand are annotated separately (**) due to sparse responses that led to unanticipated exclusions for some segments of the target population, as described above. As a result of these broad exclusions, the samples could not be adequately adjusted to represent ECEC settings and staff in New Zealand overall (see Annexes B and C). For these reasons, the respondents in the samples from New Zealand represent only respondents included in the sample and not ECEC staff and settings from the whole country. In all other countries and subnational entities included in this report, data are weighted to represent the full population targeted by the survey; that is, the samples are constructed to be representative of ECEC staff and settings in the whole country or subnational entity.
Countries and subnational entities undertook non-response bias analyses to understand the likelihood of bias in the TALIS Starting Strong 2024 samples. These non-response bias analyses did not suggest a direct link between the levels of response rates and the likelihood of bias in the samples, although in many instances this was difficult to estimate due to a lack of data on ECEC staff and leaders from sources other than TALIS Starting Strong. Given the limited data available from or about the population of ECEC staff and leaders targeted by TALIS Starting Strong, information from all participating countries and subnational entities is presented in this report, despite not achieving the technical standards with respect to participation rates for all samples. All other technical standards were met (OECD, forthcoming[2]).
Although countries and subnational entities are flagged (*) only in tables and figures and not in the text of this report (with the exception of New Zealand**), caution is required when interpreting estimates for samples that did not meet the TALIS technical standards with respect to response rates due to a higher risk of non-response bias.