Results from TALIS 2024

Annex A. Technical notes on sampling, participation rates and adjudication

Sampling procedures

The objective of the Teaching and Learning International Survey (TALIS) 2024 was to obtain a representative sample of teachers for each International Standard Classification of Education (ISCED) level in which an education system participated. The international sampling plan employed a stratified two‑stage probability sampling design. Schools served as the first-stage units (primary sampling units), randomly selected from within strata. Teachers were then randomly selected from the list of eligible teachers within those schools, serving as secondary sampling units. A more detailed description of the survey design and its implementation can be found in the TALIS 2024 Technical Report (OECD, forthcoming[1]).

Education systems had the option to limit the coverage of their TALIS 2024 implementation for reasons of practicality, safety or economy. However, they were encouraged to minimise such exclusions and ensure that their national survey population covered at least 95% of teachers. National Project Managers (NPMs) were required to document the reasons for each exclusion, along with details such as the school’s size, location and student population, for every ISCED level in which the system participated.

Sample size requirements

To ensure reliable estimation and allow for some non-response, the minimum sample size was set at 20 teachers per participating school. A minimum of 200 in-scope schools were to be selected, resulting in a nominal international sample size of at least 4 000 teachers per ISCED level.

Education systems could expand their samples by selecting more schools, more teachers per school or both. In some cases, systems were required to increase the number of teachers sampled within schools to offset selecting too many schools with fewer than 20 teachers.

For systems with a limited number of eligible schools, the sample size requirement was reduced. In a few instances, where the average number of teachers per school was lower than expected, the number of schools sampled was increased to meet the minimum required number of participating teachers.

In many systems, ISCED level divisions do not align with distinct school buildings or administrations. For example, schools covering Grades 8 to 12 span ISCED levels 2 and 3, but may not fully represent all of ISCED Level 2.

If a system participated in multiple modules at different ISCED levels, overlap control was applied to avoid selecting the same schools for multiple modules. While teachers may work across ISCED levels and belong to multiple target populations, it was not feasible for them to complete separate questionnaires for each level. Similarly, principals were not expected to complete multiple questionnaires for different ISCED levels within the same school. As a result, schools were generally selected for only one ISCED level, and only teachers at that level were listed and sampled.

However, in response to requests from two education systems, the selection of the same schools for two modules was permitted. In such cases, principals were required to complete multiple questionnaires, while teachers within each school were still sampled for only one ISCED level.

Definition of teachers and principals

A teacher at ISCED Level 1, 2 or 3 is defined as someone who, as part of their regular duties at a school, provides instruction in programmes corresponding to that ISCED level. Teachers who teach a mix of programmes across different ISCED levels within the target school are included in the TALIS target population. There is no minimum threshold for the amount of instruction a teacher must provide at any of the three ISCED levels to be included.

The international target population of TALIS 2024 includes only teachers who teach regular classes in ordinary schools, as well as the principals of those schools. The following teachers are considered out of scope for TALIS 2024:

• teachers teaching in schools exclusively serving students with special education needs1

• teachers teaching exclusively to adults

• substitute and emergency teachers

• teachers who also act as school principals

• teachers on long‑term leave (e.g. disability, sabbatical)

• teachers who had taken part in the TALIS 2024 field trial.

Non-teaching staff (nurses, school psychologists, teachers’ aides, etc.) are also out of the TALIS 2024 international target population.

A principal is defined as the individual with the highest level of responsibility for the administrative, managerial and/or pedagogical leadership of a school. This role may include overseeing students, supervising teachers, engaging with parents and guardians, and planning, preparing and implementing the school’s pedagogical activities. Principals may also dedicate part of their time to teaching.

Adjudication process

Adjudication is the process of determining whether the data released were “fit for use” as intended. The issues examined concerned, among others, the questionnaire adaptation to national context, translation and verification, quality of the sampling frame, handling of out-of-scope and refusal units (i.e. teachers and/or schools), within‑school sampling, data collection, data cleaning, the reports of quality observers, participation rates and overall compliance with the technical standards.

During the adjudication session, each individual dataset (one per module per education system) was submitted to the same examination. Principal/school data were adjudicated independently of teacher data.

Once each survey process had been assessed, a recommended adjudication rating was formulated. While the rating was mostly dictated by the participation rate thresholds set in the TALIS 2024 Technical Standards (see Table A A.1 and Table A A.2), the adjudication committee at times improved the rating of a dataset if, after expert consideration, unique and favourable conditions were met (e.g. closeness to the threshold value and/or the non-response bias analysis (NRBA) report showing evidence for negligible bias risks).

To note, the adjudication rating is made at the education system level. However, potential non-response bias is specific to each estimate. Therefore, some estimates of systems with an adjudication rating of “good” might still have a high non-response bias; correspondingly, some estimates of systems with an adjudication rating of “poor” might still be reliable. For more detailed information, please refer to the TALIS 2024 Technical Report (OECD, forthcoming[1]).

The adjudication rules, based on participation rates for principals and teachers, are displayed in Table A A.1 and Table A A.2.

Table A A.1. Adjudication rules for school or principal data

School/principal participation before replacement	School/principal participation after replacement	Risk of non-response bias	Rating
≥75%	≥75%		Good
≥50% but <75%	≥75%		Fair (A)
≥50% but <75%	≥50% but <75%	Low	Fair (C)
≥50% but <75%	≥50% but <75%	High	Poor (D)
<50%			Insufficient

Note: A school was deemed a participating school if the principal returned their questionnaire with at least one question answered.

Source: TALIS 2024 Technical Standard 3.26.

Table A A.2. Adjudication rules for teacher data

School participation before replacement	School participation after replacement	Teacher participation after school replacement1	Risk of teacher non-response bias	Rating
≥75%	≥75%	≥75%		Good
≥75%	≥50% but <75%			Fair (A)
≥50% but <75%	≥75%	≥75%		Fair (B)
≥50% but <75%	≥75%	≥50% but <75%	Low	Fair (C)
≥50% but <75%	≥75%	≥50% but <75%	High	Poor (D)
≥50% but <75%	≥50% but <75%			Poor (E)
<50%	≥75%			Poor (F)
<50%	<75%			Insufficient

Note: A school was deemed a participating school if at least 50% of the selected teachers returned their respective questionnaires with at least one question answered.

1. It refers to teachers’ participation rate in participating schools.

Source: TALIS 2024 Technical Standard 3.27.

The following bulleted list aims to help data users understand what constitutes limitations on use or quality of the data:

• Good: the data of the participating system can be used for all reporting and analytical purposes and should be included in international comparisons.

• Fair (line A): national and subnational estimates can be produced; some teacher/staff characteristics may be less precise, as indicated by a larger standard error (s.e.), hence the warning “fair”, but with no additional warnings to users deemed necessary.

• Fair (line B, only for teacher data adjudication): national and subnational estimates can be produced; some subnational estimates may be of lower precision (larger s.e.) if the sample size is locally low, hence the warning “fair”, but with no additional warnings to users considered necessary.

• Fair (line C): national and subnational estimates can be produced; some subnational estimates may be of lower precision (larger s.e.) if the sample size is locally low, hence the warning “fair”, but with the possible inclusion of a note on data quality that points to the outcome of the NRBA; school participation somewhat lower than under (B), meaning that comparison of subnational estimates needs to be done with care given that some of these results are based on just a few schools; comparison of small subnational estimates with similar groups from other education systems is unlikely to uncover statistically meaningful differences because of potentially overly large standard errors.

• Poor (line D): in addition to the warnings issued for the previous category, a note that warns users of higher risks of non-response bias in some estimates should be appended; comparisons of subnational estimates need to be limited to the groups with the larger sample sizes (because the sample at this point represents between 37% and 56% of teachers/staff from a relatively small sample of schools, comparisons with similar groups in other education systems is inadvisable).

• Poor (line E, only for teacher data adjudication): subnational estimates are not recommended; a note pointing out the difficulty of obtaining a representative sample of schools, therefore, needs to be appended.

• Poor (line F, only for teacher data adjudication): limitations similar to those for line E, but with the inclusion of a note pointing out the difficulty of obtaining at least 50% participation of the selected sample of schools; evident risk of having a non-representative sample of schools.

• Insufficient: weights should not be calculated for any official tabulations, meaning that data should not be incorporated into international tables, models, averages, etc.

Participation rates and adjudication ratings

The participation rates and adjudication ratings for each participating education system, by module and population, are presented in Table A A.3, Table A A.4, Table A A.5, Table A A.6, Table A A.7 and Table A A.8. Detailed results of unweighted and weighted participation can be found in Annex G of the TALIS 2024 Technical Report (OECD, forthcoming[1]).

Table A A.3. ISCED Level 2: Principal participation and recommended ratings

Participating education system	Number of participating principals	Estimated size of school population	Principal participation before replacement (%)	Principal participation after replacement (%)	Recommended rating
Albania	282	1 090	99.6	100.0	Good
Alberta (Canada)	139	1 152	38.5	50.3	Insufficient
Australia	181	2 575	63.0	89.9	Fair
Austria	266	1 446	83.6	96.7	Good
Azerbaijan	222	4 090	99.7	100.0	Good
Bahrain	115	116	99.1	99.1	Good
Belgium	262	1 226	65.0	73.4	Fair
Brazil	212	56 460	89.9	94.3	Good
Bulgaria	208	1 691	97.7	98.7	Good
Chile	362	5 776	70.0	84.3	Fair
Colombia	189	15 310	63.8	68.5	Fair
Costa Rica	183	939	85.7	87.1	Good
Croatia	259	857	87.8	87.8	Good
Cyprus	99	100	99.0	99.0	Good
Czechia	346	2 819	99.8	100.0	Good
Denmark	189	1 555	54.9	80.9	Fair
Estonia	192	367	94.1	94.1	Good
Finland	226	723	99.6	100.0	Good
Flemish Community (Belgium)	139	791	57.1	66.2	Fair
France	171	6 780	86.0	87.0	Good
French Community (Belgium)	123	436	76.0	83.8	Good
Hungary	231	2 665	93.5	93.9	Good
Iceland	109	140	77.9	77.9	Good
Israel1	198	1 232	92.5	93.0	Good
Italy	198	5 799	97.5	98.0	Good
Japan	200	10 277	89.5	98.1	Good
Kazakhstan	371	6 676	99.1	99.1	Good
Korea	173	3 085	65.3	85.6	Fair
Kosovo	222	630	99.1	99.1	Good
Latvia	213	514	96.3	97.9	Good
Lithuania	211	738	98.7	100.0	Good
Malta	50	55	90.9	90.9	Good
Montenegro	130	161	80.7	80.7	Good
Morocco	398	3 860	99.7	100.0	Good
Netherlands	78	1 345	35.6	52.3	Insufficient
New Zealand	90	1 871	27.2	51.4	Insufficient
North Macedonia	201	335	99.6	99.6	Good
Norway	147	1 093	44.5	69.7	Insufficient
Poland	279	12 502	93.6	99.6	Good
Portugal	224	1 195	98.8	99.3	Good
Romania	216	4 587	100.0	100.0	Good
Saudi Arabia	319	7 743	95.6	95.6	Good
Serbia	199	1 129	98.5	99.5	Good
Shanghai (China)	205	689	100.0	100.0	Good
Singapore	154	189	95.6	96.9	Good
Slovak Republic	239	1 608	95.7	99.0	Good
Slovenia2	207	443	77.4	79.1	Good
South Africa	318	7 806	84.4	84.6	Good
Spain	494	7 151	97.7	97.9	Good
Sweden	183	1 693	86.4	88.8	Good
Türkiye	226	16 723	98.5	98.5	Good
United Arab Emirates	210	875	98.8	98.8	Good
United States3	152	59 531	48.0	59.4	Insufficient
Uzbekistan	210	10 485	100.0	100.0	Good
Viet Nam	202	11 439	100.0	100.0	Good

Note: A school was deemed a participating school if the principal returned their questionnaire with at least one question answered.

1. Israel: Ultra-Orthodox schools were excluded after the survey administration due to very low participation rates. They are not considered in the participation rate but contribute to the exclusion rate.

2. Slovenia: The school sample was identical for ISCED Level 1 and Level 2. Principals were asked to participate in both questionnaires but did not in many cases. Eleven items were copied between ISCED levels in these cases.

3. United States: The national centre managed questionnaire delivery from their own servers using the platform provided by the IEA in a single participant setup. Following data collection, the United States assessed disclosure risk and applied related measures based on federal requirements (https://nces.ed.gov/fcsm/dpt/). No documentation of these measures was provided.

Table A A.4. ISCED Level 2: Teacher participation and recommended ratings

Participating education system	Number of participating schools	Number of participating teachers	Estimated size of teacher population	School participation before replacement (%)	School participation after replacement (%)	Teacher participation in participating schools (%)	Overall teacher participation (%)	Recommended rating
Albania	282	4 029	11 830	99.6	100.0	99.4	99.4	Good
Alberta (Canada)	153	1 823	11 190	46.4	56.4	82.2	46.4	Insufficient
Australia	179	3 035	122 147	66.6	88.5	86.7	76.7	Fair
Austria	247	4 335	43 477	77.5	89.8	85.9	77.2	Good
Azerbaijan	222	4 307	95 548	99.5	100.0	98.4	98.4	Good
Bahrain	116	3 104	4 581	100.0	100.0	97.7	97.7	Good
Belgium1	273	5 175	40 763	67.5	76.5	84.5	64.6	Fair
Brazil	207	3 009	590 695	89.9	92.4	95.2	87.9	Good
Bulgaria	210	3 629	21 777	97.9	99.6	99.1	98.7	Good
Chile	353	2 319	59 796	66.2	79.8	84.0	67.0	Fair
Colombia	213	3 010	167 714	71.7	77.2	91.9	70.9	Fair
Costa Rica	187	3 026	14 638	87.6	89.0	94.2	83.8	Good
Croatia	268	3 163	15 957	90.5	90.5	82.6	74.8	Good
Cyprus	99	1 793	4 151	99.0	99.0	91.2	90.3	Good
Czechia	346	6 369	49 671	99.7	100.0	98.4	98.4	Good
Denmark	189	2 488	23 083	53.5	80.0	85.6	68.5	Fair
Estonia	199	3 339	8 860	97.5	97.5	89.1	86.9	Good
Finland	226	4 300	20 980	99.6	100.0	94.8	94.8	Good
Flemish Community (Belgium)2	143	2 444	24 689	58.1	68.3	84.2	57.5	Poor
France	161	2 877	206 033	80.2	81.7	76.4	62.4	Good
French Community (Belgium)	130	2 731	16 075	81	88.4	84.7	74.9	Good
Hungary	235	3 890	43 537	95.1	95.5	92.6	88.4	Good
Iceland	131	1 427	1 982	94.2	94.2	81.6	76.9	Good
Israel3	196	3 227	35 940	92.0	92.0	87.4	80.5	Good
Italy	200	3 874	168 045	98.5	99.0	92.4	91.5	Good
Japan	201	3 558	209 963	90.2	98.5	97.7	96.2	Good
Kazakhstan	377	7 360	205 012	99.8	99.8	99.6	99.3	Good
Korea	169	3 098	76 205	65.3	83.7	91.0	76.2	Fair
Kosovo	222	3 281	7 339	99.1	99.1	91.4	90.6	Good
Latvia	215	4 088	9 219	97.2	98.6	91.0	89.8	Good
Lithuania	210	4 223	16 848	99.1	99.5	97.1	96.6	Good
Malta	48	2 263	3 265	88.6	88.6	81.3	72.0	Good
Montenegro	126	1 354	2 186	78.3	78.3	84.0	65.7	Fair
Morocco	398	6 112	71 394	99.7	100.0	98.3	98.3	Good
Netherlands	84	1 271	58 097	37.6	56.4	73.7	41.5	Insufficient
New Zealand	97	1 204	24 847	28.1	43.0	73.1	31.5	Insufficient
North Macedonia	202	3 992	7 322	100.0	100.0	94.9	94.9	Good
Norway	112	1 528	20 156	34.1	53.9	73.1	39.4	Insufficient
Poland	279	3 783	172 089	93.6	99.6	94.7	94.4	Good
Portugal	224	3 511	38 887	98.7	99.1	92.2	91.4	Good
Romania	216	3 873	64 851	100.0	100.0	97.4	97.4	Good
Saudi Arabia	326	3 306	112 606	98.1	98.1	94.6	92.8	Good
Serbia	199	4 007	23 354	98.5	99.5	95.6	95.2	Good
Shanghai (China)	205	4 078	44 556	100.0	100.0	98.4	98.4	Good
Singapore	156	3 355	10 601	96.9	98.1	97.2	95.3	Good
Slovak Republic	242	4 190	22 949	97.1	100.0	96.1	96.1	Good
Slovenia	222	2 831	8 269	82.7	84.2	89.1	75.0	Good
South Africa	361	3 301	87 309	92.8	93.3	91.2	85.0	Good
Spain	507	9 098	198 806	99.8	100.0	94.1	94.1	Good
Sweden	196	3 065	33 858	92.7	95.1	76.4	72.7	Fair
Türkiye	229	4 932	291 832	99.6	99.6	97.2	96.8	Good
United Arab Emirates	212	4 369	28 559	99.5	99.5	98.4	97.9	Good
United States4	154	1 988	1 206 856	49.6	60.2	82.8	49.8	Poor
Uzbekistan	210	4 431	306 057	100.0	100.0	98.7	98.7	Good
Viet Nam	202	4 348	305 410	100.0	100.0	99.0	99.0	Good

Note: A school was deemed a participating school if at least 50% of the selected teachers returned their respective questionnaires with at least one question answered.

1. Belgium: Some sampled schools of the Flemish Community were located in the same places. The national centre advised these schools to distribute teachers between listing forms to avoid being selected multiple times. This deviation from the procedures could not be corrected for.

2. Flemish Community (Belgium): Some sampled schools were located in the same places. The national centre advised these schools to distribute teachers between listing forms to avoid being selected multiple times. This deviation from the procedures could not be corrected for.

3. Israel: Ultra-Orthodox schools were excluded after the survey administration due to very low participation rates. They are not considered in the participation rate but contribute to the exclusion rate.

4. United States: The national centre managed questionnaire delivery from their own servers using the platform provided by the IEA in a single participant setup. Following data collection, the United States assessed disclosure risk and applied related measures based on federal requirements (https://nces.ed.gov/fcsm/dpt/). No documentation of these measures was provided. The rating was defined as “poor” although the participation rate of schools before replacement did not completely reach 50%.

Table A A.5. ISCED Level 1: Principal participation and recommended ratings

Participating education system	Number of participating principals	Estimated size of school population	Principal participation before replacement (%)	Principal participation after replacement (%)	Recommended rating
Australia	178	7 023	63.0	85.6	Fair
Belgium	333	4 241	61.1	81.0	Fair
Brazil	284	93 385	84.1	93.2	Good
Flemish Community (Belgium)	158	2 417	46.4	75.6	Insufficient
France	322	34 412	91.9	93.3	Good
French Community (Belgium)	175	1 824	76.2	86.6	Good
Japan	202	19 323	87.4	98.1	Good
Korea	198	6 133	66.1	88.4	Fair
Morocco	389	23 888	98.7	98.7	Good
Netherlands	132	5 993	43.8	68.3	Insufficient
New Zealand	141	2 044	34.8	58.1	Insufficient
Saudi Arabia	204	12 267	92.8	92.8	Good
Slovenia1	206	443	77.1	78.8	Good
Spain	487	13 736	97.2	97.2	Good
Türkiye	231	16 361	97.1	97.1	Good
United Arab Emirates	206	789	99.6	99.6	Good

Note: A school was deemed a participating school if the principal returned their questionnaire with at least one question answered.

1. Slovenia: The school sample was identical for ISCED Level 1 and Level 2. Principals were asked to participate in both questionnaires but did not in many cases. Eleven items were copied between ISCED levels in these cases.

Table A A.6. ISCED Level 1: Teacher participation and recommended ratings

Participating education system	Number of participating schools	Number of participating teachers	Estimated size of teacher population	School participation before replacement (%)	School participation after replacement (%)	Teacher participation in participating schools (%)	Overall teacher participation (%)	Recommended rating
Australia	178	3 003	137 078	63.0	85.6	89.9	76.9	Fair
Belgium	326	4 913	63 889	59.9	79.3	84.2	66.8	Fair
Brazil	279	2 994	667 700	81.0	90.9	97.4	88.5	Good
Flemish Community (Belgium)	146	2 258	38 107	42.1	69.9	88.0	61.5	Insufficient
France	291	1 993	179 824	89.1	91.2	88.9	81.1	Good
French Community (Belgium)	180	2 655	25 783	78.2	89.1	81.3	72.4	Good
Japan	202	3 356	311 128	87.4	98.1	97.9	96.0	Good
Korea	184	3 124	122 407	60.3	82.1	90.6	74.4	Fair
Morocco	394	4 905	192 370	100.0	100.0	99.1	99.1	Good
Netherlands	140	1 305	60 642	46.9	72.2	85.3	61.6	Insufficient
New Zealand	130	1 529	21 009	30.6	54.7	79.5	43.5	Insufficient
Saudi Arabia	218	3 783	213 117	99.1	99.1	94.6	93.8	Good
Slovenia	220	3 615	12 379	81.8	83.5	91.5	76.4	Good
Spain	500	8 060	201 862	99.2	99.2	94.7	94.0	Good
Türkiye	232	4 053	205 969	97.5	97.5	96.3	93.9	Good
United Arab Emirates	206	4 091	25 817	99.5	99.5	97.5	97.0	Good

Note: A school was deemed a participating school if at least 50% of the selected teachers returned their respective questionnaires with at least one question answered.

Table A A.7. ISCED Level 3: Principal participation and recommended ratings

Participating education system	Number of participating principals	Estimated size of school population	Principal participation before replacement (%)	Principal participation after replacement (%)	Recommended rating
Croatia	180	381	90.0	90.0	Good
Denmark	179	610	54.4	77.1	Fair
Flemish Community (Belgium)	137	744	47.6	65.2	Insufficient
Portugal	203	731	98.9	100.0	Good
Saudi Arabia	214	5 131	92.5	92.5	Good
Slovenia1	104	140	74.3	74.3	Good
Türkiye	197	10 741	97.0	97.0	Good
United Arab Emirates	197	629	98.3	98.3	Good

Note: A school was deemed a participating school if the principal returned their questionnaire with at least one question answered.

1. Slovenia: The rating was defined as “good” although the participation rate of schools did not completely reach 75%.

Table A A.8. ISCED Level 3: Teacher participation and recommended ratings

Participating education system	Number of participating schools	Number of participating teachers	Estimated size of teacher population	School participation before replacement (%)	School participation after replacement (%)	Teacher participation in participating schools (%)	Overall teacher participation (%)	Recommended rating
Croatia	192	3 497	14 212	96.0	96.0	85.7	82.3	Good
Denmark	179	2 550	16 672	58.3	79.3	86.1	68.3	Fair
Flemish Community (Belgium)1	141	2 427	47 252	50.5	67.1	86.5	58.1	Poor
Portugal	203	3 878	34 288	99.6	100.0	93.9	93.9	Good
Saudi Arabia	226	3 952	107 791	98.7	98.7	94.1	92.9	Good
Slovenia	113	2 717	6 076	80.7	80.7	83.4	67.4	Good
Türkiye	201	4 976	287 877	99.0	99.0	95.5	94.5	Good
United Arab Emirates	200	4 228	20 222	99.5	99.5	98.8	98.3	Good

Note: A school was deemed a participating school if at least 50% of the selected teachers returned their respective questionnaires with at least one question answered. 

1. Flemish Community (Belgium): Some sampled schools were located in the same places. The national centre advised these schools to distribute teachers between listing forms to avoid being selected multiple times. This deviation from the procedures could not be corrected for. 

References

[1] OECD (forthcoming), TALIS 2024 Technical Report, TALIS, OECD Publishing, Paris.

Note

← 1.  Similar to TALIS 2013 and 2018, teachers working with special education needs students within regular school settings were considered in scope. If a school is composed exclusively of such teachers, the school itself was considered out of scope.