Mid‑Term Evaluation of Spain’s Youth Guarantee Plus Plan 2021‑2027

Annex C. Sample-selection process for the survey of YG+P participants

Sample design

The database of people who participated in the Youth Guarantee during the evaluation period served as the sampling frame for the survey. It contains the details of 849 1691 participants, including information on: i) gender; ii) age group at registration (16‑19 years, 20‑24 years, or 25‑29 years); iii) autonomous community of residence; iv) vulnerability status; and v) type of service(s) received during participation in the Youth Guarantee, i.e. career guidance, training or employment services.2

The OECD evaluation team agreed that the survey results for each type of service would need to be analysed using the gender and vulnerability status of the participants as variables, although other characteristics could also be used. They also established that the sample had to be representative of the whole group of Youth Guarantee participants, in terms of type of service received, gender, vulnerability status, age group and autonomous community of residence.

They decided, therefore, to explicitly stratify the sample using the type of service received and participants’ gender and vulnerability status as variables. As some participants received several types of service, before the selection was carried out, each participant was allocated to one of the following four groups: i) career guidance; ii) training; iii) employment services; or iv) no service received. This ensured that each participant appeared only once in the sample. Assignment to these groups was done randomly. The group of participants who had received two types of service was split in two, and one‑half allocated to each service. The group that had received three types was divided into three parts, and one‑third allocated to each service. This allocation approach did, however, maintain the composition of each group in terms of gender, vulnerability status, age group and autonomous community of residence.

Table A C.1. shows the results of this allocation, while Table A C.2. presents the distribution of the resulting sample, by explicit stratum.

To ensure that the sample reflected the distribution of the sampling frame, implicit stratification was used. This meant that the participants for each explicit stratum were selected using systematic random sampling, working from lists that had been ordered by age group and by autonomous community.

Table A C.1. Allocation of participants to groups, by service received

Type of service received	Number of participants
No service received	146 310
Total for the “no service received” group	146 310
Participants who received career guidance only	71 345
Share of the group who received training and career guidance	11 337
Share of the group who received career guidance and employment services	127 988
Share of the group who received training, career guidance, and employment services	29 676
Total for the “guidance” group	240 346
Training only	9 567
Share of the group who received training and career guidance	11 337
Share of the group who received training and employment services	10 748
Share of the group who received training, career guidance, and employment services	29 676
Total for the “training” group	61 328
Employment services only	232 773
Share of the group who received career guidance and employment services	127 988
Share of the group who received training and employment services	10 748
Share of the group who received training, career guidance, and employment services	29 676
Total for the “employment services” group	401 185
SAMPLING FRAME TOTAL	849 169

Table A C.2. Sampling frame sorted by explicit strata

	Male		Female		Total
	Non-vulnerable	Vulnerable	Non-vulnerable	Vulnerable
No service	72 479	924	71 930	977	146 310
Career guidance	53 637	67 625	67 160	51 961	240 383
Training	13 371	18 748	15 814	13 335	61 268
Employment services	99 524	107 110	118 868	75 706	401 208
Total	239 011	194 407	273 772	141 999	849 169

Sample size

While probability sampling and stratification are important to ensure the “representativeness” of the sample, i.e. that it reflects the distribution of the sampling frame, sample size is important to ensure that the desired precision is achieved in the final estimates.

Since there was no prior information on the key indicators or comparisons of interest, the sample size was calculated using standard assumptions: an estimated proportion of 0.5, a margin of error of 5% and a confidence interval of 95%. These assumptions yielded a target sample size of n=385 per explicit stratum (Table A.C.3).

Table A C.3. Target sample size by explicit stratum

	Male		Female		Total
	Non-vulnerable	Vulnerable	Non-vulnerable	Vulnerable
No service	385	385	385	385	1 540
Career guidance	385	385	385	385	1 540
Training	385	385	385	385	1 540
Employment services	385	385	385	385	1 540
Total	1 540	1 155	1 540	1 155	5 390

Working on the assumption that the response rate would be 10%, the study increased the size of the initial sample proportionally, resulting in n=3 850 per explicit stratum. For two of the strata in the sampling frame (male and female vulnerable young people who did not receive any services), the total populations were smaller than 3 850 (see Table A C.2). These populations were therefore sampled exhaustively, i.e. all members of the population were included in the sample.

Additional samples and important considerations

Non-response was higher than expected, for various reasons.3 To increase the response rate, two additional samples were drawn, excluding the young people who had already been selected in the first sample. The first additional sample was drawn in the same way as the initial sample. The second additional sample was adjusted proportionally to the non-response rate in each stratum, based on the rates obtained after the first additional sample.

Given the need for these additional samples, total sampling fractions were very high in some strata: the average was nearly 20% and the share exceeded 20% in 9 of the 16 strata (Table A C.4).

Table A C.4. Sampled units of the sampling frame, by explicit stratum

Explicit stratum	Sampled	Not sampled	Total	Sampling rate
No service
Vulnerable males	924	0	924	100%
Vulnerable females	977	0	977	100%
Non-vulnerable males	12 268	60 211	72 479	16.9%
Non-vulnerable females	7 700	64 230	71 930	10.7%
Career guidance
Vulnerable males	17 111	50 766	67 877	25.2%
Vulnerable females	14 248	37 671	51 919	27.4%
Non-vulnerable males	11 528	42 262	53 790	21.4%
Non-vulnerable females	7 923	59 197	67 120	11.8%
Training
Vulnerable males	12 322	6 419	18 741	65.7%
Vulnerable females	10 745	2 499	13 244	81.1%
Non-vulnerable males	6 055	7 203	13 258	45.6%
Non-vulnerable females	6 091	9 634	15 725	38.7%
Employment services
Vulnerable males	18 029	88 836	106 865	16.9%
Vulnerable females	14 855	60 984	75 839	19.6%
Non-vulnerable males	13 739	85 745	99 484	13.8%
Non-vulnerable females	9 086	109 911	118 997	7.6%
Total	163 126	686 043	849 169	19.2%

Table A.C.5 shows the distribution of the final sample of respondents, i.e. those who completed the questionnaire.

Table A C.5. Final sample of respondents by explicit stratum

	Male		Female		Total
	Non-vulnerable	Vulnerable	Non-vulnerable	Vulnerable
No service	423	49	427	60	959
Career guidance	506	330	435	391	1 662
Training	480	404	454	499	1 837
Employment services	460	305	521	409	1 695
Total	1 869	1 088	1 837	1 359	6 153

The final survey response rate is low (3.8%), which creates a risk of non-response bias. Non-response bias can occur when the characteristics of the people who do not complete the survey are systematically different from those of the people who do. In the Youth Guarantee survey of participants, for example, positive opinions about the services received could be overestimated if non-response was higher among participants who received poorer quality services.

The probability of non-response bias can be assessed (although not quantified) by comparing the key demographic characteristics of respondents and non-respondents. As Table A C.6. shows, the distribution of the final sample of respondents reflects the sampling frame fairly closely in terms of gender and vulnerability status. However, given that certain unobserved characteristics may be systematically affecting participants’ probability of responding to the survey, the results should be generalised with caution.

Table A C.6. Demographic distribution of the final sample of respondents and the sampling frame

	Gender		Vulnerability status
	Male	Female	Vulnerable	Non-vulnerable
Final sample	48.1	51.9	39.8	60.2
Sampling frame	51.0	49.0	39.6	60.4

Notes

← 1.  This number is slightly higher than the total recorded in the data extracted for the analysis of the Youth Guarantee Administrative Registry (Section 2.5.1). This is because the data extractions for registry analysis were carried out in April and May 2024, while the sampling frame was defined using data extracted several months later, in August 2024. By then, a greater number of young people had enrolled in the Youth Guarantee programme, which explains the observed increase.

← 2.  Hiring or entrepreneurship.

← 3.  Reasons include: an incorrectly formatted email address, the respondent marked the email as spam, the respondent did not open the email, or the respondent opened the email but did not complete the survey.