Nóra Révai
OECD
Everybody Cares About Using Education Research Sometimes
Perspectives of Knowledge Intermediaries
Abstract
This chapter describes the different types of activities that educational organisations undertake to facilitate the use of research. It presents a heatmap of knowledge mobilisation activities across various educational organisations worldwide, which visualises these efforts from sparse or non-existent to activities that are systematically implemented. By examining this spectrum of activity, the chapter aims to identify patterns and highlight areas in need of improvement. Two case studies, one set in the Netherlands and one in the United Kingdom, provide examples of a multi-faceted approach to knowledge mobilisation.
A heatmap of knowledge mobilisation activities can support countries and organisations in improving and co-ordinating knowledge mobilisation at the system level. The international landscape suggests that a variety of organisations engage in numerous activities to support the use of research in policy or practice.
Evidence dissemination, as well as collaboration and partnership building are the dominant knowledge mobilisation mechanisms. Support for organisations in developing processes and structures for research engagement is also relatively frequent. Activities aimed at improving knowledge mobilisation at the system-level, such as co-ordination of efforts, are rarer but still surprisingly widespread.
However, there are also clear gaps: Training and capacity building are the weakest links of knowledge mobilisation, particularly for researchers and intermediaries.
Four knowledge mobilisation profiles emerge from the data: Research-focused mobilisers, who focus on evidence generation and dissemination; research funders, who fund and commission research; partnership builders who focus on facilitating collaboration among actors and creating communication channels, and capacity builders with a system focus who build actors’ research capacity and provide support for organisations and systems.
The optimal distribution of activities may depend on the unique characteristics of the different types of organisations (e.g. mission, skills, legitimacy). Some knowledge mobilisation roles can be more clearly attributed to certain types of organisations, such as developing teachers’ research engagement skills to initial teacher education institutions, whereas others depend on the context and may be different for every system.
Examining the coverage and the distribution of activities, four types of knowledge mobilisation systems can be distinguished: dysfunctional (low coverage, unco-ordinated), redundant (high coverage, unco‑ordinated), patchy (low coverage, co-ordinated) and optimal (high coverage, co-ordinated).
Numerous countries across the OECD have heavily invested in facilitating and reinforcing the use of education research in recent years. Despite this investment, ministries of education continue to report that a lack of mechanisms to facilitate research use is a major barrier to its systematic use in policy and practice (OECD, 2022[1]). At the same time, an increasing number of organisations engage in activities that mediate between research and its use in policy and practice (Burns and Schuller, 2022[2]; Oliver et al., 2022[3]). The OECD Survey of Knowledge Mobilisation in Education (henceforth, the intermediaries’ survey) collected data from 288 intermediary organisations in 34 countries to better understand how they are supporting research engagement (see Chapter 1).
The current chapter presents the global picture of knowledge mobilisation activities across different types of organisations. It first discusses the typology of activities included in the intermediaries’ survey and the underlying conceptual model. Then it presents the overall intensity of knowledge mobilisation before going into the analysis by type of organisation. The chapter includes two case studies that target system-wide improvement. It concludes with some reflection prompts for the readers.
The term “knowledge mobilisation” is used broadly in the intermediaries’ survey to refer to any intentional efforts to increase or improve the use of research evidence in education policy and/or practice. For the purposes of a survey, a list of 21 activities was identified based on a review of literature and previous surveys (Oliver et al., 2022[3]; Torres and Steponavičius, 2022[4]). The list includes well-specified and distinct activities of knowledge mobilisation, categorised in six thematic groups:
evidence generation;
evidence dissemination;
collaboration and partnerships;
capacity building;
organisational support;
local, regional or national-level support (see Chapter 1 and Annex Table 3.A.1 for more details).
In practice, these activities often occur simultaneously within a more complex initiative. For example, the Knowledge Roundabout described in the Netherlands case study later on in this chapter includes evidence generation, dissemination and capacity building within one initiative. While the list covers a wide range of activities, it is not fully comprehensive. Indeed, organisations engage in a very wide range of activities, not all of which can be captured in a survey (for more examples see (OECD, 2023[5]; OECD, 2024[6])).
Together with the conceptual and practical challenges described in Chapter 1, these complexities imply that the intermediaries’ survey is naturally limited in terms of its scope and, perhaps, conceptual clarity. In addition, as pointed out in Chapter 1, the set of respondents cannot be considered as representative of any country’s knowledge system, nor of the different types of intermediaries. While the wide range and high number of respondent intermediaries allow for an analysis of the aggregated data and some cautious implications with respect to the international landscape of knowledge mobilisation, this chapter does not offer comparative analysis across countries.
The heatmap is a reflection tool more than a presentation of statistical outcomes.
– Rien Rouw, Strategic Advisor, Ministry of Education, Culture and Science, the Netherlands
The heatmap should primarily be used as a reflection tool (see Box 3.1). The authors invite the readers to think beyond the survey results. For instance, the reader might use the list of knowledge mobilisation activities as a diagnostic tool for their own organisation or develop a heatmap for their own system (see Reflection questions and activities box at the end of the chapter).
The heatmap can help reflect on the overall distribution of knowledge mobilisation activities in two ways:
From the perspective of the activities
Looking at the rows, the heatmap can help readers to identify activities and groups/types of activities that are missing: none or very few organisations do them intensively.
From the perspective of organisations
Looking at the columns, the heatmap can help readers reflect on whether “the right organisations are doing the right activities”. It can point to activities that a certain type of organisation perhaps should be doing but is not, or those that some are doing but maybe should not.
Note of caution: No normative assumption
The heatmap should not be interpreted under the normative assumption that the darker red cells, the better. It would not be meaningful, nor cost-efficient, for all types of organisations to engage deeply in all knowledge mobilisation activities.
The heatmap of knowledge mobilisation (Figure 3.1) shows the proportion of organisations carrying out each activity intensively, i.e. to a large extent or systematically (calculated from respondents answering to what extent they carry out each activity on a 5-point Likert scale).
Notes: Data show the percentage of the given type of organisation that reports doing the given activities to a large extent or systematically. As respondents could identify as several types of organisations, the same respondent might be represented in several columns. The last column shows the overall percentage of organisations that reports doing the given activities to a large extent or systematically. The number of respondents per type of organisation is indicated in parentheses. Redder shades represent progressively higher percentages (more common activities), while bluer shades represent progressively lower percentages (less common activities).Source: OECD Survey of Knowledge Mobilisation in Education data, 2023.
Perspective of activities
The heatmap suggests that all activities are covered to some extent by these types of organisations, but the extent varies widely.
Evidence dissemination, as well as collaboration and partnership building are the dominant knowledge mobilisation mechanisms
Close to two-thirds of surveyed organisations reported establishing communication channels for dissemination, the activity most commonly carried out systematically. While this activity was classified as dissemination, it can also be a relational approach if designed to be interactive (bi-directional). Disseminating research in user-friendly formats is also a common form of knowledge mobilisation, reported by more than half of the respondents.
On average, one in three organisations reported curating and maintaining an evidence repository (i.e. a structured database or collection of research syntheses). Repositories can be a major support for research users, particularly if they are easy to navigate and provide transparent information on the type and quality of evidence (Gough, Sharples and Maidment, 2022[7]). However, having too many diverse repositories can be a barrier – users may get lost in such volume and it might be difficult to judge which repository serves what purpose and can help answer what types of questions. They might also use different standards of evidence, which not only adds to the difficulty of navigating among them, but may also create mistrust. Importantly, it may reflect duplicated effort, which raises questions about cost-efficiency, given that maintaining a repository is costly.
The Education Endowment Foundation (EEF)’s Teaching and Learning Toolkit is perhaps the best-known evidence repository and has achieved international attention, with many countries currently adapting it to their own context. It focuses primarily on meta-analyses of causal evidence (randomised control trials (RCTs) and quasi-experimental designs). Increasingly more knowledge centres focus on systematic reviews, including a broader range of methodologies (e.g. the Norwegian Knowledge Centre, DIPF | Leibniz Institute for Research and Information in Education, the EEF itself) although such form of rigorous synthesis is still more rare than in the health sector (Education.org, 2021[8]).
While a large proportion of all types of organisations facilitate collaboration and build partnerships for research use, the mapping of users or mobilisers of research evidence is somewhat less widespread. Yet, strategic knowledge mobilisation should involve a good understanding of the actors and their needs, with regular updates given the continuously shifting landscape of actors (Gough, Sharples and Maidment, 2022[7]). An example of a systematic mapping is outlined in the case study below.
Training and capacity building are the weakest links of knowledge mobilisation, particularly for researchers and brokers
Knowledge mobilisation is more impactful when research is practice-centred and tailored to policy and practice needs (Farley-Ripple and MacGregor, 2024[9]). The effectiveness of research-practice and research-policy partnerships hinges upon the knowledge, competencies and attitudes of its participants, among other factors (Phelps, 2019[10]; Schlicht-Schmälzle et al., 2024[11]). Researchers’ understanding of practitioners’ and policy makers’ context, their openness to prioritising users’ needs and ability to communicate research effectively are characteristics often less important for academic career progression compared to the number of publications and citations in academic journals (OECD, 2024[12]). The data suggest that researchers do not necessarily have learning opportunities to build this capacity. This may have deeper underlying reasons, as the following quote suggests:
Lack of training for researchers may reflect social norms: Academics might prefer to see themselves as ‘pure researchers’ and do not want to receive training for policy engagement.
– Javier González, Director of Laboratory for Research and Innovation in Education for Latin America
The term “training” suggests formal learning, e.g. in the form of courses and workshops. However, in academic and policy environments, learning is often informal and happens through experiences, as the quote below reflects. Responses may have been different if the question had been phrased differently (e.g. had included informal learning and other forms of capacity building, such as masterclasses).
I would say I have training in my job, but it is more learning by doing. We need to reflect on what we mean by training.
– Elisabeth Buk-Berge, Norwegian Ministry of Education and Research
Effective brokering requires individual competencies such as communication skills and an ability to facilitate quality interactions in challenging contexts (e.g. where power relations between researchers and teachers may be at play) (Edwards, 2012[13]). They also need to recognise opportunities for crossing the boundaries between the research and practice or policy contexts, and flexibly shift between these (Bakx et al., 2016[14]; Schlicht-Schmälzle et al., 2024[11]). Yet, the activity least commonly reported is training brokers, i.e. individuals or organisations whose role is to bridge the research-policy or research-practice gap (Akkerman and Bakker, 2011[15]). A promising initiative is a pilot project to train early-career teachers and researchers to become successful brokers (see the case study from Germany in (Schlicht-Schmälzle et al., 2024[11])). OECD work has also pointed to a lack of peer learning across intermediaries (OECD, 2023[5]). The EEF has established its Education Evidence Network to facilitate exchange and learning about evidence synthesis across organisations adapting the EEF Toolkit internationally (EEF, 2024[16]).
Support for organisations is relatively frequent, activities aimed at improving knowledge mobilisation at the system level are rarer but still surprisingly widespread
More than one-third of organisations report supporting schools and/or policy organisations in developing processes for knowledge mobilisation, or directly aiding them in research implementation. Sustained and direct support at the level of organisations is said to be more effective than one-off training for individuals (Oliver, forthcoming[17]), so this is promising news. Slightly less than one in four respondents report scaling or evaluating knowledge mobilisation initiatives (see also Chapter 6), and creating system-level infrastructure, processes and incentives (see the Netherlands case study below for examples). This still seems high, considering that the main mission of most respondents is not knowledge mobilisation per se (but teacher training, school inspection, consultancy, etc.) and thus one might think that engaging in system-level improvement of knowledge mobilisation would fall out of their scope. There is also a question of what the desirable extent of system-level improvement is. Would it be more effective to have fewer actors focusing on this kind of activity to create coherence in terms of infrastructure, processes and incentives?
Many organisations co-ordinate knowledge mobilisation at the system level
This leads to the question of co-ordinating knowledge mobilisation at the system level across different actors, which is reportedly done by more than one-third of the surveyed organisations to a large extent or systematically. What does such co-ordination involve? Do organisations co-ordinate collectively (e.g. operate as a network) or does this widespread engagement represent parallel efforts? How many actors can co-ordinate knowledge mobilisation at the same time and who should those actors be? Answers to these questions will be different depending on how education is governed in a system. In top-down, centralised systems, the national ministry might be an accepted actor to co-ordinate knowledge mobilisation, while in decentralised systems, an independent body, such as a formal intermediary or some form of a network of organisations might be better suited (see the United Kingdom case study for further examples on systemic influences).
Perspective of organisations
The heatmap shows that all types of respondent organisations engage in several knowledge mobilisation activities intensively. Most organisations reported engaging in 7 activities out of 21 to a large extent or systematically. The distribution is skewed towards the bottom, indicating that many organisations focus on a few activities intensively and there are very few with a large scope, carrying out a high number of activities systematically. The most active ones are formal intermediaries and consulting firms, closely followed by research institutions, while inspectorates and quality assurance agencies, and teacher unions, are the least active.
Four knowledge mobilisation profiles emerge: Research-focused mobilisers, research funders, partnership builders and capacity builders with a system-focus
There are four distinct knowledge mobilisation profiles that emerge from the data1:
Notes: Profiles were determined based on exploratory factor analysis conducted with the Factor Analyzer package of Python, using Promax rotation and maximum likelihood method (https://pypi.org/project/factor-analyzer/). Eigen values and factor loadings are provided in Annex Table 3.A.2.
Source: OECD Survey of Knowledge Mobilisation in Education data, 2023.
There are several implications of these knowledge mobilisation profiles. First, research producers are primarily focused on dissemination, whereas creating partnerships with users, building actors’ capacity to understand and apply research, or providing systemic support for knowledge mobilisation are not systematically done by organisations with this profile. This may reflect a traditional view of research production, which is largely isolated from its application in policy and practice, as well as insufficient capacity for substantive engagement with users, including a lack of human and financial resources or a lack of appropriate incentives for all actors. In the United Kingdom, several universities (e.g. University of Manchester, University of Nottingham and University College London) now have policy engagement centres, which focus on enabling their researchers and students to increase the impact of their research and to build their network with policy makers. The driving force behind such initiatives is primarily to promote the impact of the research undertaken within these institutions and it is therefore questionable whether they contribute to systemic support for knowledge mobilisation. However, it is also questionable whether it is realistic and desirable (e.g. in terms of resource allocation) for every research producer to build more substantive relationships and capacity with users or provide systemic support.
Second, the slight tendency of funders of research towards systems approaches shows that the sustainability and scalability of knowledge mobilisation, as well as system-level factors, are increasingly considered in funding schemes (see, for example, the Latvia case study in Chapter 5). On the other hand, funding is more rarely accompanied by evaluating knowledge mobilisation initiatives. This may prevent the field from generating evidence on knowledge mobilisation and being able to build on that in determining investment and scaling (see Chapter 6 for further detail).
Third, the fact that capacity building tends to go together with systems approaches, but less with partnership building, raises questions about the impact of partnerships. If partnership-building efforts are not integrated with capacity building, for example, to develop actors’ research use skills, these partnerships may not lead to quality engagement with research (Langer, Tripney and Gough, 2016[18]; Farrell et al., 2021[19]). On the positive side, organisations with a system focus on knowledge mobilisation recognise that skills and capacity are key pieces in a system, and tend to complement activities such as co-design, organisational and local or regional-level support with training and capacity building.
Who is doing what in knowledge mobilisation is not neutral: Some roles are uncontroversial, while some might be contentious
The ideal distribution of activities may depend on the unique characteristics of the different types of organisations (e.g. mission, skills, roles). Some knowledge mobilisation roles can be more clearly attributed to certain types of organisations, whereas others depend on the context and may be different for every system.
An example of a clear role is training teachers, in the context of the intermediaries’ survey, this pertains to developing their research engagement skills. Clearly, initial teacher education (ITE) institutions should play a major role in this with respect to teacher candidates. The 2021 OECD Strengthening the Impact of Education Research policy survey, which collected data from 37 Ministries of Education in 29 countries, demonstrated gaps in this field. Respondent ministries reported that developing such skills is required or widely covered in ITE in only slightly more than half of systems on average, and continuous professional development does not seem to compensate for this gap, with 42% of systems developing research skills in professional development (OECD, 2022[1]). Data from ITE institutions in the intermediaries’ survey corroborates ministries’ perceptions – less than half report training practitioners to help them use research to a large extent or systematically. Promising examples include the Universitat Autònoma de Barcelona (the Autonomous University of Barcelona) (see Box 3.2). Interestingly, 20 research institutions that do not function as initial teacher education institutes reported that they are active in training teachers and policy makers. Consulting firms are as active in training practitioners or policy makers as teacher education institutes, suggesting that this gap is at least partially filled by the private sector.
The Autonomous University of Barcelona offers a bachelor’s programme to prepare primary school teachers and a master’s programme to prepare teachers in secondary and vocational education. Research skills are part of the core competences to be developed and supported by various courses at both levels.
In addition, the university’s summer school offers an optional hands-on course called “Solving Socio-Educational Challenges through Research: Hands on Experience”, which equips education professionals with skills in research, evaluation and innovation through an immersive, challenge-based approach.
Source: Autonomous University of Barcelona (2024[20]), Bachelor’s degree in Teacher in Primary Education, https://web.ub.edu/en/web/estudis/w/bachelordegree-G1026?subjects; Autonomous University of Barcelona (2024[21]), Master in Teacher Training in Obligatory Secondary and Upper Secondary School Education, Vocational Training and Languages, https://web.ub.edu/en/web/estudis/w/masteruniversitari-M2002?objectivesCompetencies; Autonomous University of Barcelona (2024[22]), Solving Socio-Educational Challenges through Research: Hands on Experience, https://www.uab.cat/web/cursos/cursos-de-ciencias-sociales-y-juridicas/research-methods-in-education-1345905134700.html.
An example of a less straightforward knowledge mobilisation role is that of education consulting firms. These for-profit organisations advise education authorities, schools or other education-related bodies on educational planning, research and project development. In the intermediaries’ survey, education consulting firms reported to have as widespread knowledge mobilisation activities as formal intermediaries, i.e. organisations with an explicit mandate for this work. For example, a higher proportion of education consulting firms report organisational support activities, building leadership capacity and even co-ordinating knowledge mobilisation at the system level than formal intermediaries. If these consulting firms’ primary interest is selling their services, their high-level engagement suggests substantive demand for knowledge mobilisation services. This demand may originate from insufficient or inappropriate knowledge mobilisation support by public institutions, which creates an opening filled by private providers. It could also stem from taking a wider market-based approach to the allocation of resources in education – the extent to which countries do this varies across OECD countries (OECD, 2024[23]). There are likely some systems where for-profit education consulting actors simply have a more commonly accepted, familiar and legitimate role in supporting public education, including knowledge mobilisation. When knowledge mobilisation is based on business interest, there is a question about its neutrality. Can business interests influence the selection of research evidence to be mobilised, or the nature and quality of the different activities? To what extent could that be concerning? These questions hold for all for-profit actors, including, for example, increasingly privately-funded research institutes.
It must also be noted that all education consulting firms identified as other types of organisations as well: more than half of them also claim to be research institutes and around one-third identify as a formal intermediary (see Chapter 1). In some countries, education consulting firms compete for public research mobilisation grants. For example, in the United States, the Regional Education Labs (REL) are a mechanism to support state- and district-level decision making with educational data and evidence (see case study in Chapter 4). Some of the RELs, like Marzano Research (Box 3.3), are consultancies with a knowledge mobilisation mandate.
Marzano Research is a private consulting firm in the United States that – as part of its applied research and consulting work – supports policy makers’ and practitioners’ intentional engagement with research. The work is often carried out in the form of research-practice partnership, particularly through the Regional Education Labs (RELs) programme.
Teacher as Researcher initiative in South Carolina, United States
The Teacher as Researcher initiative implemented by Marzano is a professional learning programme that supports teachers in conducting research in their own classrooms. In the 2022/23 academic year, 172 teachers in 32 schools participated in the programme, the costs of which were covered by the state of South Carolina.
Teachers are given instruction on research design through six two-hour workshops. They then conduct quasi-experiments to test the effectiveness of specific instructional strategies (by implementing it in one classroom and keeping their “traditional” strategy in another).
Internal evaluation of the programme is done through surveys, video recordings of the research design workshops and interviews with teachers. One challenge has been the timing and development of the workshops: while short sessions seem to be more manageable than, for example, a two-day research design workshop, it is still challenging to get teachers to volunteer for this kind of professional development initiative, given the time investment needed.
Source: Interview with Marzano Research, June 2024.
In addition to organisational interests, other factors should also be considered in assessing whether “the right organisation is doing the right activity”. These include the organisational competencies and skills to conduct an activity at a high standard, as well as the perceived legitimacy of the organisation to engage in an activity (for example, its independence from government).
Bottom line: Knowledge mobilisation systems
Combining the perspective of activities and that of organisations, it is possible to examine a knowledge mobilisation system (e.g. at the local, regional or national level) along two axes: the coverage of activities and the extent to which activities are co-ordinated (i.e. well-aligned and complementary). A simplistic analysis would then distinguish between the four types of systems presented in Table 3.3. While the intermediaries’ survey data does not allow for placing countries or sub-national education systems in this framework, it can serve as a reflection-starter. Importantly, this classification does not say anything about the quality of knowledge mobilisation activities and their impact, which would be a third dimension and is addressed in Chapter 6.
The intermediaries’ survey also asked about the types of evidence organisations use in their knowledge mobilisation activities (see Figure 3.2). Typically, organisations focus on three types of research out of 6 (including “other”) to a large extent or systematically (mode = 3, mean = 2.4). Just like with activities, few organisations demonstrate a broad coverage of research types (9% of organisations cover five types). Again, there is no normative assumption that all actors should cover all research types, if organisations are transparent about the types and quality of evidence that they use (Gough, Sharples and Maidment, 2022[7]).
Note: Data show the percentage of organisations reporting that their knowledge mobilisation activities focus on the given type of research to a large extent or systematically.
Source: OECD Survey of Knowledge Mobilisation in Education data, 2023.
The focus tends to be on qualitative data and large-scale quantitative data. Big data still seems to be novel and underexploited for knowledge mobilisation purposes in education. RCTs and quasi-experimental studies are the gold standard in health research, but still rare in education research (see also Chapter 2). Perhaps what is more worrying is the relatively low focus on research synthesis, which may reflect the lack of such synthesis in education (see Chapter 2).
In addition to the types of research listed in the intermediaries’ survey, some organisations reported to also focus on mixed methods, policy reviews and critical analyses, collaborative research and curriculum policy audits. Interestingly, two organisations also mentioned other knowledge sources, such as knowledge from practitioners and from bottom-up deliberative stakeholder processes. Although these are not types of research, these data reflect the fact that quality engagement with research in reality involves combining and making sense of various sources of knowledge.
The following sections present two case studies. Both demonstrate a multi-faceted approach to knowledge mobilisation combining a number of activities in strategic ways.
The Netherlands’ efforts to improve the use of education research in policy and practice have resulted in rethinking knowledge mobilisation at different levels and in a range of institutions.
Bringing practice and research together: The Ontwikkelkracht programme
The Ontwikkelkracht (development power) Programme2, launched in January 2023, aims to develop learning capacity in the education system by building a knowledge infrastructure. The programme aims to foster interaction between educational research and practice by ensuring that insights from the classroom get a place in research, and that science insights are given a place in the classroom. Ontwikkelkracht was founded by a scientist and a teacher and is now led by the Ministry of Education, Culture and Science in collaboration with the Netherlands Initiative for Education Research (NRO), Education Lab Netherlands, Groeikracht and The Transformative School. Ontwikkelkracht follows three principles: Evidence-informed practice, working in co-creation and through a learning approach.
In 2022, EUR 332 million was allocated for this project from the National Growth Fund for the next 10 years (up to and including 2032). Of this amount, EUR 101 million has been allocated for the first 4 years. The other part (EUR 231 million) has been conditionally granted for the following 6 years.
In the pilot phase (2023/24 academic year), 41 schools participated in the Ontwikkelkracht programme. In 2024/25, almost 300 schools are joining the programme. The programme has four pillars:
Strengthen the research and school improvement culture in schools by offering approaches to school improvement.
Make knowledge from research accessible and useful for educational professionals.
Set up co-creation education labs in which education professionals and researchers co-create new knowledge for complex issues from practice, hopefully resulting in new effective interventions that can be used in the classroom.
Train and support “expertise schools” that successfully improve education in an evidence-informed way to share their knowledge with other schools.
The Netherlands Initiative for Education Research: A funder-broker approach
The Netherlands Initiative for Education Research (NRO) is part of the Dutch Research Council. The NRO was established in 2012 and has been fully operational since 2014 with an annual budget of around EUR 15 million (NRO, 2024[24]) and supplemented with a substantial amount of programme-based funding. It has three main missions:
facilitate the coherence of education research by providing co-ordination at a national level
award grants for high-quality research
facilitate knowledge use by improving the links between science, policy and practice.
In recent years, the NRO has been expanding its mission from an organisation originally focused on funding research to a knowledge intermediary with a comprehensive knowledge mobilisation strategy. It has become a member of the Education Endowment Foundation’s (EEF) Education Evidence Network (EEF, 2024[16]) and has engaged in adapting the EEF Teaching and Learning Toolkit in the Netherlands. In addition, the NRO established a Knowledge Use Council (KBR) that advises the organisation on developing its knowledge mobilisation strategy and activities. It consists of experts from the policy, practice and research communities and meets four times a year. The following are two main processes of the NRO exemplifying its research funding and knowledge mobilisation activities respectively.
Knowledge Agenda
To ensure that the financial resources for education research are spent effectively, the NRO develops its research programme, the Knowledge Agenda for Education (NRO, 2022[25]), which sets out the themes for research funds for the period 2022-2028. The programme is driven by policy and practice needs that the NRO explores in partnership with professionals, including school leaders, policy officers, research institutes, advisors, scientific researchers and other stakeholders.
Following the publication of calls for proposals, the NRO assesses the proposals and selects the ones that receive funding based on their relevance and scientific quality. The selection process involves invited national and international experts from science and educational practice. Funded research projects are closely monitored through regular consultations with the research team and the intended end users to ensure that they retain their relevance for educational practice or policy.
The NRO disseminates research widely, including through its knowledge bank (a database of educational research), and supports researcher teams in disseminating their results through organising meetings where researchers, education professionals and policy makers can exchange knowledge and experience.
Knowledge Roundabout (Kennisrotonde)
The Knowledge Roundabout is a free online support and information service for schools that answers practitioners’ questions based on scientific literature. The answers include a simple overview of the do’s and don’ts, and are made available online as well as actively disseminated through social media and events. The process takes place in four steps:
1. The professional asks the question through a form on the Knowledge Roundabout website.
2. An employee of the Knowledge Roundabout – a knowledge broker or a contracted expert –contacts the professional and helps to shape the question into a research question.
3. The employee dives into literature to formulate the answer.
4. The answer is discussed with the person who asked the question and is published online.
A guide on how to formulate research questions supports practitioners in the process (NRO, 2019[26]). The Knowledge Roundabout has answered over 800 questions since it was founded in 2016 (Kennisrotonde, 2024[27]) and has an increasing reach, based on the number of incoming questions and followers on social platforms. The Knowledge Roundabout has been evaluated three times (2016, 2020 and 2023), each time by a different independent research organisation. They all concluded that professionals who ask questions are satisfied with the Knowledge Roundabout and that the answers help professionals to question and change their practice. The Knowledge Roundabout has helped to remove barriers, such as limited time for using research, that professionals often experience. It can spark debates among professionals on social media, which leads to stronger shared knowledge backed up by research. A downside is the time it takes to answer questions (>10 weeks), although according to the last evaluation report (van den Berg and Prins, 2023[28]), most professionals do not mind that.
Challenges and limitations
The Netherlands’ substantive investment in improving knowledge mobilisation has resulted in a rich knowledge mobilisation infrastructure. As this development has been running in parallel in many organisations and at different levels, there is now a question of how all the functions and activities could be co-ordinated and consolidated. While some level of collaboration exists between the different actors, there is certainly potential for more. For example, although the two initiatives described above cooperate on knowledge dissemination activities, it is unclear how they align their research agendas and broader knowledge mobilisation strategies. To use a metaphor recently applied to co-ordinating knowledge mobilisation: the Netherlands is currently reflecting on who is leading the “jazz band” of knowledge mobilisation (Maxwell, Sharples and Coldwell, 2022, p. 25[29]). While the Ministry of Education, Culture and Science could be a natural actor, in such a decentralised system as is the Netherlands, it might be perceived as overly controlling and putting the autonomy of other organisations at risk.
Equally, several initiatives are in early stages and more time is required to assess their effectiveness and impact on the quality of education policies and practices. While most initiatives invest in evaluating their impact, the diversity of these efforts will make it more difficult to disentangle the impact of each. Strategic co-ordination of the different efforts should also include a system-level approach to assessing their impact.
Expanding the approach: Lessons for others
Despite the outstanding question of what strategic leadership of knowledge mobilisation means, the Netherlands’ initiatives can be useful for other countries who may be looking to enhance their approach. They involve substantive explorations, experimentations, advice from a range of different stakeholders and a systematic reflection on their achievements and impact. Conditions for developing such a knowledge infrastructure in the Netherlands included:
political and policy emphasis on evidence-informed education practice;
support from schools and school boards;
an emerging teacher-led research movement.
These could be important levers in both centralised and decentralised education systems. In addition, the presence of a main actor – the NRO in the case of the Netherlands – can play a major role in preparing the ground for further initiatives by raising awareness of evidence use across the board.
The ESRC Education Research Programme runs between 2021-2026 and funds nine projects implemented in the four UK nations (see Box 3.4). The programme set out to “ask new questions about how research, policy and practice can come together to empower educators and create meaningful change” (University College London, 2024[30]). While each of the nine projects has its own theory of change, they are intended to interact and partner in their knowledge mobilisation approaches and learning journeys. The projects are supported by the programme team that is conducting research to understand how knowledge mobilisation efforts could be maximised and tailored for the context of the different UK nations. The research explores what a systems approach to knowledge mobilisation might look like in the context of different policy cultures in the four UK nations. This has taken the form of a comprehensive policy landscape mapping activity led by a research fellow at University College London (UCL).
The ESRC is part of United Kingdom Research and Innovation, a non-departmental public body funded by the government. The ESRC provides funding and support for research and training in the social sciences. In 2021 the ESRC commissioned an Education Research Programme to explore two themes:
teaching and learning, focusing on the role of teachers, their recruitment, retention and professional development;
the uses of technology for teaching and learning.
The five-year programme includes funding for a cohort of nine research projects based in universities across the United Kingdom. The Programme Director set an additional overarching vision for the programme: to explore new ways of working in partnership across the boundaries between education research, policy and practice in the four nations of the United Kingdom.
The main themes emerged from consultations with the research community and the commitment of the ESRC 2019 Delivery Plan to delivering a policy-relevant research programme. They also align with the Department for Education (DfE)’s Areas of Research Interest publication, which sets out the key areas in which the DfE welcomes evidence from the research community (Department for Education, United Kingdom, 2024[31]).
Source: OECD Survey of Knowledge Mobilisation in Education 2023; Interview conducted on Teams, June 2024.
Education policy landscape mapping in the UK four nations
There is a complex ecosystem of activities in education policy making, influencing how evidence is used outside of government, yet an overview of this ecosystem is missing. The rationale for the mapping is that both the policy contexts and the actors determine how evidence is taken up. As education is a devolved competency in the United Kingdom, these elements differ between its four nations.
The mapping looks at the underlying structure of knowledge mobilisation in the four devolved education systems. It seeks to answer the following questions:
What are the underpinning structural elements in each context that make a difference to how research reaches practitioners and policy makers, and the uses to which research is put?
How do contextual factors and policy levers drive different forms of knowledge mobilisation and research engagement in each of the four UK nations?
This activity is supported by a wider mapping of the United Kingdom landscape of independent, non-governmental organisations (such as charities and think tanks) that conduct research and produce evidence and knowledge relevant to education policy. The different projects in the programme are also engaged in the stakeholder mapping.
Methodology
The mapping activity has involved a desk-based review of the relevant literature and policy documents in the different nations. Documents that were examined include the Areas of Research Interest documents that are published by the DfE in England; in Scotland, the Scottish Government’s School Research Plan; in Wales, the National Strategy for Education, Research and Inquiry. Subsequently, interviews were held with key stakeholders across the UK nations to gain their perspectives on the strengths and weaknesses of the approaches adopted.
Emerging findings on structural elements
System characteristics that make a difference to knowledge mobilisation in each of the UK devolved education systems include:
the ways in which schools are organised locally and how responsibilities for their oversight are shared;
the processes through which curriculum and assessment arrangements develop over time;
the role that school inspection and performance data play in system monitoring and prompting system change.
These different structures and processes affect the terms on which researchers, policy makers and practitioners interact. Some devolved education systems place more emphasis on deeper collaboration between these parties than others and organise accordingly. These are dynamic systems that create a sense of knowledge needs and gaps, as well as incentives to move knowledge about. England stands out for its high levels of government control coupled with a market economy in knowledge mobilisation in which different actors are encouraged to compete for market share.
Regardless of the precise arrangements adopted, each system raises questions about where research fits in. Is the function of research to generate new knowledge, be more active in knowledge mobilisation, or become more engaged in knowledge exchange?
Emerging findings on drivers of research engagement
While the shared rationale for fostering research engagement is to improve system functioning in each devolved education system, the policy levers and policy spaces that encourage this vary. In a simplified model, knowledge mobilisation can operate in two directions: using push mechanisms to disseminate knowledge to users or using pull mechanisms that link researchers, practitioners and policy makers in dialogic ways that facilitate users making use of research that is relevant to them. The initial findings show that the policy culture and differences in the policy levers in the four nations of the United Kingdom influence how research evidence is taken up and whether push or pull mechanisms are more common:
In England, the research that gets through to practitioners is strongly influenced by a combination of its “What Works” agenda and policy levers that encourage research use as part of its high-stakes accountability system. These are primarily based on push principles. The most used research is what is featured in the EEF Teaching and Learning Toolkit, which focuses on RCTs and meta-analyses of interventions.
In Scotland, there is more emphasis on pull mechanisms established through knowledge exchange in different fora. The national strategy calls for cross-sector collaboration in “identifying joint priorities and shared evidence needs; synergising and aligning research agendas; the co-design and co-delivery of research and evaluation; and sharing learning” (Scottish Government, 2023, p. 9[32]). This leads to a wider range of research entering the system and an awareness of the different audiences it can inform.
In Wales, as part of the new Curriculum for Wales, teachers are encouraged to work with the research community to design and implement a curriculum that meets their learners’ needs. The government actively supports locally-based partnerships between schools and higher education institutions to help achieve this objective. This policy environment creates room for both knowledge push and knowledge pull.
Challenges and limitations
There has been much written about the influence of the policy context on evidence uptake (see e.g. (Ehren, 2019[33]; Walker et al., 2020[34]; Greany and Higham, 2018[35])). Given this, the challenge for this programme is to move beyond descriptive analyses and propose more varied processes of knowledge mobilisation and knowledge exchange that can work practically and constructively within different policy contexts, and with the unique constellation of actors that are present in that system. This is particularly challenging for projects whose implementation plans extend across different policy or political cultures. A finding from the ESRC Education Research Programme is that better understanding of how to co-ordinate across different contexts despite policy differences depends on knowledge exchange strategies that bring researchers and local policymakers into productive conversation early on.
A specific challenge for the programme has been navigating the planning of impact generation within a traditional funding structure. The original funding arrangement required the implementers to plan for impact at the beginning of the projects. However, such an early impact plan may struggle to take into account contextual drivers and barriers that are as yet unfamiliar and also liable to change. The implementers have expressed to the funder that planning for impact through knowledge exchange throughout the project cycle would be more effective and may require more flexible funding arrangements.
Expanding the approach: Lessons for others
Ultimately, understanding the influence of different policy cultures and contexts in a politically diverse country like the United Kingdom will help to illuminate how knowledge mobilisation strategies can be tailored to such different contexts.
Similar groundwork can be beneficial in the initial planning phases of knowledge mobilisation initiatives in any context to ensure that they are appropriate to the context and effectively use the levers available. This can involve a regularly updated mapping at the level of the broader policy system (e.g. regional or national level) of:
structural elements, such as institutions and their roles;
contextual drivers and policy levers of research engagement, as well as barriers.
This can be conducted (or co-ordinated) by a central intermediary in the system, such as a research institution or formal intermediary. The central mapping could then be examined and perhaps further expanded for smaller local contexts in which intermediaries operate.
This chapter examined the international landscape of knowledge mobilisation activities carried out by different types of organisations across OECD education systems. While many types of organisations engage in numerous activities to support the use of research in policy or practice, there are also clear gaps. Very little capacity building seems to be available for researchers and intermediary actors to develop brokerage skills. Certain types of organisations do not systematically carry out activities that would suit their profile (e.g. develop teachers’ research use skills in initial teacher education), while others conduct activities that may potentially (although not necessarily) carry risks to quality evidence use (e.g. education consulting firms’ wide-ranging involvement).
Importantly, these data call for an understanding of how knowledge mobilisation can be better co-ordinated at the wider system level. Is the field characterised by duplicative efforts (e.g. many evidence repositories exist) or are these complementary? In either case, mapping these efforts and understanding how they are similar or different – e.g. what type of evidence do they aim to mobilise, for which target audiences, and how – would help to create a more efficient knowledge mobilisation system. To this end, the heatmap presented in this chapter offers a tool that education systems can adapt and use in their own contexts. Transparency of intermediary functions and communication among these organisations could save costs and help target users, such as policy makers and practitioners, to navigate the myriad of knowledge mobilisation initiatives and platforms. Knowledge sharing and peer learning among intermediaries could contribute to improving their efforts.
Co-ordinating knowledge mobilisation at the system level
What do you think the optimal distribution of activities would be among different types of organisations in your system?
Activity: With a team in your organisation, identify five actors that already play a key role in knowledge mobilisation in your system and five others that are not yet active but you think should be. Draw a map of these organisations outlining their mission, the nature of their connections and their existing knowledge mobilisation activities.
Consider the following factors:
unique skills and capacity of the different organisations/actors from the perspective of knowledge mobilisation;
level of autonomy and independence from government;
the nature of existing relationships among organisations: strength of relationship, power relations and hierarchies, dependencies and accountability, competition and collaboration;
any other important factors you can think of.
Discuss strengths and weaknesses in the map you drew up: What works well? What could be improved and how? Which actor(s) should play a role in co-ordinating knowledge mobilisation in your system?
Improving knowledge mobilisation activities
What other types of knowledge mobilisation activities (not captured in this intermediaries’ survey) does your organisation or system engage in?
Which combinations of activities are common in your system? How would you characterise your system in relation to Table 3.3?
How effective do you think these activities are? What evidence is there on their effectiveness and what is missing?
Which of the activities should be scaled up and which ones should be scaled down? How would you go about that?
References
[15] Akkerman, S. and A. Bakker (2011), “Boundary Crossing and Boundary Objects”, Review of Educational Research, Vol. 81/2, pp. 132-169, https://doi.org/10.3102/0034654311404435.
[20] Autonomous University of Barcelona (2024), Bachelor’s degree in Teacher in Primary Education, https://web.ub.edu/en/web/estudis/w/bachelordegree-G1026?subjects.
[22] Autonomous University of Barcelona (2024), Master in Teacher Training in Obligatory Secondary and Upper Secondary School Education, Vocational Training and Languages, https://web.ub.edu/en/web/estudis/w/masteruniversitari-M2002?objectivesCompetencies.
[21] Autonomous University of Barcelona (2024), Solving Socio-Educational Challenges through Research: Hands on Experience, https://www.uab.cat/web/cursos/cursos-de-ciencias-sociales-y-juridicas/research-methods-in-education-1345905134700.html.
[14] Bakx, A. et al. (2016), “Boundary crossing by science teacher researchers in a PhD program”, Teaching and Teacher Education, Vol. 60, pp. 76-87, https://doi.org/10.1016/j.tate.2016.08.003.
[2] Burns, T. and T. Schuller (2022), “History and evolution of brokerage agencies in education”, in Who Cares about Using Education Research in Policy and Practice?: Strengthening Research Engagement, OECD Publishing, Paris, https://doi.org/10.1787/b2d2c2fc-en.
[31] Department for Education, United Kingdom (2024), Areas of Research Interest, https://www.gov.uk/government/publications/department-for-education-areas-of-research-interest.
[8] Education.org (2021), Calling for an Education Knowledge Bridge: A White Paper to Advance Evidence Use in Education, https://whitepaper.education.org/download/white_paper.pdf.
[13] Edwards, A. (2012), “The role of common knowledge in achieving collaboration across practices”, Learning, Culture and Social Interaction, Vol. 1/1, pp. 22-32, https://doi.org/10.1016/j.lcsi.2012.03.003.
[16] EEF (2024), Evidence for Education Network (EEN), https://evidence.education/about-us/the-network.
[9] Farley-Ripple, E. and S. MacGregor (2024), “A mixed methods study of education researchers’ knowledge mobilization approaches”, Journal of Educational Change, https://doi.org/10.1007/s10833-024-09517-w.
[19] Farrell, C. et al. (2021), Research-practice Partnerships in education: The state of the field, William T Grant Foundation, https://wtgrantfoundation.org/wp-content/uploads/2021/07/RPP_State-of-the-Field_2021.pdf (accessed on 19 October 2023).
[33] Godfrey, D. and C. Brown (eds.) (2019), Accountability structures that support school self-evaluation, enquiry and learning, Routledge.
[7] Gough, D., J. Sharples and C. Maidment (2022), “Evidence on evidence-informed policy and practice”, in Who Cares about Using Education Research in Policy and Practice?: Strengthening Research Engagement, OECD Publishing, Paris, https://doi.org/10.1787/5ee7de7c-en.
[35] Greany, T. and R. Higham (2018), Hierarchy, Markets and Networks: Analysing the ‘self-improving school-led system’ agenda in England and the implications for schools, UCL Institute of Education Press, https://www.nuffieldfoundation.org/wp-content/uploads/2019/11/Hierarchy-Markets-and-Networks.pdf.
[27] Kennisrotonde (2024), Stel jouw onderwijs- of jeugdvraag, https://www.kennisrotonde.nl/stel-je-vraag (accessed on 16 December 2024).
[18] Langer, L., J. Tripney and D. Gough (2016), The Science of Using Science: Researching the Use of Research Evidence in Decision-Making, EPPI-Centre, Institute of Education, University College London.
[29] Maxwell, B., J. Sharples and M. Coldwell (2022), “Developing a systems‐based approach to research use in education”, Review of Education, Vol. 10/3, https://doi.org/10.1002/rev3.3368.
[24] NRO (2024), NRO Granting, https://www.nro.nl/en/granting-0 (accessed on 25 November 2024).
[25] NRO (2022), Knowledge Agenda for Education, NRO, https://www.nro.nl/sites/nro/files/media-files/Knowledge%20agenda%20for%20education_0.pdf.
[26] NRO (2019), From practical issue to research question, https://www.nro.nl/sites/nro/files/media-files/Kennisrotonde-From-practical-issue-to-research-question_0.pdf.
[23] OECD (2024), Education at a Glance 2024: OECD Indicators, OECD Publishing, Paris, https://doi.org/10.1787/c00cad36-en.
[12] OECD (2024), “The state of academic careers in OECD countries: An evidence review”, OECD Education Policy Perspectives, No. 91, OECD Publishing, Paris, https://doi.org/10.1787/ea9d3108-en.
[6] OECD (2024), “Yes Minister, Yes Evidence: Structures and skills for better evidence use in education policy”, OECD Education Policy Perspectives, No. 96, OECD Publishing, Paris, https://doi.org/10.1787/6f97bcda-en.
[5] OECD (2023), Who Really Cares about Using Education Research in Policy and Practice?: Developing a Culture of Research Engagement, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/bc641427-en.
[1] OECD (2022), Who Cares about Using Education Research in Policy and Practice?: Strengthening Research Engagement, Educational Research and Innovation, OECD Publishing, Paris, https://doi.org/10.1787/d7ff793d-en.
[17] Oliver, K. (forthcoming), “What factors influence evidence use in policy? An updated systematic review [working title]”, Evidence and Policy.
[3] Oliver, K. et al. (2022), “What works to promote research-policy engagement?”, Evidence & Policy, pp. 1-23, https://doi.org/10.1332/174426421x16420918447616.
[10] Phelps, D. (2019), “The Challenges of Bridging the Research–Practice Gap through Insider–Outsider Partnerships in Education”, Teachers College Record: The Voice of Scholarship in Education, Vol. 121/12, pp. 1-28, https://doi.org/10.1177/016146811912101202.
[11] Schlicht-Schmälzle, R. et al. (2024), “Bridging the research-practice gap in education: Initiatives from 3 OECD countries”, OECD Education Working Papers, No. 319, OECD Publishing, Paris, https://doi.org/10.1787/c0d3f781-en.
[32] Scottish Government (2023), Scottish Government’s Plan for School Research 2023-2026, https://www.gov.scot/publications/scottish-governments-plan-school-research-2023-2026/.
[4] Torres, J. and M. Steponavičius (2022), “More than just a go-between: The role of intermediaries in knowledge mobilisation”, OECD Education Working Papers, No. 285, OECD Publishing, Paris, https://doi.org/10.1787/aa29cfd3-en.
[30] University College London (2024), The ESRC Education Research Programme 2021-26, https://www.ucl.ac.uk/education-research-programme/.
[28] van den Berg, E. and H. Prins (2023), Evidence-informed beantwoorden van onderwijsvragen: Evaluatie van de Kennisrotonde (Evidence-informed answers to educational questions: The evaluation of the Knowledge Roundabout, SEO Economic Research, https://www.nro.nl/nieuws/evaluatie-kennisrotonde-succesvol-en-nog-niet-uitgegroeid.
[34] Walker, M. et al. (2020), Understanding the challenges for evidence-informed school improvement support in disadvantaged schools: An exploratory study, Education Endowment Foundation.
Notes: The exploratory factor analysis was conducted with Promax rotation and maximum likelihood method. The four-factor model was selected based on Eigen values greater than 1.
← 1. Profiles were determined based on exploratory factor analysis conducted with the Factor Analyzer package of Python, using promax rotation and maximum likelihood method. (https://pypi.org/project/factor-analyzer/). Eigen values and factor loadings are provided in Annex Table 3.A.2.
← 2. Ontwikkelkracht means “development power”, which refers to the fact that the programme is developing a major infrastructure