This chapter presents the methodology and evidence base specifically developed to assess the added value of mission-oriented innovation policies. In particular, a mission theory of action is used to systemically track the process on which the expected mission “systemic effects” rely.
Mission‑Oriented Innovation Policies for Net Zero
How Can Countries Implement Missions to Achieve Climate Targets?
Abstract
There exist very few fully fledged evaluations of mission-oriented innovation policies (MOIPs) that could provide evidence of whether they meet their ambitious objectives and, given their long-term and impact-based objectives, it is likely that there is still a long way to go before enough evaluations are available to draw robust conclusions on their ability to deliver on their many promises. However, as previously mentioned, many missions were launched three to five years ago and are facing increasing political and administrative pressures to justify their co‑ordination efforts and budgets.
In response to this emerging demand for accountability, a complementary approach is proposed: while direct evidence of MOIPs’ contributions to objectives such as greenhouse gas (GHG) emissions reductions or job creation might be elusive at this stage, it is possible to track the key processes that lie at the heart of the MOIP approach and which should lead to such impacts and assess whether they unfold as expected and produce their “systemic effects”. Rather than a substantive evaluation (is the mission being realised and the selected challenge solved?), this chapter focuses on a process-tracking evaluation in the early years of this policy approach (is the MOIP approach relevant to tackle the selected challenge?).
For that purpose, it is necessary to develop a theory of action, a tool that is increasingly used in evaluation of complex interventions, along theories of change (Box 3.1). In the case of MOIPs, the theory of action should surface the expected causal relationships between the problems MOIPs tackle and their desired goals, inputs, outputs, outcomes and impacts. This theory of action is presented in this chapter and used in subsequent chapters to analyse the database of 101 net zero missions and 17 in‑depth case studies. It is also used to present the results of this work, distinguishing between the outputs and outcomes of net zero missions and structures this report, providing insights on the contribution of net zero missions to three expected outputs, followed by three expected outcomes.
In its most general understanding, a theory of change is a set of beliefs about how change happens (Church and Rogers, 2006[1]). More precisely, in the context of a policy intervention or programme, it is the logical model that explains how the objective of the intervention is achieved, including the underlying assumptions and beliefs on causal relationships. It follows step-by-step the result chains that link the interventions’ inputs and activities to the attainment of its stated objectives (usually working backwards from the objectives). It was first proposed as a tool for evaluating complex programmes, whose basic assumptions about the relations between inputs, outcomes and impacts were most often poorly articulated (Weiss, 1995[2]).
A theory of change is usually presented in a visual diagram that provides at a glance the main expected impact linkages. It does not provide a specific implementation plan but is a tool to collectively make explicit and discuss the hypotheses that guided the design of the intervention in the first place and create a consensus on these hypotheses in the likely advent of diverging views. Once agreed upon, these hypotheses are represented as causal relationships in the diagram. The tool’s main benefits are, therefore, as much in the process as in the final results (i.e. the diagram obtained at the end).
Alongside these theories of change, ‘theories of action’ (ToAs) are proposed as a tool to track the specific mission processes and the systemic effects they are expected to generate to tackle a specific challenge. In a nutshell, a theory of action is inward-looking and process-oriented. It is “the delivery model for a Theory of Change” (Georges, 2024[3]). It describes how the mission is designed and set up to realise the mission. It maps the chains of causality that connect the way missions are designed and governed to the impact they are expected to produce. Rather than only focusing on the substantive dimension of missions (is the mission being realised and the selected challenge solved?), mission theories of action track the key processes that lie at the heart of the MOIP approach and allow assessing whether they unfold as expected and produce their ‘systemic effects’ (i.e., is the adopted approach effective to tackle the selected challenge?)
Theories of change and theories of action are therefore the two faces of the same coin, providing complementary views on the expected process of change that should lead to the realisation of the mission. It is argued that mission theories of action can prove particularly instrumental to both mission managers and evaluators in a developmental approach. In the case of MOIPs, ToC should surface the expected causal relationships between the problems these policies tackle and their desired goals, inputs, outputs, outcomes, and impacts (Janssen et al., 2021[4]; Hekkert et al., 2020[5]). These maps of mission ‘contribution claims’ can prove particularly instrumental to guide and monitor their portfolio of diverse activities, in order to check that they are consistent among themselves and aim in the right direction.
When used for evaluating a given programme, the theory of change can be mobilised to systematically examine the causal relationships at each step. Depending on the type of policy or programme, indicators or markers can be associated with the different connections.
Sources: Church and Rogers (2006[1]); Weiss (1995[2]); (Janssen et al., 2021[4]; Hekkert et al., 2020[5]).
As previously mentioned, MOIPs are an additional institutional layer to direct, co‑ordinate and integrate new – and most often already existent – interventions. To validate this policy approach, it is necessary to substantiate that they produce additional benefits relative to traditional STI policy frameworks, and that these benefits outweigh the additional transaction costs. The validation of this policy approach thus depends on them successfully passing a stringent double test: not only must they accomplish their missions, but they should do so more effectively and more efficiently than would have been the case with traditional approaches, or they should fulfil objectives that are beyond the reach of these traditional approaches (Larrue, Tõnurist and Jonason, 2024[6]).
Using the evaluation terminology, MOIPs must prove two types of additionality (Figure 3.1):
1. The “first-order” additionality relates to the additional net effects of any given intervention relative to what would have been the case in the absence of public intervention.
2. The “second-order” additionality is specific to MOIPs. It captures the additional effects relative to a counterfactual scenario that is not a “no intervention” scenario but a “traditional STI policy” scenario.
Note: STI: science, technology and innovation.
Quantitative assessments of “first-order” additionality have often proven difficult due to the difficulty to run counterfactual scenarios through pairing techniques used in random control trials, for instance in clinical, development aid or education programme evaluations. In the STI domain, the beneficiaries of an innovation scheme are most often of a different nature than those who did not apply to the scheme or were not selected. The resulting selection bias makes any comparison between beneficiaries and non-beneficiaries (constituting the counterfactual sample) hardly meaningful.
Assessing the “second-order” additionality is even less suited to quasi-experimental evaluation methods since the MOIP intervention scenario is not a substitute for but an addition to the “baseline” scenario. It is, therefore, necessary to recourse to mainly qualitative evidence drawn from desk review and interviews with mission leaders and partners.
Against this backdrop, the MOIP theory of action seeks to lay the expected additional effects on top of those produced by existing policy instruments to guide field work. It is used as a map upon which the net zero mission instruction templates and interview guidelines were directly based.
Building upon the analytical framework presented above, the net zero theory of action describes how MOIPs are expected to produce their impacts, in direct relation to their specific features. More precisely, it presents the set of expected causal relationships between the design principles of this policy approach and its intended outputs, outcomes and impacts (Figure 3.2). Along the lines of the aforementioned typology of missions (Kuittinen, Polt and Weber, 2018[7]), two types of impacts are distinguished: 1) change acceleration; and 2) system transformation. The various causal relationships are based on the results of previous OECD work on MOIPs, a review of the literature on missions and multiple interactions with scholars during the project.1
Note: MOIP: mission-oriented innovation policy.
The MOIP design principles are the starting point of the theory of action. Together they constitute the “genetic traits” of a given mission. For instance, in the strategic orientation building block, the direct expected outputs of the directionality and intentionality, legitimacy and flexibility design principles are notably clearer and flexible objectives and directions to realise the missions, a broader and higher degree of consensus among stakeholders and possibly citizens on the necessity to realise these objectives, and stronger backing of high-level political and policy officials.
The second stage of the theory of action includes the intermediate outcomes of the MOIPs. For instance, the clear and legitimate shared agenda is expected to include a broader set of potential options directed towards more ambitious and long-term objectives. In combination with elements of the policy co-ordination building-block, this shared agenda should also result in a higher budget for the mission, as well as stronger buy-in and commitment from private actors and stakeholders involved in the realisation of the mission, and the support to a more comprehensive range of activities being supported (i.e. not only technological aspects but also elements related to the needed skills, infrastructure, regulatory framework, user awareness, etc.). Finally, a wide range of actors across all silos, strongly committed to achieving the mission through the exploration of a broader set of potential options and a wider range of supported activities, benefitting from greater resources, could obtain solutions with higher performance, which can be articulated in a more systemic way. Down the causal thread, as previously mentioned, this should result in either change acceleration or system transformation.
The study proceeded through two parallel strands of field work: 1) extensive fieldwork based on the analysis of a database of over 100 net zero missions; and 2) intensive fieldwork by undertaking case studies of 17 net zero missions.
3.3.1. The Net Zero Mission Database
The OECD has identified 148 net zero missions pertaining to 43 MOIP initiatives in 23 countries based on previous work done on MOIPs and other projects under the Committee for Scientific and Technological Policy (Figure 3.3). These missions directly or indirectly aim to address climate change through mitigation or adaptation.2 The database3 provides information on each mission’s objectives, targets, funding, technological areas and application sectors, as well as more exploratory features like mission rationales.
While the Net Zero Mission Database is not exhaustive, it includes the main missions that match the OECD definition. It is, however, worthwhile keeping in mind a few caveats, as presented in Box 3.1.
Although the Net Zero Mission Database is meant to be as exhaustive as possible, some relevant initiatives are not included for different reasons. First, the mission-oriented innovation policy (MOIP) definition leaves some room for interpretation as to which initiatives are mission-oriented or not. Although it obeys clear design principles, this policy approach is not totally new and distinct vis-à-vis the existing policy landscapes. The difference is at times more a matter of degree than of nature and the question of where to draw the line between what is and what is not mission-oriented relies on the analysts’ appreciations. Furthermore, some proto-missions, still at the stage of thematic calls for proposals, evolve and become gradually closer to MOIPs. The abusive use of the term “mission-oriented”, which has now become a buzzword in some domains (“mission washing”), sometimes makes the distinction even more complicated. A key element that makes this appreciation more robust is the comparison between different initiatives to obtain a rather consistent group of policies in terms of how far they go on the different dimensions of mission orientation. Moreover, besides a few borderline cases, most identified initiatives are clearly mission-oriented, recognised as such by policy makers and analysts.
Other reasons for not including some missions are related to the nature of these initiatives or the scope of this study:
Some MOIP initiatives are too novel to be included. New initiatives arise every year, including at regional levels, and several countries and regions are currently considering the adoption of an MOIP approach or the creation of new initiatives.
Very few missions have yet been identified outside advanced economies. This does not mean that other, emerging, economies do not apply any of the MOIP design principles such as stakeholder engagement or cross-sectoral co-ordination. However, it is less likely to find initiatives that combine these principles in a consistent way due to problems of policy and governance capacity in emerging countries. As part of the Mission Action Lab, the OECD is working on the potential of the MOIP approach to support sustainable development in developing countries (Kumpf, Mucarsel and Zavarella, 23 June 2023[8]).
The database focuses on missions led by national public authorities. Only one regional mission (in Wallonia) is included, although several local initiatives come close to the design and “spirit” of ecosystem-based MOIP initiatives, in particular in the context of the new generation of Smart Specialisation Strategies that drive the allocation of European Union Structural Funds dedicated to research and innovation during the programming period 2021-27.1 Many cities also use a mission approach to drive their efforts towards carbon-neutrality, not least in Europe in the context of the EU ‘Climate-Neutral and Smart Cities’ mission. These missions at city level are not included in the database. Only the national frameworks to support these cities are included.
All missions included in the database aim to act primarily upon research and innovation to tackle climate change, as per the OECD MOIP definition. Other policies adopting an MOIP approach exist in other policy areas, involve little or no STI development.2 However, to date, these missions seem rare.
Finally, this database focuses on mission-oriented policies, not mission-oriented organisations. It, therefore, does not include the specialised mission-oriented and high-risk high-reward agencies such as the Defense Advanced Research Projects Agency (DARPA) and Advanced Research Projects Agency-Energy (ARPA-E) in the United States, and the Advanced Research and Invention Agency in the United Kingdom. These agencies operate activities that match several of the MOIP design principles, which make them close to “acceleration missions” as previously defined but remain, however, mostly confined to calls for proposals to address scientific and technological obstacles.
Notes: 1. For instance, the Research and Innovation Strategy for the Smart Specialisation of Catalonia (RISCAT3), which promotes the development of territorial shared research and innovation agendas. These shared agendas “articulate, through a participatory model of governance, the collective action of various actors aimed at addressing a common challenge (usually related to the SDGs) in the territory and the problems that this challenge may generate” (Fernández and Romagosa, 2020[9]). 2. For instance, the Delta Programme in the Netherlands was included in the mission-oriented policy observatory commissioned by the European Commission (European Commission, 2018[10]). This mission aims to protect the country from flooding, mitigate the impact of extreme weather events and secure supplies of freshwater. <!!Best practice on boxes: see page 20 of the OECD Style Guide.
As shown in Figure 3.3, most of the identified net zero missions are implemented in European countries (around 80% of missions), with a strong concentration in northern Europe. Nordic countries alone represent 33% of the net zero missions in this study (31% of MOIP initiatives). Their missions are not only more numerous, but also have a higher degree of “mission orientation” as previously defined, while missions such as the Spanish “CDTI misiones” are “borderline cases”. Outside the European Union, represented countries include Australia, Brazil, Canada, Chile, Colombia, Japan, Korea, the United Kingdom and the United States.
Note 1: MOIP: mission-oriented innovation policy. Note 2: This map includes all net zero missions identified as of October 2024. Note 3: The list of net zero missions is available in Annex B.
Source: OECD Net Zero Mission Database.
Among the net zero missions identified as of October 2024, 101 related to 30 MOIP initiatives in 20 countries have been included in the analyses of this report (Table 3.1). The Table 3.1 includes the subset of these net zero missions that are included in this report’s analysis.
Note 1: CSIRO: Commonwealth Scientific & Industrial Research Organisation; NRC: National Research Council. Note 2: Detailed information on most of these MOIPs is available on the OECD online MOIP Toolkit: https://stip.oecd.org/moip. Note 3: This table only include the 101 missions included in this report’s analysis.
Source: OECD Net Zero Mission Database.
A majority of net zero missions pertain to the most widely known “stereotypes” of MOIPs, i.e., challenge-based programmes and schemes (44 missions) and the overarching mission-oriented strategic framework (32 missions). Challenge-based programmes draw on a significant extent on the “DARPA model” characterised notably by clearly defined objectives and targets related to a challenge (often of technological nature); continuity of support across the innovation stages, including market deployment, thanks to strong linkages with potential users/buyers of the technology; and proactive management and hands‑on management of the portfolio of activities by a powerful “programme director” (Bonvillian, Van Atta and Windham, 2019[11]). Overarching strategic frameworks correspond more closely – at least in the initial intentions – to the model of MOIPs presented by Mazzucato, in which a wide range of public and private actors jointly develop broad systemic missions associated with targets and milestones and pool together their resources and modes of intervention to realise them through collective action (Mazzucato, 2018[12]). Building upon the OECD typology, Uyarra (2023[13]) added a fifth type, namely the place-based micro missions that encapsulate the many local and city missions, such as Greater Manchester’s net zero mission. These missions are numerous and extend beyond the 100 cities of the EU “Climate-neutral and smart cities”’ mission. As explained in Box 3.2, the OECD database only includes the national frameworks to support these cities, e.g. the Austrian Climate-neutral City mission, the Swedish Viable Cities mission and the EU mission itself.4
Notes: MOIP: mission-oriented innovation policy. See Chapter 2 for the definitions of the MOIP types.
While the reference to mission orientation in the STI policy area dates back several decades, the earliest “modern” mission-oriented policies, i.e., those aiming primarily to address societal challenges through a targeted and co-ordinated approach, were launched in the 2010s. Most net zero missions were launched between 2018 and 2021 (81%) (Figure 3.4). This coincides with the growing communication around this new policy approach in relation to the launch of the EU missions as part of Horizon Europe. This sudden surge of MOIPs is also related to the general increase of funding for research and innovation activities aimed at mitigating GHG emissions. However, a similar trend for missions in health and other areas can be observed, which indicates that it is more related to a policy rather thematic dynamic. Only a limited number of missions were launched before 2017 (2% of net zero missions). These “early” MOIPs, which were developed even before the policy approach itself gained salience, are not only less numerous, but also more remote from the specific, target or metrics-driven mission, along with strong governance structure and measurable progress.
There has been a significant slowdown in new missions since 2022 as countries strive to implement existing missions, incrementally increase their “degree of mission orientation”, and wait for the results of assessments and evaluation before launching new ones. However, some missions have been launched more recently (for instance two MOIP initiatives have been launched in Brazil)5 or are being developed (such as in the United Kingdom, where the new government elected in 2024 intends to use a ‘mission-driven government’ approach).
Source: OECD Net Zero Mission Database.
3.3.2. In-depth case studies of net zero missions
Seventeen net zero missions have been selected to form a sample that reflects the diversity of MOIP types and thematic areas. The case studies included two main steps. First, the OECD pre-filled a dedicated template for individual cases based on available desk research and materials.6 This template was used as the main background document to support interviews with one or several persons directly involved in the mission, including the identified “mission leader” then relevant partners to fact-check and complement the template. About 40 interviews had been performed in the context of this study, representing 63 interviewees. The interview guidelines were structured along the net zero mission theory of action.
A series of dedicated sessions within the Committee for Scientific and Technological Policy meetings enabled a comparison of some net zero missions and provided a platform for discussion among policy makers and the relevant stakeholders.7
References
[11] Bonvillian, W., R. Van Atta and P. Windham (2019), The DARPA Model for Transformative Technologies: Perspectives on the US Defense Advanced Research Projects Agency, Open Book Publishers, Cambridge, United Kingdom, https://doi.org/10.11647/OBP.0184.
[1] Church, C. and M. Rogers (2006), Designing for Results: Integrating Monitoring and Evaluation in Conflict Transformation Programs, Search for Common Ground, Search for Common Ground, Washington, DC, https://documents.sfcg.org/Documents/manualpart1.pdf.
[10] European Commission (2018), Mission-oriented Research and Innovation Inventory and Characterisation of Initiatives: Final Report, Publications Office of the European Union, https://doi.org/10.2777/697082.
[9] Fernández, T. and M. Romagosa (2020), Articulating Shared Agendas for Sustainability and Social Change: A Contribution from the Territory to the EU Debate on Transitions to Sustainability, http://catalunya2020.gencat.cat/web/.content/00_catalunya2020/Documents/angles/fitxers/shared-agendas.pdf.
[3] Georges, R. (2024), What is a theory of action, https://intdev.tetratechasiapacific.com/what-is-a-theory-of-action/#:~:text=the%20program%20theory.-,A%20Theory%20of%20Action%20is%20the%20delivery%20model%20for%20a,for%20individuals%2C%20groups%20or%20communities.
[5] Hekkert, M. et al. (2020), “Mission-oriented innovation systems”, Environmental Innovation and Societal Transitions, Vol. 34, pp. 76-79, https://doi.org/10.1016/j.eist.2019.11.011.
[4] Janssen, M. et al. (2021), “The promises and premises of mission-oriented innovation policy: A reflection and ways forward”, Science and Public Policy, Vol. 48/3, pp. 438-444, https://doi.org/10.1093/scipol/scaa072.
[7] Kuittinen, H., W. Polt and M. Weber (2018), “Mission Europe? A revival of mission-oriented policy in the EU”, in RFTE (ed.), RE:thinking Europe.
[8] Kumpf, B., L. Mucarsel and A. Zavarella (23 June 2023), “Mission-oriented innovation: A suitable approach for just net zero transitions in the Global South?”, OECD Development Matters Blog, https://oecd-development-matters.org/2023/06/23/mission-oriented-innovation-a-suitable-approach-for-just-net-zero-transitions-in-the-global-south (accessed on 23 June 2023).
[6] Larrue, P., P. Tõnurist and D. Jonason (2024), “Monitoring and evaluation of mission-oriented innovation policies: From theory to practice”, OECD Science, Technology and Industry Working Papers, Vol. No. 2024/09, https://doi.org/10.1787/5e4c3204-en.
[12] Mazzucato, M. (2018), “Mission-oriented innovation policies: Challenges and opportunities”, Industrial and Corporate Change, Vol. 27/5, pp. 803-815, https://doi.org/10.1093/icc/dty034.
[13] Uyarra, E. (2023), Creating National Governance Structures for the Implementation of EU Missions, Discussion Paper for the Mutual Learning Exercise on EU Missions Governance, PSF Challenge – Horizon Europe Policy Support Facility, Directorate-General for Research and Innovation.
[2] Weiss, C. (1995), “Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families”, in Connell, J. (ed.), New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts, pp. 65-92, Aspen Institute, https://canvas.harvard.edu/files/1453087/download?download_frd=1&verifier=IVZpf0ynt3iriSXpb8lE7WirRBXUHfbceDQUHleG.
← 1. As a theory of action is a living tool that can evolve though interactions and reality checks, the present version should be considered as a milestone, not a final product.
← 2. Detailed information on most of these MOIP initiatives and their missions is available on the OECD MOIP Online Toolkit. This platform went through a major update in October 2024 and is updated regularly. See https://stip.oecd.org/moip/.
← 3. As of 30 October 2024. See annex B.
← 4. These missions are, therefore, included in the database as “overarching mission-oriented strategic frameworks”. However, each city involved in these frameworks should, in principle, be considered as a mission in itself since they all have their own objectives, cross-departmental governance structure and portfolio of interventions.
← 5. The ten structuring and mobilizing programmes are a new approach to focus the investments that the National Fund for Scientific and Technological Development (FNDCT) funds on national sustainable development priorities. In addition to the FNDCT’s mission-oriented approach, Brazil has another set of six missions that have been guiding its new industrial policy – Nova Indústria Brazil – since January 2024.
← 6. Completed templates are about 20-25 pages long. They are structured around four sections: 1) the mission-oriented policy initiatives section provides synthetic information on the policy initiative that “hosts” the net zero mission, including the overall context, how and why it was launched, how it is governed, funded, implemented and evaluated; 2) the net zero mission basic characteristics section contains information on the main features of the net zero mission, including its objectives, timeline, scale and scope, and its evolution; 3) the net zero mission process section provides detailed information on the main processes and mechanisms to design/select the mission, govern and implement it, manage its portfolio, and evaluate it; and 4) the net zero mission strengths and weaknesses section.
← 7. A first “net zero mission seminar” was organised as part of the 119th meeting of the CSTP on 20‑22 October 2021. The mission leaders of three missions were invited to present and discuss the strengths and weaknesses of their mission to tackle the climate neutrality challenge: V. Bryony Livesey, Director at the Industrial Strategy Challenge Fund (Industrial Decarbonisation Challenge); Stine Jørgensen, Deputy Director at the Danish Agency for Education and Research (Green Missions); and Lisa Higgins, Director at Research Ireland (Net Zero Research Challenge). A second “net zero mission seminar” was organised at the 121st meeting of the CSTP. It convened Joanna Franzén, Strategic Area Lead Sustainable Food Systems, Innovation Management, Vinnova (Healthy Sustainable Food, Vinnova’s pilot missions); Guus Mulder, Program Manager, Top Consortium for Knowledge and Innovation Urban Energy (A Carbon-free Built Environment by 2050, Mission-driven Top Sectors and Innovation Policy); and Hans‑Günther Schwarz, Federal Ministry for Climate Action, Environment, Energy, Mobility, Innovation and Technology, DG Innovation (Climate-neutral Cities, Austrian Transition Missions).
A panel on mission-oriented policy for net zero was also organised on 25 October 2022 as part of the Science, Technology and Innovation Outlook 2023 High-level Debate, using the synthesis of the net zero mission study published in the Science, Technology and Innovation Outlook (OECD, 2023) to feed discussions on the added value of the mission approach to reduce greenhouse gas emissions. The panel was composed of Christina Abildgaard, Executive Director, Research Council of Norway; Julien Guerrier, former Director for Research and Innovation, European Commission; and Jeroen Heijs, Director for Innovation and Knowledge, Ministry of Economic Affairs and Climate, Netherlands. The panel was moderated by Christian Naczinsky, Head of EU and OECD Research Policy Department, Austrian Ministry of Education, Science and Research.