This chapter sets out a 5 step toolkit covering the key questions that governments and QI bodies should gather data on to enable a thorough understanding of their standards and quality assurance landscapes in their national contexts, and how that system interacts with international co-operation on standards and quality assurance. This toolkit can help governments to understand the gaps and challenges in their standards and quality assurance system, and identify areas of reform to enable effective use of the QI system for global challenges. This toolkit builds on a range of existing guidance on effective utilisation of standards and quality assurance (QI) systems to meet public policy goals.
Reinforcing Regulatory Frameworks through Standards, Measurements and Assurance
Making Better Use of Quality Infrastructure in Policymaking
Abstract
Although the QI system exists across the world and underpins trade, there remains challenges to ensure the most effective use of the system to enable optimised interactions between QI and regulation, nationally and internationally, to enable the best outcomes for public policy goals.
One of the biggest challenges is that the QI system operates and is used somewhat under the radar. This poor visibility means policymakers may not have full knowledge of their national QI systems or of the international co-operation mechanisms available to them via their national QI bodies. This means that QI is currently not necessarily used in the most optimised way possible, and potentially regulatory processes are missing out on efficiencies as a result.
Another key challenge is the heterogeneity of the QI systems around the world. Countries have not set up their QI systems the same way. However, the international co-operation built into the QI system is critical to ensuring that governance of product and services is recognised across borders, and there is alignment in best practices across the inherent heterogeneity of the QI system – i.e. a system that works practically locally, but aligns internationally. This has not yet been utilised to its full by regulatory and policy objectives on urgent global challenges like environmental governance and the digital transformation.
This checklist aims to foster a deeper understanding among policymakers of their QI systems, and increased interactions between QI and regulatory stakeholders to optimise both QI and regulation for better policy outcomes. This document builds on the comprehensive guidance documents that exist on better regulation and standards and quality assurance systems, referred commonly in literature as quality infrastructure or QI, which will be referenced throughout the checklist, and specifically adds guidance on the interlinkages between QI and regulation.
This checklist has been developed to provide practical guidance and advice on reforming standards and quality assurance system and its interactions with regulation to create a more effective and efficient QI system, including in the context of global challenges facing governments today, such as environmental governance and the acceleration of AI technologies.
Based on analysis led in 2023-24, this checklist proposes questions to help central government policymakers, regulators and QI bodies to reflect on their governance and processes to better leverage the existing actors around them, collaborate more efficiently, and take forward the recommendations set out in this report to enable better policy design, implementation and reform.
The audience for this checklist are:
Governments, including policymakers, regulators and civil servants: To help policymakers at all levels assess their current QI system, its governance and institutions, and determine how this system can be more efficiently leveraged in interactions with regulations to deliver public policy objectives.
Standards, measurement, accreditation and market surveillance bodies (Quality Infrastructure or QI) institutions): To help QI institutions assess how they interact with and support regulation, identify their channels of communications with regulatory bodies at all stages of the regulatory cycle, and address any gaps and challenges in collaboration with government to enable a more efficient QI system.
This checklist covers five steps to map out the key actors of the QI and regulatory system, considering the governance and design of QI and regulation, the enforcement and monitoring of regulatory outcomes delivered via QI, and future gazing reform and emerging challenges (Figure 5.1).
The key steps this checklist covers are:
Step 1 - Who are the actors? Clarifying the existing national standards and quality assurance (QI) system and how it operates.
Step 2 – What are the common objectives between regulatory actors and QI actors? Defining the common objectives and normative policy basis for QI and regulation collaboration, and national and international policy priorities on strategic and emerging challenges.
Step 3 - How are these actors currently interacting? Clarifying how QI bodies and government bodies interact currently, in a formal or informal capacity.
Step 4 - How can policies be better designed through improved interactions between actors? Identifying how regulation supports QI, and QI supports regulation to implement better policies, and the design of QI-regulation collaboration can enable better outcomes and increased growth through international co-operation.
Step 5 - How can we implement, monitor and reform QI-regulations interactions for better regulatory outcomes? Monitoring and reforming the delivery of regulation, and the QI tools and services utilised, to enable better collaboration, and better regulatory outcomes.
For each step, there are overarching high level questions that are crucial for governments and QI bodies to consider, and further guidance is signposted for a detailed deep dive into specific issues.
Step 1. Who are the actors in your national QI and regulatory systems?
Context
To ensure effective interactions between QI and regulations, it is imperative to first understand the National QI system, as the framework of bodies delivering QI services varies between countries, dependent on resourcing and market demand.
Questions
To aid a more rapid assessment of the national standards and quality assurance system, it is recommended that governments collect information relating to the following questions centred around each pillar of QI:
1. Standards: Does one or more government recognised National Standard Body exist? If more than one government recognised standards bodies exist, do they co-ordinate nationally? Are they independent of government or situated within government?
2. Metrology: Is there a government recognised National Metrology Institute? Are there any other metrology institutes recognised by government? Are these bodies independent of government or situated within government? If there are multiple bodies related to metrology, does the National Metrology Institute co-ordinate them?
3. Conformity Assessment: How is conformity assessment or assurance (testing, inspection, certification) conducted in your country? Is it only government recognised conformity assessment bodies, or a mixture of public private bodies, or only private sector bodies?
4. Accreditation: Does one or more government recognised National Accreditation Body exist? Are they independent of government or situated within government?
5. Market Surveillance: Is there a single government department or regulator that oversees market surveillance? If not, are there clusters of government departments or regulators that oversee market surveillance together? Are these market surveillance related activities sitting at regional/state level or national level government departments and regulators?
6. Other actors: Once key components of QI are identified, consider what other organisations outside of the traditional QI system may be relevant? Examples include organisations which consider intellectual property, or public advice bureaus (e.g. councils on consumer safety, councils on the environment etc)?
Outcome of Step 1: Answering these questions can help you build a connective map (see example Figure 5.2) of your country’s standards and quality assurance (QI) system, how each component is set up, whether there are multiple bodies for each QI pillar.
Source: Author’s elaboration on core pillars of QI, and their interactions with regulatory and government bodies.
Step 2. What are the common objectives between regulators and national standards and quality assurance (NQI) systems?
Context
Using the outputs from Step 1, the following questions help build out the details of relationships between regulatory actors and the national standards and quality assurance system, commonly known as the national quality infrastructure (NQI) systems. Understanding current NQI governance structures can help governments and regulators understand where in the regulatory cycle they are already interacting with QI bodies, and if reforms would increase efficiency and deliver positive impacts for policy delivery. It can also help policymakers better understand how the QI-regulation interactions can be leveraged for global challenges such as environmental governance and the advent of AI technologies.
Questions
Define the common objectives, policy or normative basis for QI and regulators collaboration:
1. Does the government have a cross-sectoral national strategy or national policy on standards and quality assurance in place (often referred to as “national quality policy”), and is it regularly updated? Are there implementation plans that show progress relative to the strategy?
2. Are there existing legislation and regulations which govern QI pillars (e.g. a Standards Act, a technical regulation framework, an Accreditation Act, a Metrology Act)? Are these referenced in the national quality policy if one exists?
3. If there is existing legislation as per above question, is it tied into the policy design and development processes, including regulatory impact assessments?
4. Are there sectoral policies defining roles and responsibilities for regulators and standards, measurement, conformity assessment, accreditation and market surveillance organisations respectively?
5. Does your government have an overarching policy or strategy on better regulation? If so, does this have any references to national standards and quality assurance (often referred to as quality infrastructure or QI)?
6. Does your government have a technical regulation framework? If so, does this link directly with the national QI policy or reference national standards and quality assurance (often referred to as quality infrastructure or QI) organisations?
7. Is there a government body making sure that regulatory proposals are aligned to national and international good practices on regulatory policy?
8. Does the national government require alignment of its national standards and quality assurance (often referred to as National QI or NQI) bodies to international QI fora to enable trade and foster international co-operation? (E.g. Via a National Quality Policy). If so, is there an oversight by central government of NQI positions internationally?
9. Are there international co-operation policies defining roles and responsibilities for regulators who are working with QI bodies to enable trade and international co-operation?
10. Are there processes in place to link sectoral interactions between QI, regulators and businesses with a National QI policy, and national and international best practices on regulatory design (see Figure 5.3)?
Define the common objectives for strategic priorities and emerging challenges:
11. What national priorities or emerging challenges are the government’s priority where governance will urgently need to be reinforced to allow trust in the delivery of solutions (e.g. health, AI)? Are these high, medium or low risk areas?
12. What policy levers and standards and quality assurance (QI) solutions might be needed to deliver against these emerging challenges? What non-regulatory governance approaches are addressing these policy areas currently (e.g. QI such as certification, fully private such as NGO or industry led standards, or other)?
13. Which standards, measurement, conformity assessment, accreditation and market surveillance (QI) bodies are already active in these policy areas sub-nationally, nationally, regionally, and internationally?
14. Are there formal or informal co-ordination networks setup already between the government, regulators and recognised QI institutions to enable common reflections on sector-specific opportunities or emerging challenges? Are there channels of communication between local, regional and national level of QI bodies and regulators for a multi-level collaboration?
15. Are there open communication channels between standards, measurement, conformity assessment and accreditation bodies on a regular basis, both at senior and at working levels?
16. Does the NQI have resources and capability to adequately support regulatory authorities in addressing these emerging policy priorities?
Step 3: How do regulatory actors and QI actors currently interact?
Context
Using the outputs from Step 2, consider the different QI pillars and how they interact, formally or informally with the regulatory system.
Questions
Clarify the interactions between QI, regulators and government actors:
1. Is there a single overarching government department responsible for standards, measurement, conformity assessment, accreditation and market surveillance services? Or are there different departments, bodies, regulators which contain QI services within their remit?
2. For each QI pillar (standards, metrology, conformity assessment accreditation and market surveillance), the following questions clarify their endorsement from government (see Figure 5.4 for example on accreditation):
a) Which government departments or regulators are the primary holders of relationships with NQI bodies? What are the main levers that formalise this relationship – such as contract, agreement, recognition, legislation, regulation or policy? (e.g. they have contracts with QI bodies, or teams dedicated to QI).
b) Are there secondary government departments which work with NQI? What are the main levers through which this relationship is formalised – such as inclusion of accredited conformity assessment in policies, recognition of specific certification schemes? (e.g they work with QI bodies to deliver their own policies).
c) Are there any government departments that do not significantly interact with QI bodies? Why might these departments not have used QI services in the past?
d) Do ministries and regulators designated as key points of contact for QI services, have capacity, access, knowledge, and resources to facilitate effective collaboration between QI and regulatory systems?
Independence, Impartiality and Integrity of NQI:
3. For each QI pillar (standards, metrology, conformity assessment accreditation and market surveillance), based on the relationships and setup they have with respect to their government bodies, what is the mandate on independence of QI bodies? Are they wholly independent from government, or operating as quasi-independent bodies?
4. For each QI pillar (standards, metrology, conformity assessment accreditation and market surveillance), what is the mandate on impartiality in place for these bodies? Are they tasked with taking a wholly impartial, evidence or stakeholder consensus-based view? What mechanisms are in place to ensure impartiality (E.g. contract with government, mandate from government)?
5. For each QI pillar (standards, metrology, conformity assessment accreditation and market surveillance), what processes are in place to ensure integrity of QI bodies, such as avoiding bias towards specific stakeholders or avoiding capture of processes (e.g. consensus building) by specific interest groups?
Mitigating for key gaps in the NQI
6. What are the key gaps identified from mapping the national standards and assurance (NQI) system in Step 1 and 2, and its interactions with regulation? Are there aspects of standards and quality assurance (QI) services missing from your country? And if so, are they delivered via private sector or QI services of a different country? A broad dataset on global QI systems is available from the UNIDO Quality Infrastructure for sustainable development index (UNIDO, n.d.[1]) and the Global Quality Infrastructure Index Program (Mesopartner et al., 2024[2]) which can be used as a starting point to assess existing data on QI services in a country.
7. If a National Quality Policy does not exist, should one be developed, based on the answers gathered in prior questions to enable shared objectives and collaboration between QI and regulation on key national priorities? Several toolkits and guidance on Quality Policy are available to guide this process, such as from PTB-World Bank ( (World Bank and PTB, 2019[3]), the Pan African Quality Infrastructure network (PAQI, 2023[4]), and UNIDO (UNIDO, 2017[5]).
8. If coherent co-ordination channels are missing, at what level (national, regional) should co-ordination channels be set up, and what should this look like (e.g. working groups, networks)?
Outcome of Step 3: Based on the questions in Step 3, a map of interactions for each NQI institution (standards, metrology, accreditation, market surveillance) and their regulatory counterparts can be developed. An example is provided in Figure 5.4. for a hypothetical national QI body, and its governance and relationship links with relevant ministries, regulators and other public bodies in government.
Step 4. How can we design better QI-Regulation co-ordination to maximise positive impact of policies?
Context
Once the actors, their objectives and their interrelations are clearly identified and, established, there are several steps throughout the regulatory process in which standards and quality assurance (QI) systems can be further leveraged to improve regulatory outcomes.
Questions
How does regulation consider QI?
Considering their country’s existing regulatory governance (particularly based on OECD 2012 Recommendation on Regulatory Policy and Governance and related documentation), institutional context and economic and social priorities (e.g. based on OECD Product Market Regulation), each government can use the following questions to introduce QI considerations as a support for better regulatory design (Figure 5.5):
1. Is the QI system (standards, metrology, conformity assessment, accreditation, market surveillance) included within the formal regulatory impact assessment processes in place when developing a new policy or regulation, including systematically considering the costs, benefits and risks of QI pathways?
2. Are considerations given to alternative pathways to regulation to deliver policy outcomes? Such as utilisation of voluntary QI tools and services (e.g. standards, accreditation), including assessing the cost of QI pathways versus mandatory pathways? Is there a requirement to consider relevant international standards and frameworks in policy development?
3. How are regulators incentivised to use metrology as a basis for regulation (e.g. identifying calibration or testing laboratories within technical regulation)? Is there enough capacity and competency in those institutions delivering metrology solutions – such as testing labs – to support the implementation of regulation?
4. Are there policy remits where the system of standards and quality assurance (QI) is not usually considered? These may be sector, department, region or context specific.
5. How are the potential burdens (e.g. cost) associated with standards and accredited conformity assessment estimated, and who conducts this analysis? Are ex ante impact assessments used to estimate the potential costs or benefits of utilising QI?
6. Are considerations on risk included in developing regulatory pathways, including where the system of standards and quality assurance (QI) is utilised, and the role this system may play in mitigating for risk in regulatory system?
7. Are there ex post evaluation processes in place which systemically review and reform the effectiveness of regulation, and the QI tools and services linked to regulatory delivery (e.g. standards)?
How does QI consider regulation
The following questions help governments consider whether the cycle of processes in the standards and quality assurance (QI) systems– i.e. the development of standards, implementation of accredited conformity assessment, underpinned by measurement – effectively interacts with the relevant national and international policy and regulatory instruments:
1. How are regulatory priorities at the national and international level considered by National QI bodies explicitly in their respective QI pillars (standards, conformity assessment, measurement, accreditation and market surveillance), from development through to implementation?
2. In designing standards and accredited conformity assessment, do relevant standards, conformity assessment and accreditation bodies refer or consider explicitly the national laws and regulations or relevant international instruments serving as a basis and guiding the public policy objectives? This may include sector specific laws or overarching priorities (e.g. AI, environmental goals, health and safety).
3. Do government officials and regulators participate in standards development processes to enable public policy objectives to be considered as part of the consensus development process for new and revised standards?
4. Are government officials and regulators engaged or consulted in development of conformity assessment or accreditation procedures for new or revised standards, including pilot schemes?
5. Are the current tools and services offered by the standards, measurement and quality assurance (QI) institutions fit for purpose for the regulatory system? Or is there scope or need to innovate the QI services present at national and international level? (e.g. faster standard development, improved capacity in testing and calibration labs, accessibility of QI solution, increased diversity of stakeholder engagement).
Enabling growth through international co-operation
The following questions help clarify the international co-operation that QI bodies of a country already participate in:
1. Standards: Are your national standard bodies represented at international forums on standardisation and supra national regional forums on standardisation? What is the level of participation from your country? Is your participation aligned to specific national needs? (Examples of international standardisation include ISO, IEC, while regional networks vary).
2. Accreditation: Are your national accreditation bodies represented at international forums on accreditation and regional forums (e.g. ILAC or IAF internationally, or regional forums such as European Accreditation or African Accreditation Cooperation among others)? What is the level of participation from your country? Does your country participate in mutual recognition agreements on accreditation?
3. Metrology: Are your national metrology institutes represented at international forums on metrology (e.g. part of the CIPM-Mutual Recognition Agreement, or a member of BIPM, or member of OIML)? What is the level of participation from your country, such as the type of memberships or participation positions held in these fora?
4. Conformity Assessment Bodies: Are the inspection, testing and certification bodies accredited under internationally recognised accreditation schemes that enable mutual recognition of conformity across borders?
5. Policy and Regulation: Are QI bodies and their expertise utilised in trade or other such multi-lateral agreements in your country, to enable mutual recognition of governance levers (e.g. certification, testing) across borders?
6. Co-operation to enable increased trust and reduction in costs: Using the answers to the questions above, consider the avenues within each QI pillar where there are mutual recognition agreements in place to reduce trade barriers? Are there gaps in sectors where such international co-operation can bring benefit and reduce costs?
Note: Organisations highlighted are examples, and will vary for each country conducting this assessment of their participation in international QI fora.
Source: Author’s elaboration.
Outcome of Step 4: Mapping of international co-operation forums which NQI bodies and policy have access to. An example map of a hypothetical country is provided in Figure 5.6 which can be used as a template.
Step 5. How can QI-Regulations interactions be implemented, monitored and reformed for better regulatory outcomes?
A) Delivery of regulations
Context
The enforcement and compliance of regulation is critical to ensuring the regulatory system is working as intended. There are already established global best practices (see (OECD, 2014[6]) and (OECD, 2018[7]) on the level of inspection and enforcement required to ensure positive policy outcomes, and enable trust in the delivery of products and services). In the context of QI-regulation interactions, this may require following best practice principles on enforcement and inspections related to a policy, and critically assessing the gaps and challenges in enforcement, reforming where needed.
Questions
1. Are there systemic reviews of market surveillance procedures and their efficiency in delivering enforcement?
2. Do you have a strategy of outreach with consumers to gather information about compliance of products and services?
3. Do your testing, certification or market surveillance authorities share information with governments and regulatory authorities about non-compliant actors, unsafe products, unsafe use of products and location when available? If so, is this used by governments/ regulators to review and improve regulations and to inform the Market Surveillance system? Is there a framework for feedback between regulators and the national QI bodies?
4. Considering enforcement and compliance across jurisdictions, are there procedures to identify cross-border effects of enforcement, and any successes or challenges that QI levers have across borders?
5. Where gaps and challenges related to enforcement have been identified, are there procedures in place to develop reforms, utilising current best practice (e.g. using (OECD, 2018[7]), Reforming QI toolkits from World Bank (World Bank and PTB, 2019[3]) and UNIDO (UNIDO, 2016[8]), (UNIDO, n.d.[9]). Are QI levers sufficiently integrated into existing procedures?
6. What mechanisms are in place to address the gaps and challenges in enforcement and compliance across jurisdictions aligned to best practice principles on international regulatory co-operation (OECD, 2021[10])? Link back to Step 3 which assesses international co-operation in the design of QI.
B) Monitoring of regulatory outcomes and iterative reform
Context
Ex post evaluation of regulation is a critical part of the regulatory system to enable assessment of direct, indirect and unforeseen effects of regulations. For instance, a regulation may become outdated due to technological innovation, and a review process can enable identification of this issue to ensure a fit-for-purpose, efficient regulatory system.
Questions
Ex post evaluations are a useful mechanism to understand if QI-regulations interactions are achieving the policy outcome originally envisioned during development. The following questions can help governments understand whether the correct assessment processes are in place for the QI-regulation context, and if not, signpost to guidance to enable assessment.
1. The OECD has been recommending the use of ex post assessments of policies and regulation aligned to best practice since 2020 (OECD, 2020[11])? Is there periodic revaluation of policies, regulations and the mechanisms used to deliver them (e.g. certification) in place to assess continued effectiveness of policy implementation? If no formal process is in place, is there a precedent to follow specific best practice on assessment of policies?
2. Does the ex post assessment process consider whether QI tools and services were utilised efficiently, and identify unexpected challenges and gaps? This may include identifying unexpected costs, duplicative effort, missing QI services, or any other inefficiencies in processes.
3. When using QI tools and services such as standards to deliver regulatory outcomes, which have review periods built into the development (5 years for international standards), how are these reviews considered within the ex post assessment of the regulation?
4. Where gaps and challenges are identified in the effectiveness of QI-regulation interactions, are steps being taken to mitigate for these risks (linking back to the design and implementation questions in Step 3)?
5. What are the iterative processes of change already in place within QI institutions, and how are they recognised by government (e.g. standard development timelines)?
References
[2] Mesopartner et al. (2024), Global Quality Infrastructure Index, https://gqii.org/.
[10] OECD (2021), International Regulatory Co-operation, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/5b28b589-en.
[11] OECD (2020), Regulatory Impact Assessment, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/7a9638cb-en.
[7] OECD (2018), OECD Regulatory Enforcement and Inspections Toolkit, OECD Publishing, Paris, https://doi.org/10.1787/9789264303959-en.
[6] OECD (2014), Regulatory Enforcement and Inspections, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/9789264208117-en.
[12] OECD (2014), The Governance of Regulators, OECD Best Practice Principles for Regulatory Policy, OECD Publishing, Paris, https://doi.org/10.1787/9789264209015-en.
[4] PAQI (2023), Towards Free, Fair and Safe Trade in Africa: Guidelines for the alignment of national quality policies (NQPs) with the Africa Quality Policy (AQP), https://www.paqi.org/wp-content/uploads/2024/04/PAQI_Guideline_NQP_AQP13092023-2.pdf.
[5] UNIDO (2017), Guide For The Development of National Quality Policies, https://www.unido.org/sites/default/files/2017-02/GUIDE_FOR_THE_DEVELOPMENT_OF_0.pdf.
[8] UNIDO (2016), Guide for the Development of National Quality Policies, https://www.unido.org/sites/default/files/2017-02/guide_for_the_development_of_0.pdf (accessed on 23 September 2024).
[9] UNIDO (n.d.), Quality Infrastructure for Trade Facilitation (QI4TF) Toolkit: A Roadmap to Strengthen Quality Infrastructure for Trade, https://hub.unido.org/sites/default/files/publications/QI4TF_toolkit.pdf.
[1] UNIDO (n.d.), UNIDO QI Index, https://hub.unido.org/qi4sd/ (accessed on 23 September 2024).
[3] World Bank and PTB (2019), Ensuring Quality to Gain Access to Global Markets: A Reform Toolkit.