This chapter shows the technical guidelines for developing specific elements which conform the RIA. In real terms, it is the preparation manual. Thus, the methodologies for developing the following elements are presented: public consultation and engagement of stakeholders; definition of the problem; purpose of the regulation; description of the proposal; identification of alternatives; cost-benefit analysis; identification of the solution chosen; preparation for the framework of implementation and assessment.
Implementing Regulatory Impact Assessment at Peru’s National Superintendence of Sanitation Services
Abstract
The RIA is a mechanism that allows the systematisation of critical evaluation elements for the elaboration of legislative projects, either with a new preparation or an amendment. There are certain elements that define the stages of RIAs and that must be addressed during the elaboration process to ensure the highest possible regulatory quality. Although these elements may vary according to the country, in general, most of them address the following elements considered by the Australian government (Commonwealth of Australia, 2020[1]).:
Which is the issue intended to be solved?
Why the government intervention is needed?
Which political options have been considered?
Which is the potential net benefit of every option?
How was the public consultation carried out and how was it integrated to insights?
Which is the best option of those considered?
How will the implementation and assessment of the selected option be conducted?
Questions are directly related to the analytical elements of the proposed technical guidelines for their implementation at Sunass. These must be fulfilled in the intended order, since the content of each stage informs accumulatively the subsequent ones.
The process must start with a broad definition of the problem, the objectives and the possible solutions, and then limit them. The European Commission (European Commission, 2017[2]) recommends taking into account the following considerations before starting the RIA:
When deciding on the focus and depth of the RIA, the analysis should focus on what is relevant to inform decision-making, leaving out what is not.
The most appropriate methods for data collection and consequence analysis should be determined. When necessary, external studies can be contracted to provide input on specific elements, although, to the extent possible, it is recommended that these are conducted by the internal areas of Sunass.
A public consultation strategy must be designed, by acknowledging the need for consulting on all the key issues related with the RIA. The conclusions of the RIA report outcomes must be supported with the analysis of comments from stakeholders and the rationale when significant differences exist. The outcome matrix summarizing the consultation of stakeholders must be integrated in the final RIA report, as a mandatory annex.
Throughout the RIA, conclusions reached by Sunass should be supported with evidence (e.g., data, estimates, scientific findings) along with appropriate citations and, if this is not possible, should explain why. Sunass should also consider referencing stakeholder comments.
The guide will then present recommendations for carrying out the elements that are part of the development of the RIA.
The involvement of stakeholders in the process of creating regulation is one of the fundamental elements of any regulatory quality process, particularly of RIA, as it improves the transparency, accountability, efficiency and effectiveness of regulatory decisions. Therefore, it is important to define the timing, form and scope of public consultation in the regulation-making process.
There is no specific model for conducting consultation since several factors depending on the legislative proposal intended to implement must be considered. Not all the public consultation processes must be carried out in the same manner, this process must be fitted for each case, since each one has its specific elements requiring comments from stakeholders at the different points of the process.
This section develops the overall guidelines recommended by OECD to Sunass for implementing the public consultation processes with stakeholders. However, it is important to consider that throughout the consultation process, the following general principles must be observed:
Stakeholder participation: Adopt an inclusive approach that allows for the widest possible consultation, ensuring the involvement of all sectors interested in or affected by the regulatory proposal.
Transparency and responsibility: Make the parties involved aware of the public consultation process, as well as the available means for their participation and the impact of their involvement in the regulation development.
Effectiveness: Perform a public consultation at some point in the regulation development process where the insights of the involved parties can influence the regulation, by respecting the proportionality criteria and the specific restrictions for each case.
Coherence: Warrant the consistency of the consultation process, as well as the evaluation, review, and quality control.
Efficiency: Develop the necessary tools to carry out a resource-efficient consultation which allows a higher participation of the stakeholders.
There are two moments in which Sunass can carry out the public consultation process: before having a regulatory proposal and once the process of elaborating the regulation has been initiated. That is, carry out an early public consultation and a second public consultation once Sunass has a defined legislative project.
Early public consultation process
Conducting an early consultation will allow Sunass to obtain information that will allow the identification of public policy problems and more technical information for the development of new legislative projects. For such purpose, it is recommended to carry out public consultation processes on a regular basis with the main stakeholders. In order to achieve more effective consultations, it is recommended that Sunass publishes advance agendas with specific topics, so that stakeholders can prepare as much information as possible.
Once a potential public policy issue is noticed, it is convenient to perform a focused public consultation, addressing the specific topic and a first group of affected parties, who can provide better information. This stage is prior to the design and presentation of the legislative proposal.
Early consultation should take place before identifying the need to regulate, at the stage where the problem is still being identified, so that it helps to determine whether Sunass intervention is really necessary. Early public consultation allows obtaining information and evidence on the identified problem. It also helps to define if the intervention of the regulator is necessary through regulation and not through other mechanisms.
European Commission proposes methods for executing early consultations with stakeholders. Table 4.1 gives an example of these methods and tools for public consultation.
Source: Adapted from (European Commission, n.d.[3]). Better regulation: guidelines and toolbox. Stakeholder consultation https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-guidelines-and-toolbox_en.
Public consultation during the preparation process of the regulation
Once Sunass decides undertaking a project for issuing or amending a legislative project, it must submit the proposal to an open and transparent public consultation process. This, with the purpose of obtaining information contributing to the improvement of the legislative project.
It is important for Sunass to take into consideration that the public consultation process is not an element that is carried out only once per regulatory proposal, but that it may have to be carried out on more than one occasion during the entire life cycle of the regulatory proposal, according to the existing needs in each case and depending on the type of information sought. Thus, the same project may require a public consultation process both in its preliminary stage when a possible problem is hardly noticed and in the preparation process itself. Such decision must be adopted by those responsible for the legislative proposal. Performing public consultations will also help the regulated parties to understand the legislative projects and to have the enough time and elements to implement any change demanded by the regulation.
Public consultation elements
Public consultations must have an appropriate design. OECD recommends dividing the consultation in three stages, described at Figure 4.1.
I. Preparation stage of the public consultation
The preparation stage of the public consultation is the baseline for developing the consultation. This stage sets the basic elements to be used throughout the public consultation process. However, this does not limit Sunass to be able to modify these elements during the subsequent stages of the process, once there is more information. In the preparation stage, it is recommended that Sunass prepares a guidance document for the consultation. Going beyond just presenting a legislative project will help the population to really understand what is intended, and Sunass will be able to have more productive discussions. The guidance document must be adapted depending on the stakeholders engaged in the consultation. This can range from very basic information to preparation of technical documents.
For the preparation stage, it is necessary:
1. To define the objective
This implies clearly setting the elements that Sunass seeks to achieve with the consultation (data, facts, technical information, opinions, and points of view, etc.). For such purpose, for early consultation, the current context and magnitude of the problem being presented must be considered; while in the case of consultation during the process, the current context, scope, expectations and impact of the initiative, as well as the timing of the consultation, must be taken into consideration.
2. Selection of stakeholders
Having clearly defined the objective of the consultation, as well as the moment in the regulatory cycle when it will take place, the stakeholders that may be consulted must be determined.
The more adequate is this selection, the higher success the consultation will have. However, a listing of the potential stakeholders will be presented below:
Subjects affected by the potential problem in the case of early consultation, or by the regulation in the case of consultation during the process
Subjects obliged to implement the regulation
Subjects with a declared interest on the regulation
In the case of early consultation, it is essential to identify the subjects obliged to implement the public policy response, since they are the ones who will be able to contribute more to determine the existence and magnitude of the problem. Identifying specific groups will help Sunass to obtain valuable information for the development of the regulation. However, once Sunass has a legislative project, the consultation should include the publication of the RIA and regulatory project on its website, open to any person interested in submitting comments.
In the case of consultation during the process, to help identify the level of influence and participation of stakeholders, the European Commission generated a list of questions to help identify stakeholders for each regulatory project. Table 4.2 shows a guideline of questions to determine the stakeholders.
3. Selection of methods and tools
The third element of the preparation stage of the public consultation is the selection of the method and tools for public consultation. There is a great diversity of methods and tools that can be used for public consultation, therefore, the objectives intended to be reached with the consultation must be considered in order to be able to select the most adequate. Table 4.3 describes the types of consultation performed by Australia, depending on the characteristics of each legislative project.
* Complete public consultation is the predefined option.
Source: Adapted from (Commonwealth of Australia, 2020[1]), The Australian Government Guide to Regulatory Impact Analysis, www.pmc.gov.au/regulation.
Once the type of consultation to be carried out is defined, Sunass must specify the best means to conduct the public consultation. It is important to consider the access to different means for stakeholders; each public consultation process must choose the appropriate means for its conduction. For instance, it is not adequate to choose a virtual media for a public consultation which will be conducted in a community with scarce access to electronic media. The means of conducting the consultation can be decisive for the final outcome of the public consultation process.
Considering the means to be able to participate, the consultation can be:
Virtual: Through any electronic communication media.
Physical: The person must be physically at the time of the consultation.
Written: Every interaction between the regulator and stakeholders is in written.
Considering the way in which comments can be issued, the participation can be as follows:
Written: Responses or opinions are made through any written media.
Orally: The participation and comments are done verbally.
Due to the number of tools that can be used for performing the consultation process, it is important to consider the following conditions to choose the correct tool: i) proportionality; ii) the extent of interaction needed with stakeholders (written consultations vs. events with stakeholders, online discussion forums, or other Internet-based tools); iii) accessibility considerations (language used, participation of people with disabilities, etc.), and; iv) time requirements.
Considering the current regulations applicable to Sunass, the tools can be:
Public notice for comments, for written comments.
Public audience for verbal comments.
The following tools may also be considered:
Informal consultation: This type of consultation is intended to initiate contact with stakeholders. It can be carried out in different ways: telephone calls, written communications, informal meetings, and at any time during the process of designing and drafting the regulation. Favorable aspects of this tool are the speed of its execution and the variety of interests of the participants. However, considering that it would be the regulator who defines the groups called for this type of consultation, there is a risk of lack of transparency and possible lack of impartiality. This tool could be used during or before the process of developing the regulation.
Circulation of the work document or the legislative proposal for comments: It is a tool that does not generate higher costs and allows to receive information on the subject matter consulted. It is flexible in terms of time, scope and forms of response. The circulation of the working document or proposal is carried out in a systematic and structured manner and is carried out in view of what establishes some mandatory regulation. It can be used at any stage of the regulatory process, but it is more common in cases where an elaborate proposal is available. Difficulties in using this tool focus on discretion in determining who will share the working document or proposed regulations. Although the most important groups will be considered, there is the risk that the less organised sectors are not included.
Public notice for comments: This option is more open and inclusive than the circulation process for comments, and usually is more formal and structured. In this event, all stakeholders have the chance of knowing the work document and the legislative proposal and of stating their opinions.
Public hearings: This is a meeting where stakeholders can comment on the working document (in case of early consultation) or the regulatory proposal (in case of consultation during the process) in person, facilitating the dialogue. During these meetings, the regulator may be able to clarify some points, ask questions and gain a better understanding of the positions of the groups involved. There is a possibility that those responsible for addressing the public policy problem will ask the stakeholders to submit information and data. This tool usually is not used in an independent manner, on the contrary, it is applied as a complement to other consultation procedures. This, due to the fact that the presence of diverse groups can complicate the discussion and, consequently, there are sectors that are unable to express their position, so it is suggested to seek options to compensate this deficiency.
Advisory bodies: These are also called committees, commissions or working groups. They are characterised by being in charge of a defined task within the public policy process, such as providing expertise or encouraging consensus among the parties involved. Depending on their status, authority, and position in the public policy process, they can influence decision-making or act as a source of information. Advisory bodies are involved in all stages of the regulatory development process, but it is more common in the initial stage to assist in defining positions and providing alternatives.
II. Execution stage of the consultation.
The second stage of the public consultation process is the execution process of the consultation. This stage includes the spread of the public consultation process, the reception of comments, and the development of the matrix of comments.
Once Sunass has defined the tools, methods, and target population to carry out the consultation, the execution stage must start. Thus, the following stages must be considered:
4. Communication and diffusion plan of the public consultation
The design of the communication and diffusion strategy of the public consultation process is essential for ensuring a high level of participation. The communication plan should facilitate the involvement of as many stakeholders as possible, so it is important to consider the following:
The characteristics of the stakeholders to whom the consultation will be directed, in order to determine the ideal means of communication and dissemination in each case.
The incorporation of several communication channels: e-mail, letters, press, social media, web pages, etc.
The consideration for sending individual communications to stakeholders, notifying the onset of the public consultation process. In the case of early consultation, sharing with them the problems identified and, in the case of consultation during the process, the proposal to be analyzed. This is to provide them with the basic information to participate in an efficient way.
The use of common language for citizens will allow a higher engagement, as long as the consultation is not addressed to technical groups, where it is recommended to use specialised language.
If the work document is addressed to several stakeholders, it is important to consider having a complete and a short version of the same, that could be easily readable and understandable for all the participants of the consultation.
5. Deadline and treatment of the comments received
For early consultation, the deadline for receiving comments will depend on the type of consultation, tool and groups consulted to which the consultation is defined to be directed. Sunass should ensure that the actors consulted have sufficient time to analyze the information provided to them, as well as to provide feedback. For consultation during the process, the deadline for stakeholders to submit their comments must be at least 30 days, counting from the pre-publication in the official newspaper El Peruano, from the publication of the regulatory project on the Sunass website or, if applicable, from the date that Sunass establishes.
To establish a standardised procedure for the treatment of comments received, it is important to consider the ways in which the information in Table 4.4 is received:
Sunass can establish a standardised format for the reception of the comments, in order to adequately address all comments received.
Finally, during the time established for the consultation it is important to designate a person as responsible for providing the required clarifications and attending to the doubts sent by the participants of the consultation. This person must attend, within an established period of time, in a precise and simple way the communications received. This will contribute to the transparency of this process, generating confidence in the participants.
6. Preparation of the matrix of comments
The consultation process is not only about receiving comments from stakeholders, but Sunass must give a response to all comments received. This does not imply that all comments received should be accepted; however, it does imply that all comments should be addressed by Sunass. In the case of early consultation, if they are addressed in a positive way, the regulator must signal the acceptance of the comment, and, in the case of public consultation during the process, it must also make the changes leading to the regulatory proposal. If it is in the negative sense, it must indicate the reason why the comment is not appropriate. This practice generates confidence and promotes certainty for the participants in the public consultation process.
A database with this information must be created, with a proposed form to manage systematically the information received, which will allow managing the comments during the period set for such purpose. It is suggested to have in mind the information of Table 4.5.
III. Assessment stage of the consultation.
The objective of this last phase is to evaluate whether the public consultation process satisfactorily met the proposed objectives, as well as to determine the quality of the comments received and their impact on the public policy decision made by Sunass. This stage allows assessing the effectiveness of the consultation, the comments received, and the changes performed to the legislative proposal based on the latter. Thus, this stage is essential to strengthen the trust and certainty of Sunass stakeholders.
7. Assessment process of the comments received
For early consultation, an analysis of the comments received should be made in order to determine the existence of a problem, the magnitude of the problem, and the possible regulatory and non-regulatory alternatives to attack the identified problem.
For the consultation during the process, an analysis should be made to assess the impact of the comments on the regulatory proposal. This process will allow for a thorough evaluation of all comments received that are appropriate to address, as well as how they may affect the original legislative proposal. The analysis of comments received will be used for this evaluation.
The analysis of comments received leads those responsible for the regulatory proposal to review all comments received and determine the relevance of incorporating them into the proposal, according to the support provided by the participants of the consultation. Likewise, and in the event that no comments are received, they must also provide a response.
The period to carry out this analysis should be sufficient to address all comments and assess the relevance of modifying the regulatory project derived from them. After this period, the decision taken regarding the comments should be communicated to the participants. This communication may be personal or through a general publication through the web page.
The responsible bodies for the legislative proposal must be also those responsible for determining if the review of the comments will be as they are received or at the end of the established period. This will be done considering the different workloads that exist in the area responsible for the regulation and without compromising the time frame established for the analysis.
8. Assessment of the public consultation process
In order to make an optimal assessment of the fulfillment of the objectives set out in the consultation, Sunass must carry out a critical assessment. It is necessary that the responsible area performs a series of questions to define the effectiveness of the public consultation process. The assessment of the public consultation must be a qualitative process serving as a feedback to increase the effectiveness on future public consultations. Table 4.6 shows examples of questions that can help to assess the process of public consultation.
9. Final report
Once the analysis of comments received is completed, Sunass must prepare a final report. This report should contain elements that will allow stakeholders to consult information from the public consultation process more easily. The report is important because it promotes transparency of information and stakeholder certainty about the legislative process. The final report of the public consultation process is the tool that allows for a clear and concise history of the public consultation processes of the regulator's legislative proposals. The following points are the minimum elements that the final report of the public consultation process should contain:
Preparation date
Title of the Legislative project
Description of the public consultation process
Objectives of the public consultation
Bodies and/or directorates responsible for the public consultation
Description of the participants in the public consultation
The responses of the government agency or institution to each of the comments received, specifying whether or not they were useful.
Description in case the consultation has fed the legislative project or if it resulted in substantial changes to the selection of the legislative project.
This report should be published in the institutional web site of Sunass and this will be communicated to the participants so that they can consult the status of their comments.
Which are the objectives of the public consultation?
Which type of public consultation must be carried out?
Which group of stakeholders are essential for the process of public consultation?
Is it necessary to carry out a confidential or targeted consultation?
Have informal hearings been performed for addressing the topic? What relevant information was obtained from these hearings?
Which are the adequate diffusion tools to carry out the public consultation?
How much time is necessary to carry out an adequate consultation?
Did the original proposal was changed derived from the public consultation?
Were all the comments received during the consultation period responded?
Problem definition is the first step in the RIA development process. Its importance lies in the fact that a good problem definition determines the direction and quality of the result of the RIA, since it is the basis for all the other elements of the process. If the problem is not properly defined, the regulatory proposal may not mitigate or satisfactorily resolve the public policy problem.
If Sunass has conducted an early public consultation, this interaction with stakeholders can provide the inputs to develop the approach of the public policy problem. The public consultation process is a valuable tool to obtain insights and information from stakeholders, which is essential to properly define the public policy problem. However, Sunass must be careful in taking these comments to ensure an objective analysis to evaluate the issue at hand.
Additionally, Sunass must consider a collaborative process among the internal directorates, including those decentralised offices, in the stage of the definition of the issue. Having as much information as possible is essential for the preparation of a legislative project, and in most cases, the decentralised offices have the greatest interaction with users and service providers in regions outside Lima.
In order to define the problem adequately, Sunass must carry out a categorisation of the elements that must be included in the problem definition: delimitation, causes-effects, and magnitude.
Delimitation
The first element to properly categorise the public policy problem is the delimitation of the problem. The delimitation must address a first context of the perceived problem. In this phase, certain aspects of the problem must be defined; one aspect is the concrete definition of the problem, where the question "why is it problematic?” Although it may sometimes seem intuitive, an exercise of explicit definition must be carried out in order to lay concrete foundations for the rest of the RIA. Another aspect that should be considered in the delimitation is to define if there are related problems in order to have a complete perspective of the situation. Finally, in the delimitation, Sunass must establish a base line of estimation of the problem. This will serve both to establish the objectives and to achieve a measurement of the evaluation of the regulatory proposal.
Causes and effects
It is recommended that a standard logical framework methodology be established to define the causes and effects of the problem to be addressed. Using a decision tree such as the one shown in Figure 4.2, Sunass should establish the causes and effects of the identified problem. The tree has two levels of causes and two levels of effects, where the effects can be more specific or general. For example, a general cause may be the culture of corruption, and a specific cause may be extralegal charges for public sewer cleaning. Specific causes or effects are those which are the direct causes or effects of the public policy problem. For example, a public policy problem is the constant blockage of public sewerage drainage, a specific possible cause may be high maintenance costs, a specific possible effect may be water-borne diseases, and a general effect may be higher public health costs.
Sunass must be accurate when establishing causalities. On one hand, two general causes can lead in the same specific cause, additionally to the fact that specific causes can produce a feedback effect with general causes. If causes are not identified correctly, the objectives will not be specific.
Expected magnitude
Finally, determining the expected magnitude of the problem serves as a basis for the legislative design, so that it is proportional to the problem. If the magnitude of the problem is correctly captured, two potential problems are avoided: introducing limited regulation for a major problem or introducing strict regulation for a minor problem. The definition of the magnitude can be built from the following three variables:
Affected parties: First, the kind of affected parties must be defined. These can range from people, companies, public infrastructure, environment, etc. Once defined, Sunass must quantify the number of affected parties. In this delimitation, definitions for geographic staggering must be considered. The problem would affect only some regions of the country or even certain areas of any city. Lastly, it must be specified if the public policy problem affects any vulnerable group (indigenous peoples, elderly population, disabled individuals, etc.).
Severity: Severity refers to the degree of impact, that is, if the public policy problem is causing the damage to the affected groups. The regulation must consider the direction of severity; if the severity is as high as death of humans, this must be deeper than the scenario when the severity are delays on the delivery of formalities.
Occurrence probability: The occurrence probability is a core part for designing preventive regulations. When the probability of a problem is understood, the efficiency of resources displayed by the regulation is achieved. One example is the construction requirements for areas with earthquakes. In a city with constant earthquakes, buildings must have the necessary infrastructure for preventing collapses, although this represents a higher cost. However, it would not have any sense doing it in cities where this is not a risk.
The European Commission (2017[2]) has identified four factors as the main causes of public policy problems. These are the existence of market failures, behavioral biases, regulatory inefficiencies, or failure to respect fundamental rights (see Table 4.7).
A public policy problem can have negative consequences or only represent a risk for such consequences. It is important to clearly understand these underlying factors, using specialised knowledge of internal and external parties.
It is likewise important to clarify in the RIA the way in which individuals, users, companies, or other bodies are affected by the problem:
To which extent does the problem affect daily life?
Who must change his/her/its behavior to improve the situation?
Addressing these questions will ensure that the analysis remains concrete, focused, close to stakeholder concerns, and aware of the practical consequences of any regulatory initiative. This will help facilitate subsequent identification of alternatives and analysis of impacts. Table 4.7 gives a brief description of the factors causing the public policy problems identified by the European Union.
Source: Adapted from (European Commission, 2017[2]), Better Regulation Guidelines, https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines.pdf.
Guideline of questions
What is the identified public policy problem and is there sufficient evidence?
What are the causes and consequences? Are they relevant?
Who is affected by the problem? How are they affected?
Is this the first time that it occurs? If not, what have been the previous strategies for solving the problem?
What is the magnitude of the problem?
What risks are involved in the problem?
Are there any regulations that address the problem?
Is government intervention necessary? Why?
Can the problem be quantified? What are the economic, political, social and environmental costs?
Note: There are other questions that Sunass can respond, this listing is only an example of the concepts that must be covered by the analysis for the appropriate definition of the problem.
Example: Problem tree prepared by sunass for the assessment of the situation in rural areas
Once the public policy problem has been identified as well as the possible effects and consequences and the regulatory intervention has been justified, the next step in the development of the RIA is the development of objectives. Sunass must outline the specific objectives intended to solve the causes of the problem. The public policy objectives must be clearly identified, including the level of political ambition intended to be proposed, as well as the criteria by which the different alternatives will be compared.
To develop good objectives, which are specific and operational, Sunass can take into consideration the SMART methodology, prepared by the European Commission (2017[2]), referring to Specific, Measurable, Achievable, Relevant, and Time bound. Following, each one of them is briefly described.
Specific: the objective must be specific enough so that it is not open to interpretation and that it seeks to solve concrete problems and not macroeconomic situations.
Measurable: at this point of the RIA, one should start thinking about the monitoring and evaluation plan, so an objective should be set that can be subject to evaluation to determine whether the implemented regulation is working or needs to be reconsidered.
Achievable: objectives that can reasonably be met should be set, avoiding optimistic biases that end up not being achievable. At this point, the economic resources involved in implementing the regulation must be taken into consideration.
Relevant: the objective must be explicitly linked to the problem and its causes.
Time-bound: it must be specified the times in which the objectives are expected to be achieved. It is very important to consider the development of a regulatory implementation plan.
Box 4.3 includes a series of questions which may serve as a guide to define the objectives of the regulation according to the criteria pointed above.
Considering the public policy problem identified, which objectives should the regulation seek?
How do the identified objectives align to the elements of the SMART methodology?
Are there objectives that have been established for similar situations? Did they work for achieving the intended purposes?
Is the compliance of objectives plausible?
How are objectives related to the causes of the defined problem?
Which is the acceptable timeframe for attaining the objectives?
Are objectives consistent with the strategic government policies?
Which indicators can be associated with the compliance of the goal? Which is an acceptable progress of these indicators?
Example: Provision for specific water and sewage infrastructure projects in England; project developed by Ofwat
The UK water and sewage regulator, Ofwat, outlined a problem regarding the delivery of large or complex infrastructure in the industry. With climate change causing higher precipitations and on a greater scale, the current infrastructure is not enough to cover the needs. For example, this affects directly in London, where excessive raining has caused the overflow of the sewage network, thus, polluting River Thames, and causing fish death, and human intoxications, for which the European Commission has already imposed sanctions.
However, due to the fact that large infrastructure projects require high capital, this may cause Service Providers Companies (EPS) to have resources focused on the delivery of services for the construction of large infrastructure and then fail to provide the services that they are legally obliged to do or users may be affected through significant increases in service fees. For that reason, OFWAT has determined that a legislative amendment is necessary, which is presented below.
Public policy objective
The public policy proposal aims to facilitate the delivery of large or complex infrastructure, while containing and minimizing the risks to water or sewerage customers and to UK taxpayers, which are associated with such delivery.
Another objective of the policy is to promote innovation in the infrastructure provision of high-risk water and sanitation water supply services. For achieving these objectives, the policy would allow financing and delivering these projects through an independent Infrastructure Provider (IP). This would isolate and contain the associated risks and consequent costs of financing and delivering these types of projects, and should to some extent encourage new entrants to deliver more innovative or cheaper infrastructure than the existing monopoly system. However, the policy would only affect large projects (of which there are few), so it will not have an immediate widespread impact on competition as a result of the introduction of the new regime.
The pilot test would be done with the construction of the Thames Tidal Tunnel, which is the only large or complex infrastructure project in the strategic vision for the next 10 years.
The objective is then evaluated with the SMART criteria.
Specific: The objective presented is specific because it only refers to large or complex infrastructure projects, which require a high investment.
Measurable: The efficiency of the public policy would be measurable through the costs that are generated throughout the project, once the pilot project is completed. Monitoring indicators can be generated to determine the impact on the tariff cost for water and sewage service users.
Achievable: The objective is achievable since it would start with a pilot project that would assist to determine the feasibility for similar future projects. In the example, a specific case where the proposed regulation would be applied is outlined.
Relevant: The objective of this regulatory proposal is directly linked to the problem, since the generation of large and complex infrastructure projects would help deliver better water and sewage services to users. In addition, it would allow the consequences of climate change to be addressed in the water and sanitation sector.
Time-bound: The proposed regulation proposes as a pilot test the Thames Tidal Tunnel, which is a project that is estimated to be the only one to be implemented in the next 10 years.
In order to carry out an effective evaluation of public policy alternatives, it is important that Sunass considers different legislative options and different types of public interventions. This is one of the fundamental steps in the process of developing the RIA. Sunass must weigh the different alternatives before having a well-defined legislative project. On the contrary, there will be a natural trend in the legislative proposal for biases in the selection. It is important that Sunass maintains an open mind to receive new proposals for alternatives from different stakeholder groups. This may result in a better selection of legislative alternatives. It is important that when a legislative alternative is discarded, a solid justification is given, justified in terms of legislative impacts. Table 4.8 shows some examples of public interventions that can be taken as alternatives.
The consideration of alternatives must be carried out through an iterative process:
1. Start by considering the widest possible range of legislative alternatives, both in terms of content and instruments. The Sunass must consider both legislative and non-legislative alternatives. The status quo should be always considered as an alternative, that is, not conducting a legislative project. This serves to establish a baseline to counterbalance costs and benefits of other legislative proposals.
When options are being defined, the guideline principle should be if a certain measure might influence on the causes of the issue and change the pertinent behaviors for achieving the desired objectives.
As mentioned, when considering legislative alternatives, the following should be always considered:
The main alternative must be always the status quo, that is, not taking any action and let the things as they are. This base scenario should be used as a starting point against the different alternatives. This option should try to foresee the technological and social progresses, such as the role of Internet and the electronic government.
The option of improving the implementation or application of the current legislation; or, otherwise, simplifying the existent regulation.
Consider the options that involve new technological developments and TICs, to reduce implementation costs and administrative burdens to users or companies. Theoretically, all new proposals must consider their digital implementation, where applicable.
Alternative public policy instruments such as the following: non-legislative alternatives, self- and co-regulation; market solutions; international standards; or a mix of several.
Alternative scenarios such as considering the simplest option.
2. Examine previously identified public policy alternatives. In many cases, very little analysis will be needed to rule out several alternatives, for example, those that are not technically feasible, are not legally viable, are difficult to implement, do not respect fundamental rights, and have other unacceptable repercussions. Those options that go beyond what is needed to satisfactorily achieve the objectives should also be discarded, as they also fail to comply with the principle of proportionality.
3. Once the alternatives have been examined, the most relevant ones should be studied together with the baseline scenario.
More costly or inefficient alternatives must be avoided since they are kept only to highlight the benefits of the preferred alternative. Maintaining such alternatives can undermine the credibility of RIA.
It is difficult to identify at least two credible alternatives in addition to the baseline scenario. If no other alternatives exist, the focus of subsequent analysis should be on determining the detailed design of the retained options, for instance, considering sub-options within these alternatives for certain elements.
After the first impact assessment, modifications to the original alternatives may be necessary. This usually happens when the selected alternatives fail to meet the objective in the first place.
The RIA does not require a detailed description of the alternative selection process. However, it must demonstrate that all relevant alternatives were considered. What the final RIA report should include is a description of the various alternatives retained. Box 4.4 includes a series of questions that can guide Sunass in identifying regulatory alternatives.
Which are the consequences of not performing a project of legislative issuance or amendment?
What sort of alternatives of intervention, in addition to the traditional regulation, can be used to solve the problem?
Have the ways in which similar situations have been addressed in the country and in other countries been analyzed?
What is the current legal framework for each of the alternatives identified?
How would compliance with each of the alternatives be reviewed? Are they feasible?
How long would it take to implement each of the alternatives?
Was the participation of other stakeholders taken into consideration to assess the universe of alternatives?
Example: Provision for specific water and sewage infrastructure projects in england; project developed by Ofwat
Ofwat's definition of the problem:
Climate change, population growth, and higher expectations of users regarding environmental standards anticipate that a bigger and more complex water and sewage infrastructure will be required. For instance, it is expected that changes on precipitations result on more humid winters and drier summers, and that water scarcity aggravates in the south and east of United Kingdom.
In addition, episodes of heavy rain are likely to be more frequent. In London, these events will strain an already overburdened sewerage system, leading to more untreated wastewater discharges into the River Thames. Just over 18 million cubic meters of wastewater will enter the Thames each year when storm water exceeds capacity. These discharges occur, on average, once a week and have a significant environmental impact on the river. These discharges increase the likelihood of fish dying, create a greater health hazard for river users and damage the aesthetic appeal of the Thames.
Therefore, the water and sewage regulator of United Kingdom, OFWAT, outlined the possibility of a new regulation for the delivery of a large or complex water and sewage infrastructure through a third party called Infrastructure Provider (IP) instead of the Service Provider Companies (SPC). For them, it developed the advantage and disadvantages of the main alternatives to this public policy issue.
Have in mind that the “delivery” of the infrastructure may mean the design, financing, building, and/or maintenance of these projects; in some cases, it can also include the operation.
To solve this issue, Ofwat outlined three alternatives:
1. Not performing any legislative change,
2. Implementing a new legislation applicable to all SPC enabling the creation of independent IP regulated directly, financing and supplying large and complex infrastructure projects.
3. Modifying the exploitation license of a single SPC to create a separate IP (Infrastructure Provider) financing and performing a specific large project (for instance, the Thames Tideway Tunnel) on behalf of the service provider company.
Below, a summary submitted by Ofwat about the advantages and disadvantages, as well as a brief description of each is presented.
Option 1 - Service Provider Companies (SPC) keep financing and deliver all the water and sewage infrastructure projects under the existing legislative regime.
Under this "do nothing" option, all water and sewage infrastructure would continue to be financed and delivered by the Service Provider Companies under the existing legistlative regime. This establishes companies with a protected monopoly in their designated service areas, including the delivery of infrastructure. The regime has allowed SPC to lure sufficient capital to finance almost 108 000 million GBP of infrastructure (at current prices) since privatisation in 1989. For most future infrastructure projects, the current regime will be enough. Table 4.9 lists the advantages and disadvantages of this legislative option.
Option 2 – To carry out a new legislation applicable for all SPC allowing the creation of independent and directly regulated IPs, financing and supplying large or complex infrastructure projects.
Under this option, a new regulation would be developed under the section 36A of the Water Industry Act of 1991. The regulation would apply to all SPC and would allow the creation of independent IPs established through competitive bidding to finance and deliver large or complex infrastructure projects within the "normal geographic" areas of existing companies. An IP would exist during the construction and operation phase of a project, which can be regulated directly by OFWAT as an entity distinct from the parent company. Table 4.10 shows the advantages and disadvantages of this option.
Option 3 – Modify the operating license of a single EPS to create a separate IP to finance and implement a particular large project (for instance, the Thames Tideway Tunnel) on behalf of the service provider company
In this option, the OFWAT would modify the operating license of an SPC to allow the financing and delivery of a project through a competition. This would allow competition in the provision of infrastructure and give OFWAT an objective means of assessing whether the project costs are appropriate and reasonable. Table 4.11 lists the advantages and disadvantages of Option 3.
The impact evaluation is a stage that allows the regulator to explore the consequences of the legislative project proposals. This stage is the core of the RIA analysis, by being a space where direct and indirect impacts are identified, and legislative alternatives are contrasted in a qualitative or quantitative manner.
Although the most commonly used methodology in the RIA for measuring the impacts of the legislative proposal is the Cost-Benefit Analysis (CBA), this is not the only one used, and not necessarily the most appropriate in all cases. Sunass must consider the magnitude of the public policy problem in order to adequately select the impact evaluation method. While a detailed quantitative analysis is always valuable, not all legislative proposals require it. This would result in regulatory output cost overruns.
When it comes to low-impact regulation or in the face of a relevant shortage of information, it is not necessary or not possible to quantify and/or monetise the impacts. Given these scenarios, the desirable exercise is a qualitative identification of costs and benefits in order to carry out a reasonableness analysis of the regulation.
The CBA methodology compares the expected net impact of different regulatory and non-regulatory alternatives, through a detailed quantification and monetisation of the direct and indirect costs and benefits of the impacts. The complexity of CBA can vary, mainly for two reasons: the magnitude of the regulation (or policy problem) posed and the availability of information.
This section develops a practical guide for carrying out a CBA that considers the quantification and monetisation of impacts, based on the following elements:
1. Identification of costs and benefits of the regulation;
2. Presentation of the impacts;
3. Assessment of impacts; and,
4. Comparison of legislative alternatives.
It also summarises two alternative regulatory impact methodologies that are also commonly used in the RIA: Cost-Effectiveness Analysis (ACE) and Multi-Criteria Decision Analysis (MCDA).
Identification of costs and benefits
For performing any type of cost-benefit analysis, either quantitative or qualitative, the first step must be to identify direct and indirect costs and benefits of the regulation. Technically, any regulatory activity implies a cost and should imply a benefit. Although indirectly, in almost all cases, the state, users, and companies are affected. In Figure 4.4 the main categories of costs derived from regulations, collected from the Centre for European Policy Studies (CEPS) are identified.
Source: Adapted from (CEPS, Renda and University, 2015[5]), Análisis normativo: Experiencia de la Unión Europea (Regulatory Impact Analysis: the European Union experience). https://colaboracion.dnp.gov.co/CDT/Mejora%20Regulatoria/Presentaciones/Cierre%20Pilotos%20Sept%202015/3.%20RIA%20Union%20Europea%20Andrea%20Renda.pdf.
The explanation of these is completed with elements of the Standard Cost Model (SCM) (SCM Network, 2004[6]), whose information is exposed in the following Box 4.5.
The Standard Cost Model (SCM) is the more frequently used methodology to measure administrative burdens. It consists of measuring the costs associated to the activities that companies and/or citizens must carry out to comply with the regulation, named administrative burdens. SCM is not focused in the policy objectives of each regulation, but the measurement is focused on the administrative activities derived from the compliance of the regulation.
The SCM is based on the fact that for the compliance of the regulation there are information obligations (IO), that is, the information requirements derived from regulations must be provided by the regulated party to the regulatory authority. Each IO has one or more data requests, which are each of the information elements that must be provided to comply with an IO. In order to provide information for each data request, a series of specific administrative activities must be carried out.
The SCM calculates the costs of carrying out each of these activities, which can be done internally or outsourced. Also, acquisitions made to complete a specific activity should be included in the calculation if they were acquired solely to meet the regulatory requirement. For each administrative activity it is necessary to collect a series of parameters to define the cost. These parameters are: Time, Price, and Quantity (Population and Frequency). By combining these elements, the basic formula of SCM is obtained:
Cost per administrative activity (or per data requirement) = Price x Time x Quantity (Population x Frequency)
The core concept of SCM is a normally efficient business (NEB). This refers to those companies which solve their administrative tasks in a normal manner, that is, they are not the most efficient nor the most inefficient for solving. Thus, it is possible to identify general contexts that can be derived directly from the regulation. In order to specify the NEB, a series of interviews must be carried out with companies in the target group for each of the administrative activities. In this way it is possible to find out how much time they invest in a specific activity associated with a data request.
Source: Adapted from (SCM Network, 2004[6]), International Standard Cost Model. http://www.oecd.org/gov/regulatory-policy/34227698.pdf.
However, each of the different types of costs and benefits in Figure 4.4 will be explained below.
Direct costs of regulation
Direct compliance costs
Direct compliance costs are the group of costs incurred by parties to comply with regulatory obligations. These can range from specific financial charges to long-term investments or industry restructuring. Costs can be categorised into the following four items:
Collections: Result from a transfer of financial resources directly from companies or consumers to the State, to solve obligations explicitly marked in the regulation. They are the type of costs that are easy to identify, by definition. They usually have several names, including: taxes, fees, payment of duties, among others.
Substantive compliance costs: They derive from legal obligations imposed to companies to operate through any legal instrument in general. They reflect the resources that companies or citizens must assign to perform their activities in compliance with the requirements. These costs can be subdivided in one-time costs, recurrent costs, capital costs, operational and maintenance costs, and financial costs (costs related to financing investments).
Administrative burdens: Costs derived from the collection, production, maintenance, or delivery of information derived from regulatory requirements.
Long-run structure costs: These are the costs for the economy in general, resulting from an essential change in industries or complete sectors derived from the regulation; which are very difficult to measure due to their diffuse nature and the long periods of time in which they occur.
Irritation costs
Costs for “discomfort” or irritation are mainly derived from administrative burdens, and because they are subjective in nature and their specific origin is unclear, they are difficult to quantify or monetise (CEPS, 2013[7]). These costs include corruption costs, excessive waiting times for procedures (for instance, to receive a response in 60 days instead of 20 as regulations state), and the inconvenience of perceived regulatory overload. These elements can be used as a proxy to measure administrative burdens, as the SCM does; however, some countries separate them as an additional element. Usually these costs serve as an element for qualitative analysis.
Government implementation costs
Government implementation costs refer to the additional costs that the State has to incur in order to effectively implement the regulation. These consist of the following:
Adaptation costs: Costs incurred to update human or material resources to implement a regulation (for instance, training courses for the personnel on new guidelines or purchasing computer equipment for digital procedures).
Information costs: Costs generated in the production of statistical data to assess compliance with the regulation.
Monitoring costs: Costs related to human and material resources to monitor regulatory compliance (for instance, costs to create a new crew of inspectors, including the necessary vehicles, salaries, etc.)
Adjudication costs: Costs of using the legal system to resolve disputes over the new regulation.
Indirect costs of regulation
Indirect compliance costs
Indirect compliance costs arise when the prices of goods and services increase because companies must incur higher development or production costs due to new requirements that may be imposed by a given regulation. These costs might have a domino effect on the related goods. A clear example is when an aluminum production company raises the cost of electricity, it will pass that price increase on to the final cost of its product.
Other indirect costs
The costs categorised as “other” in Figure 4.4 mainly refer to those generated by changes in the behavior of individuals or firms in the market, either by direct effects of market rules or by changes in incentives. These are divided into the following categories:
Substitution effect: A regulatory intervention can usually change people's behavior patterns, which can generate unintended costs. For example, if regulation results in an increase in the price of a product, people will usually respond by consuming less of that product or by switching to the purchase of a substitute product.
Affectations to competition: Regulation can affect competition in three main ways: rules that make it difficult for new competitors to enter the market, mainly for small businesses; rules that reduce aggressive competition between competitors; and, rules that induce collusion, for example, by imposing changes in price.
Reduced market access: There are regulations blocking the possibility for companies seeking market access. One example is when public bidding procedures have a bad design where a company trends to corner the market.
Investment and innovation restrictions: The regulation might affect incentives for investing resources on research and development. One example is the absence of regulation associated with intellectual property or patents.
Uncertainty: When the regulation is not clear enough, uncertainty can emerge regarding the scope of permit for the business activity. This can inhibit the investment due to the risk aversion of entrepreneurs. Additionally, uncertainty can arise when regulation is constantly changing.
Table 4.12 includes a listing of the potential costs of regulations, classified by category and with examples.
Source: Adapted from (CEPS, 2013[7]), Assessing the costs and benefits of regulation. https://ec.europa.eu/smart-regulation/impact/commission_guidelines/docs/131210_cba_study_sg_final.pdf.
Regulation benefits
As mentioned above, the benefits are more difficult to categorise because they are generally presented differently in each case, depending on what the objectives of the regulation seek (CEPS, 2013[7]). In addition, some benefits are difficult to quantify since public policy often addresses problems that are difficult to monetise such as human health, environment, safety, etc. However, it is important to be able to identify both direct and indirect benefits in order to justify regulation. This is relevant in two ways: to ensure that the best alternative is taken and to defend regulatory proposals in public consultations.
Table 4.16 includes a list of potential benefits from regulations, classified by category.
Source: Adapted from (CEPS, 2013[7]), Assessing the costs and benefits of regulation. https://ec.europa.eu/smart-regulation/impact/commission_guidelines/docs/131210_cba_study_sg_final.pdf.
Cost-benefit analysis
The Cost-benefit analysis (CBA) is one of the main tools used to analyze the regulation impact. The CBA is one tool for economic analysis that requires that the positive (benefits) and negative (costs) effects that public policies create are previously quantified in a monetary manner, in order to be able to compare them, mainly through two criteria: Cost-Benefit Ratio (CBR) and net benefits.
CBR is defined as:
If the CBR is higher than one, it implies that the project will bring, in general, more benefits than costs, since the present value of benefits is higher than the present value of costs. If during the RIA process there are different alternatives with the CBR higher than one, the difference on present net benefits must be considered.
On the other hand, net benefits are the difference between benefits and costs brought to present value. In general, Sunass should only consider those projects with a positive net benefit. When considering different public policy alternatives and if only one can be implemented, the net benefits should be a main driver of decision. However, other variables must be considered, such as implementation feasibility, possibility of inspection, political feasibility, etc. In general, the net benefit criterion is more widely used than the CBR, which is more frequently used in cost-effectiveness analysis. The following steps describe the application of the CBA:
1. Identify the Direct and Indirect Impacts of Regulatory Alternatives
The first step of the CBA is to identify the costs and benefits of regulation. It is important to consider all impacts of the regulatory proposal, both positive and negative. An incorrect identification of costs and benefits could lead to wrong decisions, since the impacts of a regulatory alternative could be under- or over-estimated, making the comparison between regulatory options invalid.
2. Quantification and monetisation of costs and benefits
CBA implies that all impacts of regulation must be quantified in monetary terms. In some cases, mainly with the costs of regulation, monetisation is relatively simple since the existence of a market allows the use of market prices in this step. However, it can be difficult to identify a market for the benefits, so other methods of quantification are needed. Among the main methods for assigning a monetary value to impacts that do not have a specific market are hedonic pricing and contingent valuation, however, these are not the only options available.
In order to have an objective comparison of the quantification of costs and benefits, the following considerations must be taken into account:
Exchange rate: Sometimes the inputs for companies to comply with regulation or for governments to enforce it are in foreign countries with different currencies. In these situations, costs should be expressed in a single currency to achieve comparability, preferably in U.S. dollars because of their ease of procurement and liquidity. In these cases, the exchange rate expressed for each currency and the source of the data must be clearly specified.
Average market prices: When looking for prices for products that serve as inputs for regulatory compliance, an average market price should be approached. By obtaining quotations for the material good or service, the average price should be presented in a final manner; however, there should be total transparency when doing so, if possible in a RIA methodological annex. Publication of the price of a single brand should be avoided at all costs to avoid unfair market practices.
Standardisations: The correct use of a method to standardise variables must be ensured. In particular, this is relevant for standardizing the temporality of the variables, for example, converting quarterly values into annual values. A common example is the calculation of financial annuities, such as the value of a credit. In this case it can be confusing to manage compound vs. simple interest. These conversions can usually be done through calculators on specialised websites.
3. Determine cash flows
To achieve comparability among alternatives, the same units of measurement and time must be established. This must be observed since it would be impossible to compare quarterly costs vs. annual costs of a compliance cost. For effective data presentation, the example of Table 4.14 can be followed, where costs and benefits per year are broken down. The following step is to define the time horizon when the regulation will be evaluated.
4. Define the evaluation horizon
The definition of the horizon to be considered for the determination of cash flows can be carried out in different ways. It is important to emphasise that the choice of the evaluation horizon will impact the effects to be considered in the evaluation, as well as the preference of one regulatory alternative over another.
The horizon should reach the point where the contribution of discounted net benefits begins to be negligible. That is, when the benefits and costs, brought to present value, provide minimum amounts to the total net benefits. When there is not enough information to identify the periods in which the benefits and costs will occur, it is recommended that the period of time be long and use a perpetuity to discount the flows.
Validity period of the legislative measure. This point is particularly relevant for regulations that have an established expiration period.
Clearly identify the periods of costs and benefits. Budget restrictions have also temporary restrictions, so the feasibility of a regulatory project has to consider this as well.
Expected life of capital investments required by the regulatory policy or the physical effects that cause the benefits.
5. Discount of Cash Flows
In general, regulatory alternatives generate different cash flows in different periods which must be compared when selecting the best intervention. In order to make an adequate choice, it is necessary to bring all cash flows to the present, that is, future income and expenses must be discounted to know what their value would be today. Cash flows are discounted with the following formula:
Where:
V0: Present value of the discounted cash flow
R: Discount rate
Vt: Cash flow expected to be received (or expended) in the period t
T; Number of periods
Since all cashflows, whether costs or benefits, must be discounted in order to be comparable, it is necessary to use the Net Present Value (NPV). The NPV allows to compare projects with different durations and different cash flows, and is given by the following formula:
There are several methods to estimate the discount rate (r), including: the temporary preferential social rate, the discount rate of the weighted average cost of capital (WACC), the hybrid discount rate, and the shadow price of capital. The selection of any estimation method is subject to the judgement of Sunass and experts. Some countries or jurisdictions have defined discount rates that are used in most evaluations. Europe uses 4% in general, Australia 7%, and United States makes a sensitivity analysis using values between 3% and 10%. Regardless of which value is selected for the discount rate, a sensitivity analysis is recommended in order to obtain more robust results.
6. Impact assessment
In most cases, the costs and benefits that are part of the analysis involve uncertainty: costs may be higher than anticipated, benefits lower, etc. That is why it is recommended to include a scenario-based sensitivity analysis. To build the different scenarios, key assumptions must be taken into account that divide the results, traditionally into high risk, medium risk and optimistic. Some variables that regularly affect scenarios include interest rates, inflation, employment, economic growth, tax rates, foreign trade tariffs, etc.
The United Kingdom uses this method for estimating impacts for comparing several scenarios, and a reproduction of its presentation can be observed in Table 4.15. There are two scenarios for each type of impact, the optimistic (low costs) and pessimistic (high costs) costs and the high and low benefits. Here again, there should be full transparency in the calculation of the scenarios.
Table 4.15 shows the distinction between annual average and total cost of regulation. The annual average refers to all the costs and benefits that occur in a particular year. Tabulating this allows us to understand the gap between costs and benefits, and the amount of investment resources needed over the life of the project. This comparative element is relevant in cases where the regulatory alternatives under analysis have significantly different validities, thus causing a biased comparison with the total cost.
Source: Adapted from (UK Government, 2020[8]), Impact Assessment Calculator https://www.gov.uk/government/publications/impact-assessment-calculator--3.
As a summary and to assess the impacts among the alternatives, impacts can be presented as suggested in Table 4.16.
Source: Adapted from (UK Government, 2020[8]), Impact Assessment Calculator. https://www.gov.uk/government/publications/impact-assessment-calculator--3.
Last, the differentiated impacts for identified groups must be considered. This can include small and medium companies, elderly population, geographic areas, etc. This is relevant as a decision criterion, especially when the alternative with the best outcome in the cost-benefit analysis implies a high cost for a vulnerable group.
In the report from CONAMER (2013[9]), Guía para evaluar el impacto de la regulación Vol. II, Casos de Estudio [Guideline for assessing regulation impact Vol. II, Case studies], real examples of RIA where the CBA is used can be found.
Alternate methodologies for estimating the impact
Cost-effectiveness analysis
The Cost-Effectiveness Analysis (CEA) is used when the costs of alternatives can be quantified and expressed in money terms, while the benefits, although they can be quantified, are difficult to be expressed in monetary terms. Thus, the CEA estimates the profitability of different public policy options and then, compares the results for choosing the most efficient option. It is used most often to analyze social regulations, specifically those regulating public health and safety issues, in which valid measures of effectiveness can be developed.
This method is often used to compare a set of regulatory options with similar objectives. Therefore, while CBA may result in the rejection of alternatives if costs exceed benefits, this tool focuses on choosing the best alternative. Thus, CEA involves a more limited analysis than CBA and is less demanding in terms of resources and specialisation, so it may be easier to apply in a context of limited institutional capacities.
The steps to be followed for the application of cost-effectiveness analysis are detailed below:
1. Quantify costs: The categories and strategies presented above can be used to quantify the costs of each regulatory alternative. The costs must be in net present value and it must be specified to what extent the analysis was done (direct, indirect, etc.) and which categories were taken into account.
2. Identify benefits: Since it is not possible to monetise the benefits of regulation, an approximation is made to quantify them. However, it is strictly necessary that the benefits of different regulatory alternatives have the same measure of unity in order to make comparisons.
3. Assess alternatives: Once the direct costs and benefits of the legislative alternatives are obtained, the formula of the cost-benefit analysis is applied. Specifically, the Cost-Effectiveness Ratio (CER) is obtained by dividing the present value of the regulatory project costs by the quantitative measure of the benefits:
The regulator must choose the alternative with the lowest CER.
An example of this type of impact assessment is the case where costs are expressed in monetary terms but benefits in units. In this case one scenario may be the number of poisoning deaths prevented by a new regulatory program, such as limiting the number of wastes disposed of in water reserves.
CER is an estimate of the cost expressed in monetary values incurred per unit of benefit achieved by the implementation of the legislative alternative. The analysis does not evaluate the benefits in monetary terms but is an attempt to find the lowest cost option to achieve a desired quantitative outcome.
After applying the CER formula, the regulator must classify the alternatives considering their effectiveness. Thus, the criterion to be used will always be to choose the lowest CER, that is, the one that reflects the lowest cost option among the proposed alternatives.
In the report from CONAMER, Guía para evaluar el impacto de la regulación Vol. II, Casos de Estudio [Guideline for assessing regulation impact Vol. II, Case studies] (COFEMER, 2013[9]), real examples of RIA using the CEA can be found.
Multi-criteria decision analysis
Unlike the previous cases, not all the costs and benefits derived from different options can be quantified and/or monetised. In these circumstances, Multi-criteria Decision Analysis (MCDA) is the appropriate analytical tool since it is a technique for making decisions considering different criteria simultaneously.
MCDA involves identifying the objectives of the intervention and determining all the factors (criteria) that would indicate that those objectives have been met. There is no rule about the number of criteria to be selected, everything will depend on the problem to which the regulation is oriented, as well as the elements that allow understanding how the different options would operate in the face of the problem. This analysis can combine quantitative and qualitative elements to weight the criteria, thus reducing the subjectivity of the analysis.
As a first step, the objectives of the regulation must be listed, which are used to create the set of weighed criteria and must be in importance order. This creates the context to determine the preferences among the alternative options. The performance of each alternative is identified and then, it is assessed based on the criteria listed. The contribution to the criteria is assessed normally using a scoring factor. Weights and scores for each one of the alternatives are added to obtain a global value, by providing a classification of the different options.
Consequently, for conducting the MCDA, the following steps must be followed:
1. Identify the objectives: the purpose of a MCDA: find the option that better complies with the outlined objectives.
2. Establish the evaluation criteria: The criteria serve to weigh the compliance of secondary objectives.
3. Identify the options that will be assessed.
4. Scoring and assessing the expected performance of each option according to the evaluation criteria: The performance assessment can be summarised by means of a matrix, in which the evaluation of each option is presented according to the defined criteria (which can be quantitative and/or qualitative, see Table 4.17).
5. Weighting of criteria: Weights are assigned to each criterion in order to reflect its relative importance in the final decision.
6. Use a decision mechanism to identify the best option (usually the one with the highest score).
To be effective, the MCDA must meet the following standards:
Assign an appropriate weight to the criteria: The weights assigned might have a significant effect on outcomes, for example, a high weight of the criteria regarding the benefits related to the costs causes biases on the results versus options with relatively low costs. Because of that, neutral weights of 50% must apply for criteria related with costs and 50% for criteria related with benefits, unless appropriate alternative weights can be warranted.
Scoring scale to be used: A symmetric scale ranging from -10 to +10 is easy to apply and understand, as well as it allows a sufficient margin to differentiate the different options. It is recommended to use this range in the impact assessment.
Indications on appropriate/inappropriate criteria: Criteria for MCDA should be closely linked to the problems and objectives identified. Cost criteria should be defined as "cost" rather than "cost minimisation". This specification allows costs to be properly evaluated relative to the baseline (a more costly option than the status quo will receive a negative score).
As part of the MCDA the following elements must be addressed:
Clearly explain the justification for the weights assigned to each criterion.
The relative scores assigned to the criteria of each option must be consistent with the relative effects (for instance, if an option represents costs 10 million higher than baseline, while another represents 1 more million, it would be appropriate to assign scores of -10 to the first one and -1 to the second one).
Consider whether the weighted total scores of some options are close, as in these cases the MCDA results are very sensitive to the weights chosen.
When an option imposes a compliance cost (that is, a negative weighed score is assigned), the impact assessment should at least estimate the magnitude of those costs.
In the report from CONAMER, Guía para evaluar el impacto de la regulación Vol. II, Casos de Estudio (Guideline for assessing regulation impact Vol. II, Case studies) (COFEMER, 2013[9]), real examples of RIA using the MCDA can be found.
Summary of methodologies
Table 4.18 summarises the different methodologies presented in the selection.
Example:
Following the example presented by Ofwat, its RIA carried out a CBA, including a sensitivity analysis to measure three net benefit scenarios for each of the two public policy proposals. Measured as low, high, and “best” estimates, Ofwat presents a range of comparison between them. Thus, it ends up choosing policy 2 as the best option since its expected net benefit from the best estimate is £237 million while the expected estimate from policy 3 is £87 million. The following is a summary of the results of the CBA.
Option 1 Make new regulations applicable to all SPC that allow the creation of independent and directly regulated IPs that finance and supply large or complex infrastructure projects.
Assumptions:
Base year: 2010
Period: 30 years
Discount rate 3.5%
Costs
All estimations refer to the only major infrastructure project planned for the next 10 years - the Thames Tidal Tunnel. OFWAT's costs in terms of additional regulatory effort amount to GBP 5 million. With an average of 0.08 million GBP per year during the period. The remaining annual costs (2.3 million GBP per year, according to the best estimation, between 1.8 and 4.2 million GBP) are related with the management of IP as additional companies. Transition costs represent the cost for SPC bidding for IPs: They are estimated in 17 million GBP split in two years (which represents 0.4% of the total cost of the project Thames Tideway Tunnel). Table 4.19 presents 3 scenarios of the total costs of option 1.
Benefits
The benefit is the isolation of the project risk within the independent IP, by preventing that this “spreads” to the SPC. Such risk might express financially and/or in the attention diversion of the administration, with a higher risk for regulatory and financial outcomes. This can lead the market to reassess the credit solvency of the SPC, which entails an increased capital cost. The monetised benefit (for information only) is referred to prevent an increase of the capital cost for the main business of Thames Water of 0.25% - 1% during the building period of the TTT (see Table 4.20). Note: Data are for information only, but based on market confidential information. Table 4.21 presents three scenarios of net benefits of option 1.
Option 2. Modify the operation license of a sole SPC to create a separate IP financing and performing a large regulatory project (for instance, the Thames Tideway Tunnel) on behalf of the SPC.
Assumptions:
Base year: 2010
Period: 30 years
Discount rate 3.5%
Costs
Like Option 1, the estimates refer to the Thames Tideway Tunnel. Costs for OFWAT in the negotiation of the changes of license and the contractual terms amount to 5 million GBP, with an average of 0.08 million GBP per year. The remaining annual costs (best estimate of GBP 2.3 million; range GBP 1.8-4.2 million) are related with IPs in operation. Transition costs accumulate for water companies in the tender of IPs: They are estimated on 17 million GBP in two years. In general, costs are like those of Option1, with the OFWAT’s regulatory costs replaced by licensing costs and negotiation of contracts of a similar magnitude. Table 4.22 presents the total costs estimated from Option 1.
Benefits
The benefits in concept are similar to those of option 1, but option 2 (see Table 4.23) will be much less effective in isolating project risk within the IP, because the IP will not be a truly separate and directly regulated entity. This means there is a greater likelihood of risk "contagion" to the linked SPC, for example, through “market consolidation” (that is, including the IP value in the accounts of the SPC). As an assumption, the benefits of risk isolation (estimated in terms of capital cost reduction for the SPC) are half of those in option 1, due to the cost of a similar option. However, in practice, benefits can be even more limited than this. Table 4.24 presents 3 scenarios of net benefits from option 2.
When comparing the range of benefit estimates, Option 1 goes from GBP 53 to GBP 547 million, with the better estimate in GBP 237 million, while in option 2 the range passes from –GBP 97 to GBP 247 million with the better estimate in GBP £87 million. Option 1 not only has a higher net expected benefit (better estimate) than Option 2, but the variability is less risky. A key point is that even the worst-case scenario yields a positive net benefit, contrary to Option 2. In this case it is important not only to look at the expected net benefit, but also at the range of uncertainty. There could be a case where the best estimate is higher, but the range of variability is high with negative values. In these cases, the probability of the range of variability should be assessed.
Which is the most appropriate methodology to assess impacts?
What are the costs and benefits of each alternative?
What costs and benefits are direct or indirect?
What is the most appropriate way to quantify costs and benefits? Is there a bias in the quantification method?
What are the groups facing costs and benefits? Does any particular group suffer disproportionate costs?
What is the temporality of costs and benefits?
What variables affect the estimated projection of costs and benefits?
What is the level of uncertainty associated with impacts? Does uncertainty significantly affect the alternative selection?
The impact of a regulation is directly related to the extent to which regulated regulators comply with it. A regulation without compliance will not solve the public policy problem that originated it, and therefore will not achieve the objectives set out. It is essential that Sunass design a regulatory compliance plan before it is approved. The RIA process loses value when a compliance strategy is not designed, as well as monitoring and assessment, since the design of the regulation can be very sophisticated, but when implementing the regulation, there are not sufficient human or material resources, or there is no body responsible for monitoring the standard.
The establishment of a correct compliance strategy should include the following:
It minimises costs and efforts for the regulated subjects and the government
It creates incentives so the regulated subjects comply with the regulation
It establishes the adequate guidelines for those who oversee the regulation.
To determine the compliance with a regulation, the extent of voluntary or mandatory compliance that the regulation will have must be considered, that is, how easy or difficult it will be to comply with the regulation.
Voluntary compliance: Encourages obligated subjects to change their conduct and comply with the obligations imposed. This compliance is related to the cost of compliance with the regulation to the subjects, as it is assumed to be minimal and therefore motivates compliance without the need for additional measures. It is assumed that both risk and compliance costs are low, which encourages compliance with the obligations imposed by the authority.
Mandatory compliance: It will impose more work of surveillance and control from authorities. It is presupposed that both, costs, and administrative burdens imposed by the regulation for subjects obliged are high, which encourages to prevent its compliance.
The inspection and auditing strategy of the selected regulatory proposal should consider the following principles, which will guarantee an increase in compliance (OECD, 2018[10]):
Evidence-based application: The assignment of a budget for inspections must be conducted using a cost-benefit analysis and depending on the level of risk of the topic. Regulators must consider evidence for assigning resources in an efficient manner.
Selectivity: Having a budget restriction, regulators should select the subjects to be inspected through a risk assessment.
Focus on risks and proportionality: Supervise with greater periodicity and/or rigidity the instances that imply greater risk. On the contrary, supervise with less frequency and requirements those that imply a lower risk.
Responsive regulation: Determine the level of supervision considering the profile and specific behavior of the regulated subject.
Long term vision: The regulatory compliance policy must be based on fulfilling general and specific goals.
Coordination and consolidation: Inspection functions must be coordinated and consolidated, avoiding duplication and waste of public resources. Coordination between government agencies is necessary to avoid duplication.
Transparent governance: Warrant stability with professional careers, as well as the interference of political cycles.
Information integration: Use information technologies to share information, maximizing the risk focus and coordination.
Clear and fair process: Parameters should be established to establish a fair process, as well as publish the requirements for each inspection process.
Encourage the compliance: The government should implement the necessary mechanisms to achieve active compliance with the regulation.
Professionalism: Ensuring the inspectors professionalism is critical to effective and transparent inspections that build trust.
Which are the incentives of the regulated parties for complying with the regulation?
Is the strategy of a voluntary compliance reasonable, or is it necessary to appeal to traditional measures of mandatory compliance?
What are the risks in case of non-compliance?
What are the best tools to ensure compliance?
What risk indicators can feed the compliance strategy?
Can the current supervisory staff reasonably monitor regulatory compliance? Otherwise, what is the additional need for staff?
What supervising methods, beyond inspections, can be used?
Which is the necessary budget, including staff and technological tools, to achieve an adequate supervision of the regulatory compliance?
The monitoring and assessment of the regulatory proposal allows to clearly identify whether the public policy objectives are being achieved, as well as to determine whether the proposed regulation is necessary or how it can be more effective and efficient to achieve the proposed objectives. Monitoring and assessment are usually underestimated steps in the public policy development process, but they are a fundamental part of the implementation of the regulatory proposal to achieve its objective.
Monitoring
Monitoring is a systematic process for the collection, analysis and use of information to measure the progress of public policy objectives established in the legislative proposal. Monitoring a legislative proposal allows to identify if a regulation is being applied as expected, or if applicable, if there is the need to implement other measures. In order to carry out adequate monitoring of our regulatory proposal we need to identify:
What evidence do we need?
When and how should we collect it?
When to collect it and from whom?
To do this we must establish indicators that allow us to measure the performance of a legislative proposal. These indicators must be defined, measurable, and time-dependent. To obtain the information we must consider the following characteristics:
Information research must be exhaustive, that is, we must consider qualitative and quantitative information.
The cost for data collection should be proportional to the expected benefit of obtaining that information.
Avoid requesting duplicate information, particularly information that has been previously requested. Internal coordination between Sunass areas is very important for this purpose.
The collection and use of the information collected must be timely, otherwise there is a risk that the information will not be useful and excessive costs will have been generated.
Finally, during this process, the transparency and usefulness of the information must be guaranteed.
As mentioned, monitoring is a fundamental element for complying with the regulatory objectives. Implementation and compliance are integral parts of the RIA process. However, there is evidence that most of the time, they do not get the due attention. Several analyses of RIA take for granted that the implementation and compliance are intrinsically involved in the process, something that will occur automatically. For the adequate implementation and compliance and a better achievement of the balance between efficiency and effectiveness, it is necessary that Sunass considers if inspections would be necessary, as well as the way in which they must be organised and planned within the budget (OECD, 2018[10]).
Assessment
The assessment is defined as an evidence-based judgement about a measurement in which a legislative proposal has been effective and efficient, relevant, and coherent. Mechanisms for regulatory evaluation need to be established from RIA´s part process where the regulatory proposal and implementation plan are being designed.
The assessment of the legislative proposal allows to identify if the decision was adequate for managing the initial public problem, if the intervention was enough, and if other ways to reach the same outcome exist in an equally or more efficient manner. With the evaluation, the regulatory cycle is closed, and it is what is known as ex post assessment. The ex post assessment must be performed sometime after the legislative proposal has been implemented, usually a 5-year period, but it will depend on the impact of the regulation.
For conducting an ex post assessment, it is first necessary to identify and gather the information on indicators which allow to measure to which extent the objectives have been reached, as well as the level of compliance, and the effects of the regulation. Such indicators should normally be determined as part of an ex ante evaluation when applying any of the established methods for estimating impacts.
The objectives outlined with the regulation should fulfill two functions that should be covered when a regulation assessment is carried out. First, they must be achieved, that is, complying with the specific objective for which they were designed. If an objective has not been achieved, a decision must be made about changing the strategy for its compliance or the relevance of designing different objectives. If the objective has been achieved, Sunass must evaluate its contribution to solve the central problem identified in the RIA. If it has not been successful in contributing to solving the problem, it may be that the objective was poorly stated or that the identified cause of the problem may not have had any impact. In these cases, the objectives must be rethought. Second, it is required to use a methodology that allows to state if there is causality link between the estimated effects and the regulation.
Ex ante and ex post evaluation are similar in the sense that in both processes the quality of the regulation is evaluated. However, while the ex ante evaluation is a process based on available information and leaves some uncertainty about its effectiveness in the air, the ex post evaluation is a verification exercise that requires the collection of data regarding compliance and effects on the agents or market of the regulation over a given period.
Thus, while the ex ante evaluation aims at predicting the impact of a regulation on the basis of a prospective analysis, the ex post evaluation is defined as a critical judgment, based on evidence, of whether a regulation has satisfied the needs it was intended to satisfy and whether it has achieved the expected effects. Ex post evaluation goes beyond an assessment of whether something happened or not, and usually focuses on the cost and effectiveness of the regulation as a whole (OECD, 2018[10]).
What are the key indicators for monitoring the status of the regulatory implementation?
What are the potential costs of generating the information needed to monitor regulation?
How often will monitoring be?
Which are the indicators that would be used to assess the effectiveness of the regulation?
How long after the regulation has been implemented will it be subject to assessment?
What criteria would be used to consider if the regulation met its objectives?
References
[7] CEPS (2013), Assessing the costs and benefits of regulation, Center for European Policy Studies, Brussels, https://ec.europa.eu/smart-regulation/impact/commission_guidelines/docs/131210_cba_study_sg_final.pdf (accessed on 9 July 2020).
[5] CEPS, A. Renda and D. University (2015), Análisis de impacto normativo: la experiencia de la Unión Europea, https://colaboracion.dnp.gov.co/CDT/Mejora%20Regulatoria/Presentaciones/Cierre%20Pilotos%20Sept%202015/3.%20RIA%20Union%20Europea%20Andrea%20Renda.pdf (accessed on 8 July 2020).
[9] COFEMER (2013), Guía para Evaluar el Impacto de la Regulación, http://www.cofemer.gob.mx/presentaciones/Espa%F1ol_Vol%20I.%20Metodos%20y%20Metodologias_FINAL.pdf (accessed on 8 September 2017).
[4] Comisión Europea (n.d.), Better regulation Toolbox, https://ec.europa.eu/info/sites/info/files/better-regulation-toolbox.pdf (accessed on 8 September 2017).
[1] Commonwealth of Australia, D. (2020), The Australian Government Guide to Regulatory Impact Analysis, http://www.pmc.gov.au/regulation.
[2] European Commission (2017), Better Regulation Guidelines, European Commission, Brussels, https://ec.europa.eu/info/sites/info/files/better-regulation-guidelines.pdf (accessed on 7 July 2020).
[3] European Commission (n.d.), Better regulation: guidelines and toolbox. Stakeholder consultation, https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-regulation-why-and-how/better-regulation-guidelines-and-toolbox_en (accessed on 23 July 2019).
[10] OECD (2018), OECD Regulatory Enforcement and Inspections Toolkit, OECD Publishing, Paris, https://dx.doi.org/10.1787/9789264303959-en.
[6] SCM Network (2004), International Standard Cost Model Manual, http://www.oecd.org/gov/regulatory-policy/34227698.pdf (accessed on 2 August 2017).
[8] UK Government (2020), Impact assessment calculator - GOV.UK, https://www.gov.uk/government/publications/impact-assessment-calculator--3 (accessed on 9 July 2020).