This chapter describes the current state of play of innovation procurement in Croatia focusing on the three enabling elements of innovation procurement as the building blocks of innovation procurement. (i) regulatory and institutional framework, (ii) strategy and (iii) monitoring system. The chapter starts by overviewing the regulatory frameworks and the institutional frameworks including innovation ecosystem of innovation procurement in Croatia. Then, the chapter examines the relevance of strategies of innovation procurement aligned with other sectoral policies. Lastly, the chapter introduces a monitoring system of innovation procurement with key performance indicators which were proposed to Croatia by the OECD.
Innovation Procurement in Croatia
Boosting Innovation in the Public Sector
Abstract
Increasing the uptake of innovation procurement requires multidimensional efforts, from developing an ecosystem conducive to innovation to mobilising stakeholders towards shared objectives and upskilling the procurement workforce to navigate more complex options and policy trade-offs. This Chapter discusses regulatory and institutional frameworks, strategies, and monitoring system of innovation procurement.
2.1.1. The procurement regulatory framework allows to support innovation, but its application to different types of innovation could be clarified
The Croatian legal basis for innovation procurement is the Public Procurement Act (PPA, Official Gazette No. 120/2016), which was adopted by the Croatian Parliament on 9 December 2016 and entered into force on 1 January 2017. The PPA transposed the 2014 EU Directives. Following the European regulatory framework, the PPA provides several options that could be used to implement the innovation procurement. Table 2.1 lists the procurement procedures and methods that can be used in innovation procurements:
While different procurement procedures and methods can be used for the two concepts of innovation procurement (buying the process of innovation – R&D- and buying the outcome of innovation as PPI), Figure 2.1 identifies the most common procedures and methods used for each concept of innovation procurement.
Source: Prepared by the author
It is worth noting that while procedures used to procure the outcomes of innovation (PPI) are governed by the procurement regulatory framework, R&D procurement such as Pre-commercial procurement (PCP) used to procure the process of innovation fall outside this regime. PCP is the procurement model of R&D that uses the exemptions related to the purchases of R&D services stipulated in Article 14 of the EU Directive and Article 30 (17) of the PPA, As such, PCP is applicable and not considered as a state aid when: (i) the benefits do not accrue exclusively to the contracting authority1 for its use in the conduct of its own affairs, or (ii) the service provided is not wholly renumerated by the contracting authority (Croatian Parliament, 2016[2]).
PCP may be suitable when a considerable amount of research and development is needed to develop a new product or solution and seek radical innovation. The contracting authority has large flexibility in carrying it out provided that the principles of transparency, equal treatment and non-discrimination of the Treaty on the Functioning of the European Union (TFEU) are respected. As various procurement procedures and methods can be used for innovation procurement, Croatia could benefit from continuing to clarify to contracting authorities a legal aspect of innovation procurement, in particular, R&D procurement.
Various procurement procedures and methods, foreseen in the PPA, can be used for carrying out innovation procurement. However, it should be noted that the majority of public procurement is carried out through open procedure in Croatia. In 2023, open procedure accounted for 89.33% in terms of the number of procedures and 90.20% in terms of procurement value. Three contracts were awarded through competitive procedure with negotiations, and no contracts were awarded through competitive dialogue and innovation partnership (MINGO, 2024[3]).
Open procedure is not always the most appropriate method for innovation procurement since it often involves more prescriptive requirements and less flexibility. Yet, innovation procurement can be carried out through well-designed open procedures by using instruments such as early market engagement, functional specifications and innovation-friendly contract conditions. To this end, Article 218 of the PPA stipulates that contracting authorities can set up special conditions for contract execution for features related to innovation, while Article 284 stipulates that the quality criteria could include innovative characteristics. However, there are many restrictions in using open procedures to spur innovation. Indeed, other procurement methods such as competitive procedure with negotiation, competitive dialogue and innovation partnership, which are traditionally rarely used in Croatia, are deemed more appropriate and innovation friendly as they allow for more flexibility in the design of the solution and dialogue with the market.
While innovation procurement is not mainstreamed in Croatia, the country already had experience in this context such as the innovation procurement carried out through competitive procedure with negotiations by the City of Koprivnica, the winner of the award “Procura+ Award 2020” in the category “Procurement of Innovation of the Year.” The City of Koprivnica carried out one of the first innovation procurement in Croatia, in collaboration with the Regional Energy Agency North, to fully transform a 35-year-old kindergarten building. The City of Koprivnica sought innovative solutions through a simple combination of existing materials and techniques (incremental) rather than a newly developed concept/ innovation (transformative). Therefore, innovation arose from the combination of existing materials and basic techniques, when such a combination was not previously offered nor used in the market.
The City of Koprivnica was seeking the innovative solution to reconstruct a 35-year-old prefabricated wooden kindergarten. Loptica kindergarden was considered to be almost ending its useful lifetime and was suffering from serious structural defects that could have led to its demolition eventually. None of major renovation works had been made, leading to many issues such as energy inefficiency.
The idea was to avoid both the demolition of the kindergarten and the construction of a new one, which would imply a substantial cost and also be a burden to the environment. The City of Koprivnica desired to design and implement deep transformation of the kindergarten that would result in significant improvement of the energy efficiency, indoor space functionality, childcare, learning and play environment quality. In addition, the City of Koprivnica needed an innovative solution that could be replicated on similar buildings in the city and in the country, which could be of benefit to both the procurer and the bidder. Therefore, the city decided to carry out innovation procurement to solve their challenge.
In 2017, the Regional Energy Agency North and the City of Koprivnica started to prepare the first innovation procurement process in Croatia to fully transform the 35-year-old kindergarten building. The whole process started in January 2017 by needs identification and continued through March 2018 by publication of Prior Information Notice (PIN) in TED (Tenders Electronic Daily) and National Official Procurement Portal (EOJN), which marked the start of the open market consultation phase. By the end of 2018, the competitive procedure with negotiation was selected as the most suitable procurement procedure to finalise the process. The procurement procedure was published on 31 January 2019. The contract was signed on 24 April 2019 and the innovative solution was delivered and implemented by 31 August 2019. The innovation arose from the combination of existing materials and basic techniques, where such a combination was not previously offered nor used in the market. The final solution completely transformed the kindergarten building with substantially extended lifetime.
This case of innovation procurement showed the potential of innovation procurement in the country with several impacts. Obviously, the final impact of the innovative solutions through this innovation procurement was the improved building lifetime of 25 more years for the cost of less than EUR 500 per square meter, which is approximately 50% of a cost of a new building.
The solution also resulted in 61% savings on heating energy, 66% savings on primary energy and 66 % less CO2 emissions per year.
Instead of demolishing the old building and constructing new one, the project transformed and upgraded the building which was considered to be near the end of its useful lifetime.
Source: Prepared based on (City of Koprivnica, 2018[4]) and the information provided by the Regional Energy Agency North
The competitive procedure with negotiations carried out by the city of Koprivnica showcased an objective to seek incremental innovation by combining existing materials and techniques, which would have been challenging under open procedure as it does not allow for dialogue and negotiations with bidders. In the short- and medium-term, Croatia could focus on promoting innovation procurement that seeks incremental innovation rather than transformative innovation. This will allow for a gradual paradigm shift among procurement officials by moving from traditional procurement such as open procedure to non-traditional procedures that require dialogue and negotiation such as competitive procedure with negotiations and competitive dialogue.
In the long-term, Croatia could promote innovation procurement which involves R&D to seek more drastic innovation such as innovation partnership. Indeed, Croatia has experience in participating in PCP project (Pre-commercial Procurement of innovative ICT-enabled monitoring to improve health status and optimise hypertension care) during the years 2019 – 2023, which was funded by the EU’s Horizon 2020 programme. However, innovation procurement which involves R&D such as PCP and innovation partnership requires relatively extended timeframes and often years. Therefore, a country like Norway developed a procurement method called StartOff model to support smaller R&D services and to reach out to more start-ups by using the same exemption of the EU Directive and the PPA applicable to PCP (See Box 2.2 for more details on the StartOff model).
There is flexibility in the approach to R&D procurements outside the scope of the EU Directive and PPA. The Norwegian StartOff model is one such adaptation to facilitate fast-track projects especially targeting start-ups. Start-ups often have the most innovative solutions, but traditional procurements are hard to navigate, demand a lot of resources, and there is no way of guaranteeing a contract. The public sector on the other hand is not familiar with the innovative solutions offered by start-ups.
The StartOff model is a fast-track (6 months) R&D-procurement inspired by PCP but consists of only two phases and only one bidder is awarded a contract for up to phase 2 (developing a minimum value product: MVP) over three months. Figure 2.2 provides a general overview.
The process starts with an open call, especially targeting (but not limited to) the start-up community through social media, using promo videos and direct targeting. The bid offer follows a maximum 2 000 words idea sketch template that is evaluated to select a maximum of 6 bidders to be invited to a 45-minute digital interview to present the proposal and answer questions.
Based on the idea sketch and interview, three suppliers are then selected to further develop their idea over 3 weeks with a compensation of EUR 5 000. Then, the suppliers present their solution design in writing and in a pitch session. Based on the same evaluation criteria as in the previous phase (quality and innovation potential 60%, team capacity and skills 30%, and commercial potential 10%), one supplier is selected to develop the MVP with a compensation of EUR 45 000. The development happens in close co-operation with the public buyer. The supplier retains all IPR-rights to what is being developed, while the public buyer gets extended license-free user rights.
After development, the supplier demonstrates the solution in a Demo-day, in which interested buyers, investors and others attend. In addition, the parties explore the possible ways forward: a subsequent procurement, continued testing or development, buying the MVP or other.
The StartOff model is a co-operative initiative of the StartOff team which is composed of the Norwegian Supplier Development Programme (LUP), the Norwegian Digitalisation Agency and the Norwegian Agency for Public and Financial Management (DFØ). The StartOff team assumes project management duties, allowing the public client to concentrate on learning and identifying optimal solutions.
Fast-track small scale innovation procurements also have the following benefits:
Less Complexity: Larger innovation projects often involve numerous stakeholders, dependencies, and complexities. Smaller projects facilitate an easier path to a successful project and later adoption of the solution.
Reduced Development Risk: Large-scale innovation procurements can span 3-5 years and risk rendering solutions outdated due to continuous technological advancements, change of key personnel, or shifting priorities. Fast-track projects lasting 6-12 months are less susceptible to these risks.
Market interest: Start-ups/ SMEs cannot always deliver on the full range of deliverables required in a complex project. Limited project duration is often a better fit for startups with their desire for quick product development and verification.
Source: Prepared based on the information provided by the Norwegian Agency for Public and Financial Management (DFØ).
2.1.2. Strengthening institutional co-operation is pivotal for better innovation procurement policies
Besides opportunities provided by the legal and regulatory framework, it is useful to understand the roles of different institutions in the Croatian public procurement system, since each institution has, to a varying extent, an influence on the uptake of innovation procurement.
In Croatia, key functions of public procurement system are clearly defined and assigned to competent authorities in a comprehensive and coherent manner. The Directorate for Public Procurement Policy (DPPP) within the Ministry of Economy of the Republic of Croatia (MINGO) is the main public procurement authority in Croatia. As a policy maker, MINGO is in charge of many key functions within the public procurement system in Croatia. Its mandates include (i) drafting of laws and other regulations related to public procurement, (ii) development and improvement of the public procurement policy, (iii) administration of a national e-procurement system EOJN RH (Elektronički oglasnik javne nabave RH), (iv) administration of the national certification framework for the public procurement workforce, (v) provision of methodological assistance (e.g. capacity-building activities, technical opinions and guidelines), (vi) monitoring of the public procurement system including the preparation of annual statistical reports of public procurement and ex-ante controls (Croatian Parliament, 2016[2]).
The main oversight bodies regarding public procurement are MINGO (administrative monitoring and random ex-ante controls of tender documents) and the State Audit Office. The State Audit Office performs ex-post audits to ensure the efficiency and effectiveness of the use of public funds including through public procurement. Under the context of the EU-funded projects targeting innovation, the Croatian Central Finance and Contracting Agency for European Union programmes and projects (SAFU) is a key actor in the management of European Structural & Investment Funds (ESIF) programmes. SAFU was founded in 2007, and its main mandate is to ensure the legality of EU investments in Croatia and to support managing programmes and projects financed by the European Union budget in Croatia (SAFU, n.d.[5]).
The State Commission for the Supervision of Public Procurement Procedures (DKOM), as an autonomous and independent body, handles review procedures related to public procurement. The decision of the DKOM is final and cannot be appealed. However, an administrative dispute can be initiated before the High Administrative Court of the Republic of Croatia after the decision of the DKOM (Croatian Parliament, 2016[2]).
The Central State Office for Central Public Procurement is a national Central Purchasing Body (CPB) established in 2010. It is in charge of central public procurement for the central government comprised of thirty entities (e.g. Office of the President, the Parliament, ministries). It operates 132 active framework agreement for a wide range of product categories (e.g. office and ICT equipment, travel services, vehicles, fuel cleaning services, maintenance service). Other contracting authorities also have the option to use these framework agreements (Central State Office for Central Public Procurement of the Republic of Croatia, n.d.[6]).
Finally, an important actor of the procurement system are the public procurement officials themselves within contracting authorities, who are responsible for carrying out public procurement procedures in accordance with the established institutional and regulatory frameworks. It is estimated that there are approximately 12 000 contracting authorities and contracting entities registered in EOJN RH.
Source: Created by adapting (OECD-SIGMA, 2016[7]) based on (Croatian Parliament, 2016[2]) and the information provided by Croatia
When it comes to innovation procurement, all actors within the procurement system play important role in innovation procurement under their respective mandates and capacity. For example, MINGO2 is in charge of taking initiatives to promote innovation procurement as public procurement authority of Croatia. In addition to these actors in the public sector, the Croatian Agency for SMEs, Innovations and Investments (HAMAG-BICRO) acts as key institution to promote innovation procurement as a competence centre of innovation procurement in Croatia. HAMAG-BICRO was founded in 1994 under the auspice of the Ministry of Economy of the Republic of Croatia to support the development of SMEs, innovation process and investments in Croatia. Currently, HAMAG-BICRO functions as a competence centre of innovation procurement in Croatia. However, Croatia does not have a formal institutional mechanism to co-ordinate innovation procurement policy at high level (e.g. inter-ministerial committee), although the key stakeholders have informal mechanisms to collaborate (e.g. training of innovation procurement organised in collaboration among MINGO, HAMAG-BICRO, Central State Office for Central Public Procurement and SAFU). Therefore, Croatia could consider the possibility of establishing a formal institutional mechanism to co-ordinate innovation procurement policy.
Economic operators also play key role in public procurement. Under the context of innovation procurement, they are even more relevant actors in that they provide innovative solutions based on the needs and problems described by the public sector. In addition, the innovation ecosystem in Croatia contributes to boosting innovation in the country. These include universities, research centres, incubators and accelerators, and brokers. Figure 2.3 presents the stakeholders that play a key role in the successful implementation of innovation procurement in Croatia.
Source: Created by the author
More details and issues of the key stakeholders of innovation procurement in Croatia will be discussed later at relevant sections of the report. Section 3.1 discusses contracting authorities from the viewpoint of their capability to carry out innovation procurement. Part 3.1.3 overviews the function of HAMAG-BICRO as a competence centre of innovation procurement in Croatia to reinforce its current functions. Section 3.2 focuses on economic operators, DKOM and SAFU as obtaining their buy-in is indispensable to promote innovation procurement in Croatia.
2.1.3. Croatia could benefit from a national strategy to guide and strengthen innovation procurement efforts
It is essential to develop a strategy to show clear vision and commitment as well as to define activities and their targets within a specified timeline (OECD, 2023[8]). Currently, Croatia does not have a national public procurement strategy nor a strategy of innovation procurement, whether it is stand-alone or part of the national public procurement strategy. Indeed, the lack of the strategy of innovation procurement and its linkage with other sectoral strategies was pointed out as one of the weaknesses of Croatia’s policy framework of innovation procurement by the EC report (European Commission, 2021[9]).
Croatia developed several strategic documents, including the Strategy for the Development of the Public Procurement System in the Republic of Croatia (2008 June) and its action plan, as part of Croatia’s accession process to the EU, E-procurement strategy (2013-2016) and First National Action Plan (NAP) of Green Public Procurement for period 2015 – 2017 with a view to 2020.
Currently, regardless of the absence of a national public procurement strategy, reform measures related to public procurement system specified in the recovery and resilience plan function as a strategy or action plan including promoting innovation procurement. Croatia submitted its recovery and resilience plan (RRP) to the European Commission (EC) on 14 May 2021 to request for financial support from the Recovery and Resilience Facility (RRF) to support the economic recovery after the Covid-19 crisis and the earthquakes that hit the country in 2020. In the RRP, Croatia aims at boosting economic growth and creating jobs while achieving its key challenges: (i) green transition, (ii) digital transition, and (iii) economic and social resilience. Operational Arrangements between the EC and Croatia for the RRF specify the implementation of the measures related to public procurement reform including, among others, promoting innovation procurement (No. 263: Programmes and activities aimed at designing and managing innovative public procurement and 264: Training of procurement officers on innovation public procurement procedures) (European Commission, 2022[10]).
Croatia could benefit from developing a strategy of innovation procurement, whether it is stand-alone or part of the national public procurement policy. First, the two reform measures of the RRP related to innovation procurement (No. 263: Programmes and activities aimed at designing and managing innovative public procurement and 264: Training of procurement officers on innovation public procurement procedures) will be due by the end of 2025 and further measures are needed to increase the uptake of innovation procurement in the upcoming years. In addition, a more comprehensive strategy shows strong commitment of government of the government with specific actions to promote innovation procurement. Indeed, the survey on innovation procurement carried out to economic operators in Croatia showed that the clear vision of the government is the most relevant initiative to promote innovation procurement (See Figure 3.15).
The strategy can include, but not limited to, the following elements such as:
Background / Context
Key challenges and gaps identified.
Vision / objective.
Specific activities and targets.
Implementation period.
Leading entity and key stakeholders (roles and responsibilities).
Alignment with other sectoral strategies.
Financial resources.
The strategy could be accompanied by an action plan that summarises key initiatives to be taken, responsible parties, key performance indicators, and timeframe (OECD, 2023[8]).
Specific activities and targets could include promoting innovation procurement such as organising training and events and reinforcing/promoting the function of a competence centre of innovation procurement, and establishing a monitoring mechanism of innovation procurement, to name a few. Recommendations provided in this report and the action plan in Chapter 4 will feed inputs to potential activities which could be specified in the future strategy of innovation procurement.
In addition, some countries have decided to set up quantitative targets of innovation procurement to measure the level of uptake. As such, Croatia could also consider the possibility of setting up quantitative targets of innovation procurement. However, to avoid setting over ambitious targets from the beginning, Croatia could start from specifying the number of innovation procurement procedures rather than the share against the total of public procurement procedures.
Multiple stakeholders need to be involved and/or consulted when developing a strategy and/or action plan. For example, Spain established an interministerial commission of innovation procurement to co-ordinate innovation procurement initiatives and prepare the innovation public procurement plan (See Box 2.3). This high-level initiative also contributes to communicating a clear vision and commitment of the government to promote innovation procurement.
Spanish Science, Technology and Innovation Strategy 2021-2027 (EECTI), a national innovation policy of Spain, clearly recognises public procurement as a strategic tool to stimulate innovation. It mentions that innovation procurement will be promoted from the public sphere (central and local entities, public companies, universities, etc.) that will turn the public sector into driving elements of innovative activity in areas such as health, mobility or the smart and sustainable agri-food chain.
On 9 April 2024, Royal Decree 364/2024 was approved to create the Interministerial Commission for the Incorporation of Innovation Criteria in Public Procurement and regulating the Innovation Public Procurement Plan. This Interministerial Commission is attached to the Ministry of Science, Innovation and Universities with the Presidency held by the Ministry of Science, Innovation and Universities, and the second Vice Presidency held by the Ministry of Finance.
The Interministerial Commission, in collaboration with the State Procurement Advisory Board and in accordance with the National Public Procurement Strategy, will be in charge of:
Drafting and reviewing the Innovation Public Procurement Plan, which shall be aligned with the Spanish Strategy for Science, Technology and Innovation, the State Plan for Scientific, Technical and Innovation Research and the National Public Procurement Strategy.
Monitoring the implementation progress of the Plan.
Designing training initiatives for the public procurement workforce to carry out innovation procurement.
The Innovation Public Procurement Plan will have the duration of seven years, and determine the categories in which contracting authorities within the public sector are going to promote public procurement that takes into account innovation parameters, as well as the criteria. The criteria may be incorporated into procurement conditions such as technical specifications, selection criteria, contract award criteria and/or special conditions of execution.
Similarly to the example in Spain, it is essential to maintain policy coherence and linkages among different strategies. Indeed, many sectoral strategies recognise public procurement as a strategic tool to stimulate innovation in the economy of Croatia:
The National Development Strategy of the Republic of Croatia until 2030, under the strategic goal 1. Competitive and innovative economy, mentions the potential to use innovation procurement and green public procurement more intensively to encourage innovation and facilitate the commercialization of innovation (Croatian Parliament, 2021[17]).
The Education, Science and Technology Strategy of the Republic of Croatia further includes innovation procurement as one of the indicators in the Measure 6.5 Promote investment of the innovative economy in research, development and innovation: i.e. “Public procurement as a tool that encourages innovation and innovation partnership (number of tenders and amount of funds”) (Croatian Parliament, 2014[18]).
The Digital Croatia Strategy for the period until 2032 encourages the use of innovation procurement procedures (e.g. innovation partnership, competitive dialogue) to procure advanced digital solutions in the public sector and the modernisation of information infrastructure (Croatian Parliament, 2023[19]).
The Integrated National Energy and Climate Plan for the Republic of Croatia for the period 2021 - 2030 mentions the linkage of green public procurement with innovation procurement, by showing the benefits of procuring innovative low-carbon products and services through green (and innovation) procurement in order to further boost their entry into the market, with the public sector serving as a good example. (Ministry of Economy and Sustainable Development of the Republic of Croatia (MINGOR), 2023[20]) The Croatian Smart Specialization (RIS) Strategy 2023–2029 (S3) clearly recognises public procurement as a strategic tool to promote innovation (MINGOR, 2023[21]) It mentions the concepts of innovation procurement including R&D procurement (e.g. PCP) and PPI. It defines seven thematic priority areas under the context of boosting innovation in Croatia:
Personalised Health
Smart and Clean Energy
Smart and Green Transport;
Security and Dual Use – Awareness, Prevention, Response, Remediation
Sustainable and Circular Food
Customized and Integrated Wood Products
Digital Products and Platforms
As such, when developing its strategy for innovation procurement, Croatia should ensure that the strategy aligns with these existing sectoral policies through the institutional coordination mechanism.
Performance measurement frameworks of public procurement contribute to i) assessing progress and achievements periodically and consistently and ii) identifying potential gaps against objectives and targets. This approach enables governments, contracting authorities and other key stakeholders to take relevant actions and/or to tailor specific strategies. The OECD Recommendation on Public Procurement highlights the need to drive performance improvements through evaluation of the effectiveness of the public procurement system, from individual procurements to the system as a whole, at all levels of government where feasible and appropriate (OECD, 2015[22]) (see Box 2.4).
I. Assess periodically and consistently the results of the procurement process. Public procurement systems should collect consistent, up-to-date and reliable information and use data on prior procurements, particularly regarding price and overall costs, in structuring new needs assessments, as they provide a valuable source of insight and could guide future procurement decisions.
II. Develop indicators to measure performance, effectiveness and savings of the public procurement system for benchmarking and to support strategic policy making on public procurement.
Source: (OECD, 2015[22])
Croatia does not have a formal public procurement performance measurement framework. However, MINGO, as one of its mandates specified in article 436 of the PPA, prepares and issues a Statistical Annual Report on Public Procurement of the Republic of Croatia in its website each year since 2007. The latest statistical annual report was published for the year 2023 on 19 August 2024. The report includes a wide range of information and statistics on public procurement procedures implemented in Croatia: the number and value of procurement procedures by various classifications (e.g. types of procurement procedure, types of public entities, sector, subject matter of contract, county, FA/DPS), the average number of bids, statistics on contracts awarded to SMEs, and contracts awarded through BPQR criteria, the top 10 largest contracting authorities, and information on green public procurement (number and value) (MINGO, 2024[3]). MINGO worked with the World Bank on measuring performance of public procurement. This collaboration led to a proposal for some key performance indicators (KPIs) related to cost efficiency, timely delivery, and transparency, quality and fairness (The World Bank, 2021[23]).
Croatia recognises the relevance and the need for setting up key performance indicators to measure the performance of innovation procurement in Croatia and using the results for future policy making. This section discusses the development of a monitoring system of innovation procurement in Croatia. It describes the key performance indicators of innovation procurement identified by MINGO and the OECD.
2.2.1. Developing key performance indicators to measure the performance of innovation procurement in Croatia
As part of the project Supporting implementation of innovation procurement in Croatia, the OECD, together with MINGO, identified potential KPIs of innovation procurement, by building upon and adapting the KPIs proposed in the OECD performance measurement framework of public procurement (See Box 2.5).
The OECD performance measurement framework consists of 259 KPIs which are highly flexible and adaptable to the individual needs of different stakeholders (a country, a contracting authority or a CPB). It considers different elements:
Assessing the performance of public procurement at three levels (tender level, contracting authority level and national level).
Identifying three dimensions in line with compliance (68 indicators), efficiency (128 indicators) and strategic (63 indicators):
Compliance KPIs aim at assessing whether procurement processes and outcomes are in line with the national or any other applicable legislation. Indicators are related to publication/transparency requirements, ex-ante control / audit findings, sanctions, integrity matters, appeals/litigation, and compliance with payment delays.
Efficiency KPIs aim at assessing whether the procurement processes enable to achieve the best procurement outcomes and effectiveness and the best “value for money”. For instance, efficiency KPIs could cover savings (in monetary value and time), level of market participation in specific procedures, duration of procurement processes (including the tender evaluation phase), the planning of procurement activities, the implementation of different efficiency tools such as framework agreements or Dynamic Purchasing Systems (DPS), contract modification, professionalisation issues and payment.
Strategic KPIs aimed at assessing how public procurement processes and outcomes contribute to achieving pressing policy goals such as fighting against climate change, promoting innovation, creating jobs, social inclusion (i.e. gender, vulnerable groups, etc.). and the development of small and medium enterprises. In this context, KPIs could include the share of sustainable goods and services, the share of procurement awarded to SMEs (in number and volume), and the share of procurement involving innovation features.
Covering the whole procurement cycle from planning to contract management (pre-tender, tender, contract management).
Usable and adaptable by different stakeholders such as contracting authorities, public procurement authorities, and CPBs.
Each KPI can be also classified into ‘core’ and ‘aspirational’ indicators by each user. Core indicators can be defined as the minimum indicators that should be tracked as part of the performance measurement. In contrast, aspirational indicators can be considered optional, or to be implemented at a later stage when the performance measurement is more mature.
Source: (OECD, 2023[24])
After a series of discussions, MINGO and the OECD agreed on 33 key performance indicators to measure the performance of innovation procurement in Croatia. These 33 indicators are grouped within 8 categories: (i) General statistics, (ii) Pre-tender, (iii) Tender, (iv) Post-tender (contract management), (v) Ex-ante controls, (vi) Ex-post controls, (vii) Remedy, and (viii) Professionalisation. Among those 33 indicators, 15 indicators were classified as core (or relevant) indicators, while the rest of 18 indicators were classified as aspirational (potentially relevant in the future) ones. This implies that 15 indicators are the minimum indicators that should be tracked as part of the performance measurement of innovation procurement, while the measurement of the remaining 18 indicators will be implemented at a later stage, when the performance measurement framework and/or the state of play of innovation procurement become more mature (see Table 2.6).
The 33 indicators consist of 56 sub-indicators. For example, one indicator on the number of innovation procurement procedures is divided into eight sub-indicators, depending on the procurement procedures and methods: (i) Open procedure, (ii) Restricted procedure, (iii) Competitive procedure with negotiations, (iv) Competitive dialogue, (v) Innovation partnerships, (vi) Design contest, (vii) Negotiated procedure without prior publication and (viii) Pre-commercial procurement (PCP). One indicator on the average time for decision on remedy is divided into two sub-indicators: (i) before DKOM and (ii) before High Administrative Court of the Republic of Croatia. A list of all indicators is available in Annex B.
Source: Created by the author
2.2.2. Ensuring the availability and access of relevant quality data to assess the performance of innovation procurement
There are many challenges in measuring the performance of public procurement system such as the lack of available, accessible, usable, quality data (OECD, 2019[25]). The lack of data often represents a bottleneck to the implementation of performance measurement frameworks. Therefore, it is necessary to map the availability of the data to calculate each indicator. This enables policy makers to identify how many indicators can be calculated under the current situation and address the data gaps by identifying the missing data necessary to calculate selected indicators. This also entails mapping relevant data sources (i.e. identifying the data owner for each indicator), clarifying the format of data and assessing the data quality.
Table 2.7 shows the number of indicators which were identified to measure the performance of innovation procurement in Croatia and can be calculable in a certain way (manually and/or digitally).
Note: Yes: Indicator is calculable at least manually (even if it takes time to collect the data), No: Indicator is not calculable even manually (no way to calculate the indicator due to a lack of data)
Source: Created by the author
Croatia is equipped for launching the performance measurement framework of innovation procurement, because almost all core indicators can already be measured (see Annex B for the data availability for each indicator). Overall, 92.9% (52 out of 56 indicators) can be calculated in a certain way (manually and/or digitally). Four indicators (2 core and 2 aspirational) are not calculable at the moment. These two core indicators are related to the pre-commercial procurement: (i) the number of innovation procurement procedures carried out through PCP, and (ii) procurement volume of innovation procurement procedures carried out through PCP. The two aspirational indicators are (i) the share of innovation partnership that procured the outcome of innovation (number) due to a lack of data entry point on the number of innovation partnership that procured the outcome of innovation, and (ii) the share of PCP that triggered PPI to procure the outcome of R&D rather than just R&D services (number) due to a lack of the data entry points.
It should be noted, however, that collecting the data on PCP is a complex task, as PCP is outside the scope of the PPA and the EU Directives, and therefore cannot be collected through the e-procurement system unlike other procurement procedures foreseen in the PPA and the EU Directives (e.g. open procedure, competitive dialogue and innovation partnerships). Therefore, in the long term, a mechanism to collect the data on PCP would become essential to measure indicators related to PCP. For example, the Government of Lithuania issued a Resolution of Government that regulates PCP and makes it mandatory for public entities to report on the implementation of PCP to a co-ordination organisation, Innovation Agency (See Box 2.6).
The government of Lithuania issued a Resolution of Government to regulate the R&D procurement including pre-commercial procurement (PCP), which is outside the scope of the public procurement law. Article 64 of this Resolution requires public entities to obtain positive feedback from the co-ordinating organisation (Innovation Agency), before starting pre-commercial procurement, regarding the compliance with regulations of PCP. Thus, public entities are not allowed to implement PCP without undergoing this process with the Innovation Agency.
This process ensures that the co-ordinating organisation (Innovation Agency) gathers all the information on the entire PCP cycle in Lithuania. An expert of Innovation Agency is also assigned to a member of the public procurement committee and gets involved in the contract implementation stage of PCP.
Source: (Government of the Republic of Lithuania, 2020[26]) and information provided by the Ministry of Economy and Innovation of Lithuania
Currently, there are only a few procurement procedures in Croatia which could be labelled as innovation procurement. The successful launch of the performance measurement framework of innovation procurement is subject to the availability of data and calculability of the indicators. It should be noted that the EOJN collects the information on innovation procurement by requiring contracting authorities to fill in the fields related to innovation procurement since 1 January 2024.
However, as Croatia has ambitions to increase the number of innovation procurement procedures (e.g. 50 procedures per year in the future), it would be useful to focus on indicators which can be calculated using digital information. Table 2.8 shows how many indicators can be calculated in a digital format without relying on time-consuming manual tasks.
Note: Yes: Indicator is calculable in the digital format, No: Indicator is not calculable in the digital format
Source: Created by the author
Overall, 83.9% (47 out of 56) of indicators can be calculated in a digital format. 19.4% (18 out of 93) of indicators can be partially calculated in a digital format. There are only 9 indicators (4 core and 5 aspirational) which cannot be calculated in a digital format at all. Four core indicators which are not calculable in a digital format include the two core indicators not calculable even manually (the number of innovation procurement procedures carried out through PCP, and the procurement volume of innovation procurement procedures carried out through PCP) and two core indicators related to appeals made before he High Administrative Court of the Republic of Croatia: (i) Number of challenged innovation procurement procedures before the High Administrative Court of the Republic of Croatia, and (ii) the share of successful decisions (in favour of CA), due to the lack of data entry point on the number of appeals of innovation procurement procedures before High Administrative Court of the Republic of Croatia. Five aspirational indicators which are not calculable in the digital format include the two aspirational indicators not calculable even manually (the share of innovation partnership that procured the outcome of innovation, and the share of pre-commercial procurement that triggered the public procurement of innovative solutions to procure the outcome of R&D) and other three aspirational indicators: (i) the share of innovation procurement procedures that held open market consultation due to the lack of data entry point on the number of innovation procurement procedures that held open market consultation, (ii) the average preparation time for the innovation procurement procedure due to the lack of data entry point on the total number of days for preparing a innovation procurement procedure, and (iii) the average time for decisions by the High Administrative Court of the Republic of Croatia due to the lack of data entry point on the total number of days for a decision by High Administrative Court of the Republic of Croatia. It should be noted again that the absence of the data availability in the digital format will not have a large impact for the time being in the future, as there are a few cases of innovation procurement and therefore it will not take time to collect the data even manually.
Defining the ownership of the performance framework innovation procurement to ensure sound and regular monitoring
It is essential to define who will own and implement the performance measurement framework of innovation procurement. This section briefly discusses different elements to be considered for the smooth implementation of the framework such as defining a leading entity and the implementation period and measures.
MINGO is the owner of the performance framework for innovation procurement, as prescribed by law that sets out that the statistic on public procurement data is the jurisdiction of this ministry. For the time being, MINGO will be able to pilot the identified indicators for all innovation procurement procedures, as these represent only a few procedures per year. As the number of innovation procurement procedures increases in the future, additional capacity may be needed to continue the monitoring exercise. Activities related to the pilot include, but are not limited to, testing the relevance of each indicator, identifying potentially relevant indicators, and checking the complexity to collect the necessary data to calculate each indicator. Overall, the performance measurement framework will be useful to showcase the uptake of innovation procurement in Croatia.
References
[6] Central State Office for Central Public Procurement of the Republic of Croatia (n.d.), Framework agreements, https://sredisnjanabava.gov.hr/okvirni-sporazumi/11 (accessed on 16 September 2024).
[4] City of Koprivnica (2018), Market Sounding Prospectus: Extensive transformation of a prefabricated building, http://www.ppi.koprivnica.hr (accessed on 11 September 2024).
[19] Croatian Parliament (2023), Digital Croatia Strategy for the period until 2032, https://narodne-novine.nn.hr/clanci/sluzbeni/2023_01_2_17.html (accessed on 21 October 2024).
[17] Croatian Parliament (2021), National Development Strategy of the Republic of Croatia until 2030, https://narodne-novine.nn.hr/clanci/sluzbeni/2021_02_13_230.html (accessed on 21 October 2024).
[2] Croatian Parliament (2016), Public Procurement Act of the Republic of Croatia, https://narodne-novine.nn.hr/clanci/sluzbeni/full/2016_12_120_2607.html (accessed on 11 September 2024).
[18] Croatian Parliament (2014), Education, Science and Technology Strategy of the Republic of Croatia, https://narodne-novine.nn.hr/clanci/sluzbeni/2014_10_124_2364.html (accessed on 21 October 2024).
[14] European Commission (2023), Poland sets targets for procurement of R&D and innovative solutions, https://digital-strategy.ec.europa.eu/en/news/poland-sets-targets-procurement-rd-and-innovative-solutions (accessed on 12 September 2024).
[9] European Commission (2021), The strategic use of public procurement for innovation in the digital economy, https://op.europa.eu/en/publication-detail/-/publication/7f5a67ae-8b8e-11eb-b85c-01aa75ed71a1/language-en (accessed on 5 October 2021).
[1] European Commission (2014), Directive 2014/24/EU of the European Parliament and of the Council of 26 February 2014 on public procurement, https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=celex:32014L0024 (accessed on 20 June 2022).
[10] European Commission, G. (2022), Recoverv and Resilience Facility: Operational arrangements between the European Commission and Croatia.
[26] Government of the Republic of Lithuania (2020), Decision on the approval of the description of the procedure for the procurement of research and development services, https://e-seimas.lrs.lt/portal/legalAct/lt/TAD/48ae1cf0391811eabd71c05e81f09716/asr (accessed on 14 October 2024).
[3] MINGO (2024), Statistical report on public procurement in the Republic of Croatia (2023).
[21] MINGOR (2023), Croatian Smart Specialization (RIS) Strategy 2023–2029 (S3), https://mingo.gov.hr/UserDocsImages/slike/Vijesti/2022/S3%20do%202029%20Tekst%20VRH%202023%2012%2013.pdf (accessed on 18 September 2024).
[12] Ministry of Economic Affairs and Employment of Finland (2022), Network-based competence centre for sustainable and innovative public procurement (KEINO), https://tem.fi/en/keino-en (accessed on 12 September 2024).
[20] Ministry of Economy and Sustainable Development of the Republic of Croatia (MINGOR) (2023), Integrated National Energy and Climate Plan for the Republic of Croatia for the period 2021 - 2030.
[11] Ministry of Finance of the Republic of Estonia (2023), Strategic principles of public procurement, https://kliimaministeerium.ee/media/12120/download (accessed on 12 September 2024).
[16] Ministry of Science, Innovation and Universities of Spain (2020), Estrategia Española de Ciencia, Tecnología e Innovación 2021-2027, https://www.ciencia.gob.es/Estrategias-y-Planes/Estrategias/Estrategia-Espanola-de-Ciencia-Tecnologia-e-Innovacion-2021-2027.html;jsessionid=95CFDAF0A0C96CDA8FB5ABD28E17C4C1.1 (accessed on 20 September 2024).
[15] Ministry of the Presidency, Justice and Relations with the Courts of Spain (2024), Real Decreto 364/2024, de 9 de abril, por el que se crea la Comisión, https://www.boe.es (accessed on 20 September 2024).
[8] OECD (2023), “Professionalising the public procurement workforce: A review of current initiatives and challenges”, OECD Public Governance Policy Papers, No. 26, OECD Publishing, Paris, https://doi.org/10.1787/e2eda150-en.
[24] OECD (2023), “Public procurement performance: A framework for measuring efficiency, compliance and strategic goals”, OECD Public Governance Policy Papers, No. 36, OECD Publishing, Paris, https://doi.org/10.1787/0dde73f4-en.
[25] OECD (2019), Reforming Public Procurement: Progress in Implementing the 2015 OECD Recommendation, OECD Public Governance Reviews, OECD Publishing, Paris, https://doi.org/10.1787/1de41738-en.
[22] OECD (2015), OECD Recommendation of the Council on Public Procurement, http://www.oecd.org/gov/public-procurement/recommendation/OECD-Recommendation-on-Public-Procurement.pdf (accessed on 28 May 2018).
[7] OECD-SIGMA (2016), “Breif 26- Organising Central Public Procurement Functions,”, http://www.sigmaweb.org/publications/Public-Procurement-Policy-Brief-26-200117.pdf (accessed on 19 September 2019).
[5] SAFU (n.d.), About CFCA | SAFU, https://www.safu.hr/en/about-cfca/ (accessed on 28 June 2023).
[13] The Government of the Republic of Lithuania (2020), Resolution on the approval of the National Progress Plan for 2021-2030, https://e-seimas.lrs.lt/portal/legalAct/lt/TAD/c1259440f7dd11eab72ddb4a109da1b5?jfwid=-whxwii77y (accessed on 13 June 2023).
[23] The World Bank (2021), “Measuring Performance of Public Procurement In Croatia: Output 1 Report on the analysis of the effectiveness of the public procurement system in Croatia”.
← 1. For the purpose of this report, “Contracting authoriy(ies)” refers to both “classic contracting authorities” and “contracting entities” under the definition of the EU Directive and the Public Procurement Act, unless otherwise stated.
← 2. MINGO in this report refers to the Directorate for Public Procurement Policy, which is one of the directorates of MINGO in charge of public procurement policy. It should be noted that within MINGO there is a Directorate of Sector for Innovation, which is in charge of overall innovation policy of Croatia.