The localisation pipeline can be divided into two main broad steps. First, using global dictionaries of placenames and text analysis of OECD Creditor Reporting System (CRS) project descriptions,1 it identifies potential cities and regions (candidate places) mentioned in each project. This step is entirely dictionary-based and does not use artificial intelligence (AI). Second, for each geographic candidate identified by the dictionary, AI is used to determine whether it is indeed a subnational location or a false positive.
The Impact of Decentralised Development Co‑operation
Driving Local Governance and the SDGs
The first broad step of the localisation methodology identified candidate city and region names in 40% to 53% of the 97 811 CRS decentralised development co-operation (DDC) projects analysed, depending on the dictionary of cities and regions used. Project descriptions were unavailable for around 3.5% of projects. Among projects with available descriptions, potential locations were detected in 38 670 projects using Natural Earth, 52 066 using GeoNames and 44 815 using the Database of Global Administrative Areas (GADM) (Figure A B.1). Because some project descriptions mention more than one place, either because projects target multiple locations or involve several providers through multilateral arrangements, the total number of place names matched is higher, reaching 80 633 with Natural Earth, 122 767 with GeoNames and 107 542 with the GADM (Figure A B.2). Project descriptions generally provide more detail on partner cities and regions than on OECD Development Assistance Committee (DAC) members, as they tend to specify where activities are implemented. Natural Earth and the GADM are more effective in identifying regions, while GeoNames captures a larger number of city names. However, the relatively high number of provider city matches when using GeoNames may reflect the presence of false positives. For the identification of partner cities and regions, the three sources appear broadly consistent, suggesting reliable results. Based on this, and given that identifiers for providers regions and cities could be collected for most DAC countries reporting DDC ODA through ad hoc requests to relevant data providers building on ECOPER’s work (2024[1]), the final analysis restricts the localisation process based on dictionaries and AI only to recipient regions and cities.
Sources: Author’s elaboration based on OECD (2025[2]), CRS: Creditor Reporting System (flows), https://data-explorer.oecd.org/vis?lc=en&pg=0&snb=18&df[ds]; Natural Earth (2025[3]), Natural Earth: Free vector and raster map data at 1:10m, 1:50m, and 1:110m scales, https://www.naturalearthdata.com (accessed on 8 September 2025); GeoNames (2025[4]), GeoNames geographical database, https://www.geonames.org (accessed on 8 September 2025); GADM (2025[5]), Database of Global Administrative Areas (GADM) version 4.1, https://gadm.org (accessed on 8 September 2025).
Sources: Author’s elaboration based on OECD (2025[2]), CRS: Creditor Reporting System (flows), https://data-explorer.oecd.org/vis?lc=en&pg=0&snb=18&df[ds]; Natural Earth (2025[3]), Natural Earth: Free vector and raster map data at 1:10m, 1:50m, and 1:110m scales, https://www.naturalearthdata.com (accessed on 8 September 2025); GeoNames (2025[4]), GeoNames geographical database, https://www.geonames.org (accessed on 8 September 2025); GADM (2025[5]), Database of Global Administrative Areas (GADM) version 4.1, https://gadm.org (accessed on 8 September 2025).
To identify subnational cities and regions in CRS data, three complementary geographic databases with worldwide coverage were used:
1. Natural Earth provides a balanced set of global locations, including countries, first- and second-level administrative regions, and major cities. It is relatively small, consistent and efficient for large-scale processing. However, it only includes more prominent places and may omit smaller cities.
2. GeoNames is an extensive database covering millions of place names, from large cities and regions to small towns and villages. Its breadth makes it valuable for capturing smaller cities but also introduces complexity and a higher risk of false matches. To minimise such errors while still accounting for small and medium-sized urban areas, the analysis was restricted to cities with 50 000 inhabitants or more.
3. The Database of Global Administrative Areas (GADM) provides detailed names and official boundaries for administrative regions, with a hierarchical structure (ADM1: provinces or states; ADM2: counties or districts, etc.). It offers precise geographic co-ordinates and broad global coverage, filling gaps where Natural Earth may be incomplete. However, the GADM does not contain city data and is therefore used exclusively for regional localisation.
For all three sources, the localisation of administrative regions focused on the first and second administrative tiers of subnational governments, similar to OECD large (TL2) and small (TL3) regions, to ensure consistency across countries.
By combining the three sources, the analysis balances consistency (Natural Earth), rich city-level coverage (GeoNames) and extensive administrative detail (GADM). This complementary approach strengthens the localisation of CRS data by enabling the identification of both cities and regions, while minimising the limitations of each individual database.
Combining the three geographic sources also allows different names or spelling variants referring to the same location to be retained as separate candidates, increasing the likelihood that at least one candidate matches the wording used in the project description and can subsequently be validated by the AI. When candidates from the different dictionaries are consolidated within a project, duplicate entries referring to the same candidate are removed, while relevant name variants are retained for the AI validation step.
Sources: Natural Earth (2025[3]), Natural Earth: Free vector and raster map data at 1:10m, 1:50m, and 1:110m scales, https://www.naturalearthdata.com (accessed on 8 September 2025); GeoNames (2025[4]), GeoNames geographical database, https://www.geonames.org (accessed on 8 September 2025); GADM (2025[5]), Database of Global Administrative Areas (GADM) version 4.1, https://gadm.org (accessed on 8 September 2025).
Assessing the performance of the geographic dictionaries
To assess the performance of the dictionary-based identification of candidate places, a random sample of 500 projects2 was drawn from projects with an available project description for which the geographic dictionaries did not identify any candidate city or region. Each project description was manually reviewed to determine whether it nevertheless contained a reference to a recipient city or region. The review found that 93% of these projects did not contain an identifiable subnational recipient in their description, while only 7% contained at least one recipient subnational location that had not been detected by the dictionaries. This indicates that missed locations were relatively uncommon among projects for which no candidate had initially been identified. In most of the remaining cases, the project description did not contain sufficiently specific information to identify a recipient city or region. Recurring reasons included very general reporting, such as references only to “decentralised development co-operation”, as well as mentions of geographic areas that do not correspond to the territorial units covered by the localisation approach, such as valleys, small islands or broad areas described as the northern part of a country. Other cases referred to very small settlements (including refugee camps), villages or neighbourhoods located within municipalities or other local administrative areas, but which do not themselves correspond to a city, municipality or local administrative unit captured by the geographic dictionaries (which cover first- and second-level administrative regions and cities of at least 50 000 inhabitants).
In the second broad step, these matches or candidate places are validated by AI to ensure that identified candidate place names genuinely refer to subnational geographic locations mentioned in project descriptions. For this purpose, an automated validation process, which uses the OpenAI API and GPT-5.2 as the underlying model, was developed. The method applies structured prompts and follows a two-step process. First, the model assesses whether each candidate string is used as an actual subnational geographic place in the project description, rather than as a country name, organisation, personal name, acronym, common word or other non-geographic reference. Second, where a candidate is identified as a place, the model classifies it as a city, region or unknown type. The prompts instruct the model to base its assessment on how the candidate is used in the specific project description, including surrounding words, rather than relying solely on general knowledge. The model is also instructed to adopt a conservative approach when the available context is insufficient and to account for multilingual project descriptions. The response is returned in JavaScript Object Notation format and includes the original candidate identifier and name, an indicator of whether the candidate is a subnational place, its place type where applicable and a brief reason for the assigned classification.
Assessing the performance of the AI validation
Does the candidate refer to a subnational place in the project description?
The performance of the AI validation step was assessed separately using a random sample of 500 projects for which at least 1 candidate place had been generated by the geographic dictionaries. As individual projects can contain several candidate places, the sample comprised 1 183 candidate-project pairs. For each pair, the AI had classified whether the dictionary-generated candidate genuinely referred to a recipient subnational place in the CRS project description (TRUE or FALSE). A human reviewer independently assessed the same question. Comparing the AI and human classifications made it possible to construct a confusion matrix and identify true positives (TP), false positives (FP), true negatives (TN) and false negatives (FN).
Across the 1 183 candidate-project pairs, 761 were true positives, 34 false positives, 335 true negatives and 53 false negatives. At the candidate level, the AI achieved a precision of 95.7% , meaning that 95.7% of the candidates classified by the AI as recipient subnational places were confirmed by the human review, and a recall of 93.5% , indicating that the AI correctly retained 93.5% of the candidates identified by the reviewer as genuine recipient places. The resulting F1 score – a measure combining precision and recall, with 100% indicating perfect performance – was 94.6% , indicating a high and relatively balanced performance in limiting both false positives and false negatives.
Project-level results were similarly positive. Among projects containing at least one true recipient location, the pipeline identified at least one correct recipient subnational location in 94.9% of cases. In 72.5% of projects, all unique city and region references identified by the reviewer were captured, while in 22.3%, some, but not all, were captured. The latter should not necessarily be interpreted as a failure to identify the recipient territory, as project descriptions may refer to the same intervention at different nested territorial levels, for example both a city and the region in which it is located. In such cases, identifying either location may be sufficient to localise the project, while the difference concerns mainly the granularity and completeness of the localisation. On average, the pipeline captured around 84.4% of the unique recipient locations identified within each project.
If so, what type of subnational place is it?
A secondary step assessed the AI’s ability to classify identified subnational locations as either a city or a region. Candidate places already carried a territorial type from the underlying geographic dictionaries, as each dictionary entry is classified as a city or region. However, the same or similar place name can sometimes appear more than once in the dictionaries and may refer to different territorial levels within the same country. The AI was therefore asked to infer the territorial type independently, based on the way the candidate was used in the project description and drawing on general definitions of cities and regions provided in the prompt, rather than to validate the type inherited from the geographic dictionary.
This assessment was conducted on the 761 true-positive candidate-project pairs, for which both the AI and the human reviewer had confirmed that the candidate was a genuine recipient location and therefore provided a place type. Overall, the AI-assigned type matched the human classification in 80.7% of cases. This relatively lower agreement partly reflects the conservative approach built into the AI prompt: where the available information was insufficient to distinguish confidently between a city and a region, the model was instructed to classify the type as unknown. In 124 cases (16.3% of true positives), the AI returned an unknown type that the human reviewer was subsequently able to classify, drawing where necessary on additional contextual knowledge and external verification. When considering only cases in which the AI itself classified the location as either a city or a region, the human reviewer agreed with the inferred territorial type in over 96% of cases. For the final classification used in the analysis, the AI-inferred type was retained whenever the model classified a location as a city or region. Where the AI returned an unknown type, the territorial type inherited from the geographic dictionary was used. Where the same place name appeared in the dictionaries as both a city and a region, the location was treated as ambiguous and labelled “region or city” rather than assigned to a single territorial level.
A recurring source of uncertainty was that a city and the region in which it is located can share the same name, particularly where a region takes the name of its principal city. In such cases, distinguishing between the two territorial levels from the project description alone may not always be possible. This ambiguity does not necessarily imply that the geographic target has been incorrectly identified, but rather affects the level of territorial granularity at which the project can be localised. To account for this issue in the quantitative analysis, regional- and city-level results are analysed separately. The regional dataset includes locations identified as regions as well as ambiguous cases that could refer to either a region or a city, while the city dataset similarly includes locations identified as cities together with these ambiguous cases. The two territorial levels are therefore not combined within the same analysis: DDC ODA localised at the regional level is linked to regional outcome indicators, while city-level ODA is analysed against city-level indicators. This approach preserves the available geographic information while avoiding the need to impose an arbitrary territorial classification in genuinely ambiguous cases.
The results above indicate that the localisation pipeline performs well in identifying and validating recipient regions and cities, while also highlighting areas where it could be further developed. First, the geographic dictionaries used in the analysis cover first- and second-level administrative regions and cities of at least 50 000 inhabitants. As the coverage and quality of global subnational geographic data improve, the approach could be extended to smaller cities and lower-level subnational administrative units, allowing a larger share of geographically specific projects to be localised.
Further improvements could also strengthen the classification and disambiguation of locations. The current approach deliberately relies primarily on information contained in CRS project descriptions and adopts conservative prompts where the available evidence is insufficient. Future versions could combine project descriptions with other CRS fields and retrieved external geographic information to improve the identification of territorial types and distinguish, for example, between a city and a region sharing the same name. More advanced AI-based pipelines could also increasingly support the initial identification and disambiguation of place names. The current dictionary-first approach was nevertheless chosen to limit the risk of hallucination by asking the AI to validate candidate locations drawn from established geographic sources rather than generate locations directly from unstructured project text.
Ultimately, the most robust solution would be for provider and recipient regions and cities to be reported directly in the CRS using harmonised geographic identifiers and classifications. In the absence of such systematic reporting, the approach developed here provides a first step towards localising CRS development finance data at the subnational level and can be progressively refined as geographic data, reporting practices and AI-based methods improve.
References
[1] ECOPER (2024), Decentralised Cooperation Report: Mapping Donor Cities, https://www.elankidetza.euskadi.eus/contenidos/documentacion/publicaciones_descentralizada/es_def/INFORME-2024-ENG-.pdf.
[5] GADM (2025), Database of Global Administrative Areas (GADM) version 4.1, University of California, Berkeley, https://gadm.org (accessed on 8 September 2025).
[4] GeoNames (2025), GeoNames geographical database, https://www.geonames.org (accessed on 8 September 2025).
[3] Natural Earth (2025), Natural Earth: Free vector and raster map data at 1:10m, 1:50m, and 1:110m scales, North American Cartographic Information Society, https://www.naturalearthdata.com (accessed on 8 September 2025).
[2] OECD (2025), CRS: Creditor Reporting System (flows), OECD, Paris, https://data-explorer.oecd.org/vis?lc=en&pg=0&snb=18&df[ds].
← 1. Project descriptions combine two variables in the CRS database: “LongDescription” (a long description of project activities) and “Geography” (the geographical target area). While the latter is intended to report the location where the project is implemented, in practice it is not always completed. Moreover, as it does not rely on harmonised lists of regions and cities, and provides limited guidance on how locations should be reported, the quality of the information varies. In some cases, the field reports the provider region or city rather than the recipient location, or simply repeats the recipient country without identifying a subnational place.
← 2. The validation exercises are based on random samples of 500 projects. Under conservative assumptions for simple random sampling, this sample size provides project-level estimates with a maximum margin of error of approximately ±4.4 percentage points at the 95% confidence level.