The impact of digital technologies on students’ learning in Ireland

2. Measuring the impact of digital technologies in education

After the introductory session, the second part of the event focused on the measurement of the impact of digital technologies in education. The complexity of this issue was a key point of discussion during the event, as participants highlighted that assessing the impact of digital technologies in education is a complex and ongoing process. As stated by several authors (Golden, 2020[6]; Hallie Preskill et al., 2013[7]), policy evaluation in education entails understanding complex contexts and relationships within education systems, as well as the specific reforms being implemented. Part of this involves the use of indicator-based frameworks, which allow policymakers to assess the availability, use, and outcomes of, in this case, digital technologies in schools. These frameworks track metrics such as ICT infrastructure (e.g., device-to-student ratios and internet access), usage patterns in classrooms, and learning outcomes like academic performance and digital skills.

For instance, international testing frameworks like PISA offer data and overviews of students’ ability to solve complex problems, think critically, and communicate effectively. The same applies for the testing framework of ICILS (International Computer and Information Literacy Study) in relation to students’ computer and information literacy (CIL) and computational thinking (CT)1. However, while these indicators provide valuable insights into usage patterns and correlations, they do not establish causality or fully capture the nuanced effects of digital technologies on educational outcomes (see Box 1). Because of this, some authors (Golden, 2020[6]; Hallie Preskill et al., 2013[7]; Daniel L. Stufflebeam, 2001[8]) stress the need for more comprehensive evaluation strategies that integrate both quantitative and qualitative methods to fully understand the complexities of educational reforms.

Qualitative and mixed methods assessments – such as interviews or focus groups with teachers, students, and parents – provide crucial context for understanding how digital technologies are perceived and used in practice. Surveys and classroom observations, for example, can complement self-assessment data by offering additional insights into how digital technologies are used and how their use correlates with student engagement and teacher practices (Joint Research Centre, 2023[9]). These methods allow policymakers to consider stakeholder needs, identify implementation barriers, and, ultimately, find ways to address them. Though these data sources can help policymakers track trends and correlations, to assess actual impact and causality mechanisms, additional research approaches – such as experimental designs (e.g., Randomised Control Trials) and longitudinal studies – are required.

Randomised Control Trials (RCTs) allow policymakers to establish causal links between technology use and educational improvements by comparing outcomes between treatment and control groups (Cukurova and Luckin, 2019[10]). Therefore, this type of study offers robust evidence to inform scalable policy decisions and identify effective practices across diverse educational contexts (Golden, 2020[6]). Longitudinal studies allow policymakers to track the long-term effects of digital technologies on students’ learning trajectories, career readiness, and overall educational equity. This methodology allows to assess the sustainability of reforms as well as their impact over time (Golden, 2020[6]).

To assess actual impact and causality mechanisms, […] research approaches – such as longitudinal studies or experimental designs (e.g., Randomised Control Trials) – are required.

On the grounds of these strategies and regular evaluations, policymakers can monitor progress and adjust policies based on evidence. By using these methodologies, policymakers can gather robust evidence to support informed decision-making, enabling governments to design policies that maximise the benefits of digital tools while addressing challenges such as inequity and misuse. This evidence-based approach ensures that investments in educational technology contribute meaningfully to achieving broader educational goals. To achieve this, comprehensive digital education strategies are essential in addition to these methods. In fact, strategies can guide policy efforts, as they set clear objectives for the integration of technologies with measurable targets (e.g., for infrastructure development, teacher education, professional learning, and student outcomes) (OECD, 2023[11]).

The experiences of Estonia, France, Sweden and Ireland

Estonia

Estonia2 is widely acknowledged as a pioneer in the integration of digital technologies into education, serving as a model for other countries. In 1996, Estonia launched the Tiger Leap programme (Tiigrihüpe), which was critical for the country’s digital policy (Tiger Leap Foundation, 2007[12]). This ambitious initiative aimed to digitise schools by providing them with robust technological infrastructure, promoting digital literacy, and enhancing teachers’ ability to effectively integrate technology into their teaching. The goal was to ensure that technology would support the achievement of learning outcomes as outlined in the national curriculum. By the early 2000s, all Estonian schools had internet access and computer labs, creating a strong foundation for the subsequent integration of digital tools (Digital Skills & Jobs Platform, 2022[13]). Building on the Tiger Leap programme, Estonia will launch the AI Leap 2025 (TI-Hüpe 2025)3 programme to provide students and teachers with free access to leading AI tools as well as equip them with the skills needed to use them effectively in education (e-Estonia, 2025[14]).

Further, since 2014, Estonia’s national curriculum has included digital competence as a core objective, further embedding technology into education. This focus has been reinforced by frameworks such as the Lifelong Learning Strategy 2020 and the ongoing Education 2035 strategy, which stresses the role of technology in improving the quality and inclusiveness of education (Figure 2). Additionally, according to the expert from Estonia, the country has made efforts to combine its infrastructure investments with initiatives to boost digital skills. For instance, the ProgeTiger programme, launched in 2012, promotes coding, robotics, and other technological competencies among students and educators. The programme also supports continuous teacher training to ensure educators can effectively integrate digital tools into their teaching practices (Education Estonia, 2020[15]).

Despite these advancements, Estonia faces challenges in measuring the direct impact of digital technologies on educational outcomes. As current metrics primarily capture access to digital technologies and digital competencies, there is a recognised need for longitudinal research and comprehensive metrics that link digital device usage to academic performance and critical thinking development (Education Estonia, 2024[16]).

Figure 2. Underlying principles and values of Estonia’s Education 2035 Strategy

Source: Ministry of Education and Research (Estonia) (2021[17]), Education Strategy 2021–2035, https://www.hm.ee/sites/default/files/documents/2022-10/haridusvaldkonna_arengukava_2035_kinnittaud_vv_eng_0.pdf

To address this gap, Estonia has initiated studies such as the DigiEfekt Study (2020–2023). Conducted by the University of Tartu, this research examines how digital technologies influence student outcomes in mathematics, science, and digital literacy (Education Estonia, 2024[16]). Early findings suggest that lessons leveraging interactive digital tools to connect prior knowledge with new concepts yield positive outcomes, whereas passive substitution of traditional tools with digital ones has limited effects (Education Estonia, 2020[15]). Crucially, this study found that the nature of technology use is more influential than frequency alone. Interactive or constructive uses of technology demonstrated positive effects on learning outcomes, while passive uses resulted in no or even negative effects depending on the specific learning outcomes measured. These findings underscore the importance of considering not just how often, but how effectively technology is used in educational settings to maximise its positive impact on student learning. Future research and assessments may, thus, need to focus more explicitly on the quality and effectiveness of technology integration to provide a more comprehensive picture of its impact on education.

France

In his intervention, the expert from France4 highlighted that his country has adopted a comprehensive, data-driven approach to integrating digital technologies in education. According to the expert, central to this strategy is EduPilote, a platform aiming to collect, aggregate, and analyse data on the use of digital tools and resources across public, lower secondary (11 to 15 years old) and upper secondary (15 to 18 years old) schools nationwide. As of April 2025, a functional prototype of the platform is being used at the Direction du Numérique pour l'Éducation (DNE)5. A first version shall be released to the public by the end of 2025. By leveraging features such as dynamic navigation and data visualisation tools, EduPilote will enable policymakers, local authorities, and educators to track and evaluate the impact of digital technologies on teaching and learning outcomes. This platform will, hence, provide detailed insights into device usage and engagement levels, facilitating informed decision-making tailored to the specific needs of individual schools and districts.

Complementary to this upcoming platform, France’s “Territoires Numériques Éducatifs” (TNE) programme supports the pedagogical use of digital resources across 12 selected regions through four main goals: 1) train teachers; 2) equip classrooms; 3) make digital resources and tools available; and 4) support parents. The programme offers webinars, training opportunities, digital resources across a variety of digital related topics, as well as a variety of “educational digital solutions” (e.g., online libraries, learning language tools, etc.) for all educational levels (Ministère de l’Éducation nationale, 2024[18]).

The French expert also mentioned that the 2015 “Digital Plan for Education” was pivotal in supporting the adoption of digital technologies at every educational level in France. According to the expert, this plan prioritised teacher training, the development of high-quality digital content, and the integration of technology into curricula to enhance teaching in primary and lower secondary schools. Complementing this effort, the ongoing “École Numérique Rurale (ENR)” programme provides funding for schools to acquire digital tools and resources. This programme has played a critical role in addressing disparities by ensuring equitable access to technology across both urban and rural areas (Ministère de l’Éducation nationale et de la Jeunesse, 2024[19]).

To assess the effects of the 2015 Digital Plan for Education, the Direction de l’Évaluation de la Prospective et de la performance (DEPP) conducted the Élaine (i.e. the Longitudinal Evaluation of Activities related to Digital Education) study in 2017. The study tracked over 5,000 students in 7th and 8th grade and focused on the Plan’s deployment of mobile digital equipment in classrooms. Specifically, it assessed two initiatives: the “1-to-1 Tablets” initiative (for both in-school and at-home use) and the “Mobile Classrooms” (MC) initiative (for individual and shared use in the classroom only). Findings showed increased tablet access and usage in schools, alongside improvements in students’ digital, mathematical, and language (reading and oral comprehension) skills. The 1-to-1 Tablets programme yielded stronger educational benefits than the MC initiative. However, impacts varied by subject, grade level, and were most significant among students from low socio-economic status and in disadvantaged schools (Direction de L’Évaluation de la Prospective et de la performance (DEPP), 2017[20]).

Currently, France’s “Digital Strategy for Education 2023–2027” – i.e. the official digital strategy developed by the then ministère de l'Éducation nationale et de la Jeunesse (now ministère de l’Éducation nationale, de l’Enseignement supérieur et de la Recherche) – provides a roadmap for integrating digital technology into the French education system for the 2023 to 2027 period. The strategy outlines the vision, priorities, and key actions set out to support digital transformation in French schools and to ensure that students and educators develop the digital skills needed in today’s world. Further, the strategy relies on four main axes: 1. A committed ecosystem in support of a shared public policy; 2. Teaching digital literacy to build citizenship and digital competence; 3. An educational community supported by a thoughtful, sustainable, and inclusive digital offering; 4. A renewed framework for a (ministerial) information system designed to serve its users (Ministère de l’Éducation nationale et de la Jeunesse, 2023[21]).

Despite significant progress in France, challenges remain. Disparities in digital literacy levels among teachers and students persist, prompting targeted professional development initiatives and resource-sharing platforms to bridge these gaps. Additionally, systemic issues such as inconsistent internet connectivity in rural areas and growing cybersecurity risks continue to hinder full-scale implementation. Programmes like “Numérique Inclusif” aim to address these inequities by promoting broader access to digital tools and skills (Ministère de l’Éducation nationale et de la Jeunesse, 2024[19]).

Sweden

Sweden6 has long been recognised for its high level of digital inclusion in education, supported by widespread access to devices and robust digital infrastructure. In his intervention, the expert from Sweden brought forward that schools in the country began introducing computers into classrooms as early as the 1980s, and that authorities (local government or municipalities) have consistently invested in digital tools and platforms to support learning. Since 2017, the National Digitalisation Strategy for the School System shapes the integration of digital resources in education with the aim of enhancing teaching quality and equipping students with critical digital competencies. The strategy focuses on three key areas: ensuring equal access to digital tools, providing adequate infrastructure, and promoting research on digitalisation's impact (Andric, 2023[22]).

One of the key characteristics of Sweden’s digital education is its effort to emphasise critical thinking and media literacy. Recognising the prevalence of misinformation online, Swedish schools have incorporated media literacy into their curricula since 2018. This includes teaching students how to critically evaluate digital content and understand programming concepts in technology subjects. These efforts align with Sweden’s broader goal of preparing students to navigate an increasingly complex digital world (European Commission, 2024[23]).

To build on its successes, Sweden is exploring ways to enhance data collection and analysis related to digital education. Initiatives aimed at linking device usage with academic performance metrics have been piloted under the National Digitalisation Strategy. These efforts seek to develop a more comprehensive understanding of how technology impacts classroom learning outcomes (Skolverket, 2022[24]). This experience underscores the importance of balancing innovation with rigorous evaluation to ensure that digital tools contribute meaningfully to student learning.

Despite these efforts, Sweden has faced challenges in the systematically measuring the direct impact of digital technologies on learning outcomes. For example, it has been difficult to examine how digital competencies – assessed by studies as ICILS – translate to improved results in subjects like mathematics or science (Fraillon et al., 2020[25]). Qualitative evidence from educators suggests that digital tools promote student engagement, creativity, and collaboration. According to the expert, many teachers report using digital technologies to facilitate group projects, enable real-time feedback, and customise lessons to meet individual learning needs. However, systematic data collection linking these practices to measurable improvements in academic outcomes remains limited. This gap is also highlighted in reports published by the Swedish National Agency for Education (Skolverket). The reports also emphasise the need for enhanced data collection mechanisms (Skolverket, 2022[24]).

According to the expert, Sweden also faces equity challenges in its digital education landscape. While most students have access to devices at school, disparities in digital literacy and home internet access persist, particularly among socio-economically disadvantaged groups. To address these gaps, the government has launched initiatives such as the provision of funding for schools to purchase devices and implement teacher training programmes on effective digital pedagogy. Additionally, the "Skoldigistöd" initiative organised by the SALAR (Swedish Association of Local Authorities and Regions) supports equitable access to technology by helping schools procure and implement digital tools effectively (OECD, 2023, p. 299[26]).

Based on the interventions from the different experts, several challenges are common when it comes to measuring the impact of digital technologies in education across Estonia, France, and Sweden. One significant issue is the lack of standardised metrics. There is a notable gap in measurements that go beyond access to digital technologies. While international assessments like ICILS (in which Ireland has not participated to date) measure digital competencies, there is a lack of comprehensive metrics assessing how the use of technology directly influences academic performance, critical thinking, and other learning outcomes, as discussed in the relevant section of this document (OECD, 2023[27]). These countries also lack comprehensive longitudinal studies that directly link digital device usage to learning outcomes, something that is missing in many other OECD countries as well (OECD, 2023[27]).

Ireland

In their turn, representatives from Ireland7 offered a presentation of their own situation. Ireland’s approach to measuring the impact of digital technologies in education reflects a structured and stakeholder-driven strategy, with room to enhance its evaluation mechanisms. A key element for this is the Digital Learning Framework (DLF), a tool which assists schools and teachers to plan and monitor their progress in embedding digital technologies into teaching and learning. The tool provides schools with clear descriptors of relevant digital competencies for teachers and school leaders, emphasising the use of technology to improve teaching, learning, and assessment outcomes. The DLF also provides a Digital Learning Planning Cycle teachers and schools can engage with as develop their own digital plan. The Cycle is composed of six steps (see Figure 3): the first three (i.e. “identify”, “gather evidence”, and “analyse and make judgements”) form the investigation phase, the fourth (“write and share reports”) represent the creation phase, the fifth (i.e. “put improvement and plan into action) represents the implementation phase, and the last (“monitor actions and evaluate impact”) represents the evaluations stage (Oide Technology in Education, 2024[28]). By focusing on the alignment of digital practices with pedagogical goals, the DLF ensures that digital tools contribute meaningfully to student achievement.

Figure 3. The six steps of the Digital Learning Planning Cycle

Source: Oide Technology in Education (2024[29]), Digital Learning Framework - Planning Resources, https://www.dlplanning.ie/

Further, the Irish Primary Curriculum Framework outlines “being a digital learner” as one of seven key competencies to embed across all curriculum areas and subjects (see Figure 4). This aims to make children curious, creative, confident, and critical users of digital technology while also fostering a responsible, safe and ethical use of technology. The framework also proposes that digital technology use would become part of the curriculum area of Science, Technology, and Mathematics with its own subject specification for third to sixth class. This approach highlights Ireland’s aim to integrate digital competencies into everyday teaching practices across different educational levels, especially with strategies aligned to the technological advancements shaping the future.

Figure 4. The Primary Curriculum Framework’s core competencies

Source: National Council for Curriculum and Assessment (2023[30]). Primary Curriculum Framework for Primary and Special Schools. https://curriculumonline.ie/getmedia/84747851-0581-431b-b4d7-dc6ee850883e/2023-Primary-Framework-ENG-screen.pdf

At the post-primary level (i.e. junior and senior cycle levels), students’ competencies are further built on this foundation. The junior cycle English curriculum supports students in using literacy skills to manage information, and to find, evaluate, synthesise, and communicate information across various media. The curriculum emphasises the use of non-literary texts – such as advertisements, media broadcasts, and opinion pieces (e.g., articles, speeches, social media posts) – to help students interpret meaning, assess effectiveness, and respond critically to digital media. The aim is for students to analyse key ideas, style, tone, content, and overall impact of these texts in a systematic way.

At the senior cycle, the Key Competencies Framework informs the development of subjects, modules, and programmes. Ireland’s National Council for Curriculum and Assessment (NCCA) has published a schedule for the redevelopment of all Senior Cycle subjects. The Framework emphasises the interconnectedness of key competencies, literacy, and numeracy, and how the development of one supports the others. This framework also underscores how key competencies are enhanced when students use digital tools to aid their learning. Additionally, in line with the Digital Strategy for Schools (DFL), there are ongoing efforts to explore how the use of digital technologies can be incorporated into the curriculum and assessment of revised Leaving Certificate subjects, as appropriate. Moreover, a revised specification for the Leaving Certificate Computer Science will be introduced in September 2027 as part of the third phase of subject revisions. The development group for this revision convened on September 2024 under NCCA structures, drawing on a brief and research from other jurisdictions to inform its work.

Finally, online safety remains a central focus of Ireland’s strategy to ensure that the benefits of digital tools are not undermined by risks like cyberbullying or distractions. To this purpose, Ireland has developed resources like the Webwise programme, which supports schools in promoting a safe and responsible use of digital technologies (Department of Education and Youth, 2024[31]). Further, in terms of funding and infrastructure, Ireland’s ICT Grant Scheme and the Schools Broadband Programme support schools in acquiring the necessary resources to enhance digital learning environments. These initiatives provide schools with the autonomy to tailor their digital leaning plans while ensuring alignment with national standards and guidelines. Lastly, recent advances in artificial intelligence, particularly generative AI, have prompted discussions about their potential impact on assessment. The State Examinations Commission (SEC) has been tasked with commissioning research on the role of generative AI in teacher-based assessment, exploring both the opportunities and challenges these technologies present.

Key considerations for Ireland

At the end of this session, several key considerations emerged. To begin with, Ireland has made notable progress in supporting the embedding of digital technologies in education. This was achieved through initiatives as the Digital Strategy for Schools, the Digital Learning Framework (DLF), the curricular reforms, and the provision of a wide range of professional learning opportunities for teachers which all support a structured and pedagogical approach to the integration of technology into teaching and learning. Additionally, the ICT Grant Scheme and Schools Broadband Programme have bolstered infrastructure, supporting schools in enhancing their digital learning environments. However, Ireland could learn from the experiences of Estonia, France, and Sweden on how to address some of its measurement gaps.

Estonia’s concrete effort to provide rigorous data on the effects of digital technologies on student learning, could offer insights that Ireland might adopt to strengthen its long-term monitoring of digital tools’ effectiveness. France’s EduPilote platform (prototype) exemplifies a data-driven initiative that links technology use to learning outcomes, a strategy Ireland could explore to improve evidence-based policy decisions. On this, Sweden has also identified the need to address the challenge of linking digital tools usage directly to academic performance; a challenge that can be tackled by developing more robust, standardised metrics and experimental studies (like RCTs) to assess the full impact of digital technologies. By building on these international experiences, Ireland could refine its approach to measuring the real-world outcomes of its digital education strategies, ensuring that technology adoption translates into tangible improvements in student outcomes.

Notes

← 1.  The International Computer and Information Literacy Study (ICILS) is a computer-based international assessment developed by the International Association for the Evaluation of Educational Achievement (IEA). It evaluates the ability of Grade 8 students (approximately 13.5 years old) to use information and communication technology (ICT) effectively for study, work, and life in a digital world. The test focuses on two main domains: 1) Computer and Information Literacy (CIL): this measures skills in using computers to investigate, create, and communicate, including tasks such as gathering, managing, and producing information digitally; and 2) Computational Thinking (CT): this assesses the ability to conceptualise problems and develop solutions using computational approaches. The ICILS also collects contextual data through questionnaires for students, teachers, school principals, and ICT coordinators to understand the environments influencing ICT education. First conducted in 2013, the study has been repeated in 2018, 2023 and is planned for 2028. ICILS provides policymakers with insights into digital literacy education across countries and tracks progress over time (IEA, 2024[38]).

← 2.  For a list of Estonia’s digital education initiatives please see Table B.1 in Annex B.

← 3.  In its first phase (from September 2025), the programme will involve 20,000 high school students (grades 10–11) and 3,000 teachers, with plans to expand to vocational schools and new incoming 10th-grade students in 2026 to add 38,000 students and 2,000 teachers to the pool. The initiative lies on a public-private cooperation where Estonia's Ministry of Education and Research is working with global AI developers to reach the programme’s goal of providing free access to AI applications and training (e-Estonia, 2025[14]).

← 4.  For a list of France’s digital education initiatives please see Table B.2 in Annex B.

← 5.  The Direction Du Numérique Pour L'éducation the directorate responsible for integrating digital technology into the educatoinal system within the “Ministère de l’Education nationale, de l’Enseignement supérieur et de la Recherche”. The directorate promotes the use of digital tools in teaching and learning, supports educational innovation, and ensures access to digital resources.

← 6.  For a list of Sweden’s digital education initiatives please see Table B.4 in Annex B.

← 7.  For a list of Estonia’s digital education initiatives please see Table B.3 in Annex B.